JP2013150069A - Imaging device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device, an image processing method and a program, capable of capturing a natural moving image having no strangeness even when two video effects having different processing speeds are imparted to a moving image frame.SOLUTION: An imaging device includes: an art effect image processing section 162a for imparting first and second special effects having processing speeds and visual effects different from each other to a plurality of frames corresponding to image data of a moving image, respectively, to generate first special effect data and second special effect data, respectively; an in-moving image special effect image processing section 163a for combining the first special effect data and the second special effect data generated by the art effect image processing section 162a to generate combined image data; and an image processing control section 291 for changing with time a ratio at which the in-moving image special effect image processing section 163a combines the first special effect data and the second special effect data.

Description

  The present invention relates to an imaging apparatus, an image processing method, and a program for imaging a subject and generating image data of the subject.

  In recent years, an imaging apparatus such as a digital camera is not limited to the function of generating a beautiful image having a natural impression, and has never been achieved by intentionally adding effects such as noise, shading, and blurring to an image. Those having a function of realizing a shooting expression that gives a unique impression are known.

  For example, Patent Document 1 below discloses a technique for generating a high-contrast image while having a grainy feeling (noise feeling) like a film. According to this technique, it is possible to take a rough and powerful image.

  Further, Patent Document 2 below discloses a technique for generating an image in which the peripheral portion is dimmed. According to this technique, an image having an impression as if taken with a toy camera, that is, an image having strong distortion or blur can be taken.

  These conventional techniques can be applied not only to still image shooting but also to moving image shooting. When shooting a moving image, a unique video expression unique to a moving image that does not exist in a still image can be performed by using temporal image transition. Conventionally, such video representation has been realized by applying a video effect that changes over time in order to produce a visual effect over a moving image frame. As a technique for realizing such a video expression, a process (hereinafter referred to as “transit”) in which a video effect applied to one image is changed to another video effect while gradually changing the ratio is known. It has been.

  FIG. 23 is a diagram for explaining an outline of transit performed by a conventional imaging apparatus. FIG. 23 schematically shows a transit that changes from art effect A to art effect B as a video effect. In FIG. 23, the processing speed of art effect A is 30 fps, and the processing speed of art effect B is 15 fps.

As illustrated in FIG. 23, when a start trigger is input, the imaging apparatus generates an art effect image A1 obtained by applying an art effect A to a captured image P1, and records the art effect image A1 as a transit image R71.
Thereafter, the imaging apparatus, at time t = t _12, performs the generation of the artistic effect image B1 subjected to the artistic effect B in the captured image P1, since the processing is not in time, to duplicate the artistic effect image A1 generated immediately before Is recorded as a transit image R72.
After that, at time t = t ₁₃ , the imaging device applies the art effect B to the signal obtained by multiplying the signal of the art effect image A1 by the coefficient b = 0.9 and the photographed image P1 to complete the processing. A transit image R73 is generated by synthesizing a signal obtained by multiplying the above signal by a coefficient 1-b = 0.1 and this transit image R73 is recorded.
Thereafter, the imaging apparatus, at time t = t _14, and generates an artistic effect image A2 which has been subjected to artistic effect A in the captured image P4, recorded as transit images R74 duplicates the transit images R73 generated immediately before.
Thereafter, at time t = t ₁₅ , the imaging apparatus generates an art effect image B1 obtained by applying the art effect B to the captured image P4. However, since the process is not in time, the transit image R74 generated immediately before is copied. Record as a transit image R75.
Thereafter, at time t = t ₁₆ , the imaging apparatus multiplies the signal of the art effect image A2 by the coefficient 0.6 and the signal of the art effect image B2 that has been processed by applying the art effect B to the captured image P4 Is combined with a signal multiplied by a coefficient 1-b = 0.4 to generate a transit image R76, and this transit image R76 is recorded.

  As described above, when a conventional image pickup apparatus records two combined images with video effects having different processing speeds within a predetermined time and records a combined image, the ratio of combining the two images is greatly changed. Shooting.

  FIG. 24 is a diagram for explaining another outline of a transit performed by a conventional imaging apparatus having two image processing units (dual engines). FIG. 24 schematically illustrates a transit that changes from art effect A to art effect B as a video effect. In FIG. 24, the processing speed of art effect A is 30 fps, and the processing speed of art effect B is 30 fps. In FIG. 24, immediately before the input of the transit start trigger, an art effect image A0 obtained by applying an art effect A to the photographed image P0 is generated, and this art effect image A0 is recorded.

When the start trigger is input, the two image processing units respectively apply a signal obtained by multiplying the signal of the art effect image A1 obtained by performing the art effect A to the photographed image P1 by a coefficient b = 0.9, and the photographed image P1. A transit image R81 is generated by synthesizing a signal obtained by multiplying the signal of the art effect image B1 subjected to the art effect B by a coefficient 1-b = 0.1, and the transit image R81 is recorded.
Thereafter, at time t = t ₁₂ , the two image processing units respectively obtain a signal obtained by multiplying the signal of the art effect image A2 obtained by applying the art effect A to the photographed image P2 by the coefficient b = 0.8, and the photographed image P2. A transit image R82 is generated by combining a signal obtained by multiplying the signal of the art effect image B2 obtained by applying the art effect B to the coefficient 1-b = 0.2, and this transit image R82 is recorded.

  As described above, the imaging apparatus generates and records a transit image by synthesizing art effect images having different effects by applying art effects at the same processing speed by the two image processing units.

JP 2010-62836 A JP 2010-74244 A

  However, in the case where transit is performed using the above-described conventional technology, when two video effects having different processing speeds are applied to one moving image frame, the other video effect is applied in synchronization with the application of one video effect. I couldn't. As a result, there is a problem in that the frame rate of the moving image is delayed, so that the captured moving image is uncomfortable.

  In FIG. 24, even when two image processing units are used, there are the above-mentioned problems when two video effects having different processing speeds are applied to one moving image frame.

  The present invention has been made in view of the above, and can capture a moving image that is natural and uncomfortable even when two moving image effects having different processing speeds are applied to the moving image frame. An object is to provide an apparatus, an image processing method, and a program.

  In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention is an imaging apparatus capable of capturing a moving image by capturing an image of a subject and generating image data of the subject. The first special effect image data and the second special effect image data are provided by applying the first special effect and the second special effect, which have different processing speeds and visual effects over a plurality of frames corresponding to the data, respectively. A special effect image processing unit that generates the image, and a special in-video image that generates synthesized image data by synthesizing the latest first special effect image data and the second special effect image data generated by the special effect image processing unit The ratio at which the effect image processing unit and the moving image special effect image processing unit synthesize the first special effect image data and the second special effect image data is changed over time. Characterized by comprising a an image processing control unit.

  In the imaging device according to the present invention as set forth in the invention described above, the image processing control unit causes the special effect image processing unit to alternately apply the first and second special effects.

  In the imaging device according to the present invention, in the above invention, the special effect image processing unit includes a first special effect processing unit that applies the first special effect, and a second special effect that applies the second special effect. And the image processing control unit supplies the first special effect and the second special effect to the first special effect processing unit and the second special effect processing unit, respectively. Is provided.

  In the image pickup apparatus according to the present invention, in the above invention, the image processing control unit is configured such that the processing speed for the same image data is greater than the first special effect when the second special effect is greater than the first special effect. The image data to which the second special effect is applied is resized and executed by the special effect image processing unit so that the processing speed is substantially the same as the effect.

  An image processing method according to the present invention is an image processing method executed by an imaging apparatus capable of capturing an image of a subject to generate image data of the subject and capturing a moving image, and includes a plurality of images corresponding to the image data of the moving image. Special effect image data and second special effect image data are generated by applying a first special effect and a second special effect, which have different processing speeds and visual effects from one frame to another, respectively. An image processing step, and a moving image special effect image processing step for generating synthesized image data by combining the latest first special effect image data and the second special effect image data generated in the special effect image processing step; , The ratio of combining the first special effect image data and the second special effect image data in the moving image special effect image processing step over time An image processing control step of changing, characterized in that it comprises a.

  The program according to the present invention is a program for capturing an image of a subject, generating image data of the subject, and causing an imaging apparatus capable of capturing a moving image to be executed, which is processed over a plurality of frames corresponding to the image data of the moving image. A special effect image processing step for generating the first special effect image data and the second special effect image data by applying the first special effect and the second special effect having different speed and visual effects, respectively; A special effect image processing step in the moving image that combines the latest first special effect image data and the second special effect image data generated in the special effect image processing step to generate composite image data; The ratio of combining the first special effect image data and the second special effect image data in the special effect image processing step over time Characterized in that to execute the image processing control step of reduction, the.

  According to the present invention, the image processing control unit causes the special effect image processing unit in the moving image to synthesize the latest two special effect images generated by the special effect image processing unit and having different processing speeds, and the ratio of the synthesis is changed over time. To change. Thus, even when two video effects having different processing speeds are applied to the moving image frame, it is possible to shoot a moving image that is natural and has no sense of incongruity.

FIG. 1 is a block diagram illustrating a configuration of the imaging apparatus according to the first embodiment of the present invention. FIG. 2 is a perspective view showing a configuration of the side (front side) facing the user of the imaging apparatus according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating an outline of art effect image processing performed by the art effect image processing unit of the imaging apparatus according to the first embodiment of the present invention. FIG. 4 is a diagram for explaining the outline of multi-echo. FIG. 5 is a diagram for explaining the outline of the one-shot echo. FIG. 6 is a diagram showing the time change of the coefficient of the SDRAM image synthesized with the latest photographed image. FIG. 7 is a diagram for explaining the outline of transit. FIG. 8 is a diagram illustrating a temporal change in the coefficient of the art effect given before the start trigger is input in the transit process. FIG. 9 is a diagram for explaining an outline of fluctuation. FIG. 10 is a diagram illustrating an operation assignment example (first example) to each arrow key of the cross key at the time of moving image recording and at the time of non-moving image recording. FIG. 11 is a diagram illustrating an operation assignment example (second example) to each arrow key of the cross key when recording a moving image and when not recording a moving image. FIG. 12 is a timing chart illustrating an example of synchronous communication between the lens control unit and the control unit included in the imaging apparatus according to the first embodiment of the present invention. FIG. 13 is a flowchart illustrating an outline of processing performed by the imaging apparatus according to the first embodiment of the present invention. FIG. 14 is a diagram illustrating a moving image screen display example on the display unit of the imaging apparatus according to the first embodiment of the present invention. FIG. 15 is a flowchart illustrating an outline of the moving image special effect control process performed by the imaging apparatus according to the first embodiment of the present invention. FIG. 16 is a flowchart illustrating an outline of image processing performed by the imaging apparatus according to the first embodiment of the present invention. FIG. 17 is a block diagram illustrating a configuration of the imaging apparatus according to the second embodiment of the present invention. FIG. 18 is a diagram illustrating an outline of a transit executed by the imaging apparatus according to the second embodiment of the present invention. FIG. 19 is a diagram for explaining the outline of the transit executed by the imaging apparatus according to the third embodiment of the present invention. FIG. 20 is a diagram for explaining the outline of the transit executed by the imaging apparatus according to the fourth embodiment of the present invention. FIG. 21 is a diagram for explaining the outline of the transit executed by the imaging apparatus according to the fifth embodiment of the present invention. FIG. 22 is a diagram for explaining the outline of the transit executed by the imaging apparatus according to the first modification of the fifth embodiment of the present invention. FIG. 23 is a diagram for explaining an outline of transit executed by a conventional imaging apparatus. FIG. 24 is a diagram illustrating another outline of transit executed by an imaging apparatus having two conventional image processing units.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as “embodiments”) will be described with reference to the drawings. In the description of the drawings, the same portions will be described with the same reference numerals. Further, the drawings are schematic, and it should be noted that the dimensions and ratios of each part are different from the actual ones. Furthermore, the drawings include portions having different dimensional relationships and ratios. Note that the present invention is not limited to the embodiments.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a configuration of the imaging apparatus according to the first embodiment of the present invention. FIG. 2 is a perspective view showing a configuration of the side (front side) facing the user of the imaging apparatus according to the first embodiment of the present invention. As shown in FIGS. 1 and 2, the imaging device 1 includes a main body 2 and a lens unit 3 that is detachable from the main body 2.

  The main body 2 includes a shutter 10, a shutter driving unit 11, an image sensor 12, an image sensor driving unit 13, a signal processing unit 14, an A / D conversion unit 15, an image processing unit 16a, and an image processing unit. 16b, AE processing unit 17, AF processing unit 18, image compression / decompression unit 19, input unit 20, display unit 21, display drive unit 22, recording medium 23, memory I / F 24, and SDRAM (Synchronous Dynamic Random Access Memory) 25, Flash memory 26, main body communication unit 27, bus 28, and control unit 29.

  The shutter 10 sets the state of the image sensor 12 to an exposure state or a light shielding state. The shutter drive unit 11 is configured using a stepping motor or the like, and drives the shutter 10 according to a signal input from the control unit 29.

  The imaging device 12 is configured using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like that receives the light collected by the lens unit 3 and converts it into an electrical signal. The image sensor drive unit 13 causes the signal processing unit 14 to output image data (analog signal) from the image sensor 12 at a predetermined timing. In this sense, the image sensor driving unit 13 functions as an electronic shutter.

  The signal processing unit 14 performs analog processing on the analog signal input from the image sensor 12 and outputs the analog signal to the A / D conversion unit 15. Specifically, the signal processing unit 14 performs noise reduction processing, gain increase processing, and the like on the analog signal. For example, the signal processing unit 14 performs waveform shaping on the analog signal after reducing reset noise and the like, and further increases the gain so that the target brightness is obtained.

  The A / D conversion unit 15 generates digital image data by performing A / D conversion on the analog signal input from the signal processing unit 14, and outputs the digital image data to the SDRAM 25 via the bus 28.

  The image processing unit 16a acquires image data from the SDRAM 25 via the bus 28, performs various image processing on the acquired image data (RAW data), and generates processed image data. This processed image data is output to the SDRAM 25 via the bus 28. The image processing unit 16a includes a basic image processing unit 161a, an art effect image processing unit 162a, and a moving image special effect image processing unit 163a. The image processing unit 16b is realized by the same configuration as the image processing unit 16a, and includes a basic image processing unit 161b, an art effect image processing unit 162b, and a moving image special effect image processing unit 163b. Therefore, hereinafter, the basic image processing unit 161a, the art effect image processing unit 162a, and the moving image special effect image processing unit 163a will be described.

  The basic image processing unit 161a performs at least optical black subtraction processing, white balance (WB) adjustment processing on image data, image data synchronization processing, color matrix calculation processing when the image sensor 12 is a Bayer array, Basic image processing including gamma correction processing, color reproduction processing, edge enhancement processing, and the like is performed. Further, the basic image processing unit 161a performs finish effect processing for reproducing a natural image based on preset image processing parameters to generate finish effect image data. Here, the parameters of each image processing are values of contrast, sharpness, saturation, white balance, and gradation.

  The art effect image processing unit 162a performs a special effect that causes a visual effect by combining a plurality of image processes with respect to one image data, and is referred to as special effect image data (hereinafter referred to as "art effect image data"). ) Is generated. The art effect image processing unit 162a has different special effect processing speeds according to the number and types of image processing to be combined with one image data.

  FIG. 3 is a diagram showing an outline of art effect image processing performed by the art effect image processing unit 162a. In FIG. 3, as art effect image processing, Fantastic Focus, Fantastic Focus + Starlight, Fantastic Focus + White Edge, Pop Art, Pop Art + Starlight, Pop Art + Pinhole, Pop Art + White Edge, Toy Photo, Rough Monochrome Ten types of diorama are described. Hereinafter, the art effect image processing will be described.

Fantastic focus is a process that gives a soft focus effect in which the entire image is subjected to a blurring process and synthesized at a certain ratio with the image before blurring. Fantastic focus is a tone curve process that brightens the intermediate brightness, leaving the details of the subject in a soft tone and creating an image with a beautiful and fantastic atmosphere as if wrapped in a happy light. Image or produce. The fantastic focus is realized by combining image processing such as tone curve processing, blur processing, alpha blend processing, and image composition processing.
Fantastic focus + starlight is a process of applying a cross filter effect that draws a cross pattern to a high-luminance portion in an image in addition to the fantastic focus.
Fantastic focus + white edge is a process in which, in addition to the phantom focus, an effect of gradually becoming whiter as it goes from the central part of the image to the peripheral part (peripheral part). Such a whiteness effect can be obtained by changing the pixel value so that the peripheral portion becomes whiter as the distance from the center of the image increases.

Pop art is a process that emphasizes colors colorfully and expresses a bright and fun atmosphere. Pop art is realized by, for example, combining saturation enhancement processing and contrast enhancement processing. The overall effect is high contrast and high saturation.
Pop art + starlight is a process of applying pop art and starlight in layers. In this case, the effect of applying a cross filter to a colorful image can be obtained.
In addition to pop art, pop art + pinhole is a process of applying toy photo (pinhole) that darkens the periphery of an image by shading and gives an effect of looking through a hole. Details of the toy photo will be described later.
Pop art + white edge is a process in which pop art and white edge are overlapped.

  Toy photo is a process in which the brightness is reduced (darker) as the distance from the center of the image increases, and the effect is as if looking into a different space from a hole. Toy photo is realized, for example, by combining low-pass filter processing, white balance processing, contrast processing, hue / saturation processing, and image processing such as shading processing that multiplies the luminance signal by a smaller coefficient toward the periphery. For details of photo and shading, see, for example, JP 2010-74244 A).

  Rough monochrome is a process of expressing the strength and roughness of a monochrome image by adding high contrast and grainy noise of the film. The rough monochrome is realized by combining, for example, edge enhancement processing, level correction optimization processing, noise pattern superimposition processing, synthesis processing, contrast processing, and the like (for details of the rough monochrome, for example, JP 2010-62836 A See the publication). Among these, the noise pattern superimposing process (noise adding process) is a process of adding a noise pattern image created in advance to the original image. The noise pattern image may be generated based on, for example, a random number generated.

  Diorama is a process that creates a high-contrast, high-saturation image on the screen by creating an atmosphere that looks like a miniature model or toy by blurring the periphery of the image. The diorama is realized by combining, for example, hue / saturation processing, contrast processing, peripheral blur processing, synthesis processing, and the like. Among these, the peripheral blur processing performs low-pass filter processing while changing the low-pass coefficient according to the position of the image so that the degree of blurring in the peripheral portion increases according to the distance from the center of the image. Note that as the peripheral blur processing, only the upper and lower sides of the image or only the left and right sides of the image may be blurred.

  The moving image special effect image processing unit 163a performs a moving image special effect that causes a visual effect over a plurality of frames corresponding to moving image data. Specifically, the moving image special effect image processing unit 163a includes the latest art effect image (first art effect image data) generated by the art effect image processing unit 162a and the art effect image processing unit 162b, and the first art effect image data. The art effect image (second art effect image data) is synthesized to generate composite image data. The moving image special effects performed by the moving image special effect image processing unit 163a and the moving image special effect image processing unit 163b include, for example, multi-echo, one-shot echo, transit, and fluctuation. Hereinafter, these special effects during moving images will be described.

  FIG. 4 is a diagram for explaining the outline of multi-echo. Multi-echo is an effect in which an afterimage remains in an image by repeating a predetermined recording image and an image of the image just taken for a predetermined period. Specifically, as shown in FIG. 4, the trajectory of movement of the ball 101 can be displayed as an afterimage.

  When the multi-echo is set as the moving image special effect, the moving image special effect image processing unit 163 receives an image of the previous frame with respect to the image generated immediately after the start trigger is input. Is processed at a predetermined ratio (multi-echo processing).

  In the case illustrated in FIG. 4, the moving image special effect image processing unit 163 combines the captured image P1 immediately after the start trigger is input with the recorded image R0 corresponding to the captured image P0 of the one previous frame. A multi-echo image R1 is generated. At the time of this synthesis, in each pixel, the signal of the captured image P1 is multiplied by a coefficient of 0.6, while the signal of the recorded image R0 is multiplied by a coefficient of 0.4.

  Subsequently, the moving image special effect image processing unit 163 generates a multi-echo image R2 by combining the captured image P2 of the next frame and the combined image R1. When generating the multi-echo image R2, in each pixel, the signal of the captured image P2 is multiplied by a coefficient of 0.6, while the signal of the composite image R1 is multiplied by a coefficient of 0.4.

  The moving image special effect image processing unit 163 sequentially generates the multi-echo images by repeatedly performing the above-described synthesis processing until an end trigger is input. When the end trigger is input, a recording image R5 corresponding to the captured image P5 is recorded on the recording medium 23.

  FIG. 5 is a diagram for explaining the outline of the one-shot echo. One-shot echo is an effect in which an image of a specific frame (instantaneous image) disappears while fading gradually. Specifically, as shown in FIG. 5, the effect is such that the ball 101 photographed at a specific time remains as an afterimage in the photographed image for a while.

  When a one-shot echo is set as a moving image special effect, the moving image special effect image processing unit 163 temporarily stores image data captured and recorded immediately after that in the SDRAM 25 when a start trigger is input. Then, a process (one-shot echo process) is performed in which the image data is synthesized so that the weight gradually decreases with time with respect to the image data captured thereafter.

  In the case shown in FIG. 5, a captured image P1 captured immediately after the start trigger is input is recorded in the SDRAM 25 (hereinafter referred to as SDRAM image S1). The moving image special effect image processing unit 163 combines the SDRAM image S1 recorded by the SDRAM 25 and the latest photographed image at a predetermined ratio.

In the case shown in FIG. 5, immediately after the start trigger is input, a one-shot echo image R11 corresponding to the captured image P1 is displayed.
Thereafter, at time t = t ₂ , the moving image special effect image processing unit 163 generates the one-shot echo image R12 by synthesizing the SDRAM image S1 and the captured image P2. When this synthesis is performed, in each pixel, the signal of the SDRAM image S1 is multiplied by a coefficient a ₁ = 0.8, while the signal of the captured image P2 is multiplied by a coefficient 1-a ₁ = 0.2.
After that, the moving image special effect image processing unit 163, at time t = t ₃ , a = 0.4 is a coefficient that is multiplied by the signal of the SDRAM image S1, and 1−a = 0.6 is a coefficient that is multiplied by the signal of the captured image P3. Is generated as a one-shot echo image R13.
After that, the moving image special effect image processing unit 163, at time t = t ₄ , a = 0.2 is a coefficient to be multiplied by the signal of the SDRAM image S1, and 1−a = 0.8 is a coefficient to be multiplied by the signal of the captured image P4. Is generated as a one-shot echo image R14.
Thereafter, since the coefficient a = 0 at time t = t ₅ , the captured image P5 becomes the recorded image R5.

  FIG. 6 is a diagram showing a temporal change in the composition ratio of the SDRAM image synthesized with the latest photographed image. In FIG. 6, the horizontal axis t is the elapsed time from the start trigger input time (t = 0), and the vertical axis a is a coefficient to be multiplied by the signal of each pixel of the SDRAM image S1.

As shown in FIG. 6, the coefficient a is a ₁ at t = 0, and decreases smoothly with time, and becomes 0 when a predetermined time t _{5 has} elapsed from the start trigger input. That is, the proportion of the SDRAM image S1 in the composite image is set to gradually decrease with time. As a result, it is possible to create an image in which the SDRAM image S1 gradually disappears with time.

  Note that the curve shown in FIG. 6 is merely an example, and any curve may be used as long as the coefficient a attenuates with time so that the coefficient a smoothly changes so that the coefficient a becomes 0 in a certain time. For example, it may be a curve including a period in which the coefficient a takes a constant value.

  The coefficient a may be determined according to the number of frames from the start trigger. In this case, the coefficient may be set to 0 for a predetermined number of frames (for example, 120 frames) while gradually decreasing the value of the coefficient a in units of frames.

  FIG. 7 is a diagram for explaining the outline of transit. Transit is an effect in which an art effect applied to one image is gradually changed to another art effect while changing the ratio. FIG. 7 schematically shows a transit from art effect A to art effect B. In FIG. 7, the processing speed of art effect A is 30 fps (33 msec / frame), and the processing speed of art effect B is 15 fps (66 msec / frame). In FIG. 7, immediately before the start of the transit start trigger, the art effect image processing unit 162a generates an art effect image A0 obtained by applying the art effect A to the photographed image P0. It should be noted that the time required from the start of art effect switching to the end of switching in transit can be set as appropriate.

When the start trigger is input, the image processing control unit 291 causes the art effect image processing unit 162a and the art effect image processing unit 162b to execute the art effect A and the art effect B on the captured image P1, respectively. Art effect image A1 and art effect image B1 are generated. In this case, the image processing control unit 291 does not generate the art effect image B1 by the art effect image processing unit 162b in time, so that the transitive image R21 using only the art effect image A1 is added to the moving image special effect image processing unit 163. Generate.
Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _12, the coefficient to be multiplied to the signal of the latest artistic effect image A2 that artistic effect image processing section 162a is subjected to the artistic effect A relative captured image P2 Assuming that b (= b ₀ ) = 0.9 and the art effect image processing unit 162b multiplies the signal of the latest art effect image B1 obtained by applying the art effect B to the photographed image P1, 1−b (= 1−b _0). ) = 0.1 to generate a transit image R22 by combining the two images.

Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _13, the coefficient b = 0 to be multiplied by the signal of the latest artistic effect image A3 of artistic effect image processing section 162a is subjected to the artistic effect A in the captured image P3 .8, and the art effect image processing unit 162b combines the two images with a coefficient 1-b = 0.2 by which the signal of the latest art effect image B1 obtained by applying the art effect B to the photographed image P1 is combined. An image R23 is generated. At this time, the art effect image processing unit 162b starts processing of the art effect B applied to the captured image P3.
Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _14, artistic effect image processing unit b = 0 the coefficients to be multiplied to the signal of 162a is artistic effect image A4 subjected to artistic effect A in the captured image P4. 7, the art effect image processing unit 162b combines the two images by setting the coefficient by which the signal of the art effect image B2 obtained by applying the art effect B to the photographed image P3 is 1-b = 0.3, thereby generating the transit image R24. Is generated.

Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _15, artistic effect image processing section 162a is b = 0 the coefficients to be multiplied to the signal of the artistic effect image A5 subjected to artistic effect A in the captured image P5. 5 and the art effect image processing unit 162b combines the two images by setting the coefficient by which the signal of the art effect image B2 obtained by applying the art effect B to the photographed image P4 is 1-b = 0.5, thereby generating the transit image R25. Is generated. At this time, the art effect image processing unit 162b starts processing of the art effect B applied to the captured image P5.
Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _16, artistic effect image processing unit b = 0 the coefficients to be multiplied to the signal of 162a is artistic effect image A6 subjected to artistic effect A in the captured image P6. 3 and the art effect image processing unit 162b combines the two images by setting the coefficient by which the signal of the art effect image B3 obtained by applying the art effect B to the photographed image P5 is 1-b = 0.7. Is generated.

Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _17, artistic effect image processing unit b = 0 the coefficients to be multiplied to the signal of 162a is artistic effect image A7 subjected to artistic effect A in the captured image P7. 2 and the art effect image processing unit 162b combines the two images by setting the coefficient by which the signal of the art effect image B3 obtained by applying the art effect B to the photographed image P6 is 1-b = 0.8, so that the transit image R27 Is generated. At this time, the art effect image processing unit 162b starts processing of the art effect B applied to the captured image P7.
Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _18, artistic effect image processing unit b = 0 the coefficients to be multiplied to the signal of 162a is artistic effect image A8 subjected to artistic effect A in the captured image P8. 1, the art effect image processing unit 162b combines the two images by setting the coefficient by which the signal of the art effect image B4 obtained by applying the art effect B to the photographed image P7 is 1-b = 0.9. Is generated.

Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _19, as a transit image R29 duplicates the latest artistic effect image B4 which artistic effect image processing unit 162b is subjected to artistic effect B in the captured image P8 Record.
Thereafter, the special effect image processing unit 163a in the moving picture at time t = t _20, records the artistic effect image B5 to artistic effect image processing unit 162b is subjected to artistic effect B in the captured image P9 as a transit images R30.

  In this way, the image processing control unit 291 synthesizes the moving image special effect image processing unit 163a while changing the ratio (coefficient) at which the latest art effect image A and the latest art effect image B are combined with time.

FIG. 8 is a diagram showing a temporal change in the art effect composition ratio (coefficient) b given before the start trigger is input in the transit process. As shown in FIG. 8, the coefficient b is b ₀ at t = 0, and decreases smoothly with time, and becomes 0 when a predetermined time t _{15 has} elapsed from the start trigger input.

  Note that the curve shown in FIG. 8 is merely an example, and any curve may be used as long as the coefficient b is smoothly changed so that the coefficient b becomes 0 in a certain time, for example, the coefficient b attenuates with time. A curve including a period in which the coefficient b takes a constant value may be used.

  The coefficient b may be determined according to the number of frames from the start trigger. In this case, the coefficient b may be reduced to 0 in a predetermined number of frames (for example, 120 frames) while smoothly decreasing the value of the coefficient b in units of frames.

  FIG. 9 is a diagram showing an outline of fluctuation. Fluctuation is an effect obtained by changing the degree of art effect applied to each frame in a moving image at random. FIG. 9 shows a case where the art effect is a cross filter as an example. In this case, the moving image special effect image processing unit 163 randomly fluctuates the cross 102 with time and gives fluctuation by randomly changing the size of the cross 102 with time.

Returning to FIG. 1, the configuration of the imaging apparatus 1 will be described.
The AE processing unit 17 acquires image data recorded in the SDRAM 25 via the bus 28, and sets an exposure condition for performing still image shooting or moving image shooting based on the acquired image data. Specifically, the AE processing unit 17 calculates the brightness from the image data, and determines the aperture value (F value) setting value, shutter speed, and the like based on the calculated brightness, thereby automatically detecting the image capturing apparatus 1. Perform exposure.

  The AF processing unit 18 acquires image data recorded in the SDRAM 25 via the bus 28, and adjusts the automatic focus of the imaging device 1 based on the acquired image data. For example, the AF processing unit 18 extracts a high-frequency component signal from the image data, performs AF (Auto Focus) calculation processing on the high-frequency component signal, and determines the focus evaluation of the imaging device 1 to perform imaging. The automatic focus of the apparatus 1 is adjusted.

  The image compression / decompression unit 19 acquires image data from the SDRAM 25 via the bus 28, compresses the acquired image data according to a predetermined format, and outputs the compressed image data to the SDRAM 25. Here, the predetermined format includes a JPEG (Joint Photographic Experts Group) system, a Motion JPEG system, and an MP4 (H.264) system. The image compression / decompression unit 19 acquires image data (compressed image data) recorded on the recording medium 23 via the bus 28 and the memory I / F 24, expands (decompresses) the acquired image data, and stores it in the SDRAM 25. Output. Note that a recording unit may be provided inside the imaging apparatus 1 instead of the recording medium 23.

  The input unit 20 includes a setting signal input unit 201 that receives input of various setting signals including a still image shooting condition setting signal that sets shooting conditions in still image shooting and moving image shooting, and starts moving image shooting that instructs at least the start of moving image shooting. A moving image shooting signal input unit 202 that receives a signal input.

  The input unit 20 is realized using an operation signal input user interface provided on the surface of the main body unit 2. Hereinafter, the configuration of the user interface that forms part of the input unit 20 will be described.

  The imaging apparatus 1 has a power button 41 for switching the power state of the imaging apparatus 1 to an on state or an off state as a user interface for operation input, and a release button 42 for receiving an input of a still image release signal for giving a still image shooting instruction. A mode dial 43 for switching various shooting modes set in the imaging apparatus 1, an operation button 44 for switching various settings of the imaging apparatus 1, a menu button 45 for displaying various settings of the imaging apparatus 1 on the display unit 21, and The playback button 46 for displaying an image corresponding to the image data recorded on the recording medium 23 on the display unit 21, the moving image button 47 for receiving input of a moving image release signal for giving a moving image shooting instruction, and various functions of the imaging device 1. A function key 48 to be set, an erase button 49 for erasing data, and a display image on the display unit 21 Again it provided, and a touch panel 50 that receives an input signal corresponding to the contact position from the outside.

  The release button 42 can be advanced and retracted by pressing from the outside. When the release button 42 is pressed halfway, a 1st release signal that instructs a shooting preparation operation is input. On the other hand, when the release button 42 is fully pressed, a 2nd release signal instructing still image shooting is input.

  The operation button 44 includes a cross key 441 that has a cross shape in the up / down / left / right directions in order to perform selection setting input on a menu screen or the like, and a determination button 442 for confirming selection by the cross key 441. The cross key 441 includes an up arrow key 443, a down arrow key 444, a left arrow key 445, and a right arrow key 446.

  Of the user interfaces described above, buttons other than the moving image button 47 form part of the setting signal input unit 201. The moving image button 47 forms a part of the moving image shooting signal input unit 202.

  FIG. 10 is a diagram illustrating an example of assigning operations to the arrow keys of the cross key 441 when the moving image is not being recorded (when the moving image is not recorded) and when the moving image is being recorded (when the moving image is being recorded). When a moving image is not being recorded, an exposure correction operation is assigned to the up arrow key 443. The down arrow key 444 is assigned continuous shooting / single shooting operation. An AF target operation is assigned to the left arrow key 445. A flash setting operation is assigned to the right arrow key 446.

  Next, a case where a moving image is being recorded will be described. In this case, a one-shot echo start operation is assigned to the up arrow key 443. The down arrow key 444 is assigned a transit start operation. The left arrow key 445 alternately assigns start and stop operations of multi-echo and functions as a toggle key. A fluctuation start operation is assigned to the right arrow key 446. Note that when setting the end of fluctuation to be arbitrarily selected, the right arrow key 446 may be made to function as a toggle key capable of alternately operating the start and end operations of fluctuation.

  The assignment of input buttons at the time of moving image recording is not limited to that shown in FIG. For example, as shown in FIG. 11, an effect start operation may be assigned to the up arrow key 443, an effect stop operation may be assigned to the down arrow key 444, and an effect switching operation may be assigned to the left arrow key 445 and the right arrow key 446. . In this case, it is more preferable to display the special effect being set on the display unit 21.

  Also, the operation of the cross key 441 may be assigned according to the ease of use or the frequency of use of the operation, or may be set by the user independently. Further, when two adjacent keys (for example, the up arrow key 443 and the right arrow key 446) are both pressed, another setting may be performed.

  The display part 21 is comprised using the display panel which consists of a liquid crystal or organic EL (Electro Luminescence). The display drive unit 22 acquires image data recorded by the SDRAM 25 or image data recorded by the recording medium 23 via the bus 28 and causes the display unit 21 to display an image corresponding to the acquired image data. Here, for image display, a rec view display that displays image data immediately after shooting for a predetermined time (for example, 3 seconds), a reproduction display that generates image data recorded on the recording medium 23, and an image sensor 12 are continuously displayed. Live view display that sequentially displays live view images corresponding to image data to be generated in time series. The display unit 21 appropriately displays operation information of the imaging device 1 and information related to shooting.

  The recording medium 23 is configured using a memory card or the like mounted from the outside of the imaging device 1. The recording medium 23 is detachably attached to the imaging device 1 via the memory I / F 24. Image data processed by the image processing unit 16a or the image processing unit 16b and the image compression / decompression unit 19 is written on the recording medium 23 by a read / write device (not shown) according to the type, or is written on the recording medium 23 by the read / write device. The recorded image data is read out. The recording medium 23 may output the imaging program and various types of information to the flash memory 26 via the memory I / F 24 and the bus 28 under the control of the control unit 29.

  The SDRAM 25 is configured using a volatile memory. The SDRAM 25 is a primary recording unit that temporarily records image data input from the A / D conversion unit 15 via the bus 28, processed image data input from the image processing unit 16, and information being processed by the imaging apparatus 1. As a function. For example, the SDRAM 25 temporarily records image data that the image sensor 12 sequentially outputs for each frame via the signal processing unit 14, the A / D conversion unit 15, and the bus 28.

  The flash memory 26 is configured using a nonvolatile memory. The flash memory 26 includes a program recording unit 261, a special effect processing information recording unit 262, and an image processing information recording unit 263. The program recording unit 261 records various programs for operating the imaging apparatus 1, imaging programs, various data used during execution of the programs, various parameters necessary for image processing operations by the image processing unit 16, and the like. The special effect processing information recording unit 262 records image processing combination information in each art effect processing performed by the art effect image processing unit 162. The image processing information recording unit 263 records image processing information in which processing time is associated with image processing that can be executed by the image processing unit 16. Further, the flash memory 26 records a manufacturing number for specifying the imaging device 1.

  The main body communication unit 27 is a communication interface for performing communication with the lens unit 3 attached to the main body unit 2. The main body communication unit 27 also includes an electrical contact with the lens unit 3.

  The bus 28 is configured using a transmission path or the like that connects each component of the imaging device 1. The bus 28 transfers various data generated inside the imaging apparatus 1 to each component of the imaging apparatus 1.

  The control unit 29 is configured using a CPU (Central Processing Unit) or the like. The control unit 29 includes an image processing control unit 291 and a display control unit 292.

  The image processing control unit 291 sets the content of image processing to be executed by the image processing unit 16 in accordance with an instruction signal from the input unit 20 input via the bus 28, and performs image processing according to the set content. Is executed by any one of the basic image processing unit 161, the art effect image processing unit 162, and the moving image special effect image processing unit 163. In addition, the image processing control unit 291 controls the application mode of the first and second art special effects in the moving image special effect image processing unit 163. Specifically, the image processing control unit 291 has a ratio at which the moving image special effect image processing unit 163a combines the first special effect image data (art effect A) and the second special effect image data (art effect B). Is changed over time.

  The display control unit 292 controls the display mode of the display unit 21. Specifically, the display control unit 292 drives the display driving unit 22 and causes the display unit 21 to display images corresponding to various image data on which the image processing unit 16 has performed image processing.

  In response to the instruction signal transmitted from the input unit 20 via the bus 28, the control unit 29 transmits the control signal and various data to each unit constituting the imaging device 1, thereby operating the imaging device 1. Overall control.

  When a 2nd release signal is input via the release button 42, the control unit 29 performs control to start a still image shooting operation in the imaging device 1. In addition, when a moving image shooting start signal is input via the moving image button 47, the control unit 29 performs control to start a moving image shooting operation in the imaging device 1. Here, the imaging operation in the imaging apparatus 1 refers to the signal processing unit 14, the A / D conversion unit 15, and the image processing for the image data output from the imaging device 12 by driving the shutter driving unit 11 and the imaging device driving unit 13. An operation in which the unit 16 performs a predetermined process. The image data thus processed is compressed in accordance with a predetermined format by the image compression / expansion unit 19 under the control of the image processing control unit 291, and is recorded via the bus 28 and the memory I / F 24. To be recorded. In the first embodiment, the recording medium 23 forms a part of the recording unit, but a recording area having the function of the recording unit is secured inside the imaging apparatus 1 separately from the recording medium 23, and this recording area The compressed image data may be recorded.

  For the main body 2 having the above-described configuration, an audio input / output unit, an auxiliary light emitting unit that emits auxiliary light (flash) to the subject, and a function of bidirectionally communicating with an external device via the Internet You may provide further the communication part etc. which have.

  Next, the configuration of the lens unit 3 will be described. The lens unit 3 includes an optical system 31, a lens driving unit 32, a diaphragm 33, a diaphragm driving unit 34, a lens operation unit 35, a lens flash memory 36, a lens communication unit 37, a lens control unit 38, Is provided.

  The optical system 31 is configured using one or a plurality of lenses. The optical system 31 collects light from a predetermined visual field region. The optical system 31 has an optical zoom function for changing the angle of view and a focus function for changing the focus.

  The lens driving unit 32 is configured using a DC motor, a stepping motor, or the like, and changes the focus position, the angle of view, and the like of the optical system 31 by moving the lens of the optical system 31 on the optical axis L.

  The diaphragm 33 adjusts the exposure by limiting the amount of incident light collected by the optical system 31. The aperture drive unit 34 is configured using a stepping motor or the like, and drives the aperture 33.

  As shown in FIG. 2, the lens operation unit 35 is a ring provided around the lens barrel of the lens unit 3. The lens operation unit 35 receives an operation signal for starting an optical zoom operation in the lens unit 3 or in the lens unit 3. An instruction signal for instructing adjustment of the focus position is received. The lens operation unit 35 may be a push switch or the like.

  The lens flash memory 36 records a control program for determining the position and movement of the optical system 31, the lens characteristics of the optical system 31, and various parameters.

  The lens communication unit 37 is a communication interface for communicating with the main body communication unit 27 of the main body unit 2 when the lens unit 3 is attached to the main body unit 2. The lens communication unit 37 includes an electrical contact with the main body unit 2.

  The lens control unit 38 is configured using a CPU or the like. The lens control unit 38 controls the operation of the lens unit 3 in accordance with an operation signal from the lens operation unit 35 or an instruction signal from the main body unit 2. Specifically, the lens control unit 38 drives the lens driving unit 32 in accordance with an operation signal from the lens operation unit 35 to focus the lens unit 3 and change the zoom, and also drives the aperture driving unit 34. To change the aperture value. When the lens unit 3 is attached to the main body unit 2, the lens control unit 38 transmits the focus position information of the lens unit 3, the focal length, unique information for identifying the lens unit 3, and the like to the main body unit 2. It may be.

  The lens control unit 38 transmits and receives a lens communication signal to and from the control unit 29 of the main body unit 2 at a predetermined cycle so as to coordinate the operation with the main body unit 2. FIG. 12 is a timing chart illustrating an example of synchronous communication between the lens control unit 38 and the control unit 29. FIG. 12A shows processing in the main body 2. FIG. 12B shows a vertical synchronization signal. FIG. 12C shows the timing of imaging and reading. FIG. 12D shows lens communication. FIG. 12E shows a lens communication synchronization signal. FIG. 12F shows the lens position acquisition signal. FIG. 12G shows processing in the lens unit 3.

  First, the control unit 29 causes the image processing unit 16 to perform image processing of the live view image, calculation of the AF evaluation value, and the like based on the image data acquired in the previous frame, and also causes the lens control unit 38 to acquire lens state data. The lens state data request command is transmitted (B1, BL). At this time, during the synchronous communication mode, the control unit 29 transmits a lens position acquisition signal instructing the timing for acquiring the lens communication synchronization signal and the position information of the optical system 31 at the same cycle as the vertical synchronization signal. This lens position acquisition signal is a signal whose state changes when half of the accumulation time at the center of the image sensor 12 has elapsed, as shown in FIG.

  The lens control unit 38 acquires the position information of the optical system 31 at the timing when the state of the lens position acquisition signal changes, and detects the operation state of the lens operation unit 35 at the reception timing of the lens communication synchronization signal (L1).

  Subsequently, as a response to the lens state data request command received from the control unit 29, the lens control unit 38 receives lens state data including the position information of the optical system 31 and the operation state of the lens operation unit 35 acquired in step L1. It transmits to the control part 29 (L2).

  Thereafter, the control unit 29 performs various setting changes such as calculation of the AF evaluation value and change of the exposure value based on the lens state data sent from the lens control unit 38 (B2).

  The control unit 29 and the lens control unit 38 periodically repeat the processing described above.

  An outline of processing performed by the imaging apparatus 1 configured as described above will be described. FIG. 13 is a flowchart illustrating an outline of processing performed by the imaging apparatus 1.

  In FIG. 13, first, when the power button 41 is operated by the user to turn on the power of the imaging apparatus 1 (step S1: Yes), the control unit 29 initializes the imaging apparatus 1 (step S2). ). In this initialization process, for example, the control unit 29 resets a recording flag indicating that a moving image is being recorded to an off state, resets a special effect flag indicating application relating to a special effect in the moving image to an off state, Processing for starting lens communication (see FIG. 12) with the control unit 38 is performed. If the power of the imaging device 1 is not on (step S1: No), the imaging device 1 repeats step S1.

  Subsequently, when the playback button 46 is not operated (step S3: No) and the menu button 45 is operated (step S4: Yes), the imaging apparatus 1 displays a menu screen for changing the setting. Then, a setting process for setting various conditions of the imaging device 1 in accordance with the user's selection operation is executed (step S5). After step S5, the imaging apparatus 1 proceeds to step S7 described later.

  The contents set here are finish effect processing, art effect processing, still image recording mode, moving image recording mode, moving image special effect processing, and the like. Finishing effect processing includes, for example, natural processing that finishes an image in a natural color, vivid processing that finishes an image vividly, flat processing that focuses on the subject material, and finishes the image in monochrome. The processing includes monotone and the like, and the art effect shown in FIG. The still image recording mode is determined according to the type of still image to be recorded, and includes, for example, a JPEG recording mode, a JPEG + RAW recording mode, and a RAW recording mode. The moving image shooting mode is a mode determined according to the moving image compression format, and includes, for example, a Motion-JPEG mode, an MP4 (H.264) mode, and the like. What is set in the special effect processing during moving image is, for example, the destination of the art effect switching due to transit or fluctuation.

  When the playback button 46 is operated in step S3 (step S3: Yes), the imaging apparatus 1 performs a playback process (step S6). In step S6, the display control unit 292 causes the display unit 21 to display a list of files recorded on the recording medium 23. Thereafter, when a reproduction image is selected and input by the input unit 20, image data is acquired from the recording medium 23, and the acquired image data is expanded by the image compression / decompression unit 19 and displayed on the display unit 21. Thereafter, the imaging apparatus 1 proceeds to step S18.

  When the playback button 46 is not operated in step S3 (step S3: No), and the movie button 47 is operated when the menu button 45 is not operated (step S4: No) (step S7: Yes). The control unit 29 inverts the recording flag indicating that the moving image is being recorded (step S8). Specifically, for example, when the recording flag is in an on state, the control unit 29 inverts the recording flag to an off state.

  Subsequently, the control unit 29 determines whether or not the recording flag recorded in the SDRAM 25 is in an ON state (step S9). When it is determined that the recording flag is in the on state (step S9: Yes), the control unit 29 generates a moving image file for recording the image data in time series on the recording medium 23 and stores the moving image file on the recording medium 23. Store (step S10).

  Thereafter, the control unit 29 sets a user interface (UI) for special moving images (step S11). With this setting, for example, assignment at the time of moving image recording as shown in FIG. 10 is executed. The control unit 29 recognizes a signal that is subsequently received by the user interface set in step S11 as a special moving image signal based on FIG.

  Subsequently, the display control unit 292 switches to a screen for moving images by switching settings such as on-screen display (OSD) to be displayed on the display unit 21 (step S12). Specifically, the display control unit 292 displays, for example, a remaining time or an icon indicating that a special effect can be applied during moving image shooting. Thereafter, the imaging apparatus 1 proceeds to step S15 described later.

  FIG. 14 is a diagram illustrating a moving image screen display example on the display unit 21. As shown in FIG. 14, on the moving image screen Q, a moving image icon 111 indicating that the moving image special effect is applicable, an on-screen display 112 indicating moving image key assignment, and the remaining time for displaying the remaining time of moving image shooting. Time 113 is displayed. Among these, the on-screen display 112 may be automatically displayed after being displayed for a few seconds, or may be always displayed. Further, the display on the screen 112 may be switched between display and non-display by an input from the setting signal input unit 201.

  If it is determined in step S9 that the recording flag is off (step S9: No), the control unit 29 sets the still image user interface (step S13).

  Subsequently, the display control unit 292 switches settings such as on-screen display on the display unit 21 to still images (step S14). By this switching, the display unit 21 displays, for example, the remaining number of recorded sheets, an icon indicating that a special effect can be applied during still image shooting, and the like. Thereafter, the imaging apparatus 1 proceeds to step S15 described later.

  In step S7, when the moving image button 47 is not operated (step S7: No), the image processing control unit 291 causes the moving image special effect image processing unit 163 to perform moving image special effect processing (step S15). .

  FIG. 15 is a flowchart showing an outline of the special effect control process during moving images. In FIG. 15, when the key for starting the special effect during moving image is operated (step S31: Yes), the image processing control unit 291 turns on the special effect flag during moving image (step S32). Thereafter, the image processing control unit 291 performs effect setting (step S33).

  When the key for starting the special effect during moving image is not operated (step S31: No), when the end instruction is input (step S34: Yes), the image processing control unit 291 turns off the special effect flag during moving image. Control is performed (step S35). Thereafter, the imaging apparatus 1 returns to the main routine. Note that the image processing control unit 291 performs control to turn off the moving image special effect flag even when the one-shot echo and transit coefficients are zero.

  On the other hand, when the key for starting the special effect during moving image is not operated (step S31: No), when the end instruction is not input (step S34: No), the imaging device 1 returns to the main routine.

  When the first release signal is input from the release button 42 after the special effect control process during moving image in step S15 (step S16: Yes), the control unit 29 causes the AE processing unit 17 to execute the AE process for adjusting the exposure. At the same time, the AF processing unit 18 is caused to execute AF processing for adjusting the focus (step S17).

  Subsequently, the control unit 29 determines whether or not the power of the imaging device 1 is turned off by operating the power button 41 (step S18). When the control unit 29 determines that the power of the imaging device 1 has been turned off (step S18: Yes), the imaging device 1 ends this process. On the other hand, when the control unit 29 determines that the power of the imaging device 1 is not turned off (step S18: No), the imaging device 1 returns to step S3.

  When the 1st release signal is not input from the release button 42 (step S16: No), when the 2nd release signal is input from the release button 42 (step S19: Yes), the control unit 29 drives the shutter drive unit 11 and the image sensor. Each unit 13 is driven to perform photographing with a mechanical shutter (step S20).

  Subsequently, the image processing unit 16a performs predetermined image processing on the captured still image (step S21). Details of the image processing will be described later.

  Thereafter, the control unit 29 causes the image compression / expansion unit 19 to compress the image data in the JPEG format, and records the compressed image data on the recording medium 23 (step S22). In step S22, the control unit 29 records the image data compressed by the image compression / decompression unit 19 in the JPEG format on the recording medium 23 in association with the RAW data not subjected to the image processing by the image processing unit 16a. May be. After step S22, the imaging device 1 proceeds to step S18.

  In step S19, when the 2nd release signal is not input from the release button 42 (step S19: No), the control unit 29 causes the AE processing unit 17 to execute the AE process for adjusting the exposure, and the AF processing unit 18 is focused. The AF process to be adjusted is executed (step S23).

  Subsequently, the control unit 29 drives the image sensor driving unit 13 to execute photographing with the electronic shutter (step S24).

  Thereafter, the image processing unit 16a performs image processing based on the setting information of the imaging device 1 (step S25). Details of the image processing will be described later.

  The display control unit 292 causes the display unit 21 to display a live view image corresponding to the image data processed by the image processing unit 16a (step S26).

  Subsequently, if the imaging apparatus 1 is recording a moving image (step S27: Yes), the control unit 29 causes the image compression / expansion unit 19 to compress the image data, and the compressed image data is created on the recording medium 23. The moving image file is recorded as a moving image (step S28). Thereafter, the imaging apparatus 1 proceeds to step S18. On the other hand, if the moving image is not being recorded in step S27 (step S27: No), the imaging device 1 proceeds to step S18.

  FIG. 16 is a flowchart showing an overview of image processing. In FIG. 16, the basic image processing unit 161a performs basic image processing (step S41). The basic image processing here includes processing such as subtraction of OB (Optical Black) values, WB correction, synchronization, color matrix calculation, gamma conversion color correction, edge enhancement, and NR (Noise Reduction).

Here, the WB correction is performed by multiplying the Bayer array image data by an R gain and a B gain corresponding to a WB mode set in advance by the user, reading the WB from the flash memory of the imaging apparatus body, and multiplying the value. This is a process of correcting by this.
Synchronization is a process of converting each pixel into RGB data by interpolating data that does not exist in that pixel when the image sensor 12 is a Bayer array.
The color matrix calculation is a process of reading and multiplying the color matrix coefficient corresponding to the set WB mode from the flash memory of the main body.
In the gamma conversion color correction process, a gamma table designed in advance according to the finish setting is read from the flash memory of the main body, and the image data is gamma converted. At this time, the gamma conversion applied to the RGB data, and the RGB color space are converted into the color space represented by the luminance signal Y and the two color difference signals Cb and Cr, and then the gamma conversion is performed only on the luminance signal Y. May be. Further, in order to obtain appropriate color reproducibility, the color may be corrected using side parameters designed in advance according to the setting of the finish. The gamma curve may be changed depending on the type of art effect.
In the edge enhancement process, an edge component is extracted by a band pass filter, and is multiplied by a coefficient corresponding to the degree of edge enhancement and added to image data for enhancement.
In NR processing, noise is reduced by frequency-decomposing an image and performing coring processing or the like according to the frequency.

  Subsequently, the art effect image processing unit 162a performs art effect processing (step S42). Here, processing such as the above-described cross filter, soft focus, noise addition, shading, peripheral brightness increase, peripheral blur, and the like are performed.

  Thereafter, when the imaging apparatus 1 performs still image shooting (step S43: Yes), the control unit 29 applies a special effect (second special effect) corresponding to the special effect during the moving image (first special effect) to the still image. After performing control for prohibiting image data from being added, the process returns to the main routine.

  Here, the reason why the special effect corresponding to the special effect during moving image is not provided when the imaging apparatus 1 performs still image shooting will be described. Response speed is important for still image shooting during movie shooting. Since the number of pixels of a still image is about several times larger than the number of pixels of a moving image, if a special effect during moving image is applied to a still image, there is a possibility of affecting the resumption timing of moving image shooting after still image shooting. Therefore, in the first embodiment, application of special effects during moving images is prohibited in the case of still images. Note that it is possible to apply to still images as long as the response is not affected at high speed.

  On the other hand, when the imaging device 1 performs moving image shooting (step S43: No), the control unit 29 determines whether or not a moving image special effect is set (step S44). When the special effect during moving image is set (step S44: Yes), the process proceeds to step S45. On the other hand, when the special effect during moving images is not set (step S44: No), the imaging apparatus 1 returns to the main routine.

  When the special effect during moving image is set in step S44 (step S44: Yes), when the effect to be applied is multi-echo (step S45: Yes), the special effect image processing unit 163a during moving image For the processing result (previous frame), the current frame is synthesized at a predetermined ratio as the above-described multi-echo processing (see FIG. 4) (step S46). On the other hand, when the special effect to be applied is not multi-echo (step S45: No), the imaging device 1 proceeds to step S47.

  In step S47, the control unit 29 determines whether or not the effect to be applied is a one-shot echo. When the effect to be applied is a one-shot echo (step S47: Yes), the moving image special effect image processing unit 163a determines whether or not a release frame that is specific image data stored in the SDRAM 25 to obtain a one-shot echo effect. A synthesis process is performed (step S48). Here, in the case of the first frame after the special effect flag is turned on, the moving image special effect image processing unit 163a performs a process of storing the image of the current frame in the SDRAM 25. On the other hand, when the special effect flag is turned on and after two frames, the moving image special effect image processing unit 163a performs the combining process with the release frame stored in the SDRAM 25.

  In step S47, when the moving image special effect to be applied is not a one-shot echo (step S47: No), the imaging apparatus 1 proceeds to step S49.

  In step S49, the control unit 29 determines whether or not the effect to be applied is fluctuation. When the effect to apply is fluctuation (step S49: Yes), the moving image special effect image processing unit 163a performs a process of adding the fluctuation effect (step S50).

  A specific process of the moving image special effect image processing unit 163a in step S50 will be described. The moving image special effect image processing unit 163a adds an effect in which fluctuation is given to the image processing parameter in the art effect processing such as shading processing and cross filter processing. For example, in the case of shading processing, the attenuation characteristic from the center of the image is changed with time. If it is a peripheral blur, the blur amount and blur shape are changed. If it is a cross filter, the length and angle of the cross pattern are changed with time. Further, saturation, contrast, and white balance may be changed with time. When art effects such as shading and shading overlap, both may be given, or only the fluctuation may be given.

  In step S49, when the effect to be applied is not fluctuation (step S49: No), the imaging device 1 proceeds to step S51.

  In step S51, the control part 29 determines whether the effect to apply is a transit (step S51). When the effect to be applied is transit (step S51: Yes), the basic image processing unit 161a and the art effect image processing unit 162a respectively perform basic image processing and art effect processing according to the setting of the finish / switching destination ( Steps S52 and S53). Thereafter, the moving image special effect image processing unit 163a generates a transit image by performing a composition process of the two images for transit (step S54). After step S54, the imaging apparatus 1 returns to the main routine.

  In step S51, when the result of determination by the control unit 29 is that the effect to be applied is not transit (step S51: No), the imaging apparatus 1 returns to the main routine.

  According to Embodiment 1 of the present invention described above, the latest art effect generated by the image processing control unit 291 in the moving image special effect image processing unit 163a by the art effect image processing unit 162a and the art effect image processing unit 162b. The image A and the art effect image B are combined, and the ratio of the combination is changed over time. Thereby, even when two video effects having different processing speeds are given to the moving image frame, it is possible to shoot a moving image that is natural and has no sense of incongruity.

  In addition, according to the first embodiment, when two images to which art effects having different processing speeds are applied within a predetermined time are combined to record a combined image, the ratio of combining the two images is set to the number of frames. Since it can be changed gradually in synchronization, the switching from art effect A to art effect B can be smoothly changed.

  Further, according to the first embodiment, special effects equivalent to those that can be realized on an editing device or a PC can be executed by the imaging apparatus 1 itself. Therefore, a moving image reflecting the user's drawing intention can be created without requiring high-level specialized knowledge and without subsequent editing.

  In addition, according to the first embodiment, since the user interface constituting the setting signal input unit can have both video and still image input functions, the user has excellent operability even when shooting video. An interface can be realized. In addition, since the number of user interfaces can be reduced, the restriction on layout is reduced, and the imaging apparatus 1 suitable for downsizing can be realized.

  In general, an imaging apparatus has a small memory and inferior processing capability as compared with a PC or the like, so that it is difficult to realize editing processing of moving image data equivalent to that of a PC or the like. In addition, in the imaging apparatus 1 that can shoot still images in a moving image or still images immediately after the moving image ends, memory shortage and memory management problems are decisive. There is a possibility that problems such as responsiveness at the time of image shooting and degradation of still image quality may occur. In the first embodiment, in view of this point, it is possible to shoot a moving image to which a photographer's drawing intention is given by a special effect of temporal transition over a plurality of frames at the time of shooting without editing the shot moving image. Can be realized.

  In the first embodiment, moving image shooting is used, but a live view image displayed on the display unit 21 can also be applied. In this case, when a release signal is input during the transition from the art effect A to the art effect B, the transit image may be recorded as a still image by combining the ratios at the time when the release signal is input. Good.

  In the first embodiment, moving image shooting is used, but a live view image displayed by the electronic viewfinder can also be applied.

  Further, in the first embodiment, the display control unit 292 divides the display area of the display unit 21, displays a live view image with transit in one display area, and displays a live image in the other display area. It may be displayed as a view image. Thus, the user can take a picture while comparing the images.

  In the first embodiment, moving image shooting is used. However, continuous shooting in which continuous shooting is performed within a certain time can also be applied.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. The imaging apparatus 1 according to the second embodiment has a configuration different from that of the first embodiment described above and a transit process as a moving image special effect. Specifically, the imaging apparatus according to the second embodiment performs transit processing with one image processing unit. Therefore, in the following, after describing the configuration of the imaging apparatus according to the second embodiment, the transit process will be described. In addition, the same code | symbol is attached | subjected and demonstrated to the same structure.

  FIG. 17 is a block diagram illustrating a configuration of the imaging apparatus 100 according to the second embodiment. As illustrated in FIG. 17, the imaging apparatus 100 includes an image processing unit 110.

  The image processing unit 110 acquires image data from the SDRAM 25 via the bus 28, performs various image processing on the acquired image data (RAW data), and generates processed image data. This processed image data is output to the SDRAM 25 via the bus 28. The image processing unit 110 includes a basic image processing unit 161a, an art effect image processing unit 162a, and a moving image special effect image processing unit 163a.

  Next, transit processing executed by the imaging apparatus 100 according to the second embodiment will be described. FIG. 18 is a diagram for explaining an outline of a transit executed by the imaging apparatus 100 according to the second embodiment. FIG. 18 schematically shows a transit from art effect A to art effect B. In FIG. 18, the art effect A processing speed by the art effect image processing unit 162a is 30 fps (33 msec / frame), and the art effect B processing speed is 15 fps (66 msec / frame). It should be noted that the time required from the start of art effect switching to the end of switching in transit can be set as appropriate.

  In FIG. 18, the art effect image processing unit 162a generates an art effect image A0 obtained by applying the art effect A to the photographed image P0 before the input of the transit start trigger.

  When the start trigger is input, the image processing control unit 291 causes the art effect image processing unit 162a to execute a process (art effect B-1) that allows the art effect B to be performed between frames on the captured image P1. As a result, an art effect image B-1 that is less than one frame is generated and temporarily recorded in the SDRAM 25. Therefore, the image processing control unit 291 causes the moving image special effect image processing unit 163a to duplicate the art effect image A0 and record it as the transit image R31.

Thereafter, at time t = t ₁₂ , the image processing control unit 291 causes the art effect image processing unit 162a to execute the art effect A on the captured image P2 to generate the art effect image A1. At this time, the moving image special effect image processing unit 163a records the art effect image A1 as a transit image R32.

Thereafter, the image processing controller 291 at time t = t _13, with respect to the captured image P3, to execute the rest of the artistic effect B in artistic effect image processing section 162a (artistic effect B-2). Thereby, the art effect image B1 of one frame is generated. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the latest art effect image A1 signal to b = 0.9, and sets the coefficient to be multiplied to the latest art effect image B1 signal to 1−b = 0. Transit image R33 is generated and recorded by combining two images as 1.

Thereafter, the image processing controller 291 at time t = t _14, with respect to the captured image P4, art to effect image processing unit 162a by executing the artistic effect A to produce an artistic effect image A2. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A2 to b = 0.8, and sets the coefficient to be multiplied to the latest art effect image B1 to 1−b = 0.2. A transit image R34 is generated and recorded by combining the two images.

Thereafter, the image processing controller 291 at time t = t _15, with respect to captured image P5, to perform artistic effect image processing unit 162a to possible artistic effect B between frames handle (artistic effect B-1) . As a result, an art effect image B-1 that is less than one frame is generated and temporarily recorded in the SDRAM 25. For this reason, the moving image special effect image processing unit 163a generates the transit image R25 using the art effect image A2 and the art effect image B1. Specifically, the moving image special effect image processing unit 163a sets a coefficient to be multiplied by the signal to the art effect image A2 as b = 0.7, and sets a coefficient to be multiplied to the signal of the art effect image B1 as 1−b = 0.3. A transit image R35 is generated and recorded by combining the two images.

Thereafter, the image processing controller 291 at time t = t _16, with respect to captured image P6, art to effect image processing unit 162a by executing the artistic effect A to produce an artistic effect image A3. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A3 to b = 0.5 and sets the coefficient to be multiplied to the signal of the art effect image B1 to 1−b = 0.5. A transit image R36 is generated and recorded by combining the images.

Thereafter, the image processing control unit 291 causes the art effect image processing unit 162a to execute the remaining art effect B on the captured image P7 at time t = t ₁₇ (art effect B-2). Thereby, the art effect image B2 of one frame is generated. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the latest art effect image A3 signal to b = 0.3, and the coefficient to be multiplied to the latest art effect image B2 signal is 1−b = 0. 7, a transit image R37 is generated and recorded by combining the two images.

Thereafter, the image processing controller 291 at time t = t _18, with respect to captured image P8, art to effect image processing unit 162a by executing the artistic effect A to produce an artistic effect image A4. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A5 to b = 0.2 and sets the coefficient to be multiplied to the signal of the art effect image B2 to 1−b = 0.8. A transit image R38 is generated and recorded by combining the images.

  According to the second embodiment of the present invention described above, a transit whose effect changes with time can be performed by only one image processing unit 110. Thereby, power consumption can be reduced, and a moving image can be shot smoothly.

  Further, according to the second embodiment, since the transit can be performed by only one image processing unit 110, the imaging apparatus 100 can be reduced in size.

(Embodiment 3)
Next, a third embodiment of the present invention will be described. The imaging apparatus according to the third embodiment has the same configuration as that of the above-described second embodiment, and differs in transit processing as a moving image special effect executed by the imaging apparatus. For this reason, only the transit process executed by the imaging apparatus according to the third embodiment will be described below. In addition, the same code | symbol is attached | subjected and demonstrated to the same structure.

  FIG. 19 schematically illustrates a transit executed by the imaging apparatus 100 according to the third embodiment. FIG. 19 schematically illustrates a transit from art effect A to art effect B. In FIG. 19, the processing speed of art effect A by the art effect image processing unit 162a is 30 fps (33 msec / frame), and the processing speed of art effect B is 30 fps (33 msec / frame). It should be noted that the time required from the start of art effect switching to the end of switching in transit can be set as appropriate.

  In FIG. 19, before the input of the transit start trigger, the art effect image processing unit 162a generates an art effect image A0 obtained by applying the art effect A to the photographed image P0, and this art effect image A0 is used as a recorded image R0. It is recorded.

  When the start trigger is input, the image processing control unit 291 causes the art effect image processing unit 162a to apply the art effect B to the photographed image P1 to generate the art effect image B1. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A0 generated immediately before the start trigger to b = 0.9, and sets the coefficient to be multiplied to the signal of the art effect image B1 to 1−. A transit image R41 is generated and recorded by combining two images with b = 0.1.

Thereafter, the image processing controller 291 at time t = t _12, with respect to the photographed image P2, the artistic effect image processing unit 162a by subjected the artistic effect A to produce an artistic effect image A1. At this time, the special effect image processing unit 163a in the moving image sets the coefficient to be multiplied to the signal of the art effect image A1 as b = 0.8 and sets the coefficient to be multiplied to the signal of the art effect image B1 as 1−b = 0.2. A transit image R42 is generated and recorded by combining the images.

Thereafter, the image processing controller 291 at time t = t _13, with respect to the captured image P3, the artistic effect image processing unit 162a by subjected the artistic effect B to produce an artistic effect image B2. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A1 as b = 0.7, and sets the coefficient to be multiplied to the signal of the art effect image B2 as 1−b = 0.3. A transit image R43 is generated by combining the images and recorded.

Thereafter, the image processing controller 291 at time t = t _14, with respect to captured image P4, the artistic effect image processing unit 162a by subjected the artistic effect A to produce an artistic effect image A2. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A2 to b = 0.6, and sets the coefficient to be multiplied to the signal of the art effect image B2 to 1−b = 0.4. A transit image R44 is generated and recorded by combining the images.

Thereafter, the image processing controller 291 at time t = t _15, with respect to captured image P5, the artistic effect image processing unit 162a by subjected the artistic effect B to produce an artistic effect image B3. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A2 to b = 0.5 and sets the coefficient to be multiplied to the signal of the art effect image B3 to 1−b = 0.5. A transit image R45 is generated and recorded by combining the images.

Thereafter, the image processing control unit 291 causes the art effect image processing unit 162a to apply the art effect A to the photographed image P6 at time t = t ₁₆ to generate the art effect image A3. At this time, the moving image special effect processing unit 163a sets the coefficient to be multiplied by the signal of the art effect image A3 to b = 0.4, and sets the coefficient to be multiplied to the signal of the art effect image B3 to 1−b = 0.6. Are combined to generate and record a transit image R46.

Thereafter, the image processing controller 291 at time t = t _17, with respect to captured image P7, the artistic effect image processing unit 162a by subjected the artistic effect B to produce an artistic effect image B4. At this time, the special effect image processing unit 163a in the moving image sets the coefficient to be multiplied to the signal of the art effect image A3 as b = 0.3 and sets the coefficient to be multiplied to the signal of the art effect image B4 as 1−b = 0.7. A transit image R47 is generated by combining the images and recorded.

Thereafter, the image processing controller 291 at time t = t _18, with respect to captured image P8, the artistic effect image processing unit 162a by subjected the artistic effect A to produce an artistic effect image A4. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A4 as b = 0.2, and sets the coefficient to be multiplied to the signal of the art effect image B4 as 1−b = 0.8. A transit image R48 is generated and recorded by combining the images.

  As described above, when applying two art effects having the same processing speed by the art effect image processing unit 162a, the image processing control unit 291 causes the art effect A and the art effect B to be performed alternately, and the special effect image in the moving image. The ratio (ratio or coefficient) synthesized by the processing unit 163a is changed over time. Thereby, the transition from the art effect A to the art effect B can be performed smoothly.

  According to the third embodiment of the present invention described above, a transit whose effect changes over time with only one image processing unit 110 even when an art effect having the same processing speed is applied to a moving image frame. It can be performed. Thereby, power consumption can be reduced, and a moving image can be shot smoothly.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described. The imaging apparatus according to the fourth embodiment has the same configuration as that of the second embodiment described above, and the transit as the moving image special effect processing executed by the imaging apparatus is different. Therefore, in the following, the transit process executed by the imaging apparatus according to the fourth embodiment will be described. In addition, the same code | symbol is attached | subjected and demonstrated to the same structure.

  FIG. 20 is a diagram for explaining an outline of transit executed by the imaging apparatus according to the fourth embodiment. In FIG. 20, the transit which changes from art effect A to art effect B is shown typically. In FIG. 20, the art effect A processing speed by the art effect image processing unit 163a is 30 fps (33 msec / frame), and the art effect B processing speed is 15 fps (66 msec / frame). It should be noted that the time required from the start of art effect switching to the end of switching in transit can be set as appropriate.

  In FIG. 20, prior to the input of the transit start trigger, the art effect image processing unit 162a generates an art effect image A0 obtained by applying the art effect A to the photographed image P0, and this art effect image A0 is used as a recorded image R0. It is recorded.

  When the start trigger is input, the image processing control unit 291 causes the art effect image processing unit 163a to apply the art effect B to the photographed image P1 to generate the art effect image B21. At this time, the image processing control unit 291 thins the number of pixels of the captured image P1 in half in the art effect image processing unit 163a so that the processing time is substantially the same as the art effect A (first special effect). The art effect image B21 is generated with a size that can be processed at 30 fps. Furthermore, the moving image special effect image processing unit 163a sets b = 0.9 as a coefficient to be multiplied by the signal of the art effect image A0 generated immediately before the start trigger, and 1−b as a coefficient to be multiplied by the signal of the art effect image B21. Transit image R51 is generated and recorded by combining two images with = 0.1. In this case, when the art effect image B21 is combined with the art effect image A0, the moving image special effect image processing unit 163a combines the art effect images A21 by performing resizing processing to enlarge the size to the same size as the art effect image A0. Thereby, although the transit image R51 becomes slightly rough, it is possible to eliminate a frame delay of a moving image to be captured and to capture a smooth moving image.

Thereafter, the image processing controller 291 at time t = t _12, with respect to the photographed image P2, the artistic effect image processing unit 162a by subjected the artistic effect A to produce an artistic effect image A1. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A1 as b = 0.8 and sets the coefficient to be multiplied to the signal of the art effect image B21 as 1−b = 0.2. A transit image R52 is generated by combining the images. In this case, the moving image special effect image processing unit 163a generates a transit image R52 by synthesizing the art effect image B21 to the art effect image A1 by performing a resizing process to enlarge the art effect image B21 to the same size as the art effect image A1. And record.

Thus, the image processing controller 291 at time t = t ₁₃ ~t _20, with respect to captured image P3～P10, respectively to execute alternately artistic effect A and artistic effect B in artistic effect image processing section 162a Art effect images A2 to A5 and art effect images B22 to B25 are generated. Further, the image processing control unit 291 generates and records transit images R53 to R60 by causing the moving image special effect image processing unit 163a to combine the art effect A and the art effect B over time while reversing the ratio of combining the art effect A and the art effect B over time. .

  According to the fourth embodiment of the present invention described above, when the image processing control unit 291 causes the art effect image processing unit 163a to apply the art effect B to generate the art effect image B, substantially the same processing as the art effect A is performed. The art effect image processing unit 163a causes the art effect image processing unit 163a to thin out the number of pixels of the captured image P1 in half, thereby generating the art effect image B with a size that can be processed at 30 fps. As a result, even when an art effect having a different processing speed is applied to a moving image frame, transit with the effect changing over time can be performed with only one image processing unit 110. As a result, power consumption can be reduced and a moving image can be smoothly captured.

  In the fourth embodiment, the image processing control unit 291 causes the art effect image processing unit 163a to reduce the number of pixels of the photographed image in half, thereby generating the art effect image B with a size that can be processed at 30 fps. However, the art effect image may be generated with a data amount that can be processed at 30 fps by resizing the data amount of the photographed image in the art effect image processing unit 163a. Further, the art effect image processing unit 163a may reduce or compress the data amount of the photographed image.

(Embodiment 5)
Next, a fifth embodiment of the present invention will be described. The imaging apparatus according to the fifth embodiment has the same configuration as that of the second embodiment described above, and differs in transit processing as a moving image special effect executed by the imaging apparatus. Therefore, in the following, only the transit process executed by the imaging apparatus according to the fifth embodiment will be described. In addition, the same code | symbol is attached | subjected and demonstrated to the same structure.

  FIG. 21 schematically shows a transit executed by the imaging apparatus 100 according to the fifth embodiment. In FIG. 21, the transit which changes from art effect A to art effect B is typically shown. In FIG. 21, the processing speed of art effect A by the art effect image processing unit 163a is 30 fps (33 msec / frame), and the processing speed of art effect B is 15 fps (66 msec / frame). Furthermore, FIG. 21 illustrates an example in which the playback speed when playing back a captured image is changed from normal playback (1 ×) to 8 × playback. It should be noted that the time required from the start of art effect switching to the end of switching in transit can be set as appropriate.

  In FIG. 21, before the input of the transit start trigger, the art effect image processing unit 162a generates an art effect image A0 obtained by applying the art effect A to the photographed image P0, and this art effect image A0 is used as a recorded image R0. It is recorded.

  When the start trigger is input, the image processing control unit 291 causes the art effect image processing unit 162b to perform processing (art effect B-1) that enables the art effect B between frames on the captured image P1. As a result, an art effect image B <b> 1-1 that is less than one frame is generated and temporarily recorded in the SDRAM 25. For this reason, the image processing control unit 291 generates and records a transit image R61 by causing the moving image special effect image processing unit 163a to duplicate the art effect image A0.

Thereafter, at time t = t ₁₂ , the image processing control unit 291 causes the art effect image processing unit 162a to execute the art effect A on the captured image P2 to generate the art effect image A1. At this time, the moving image special effect image processing unit 163a performs a process of thinning out (not recording) a series of moving image shooting frames without duplicating the art effect image A1.

Thereafter, the image processing controller 291 at time t = t _13, with respect to the captured image P3, to execute the rest of the artistic effect B in artistic effect image processing section 162a (artistic effect B-2). Thereby, the art effect image B1 of one frame is generated. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A1 to b = 0.9, and sets the coefficient to be multiplied to the signal of the art effect image B1 to 1−b = 0.1. A transit image R62 is generated and recorded by combining the images.

Thereafter, the image processing controller 291 at time t = t _14, t _15, with respect to the captured image P4, P5, the artistic effect image processing unit 162a to perform the artistic effect A and artistic effect B-1. At this time, the moving image special effect image processing unit 163a performs a process of thinning out two frames from a series of moving image shooting frames without recording the two images generated by the art effect image processing unit 162a. As a result, when the captured moving image is played back, this period is tripled.

Thereafter, the image processing control unit 291 causes the art effect image processing unit 163a to apply the art effect A to the photographed image P6 at time t = t ₁₆ to generate the art effect image A2. At this time, the moving image special effect image processing unit 163a sets the coefficient to be multiplied to the signal of the art effect image A2 to b = 0.5, and sets the coefficient to be multiplied to the signal of the art effect image B1 to 1−b = 0.5. A transit image R63 is generated and recorded by combining the images.

Thereafter, at time t = t _{17 to} t ₁₉ , the image processing control unit 291 applies the art effect B-2, the art effect A, and the art effect B-1 to the art effect image processing unit 163a for the captured images P7 to P9. Are executed respectively. At this time, the moving image special effect image processing unit 163a performs a process of thinning out three frames from a series of moving image shooting frames without recording the three images generated by the art effect image processing unit 162a. As a result, when the captured moving image is played back, this period becomes quadruple speed.

  Thereafter, the image processing control unit 291 causes the art effect image processing unit 163a to execute the art effect A, the art effect B-1, and the art effect B-2 on the captured image at each time. At this time, the moving image special effect image processing unit 163a combines two images from the eight images generated by the art effect image processing unit 162a at a predetermined ratio, and records only the combined image as a transit image. A process of thinning out seven frames from a series of moving image shooting frames is performed without recording the other seven images. As a result, when the captured moving image is played back, this period becomes 8 times speed.

  According to the fifth embodiment of the present invention described above, when a captured moving image is played back by thinning out the frames of the image to be recorded, the playback speed can be gradually played back at 8 times speed. Accordingly, it is possible to smoothly switch the art effect having a high processing speed to the art effect having a low processing speed, and it is possible to shoot a more comical moving image reflecting the user's intention.

  The fifth embodiment described above can be applied even when the art effect is switched from an art effect having a low processing speed to an art effect having a high processing speed. In this case, since the intermittent speed is changed to the same speed, it is possible to shoot a mysterious movie as if it was awakened from a dream.

  Further, in the above-described fifth embodiment, the frames corresponding to each time are thinned out according to the playback speed at which the captured moving image is played back, but this frame may be recorded. FIG. 22 is a diagram for explaining the outline of the transit according to the first modification of the fifth embodiment.

  As shown in FIG. 22, the image processing control unit 291 causes the art effect image processing unit 162a to execute the art effect A, the art effect B-1, and the art effect B-2 on the captured image at each time. An effect image is generated, and the special effect image processing unit 163a in the moving image generates a transit image by synthesizing an intermediate frame between the frames to be recorded at a predetermined ratio, for example, a ratio immediately after the synthesis, and records the transit image. Also good. Furthermore, the synthesized transit image may be duplicated and recorded. Thereby, a smooth moving image can be reproduced when the photographed image is reproduced. The number of intermediate frames to be recorded may be set as appropriate according to the playback speed. For example, 2 frames may be recorded for 4 × speed reproduction, and 4 frames may be recorded for 8 × speed reproduction. Thereby, it is possible to shoot a continuously connected moving image while maintaining a comical moving image.

(Other embodiments)
Up to this point, the mode for carrying out the present invention has been described. However, the present invention should not be limited only by the first to fifth embodiments.

  For example, in the present invention, an electronic viewfinder may be provided in the main body unit separately from the display unit, and the present invention may be applied to this electronic viewfinder. In this case, it is more preferable that the special effect in the moving image is made different between the display unit and the electronic viewfinder.

  In the present invention, the main body portion and the lens portion may be integrally formed.

  In addition to a digital single-lens reflex camera, the imaging device according to the present invention is also applicable to, for example, digital cameras that can be equipped with accessories and the like, digital video cameras, and electronic devices such as mobile phones and tablet mobile devices having a shooting function. can do.

  In the description of the flowchart in the present specification, the context of the processing between steps is clearly indicated using expressions such as “first”, “after”, “follow”, etc., in order to implement the present invention. The order of processing required is not uniquely determined by their representation. That is, the order of processing in the flowcharts described in this specification can be changed within a consistent range.

  As described above, the present invention can include various embodiments not described herein, and various design changes and the like can be made within the scope of the technical idea specified by the claims. Is possible.

DESCRIPTION OF SYMBOLS 1,100 Imaging device 2 Main body part 3 Lens part 12 Image pick-up element 16a, 16b, 110 Image processing part 19 Image compression expansion part 20 Input part 21 Display part 22 Display drive part 23 Recording medium 26 Flash memory 27 Main body communication part 28 Bus 29 Control unit 31 Optical system 35 Lens operation unit 36 Lens flash memory 37 Lens communication unit 38 Lens control unit 41 Power button 42 Release button 43 Mode dial 44 Operation button 45 Menu button 46 Play button 47 Movie button 48 Function key 49 Erase button 50 Touch panel 161a, 161b Basic image processing unit 162a, 162b Art effect image processing unit 163a, 163b Special effect image processing unit during moving image 201 Setting signal input unit 202 Moving image shooting signal input unit 261 Program recording unit 262 Special effect processing information recording unit 263 Image processing information recording unit 291 Image processing control unit 292 Display control unit 441 Cross key 442 Enter button 443 Up arrow key 444 Down arrow key 445 Left arrow key 446 Right arrow key

Claims (6)

An imaging device capable of capturing an image of a subject, generating image data of the subject, and capturing a moving image,
The first special effect image data and the second special effect are provided by applying a first special effect and a second special effect, which have different processing speeds and visual effects, over a plurality of frames corresponding to moving image data. A special effect image processing unit for generating effect image data;
A special effect image processing unit in a moving image for generating composite image data by combining the latest first special effect image data and second special effect image data generated by the special effect image processing unit;
An image processing control unit for changing a ratio of the first special effect image data and the second special effect image data to be synthesized with time by the special effect image processing unit in the moving image;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the image processing control unit causes the special effect image processing unit to alternately apply the first and second special effects. The special effect image processing unit includes a first special effect processing unit that imparts the first special effect, and a second special effect processing unit that imparts the second special effect,
The image processing control unit causes the first special effect processing unit and the second special effect processing unit to execute the application of the first special effect and the second special effect, respectively. The imaging apparatus according to 1 or 2.   When the second special effect is larger than the first special effect when the processing speed for the same image data is larger than the first special effect, the image processing control unit is configured so that the second special effect is approximately the same as the first special effect. The imaging apparatus according to claim 1, wherein the image data to which the special effect is applied is resized and executed by the special effect image processing unit. An image processing method executed by an imaging device capable of capturing an image of a subject, generating image data of the subject, and capturing a moving image,
The first special effect image data and the second special effect are provided by applying a first special effect and a second special effect, which have different processing speeds and visual effects, over a plurality of frames corresponding to moving image data. Special effect image processing step for generating effect image data;
A special effect image processing step in a moving image for generating composite image data by combining the latest first special effect image data and second special effect image data generated in the special effect image processing step;
An image processing control step of changing a ratio of combining the first special effect image data and the second special effect image data in the moving image special effect image processing step with time;
An image processing method comprising: A program for imaging an object, generating image data of the object, and causing an imaging apparatus capable of capturing a movie to be executed,
The first special effect image data and the second special effect are provided by applying a first special effect and a second special effect, which have different processing speeds and visual effects, over a plurality of frames corresponding to moving image data. Special effect image processing step for generating effect image data;
A special effect image processing step in a moving image for generating composite image data by combining the latest first special effect image data and second special effect image data generated in the special effect image processing step;
An image processing control step of changing a ratio of combining the first special effect image data and the second special effect image data in the moving image special effect image processing step with time;
A program characterized by having executed.