JP2010245686A - Imaging apparatus, method of processing image, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problem: it takes time to compose two image data to obtain a composited image. <P>SOLUTION: YC processing 62b for obtaining synchronized image data is performed to RAW data obtained by photographing 61b, and processing 63b is performed for compositing the image data obtained by the YC processing 62b and image data subjected to low-pass filtering 65a after the previous photographing 61a to display on an LCD (display processing 64b). Then, low-pass filtering 65b is performed to the image data after the YC processing 62b. Processing 63c is performed for compositing the image data subjected to the low-pass filter 65b and image data, where YC processing 62c is performed to the RAW data obtained by the next photographing 61c. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for combining a plurality of images.

  When photographing with a silver salt camera, an optical filter is disposed in front of the photographing lens, so that an image having a soft flare-like soft focus effect can be taken as a whole although the outline is clear to some extent. On the other hand, in a digital camera, a method for reproducing the same effect as the soft focus of a silver salt camera by processing imaged image data without arranging such an optical filter has been proposed. Japanese Patent Application Laid-Open No. 2004-133867 proposes a method for realizing soft focus by combining image data obtained by performing filter processing on captured image data and original image data.

JP 09-251532 A

  During live view display or moving image shooting, the time from when an image is shot to when image processing is performed for display or recording is limited by the frame rate. Therefore, as the frame rate is increased, the time for performing image processing tends to be limited.

  On the other hand, when the image data is combined with the original image by performing filter processing on the image data as in the case of soft focus, the processing time increases as the image size increases. Therefore, in the above-described live view display and moving image shooting, it is necessary to reduce the image size or lower the frame rate in order to secure these processing times.

  An object of the present invention is to provide a technique for shortening the time required from shooting to combining processing when performing processing for combining a plurality of images during live view display or moving image recording.

  An imaging apparatus according to an aspect of the present invention includes an imaging unit that captures an image of a subject and obtains RAW image data, and a first RAW image obtained by continuously inputting the RAW image data and obtained at least at a first time. First image data is obtained by performing first image processing on the data, and second RAW image data obtained at a second time before the first time is set to a second value. An image processing unit that obtains second image data by performing image processing, an image combining unit that combines the first image data and the second image data to generate combined image data, and the combined image data And a display unit for displaying.

  An image processing method according to another aspect of the present invention includes a step of inputting RAW image data obtained by imaging a subject, the RAW image data being continuously input, and obtained at least at a first time. First image data is obtained by performing first image processing on the first RAW image data, and second RAW image data obtained at a second time before the first time is added to the second RAW image data. A step of obtaining second image data by performing second image processing on the image, and a step of synthesizing the first image data and the second image data to generate composite image data. It is characterized by.

  An image processing program according to still another aspect of the present invention is obtained at least at a first time by inputting RAW image data obtained by imaging a subject and continuously inputting the RAW image data. In addition, first image data is obtained by performing first image processing on the first RAW image data, and second RAW image data obtained at a second time before the first time. Performing second image processing on the second image data to obtain second image data, and synthesizing the first image data and the second image data to generate synthesized image data. It is a program for making it run.

  According to the present invention, when a process for combining a plurality of images is performed, the time required from photographing to combining process can be shortened.

FIG. 1 is a block diagram illustrating a configuration of a digital still camera that is an imaging apparatus according to an embodiment. It is a flowchart which shows the main processing flow performed with the digital still camera as an imaging device which concerns on one Embodiment. It is a flowchart which shows the detailed processing content of imaging | photography and recording operation | movement of a still image. It is a flowchart which shows the detailed processing content of imaging | photography / recording operation | movement of a moving image. It is a flowchart which shows the detailed processing content of a live view display. 6A is a diagram illustrating a time chart during live view display and moving image shooting performed by the imaging apparatus according to the embodiment, and FIG. 6B is a view during live view display and moving image shooting according to the related art. It is a figure which shows this time chart.

  FIG. 1 is a block diagram illustrating a configuration of a digital still camera that is an imaging apparatus according to an embodiment. The digital still camera shown in FIG. 1 includes a camera body 1 and an interchangeable lens 2.

  The interchangeable lens 2 includes a lens 1010, a flash memory 1011, a microcomputer 1012, a driver 1013, and a diaphragm 1014. The interchangeable lens 2 is communicably connected to the camera body 1 via the I / F 999.

  The camera body 1 includes a mechanical shutter 101, an image sensor 102, an analog processing unit 103, an analog / digital conversion unit 104 (hereinafter referred to as A / D conversion unit 104), a bus 105, an SDRAM 106, and an image processing unit 107. An AE processing unit 108, an AF processing unit 109, a JPEG processing unit 110, a memory interface 111 (hereinafter referred to as a memory I / F 111), a recording medium 112, an LCD driver 113, an LCD 114, and a microcomputer 115. , An operation unit 116, a flash memory 117, a low-pass filter unit 118, and an image composition unit 119.

  The lens 1010 focuses the optical image of the subject on the image sensor 102. The lens 1010 may be a single focus lens or a zoom lens.

  The microcomputer 1012 is connected to the I / F 999, the flash memory 1011 and the driver 1013. The microcomputer 1012 reads and writes information stored in the flash memory 1011 and controls the driver 1013. The microcomputer 1012 can further communicate with the microcomputer 115 via the I / F 999, transmits various information to the microcomputer 115, and receives information such as an aperture value from the microcomputer 115.

  In response to an instruction from the microcomputer 1012, the driver 1013 drives the lens 1010 to change the focal length and focus position, and drives the aperture 1014. The aperture 1014 is provided in the vicinity of the lens 1010 and adjusts the amount of light of the subject.

  The mechanical shutter 101 is driven in response to an instruction from the microcomputer 115 to control the time for exposing the subject to the image sensor 102.

  The image sensor 102 is an image sensor in which a Bayer array color filter is arranged in front of a photodiode constituting each pixel. The Bayer array has a line in which R pixels and G (Gr) pixels are alternately arranged in a horizontal direction, and a line in which G (Gb) pixels and B pixels are alternately arranged, and the two lines are further divided. It is configured by alternately arranging in the vertical direction. The imaging element 102 receives the light collected by the lens 1010 by a photodiode that constitutes a pixel and performs photoelectric conversion, and outputs the amount of light to the analog processing unit 103 as a charge amount. The image sensor 102 may be a CMOS type or a CCD type.

  The analog processing unit 103 performs waveform shaping on the electrical signal (analog image signal) read from the image sensor 102 while reducing reset noise and the like, and further increases the gain so that the target brightness is obtained. . The A / D conversion unit 104 converts the analog image signal output from the analog processing unit 103 into a digital image signal (hereinafter referred to as image data).

  A bus 105 is a transfer path for transferring various data generated in the digital camera to each unit in the digital camera. The bus 105 includes an A / D conversion unit 104, an SDRAM 106, an image processing unit 107, an AE processing unit 108, an AF processing unit 109, a JPEG processing unit 110, a memory I / F 111, an LCD driver 113, The microcomputer 115, the low-pass filter unit 118, and the image composition unit 119 are connected.

  The image data output from the A / D conversion unit 104 is temporarily stored in the SDRAM 106 via the bus 105. The SDRAM 106 temporarily stores various data such as image data obtained by the A / D conversion unit 104 and image data processed by the image processing unit 107, the JPEG processing unit 110, the low-pass filter unit 118, and the image synthesis unit 119. It is a storage unit that is stored.

  The image processing unit 107 includes an optical black subtraction unit 1071 (hereinafter referred to as OB subtraction unit), a white balance correction unit 1072 (hereinafter referred to as WB correction unit 1072), a synchronization processing unit 1073, a gamma / color reproduction processing unit 1074, a color matrix calculation. A section 1075, an edge enhancement processing section 1076, and a noise reduction processing section 1077 (hereinafter referred to as NR processing section 1077), which perform various image processing on the image data read from the SDRAM 106.

  The OB subtraction unit 1071 performs optical black subtraction processing (hereinafter referred to as OB subtraction processing) on the image data. The OB subtraction process is a process of subtracting an optical black value (hereinafter referred to as an OB value) caused by a dark current of the image sensor 102 from a pixel value of each pixel constituting image data.

  The WB correction unit 1072 performs a process of correcting the white balance by multiplying the image data by a white balance gain corresponding to the white balance mode. The white balance mode can be set by the user in accordance with a light source such as sunny weather, cloudy weather, a light bulb, and a fluorescent light.

  The synchronization processing unit 1073 performs a process of synchronizing image data based on the Bayer array into image data including R, G, and B information per pixel. The gamma / color reproduction processing unit 1074 performs gamma correction processing and color reproduction processing for changing the color of an image.

  The color matrix calculation unit 1075 corrects the color of the image data by performing linear conversion on the image data by multiplying the color matrix. The edge enhancement processing unit 1076 performs a process of enhancing the edge of the image data by extracting the edge from the image data, multiplying the extracted edge data by a predetermined gain, and adding it to the image data. The NR processing unit 1077 performs processing for reducing noise by processing using a filter that reduces high frequency, coring processing, and the like.

  The image processing unit 107 selects each of the units 1071 to 1077 provided therein and performs each process as necessary. The image data after each processing is performed by the image processing unit 107 is stored in the SDRAM 106.

  The AE processing unit 108 calculates subject luminance from the image data. The data for calculating the subject brightness may be an output of a dedicated photometric sensor. The AF processing unit 109 extracts a high-frequency component signal from the image data, and acquires a focus evaluation value by AF (Auto Focus) integration processing.

  The low-pass filter unit 118 generates a totally blurred image by performing a low-pass filter process for reducing high-frequency components of image data stored in the SDRAM 106. In the image subjected to the low-pass filter processing, the contour is also lost.

  The image synthesis unit 119 is totally blurred by synthesizing the image that has been subjected to the low-pass filter processing by the low-pass filter unit 118 and the image that has not been subjected to the low-pass filter processing at a predetermined ratio. Produces a clear image. Here, when the composition ratio is α (0 ≦ α ≦ 1), the pixel value of the image subjected to the low-pass filter processing is a, and the pixel value of the corresponding pixel of the image not subjected to the low-pass filter processing is b. The pixel value of the corresponding pixel of the composite image is α × a + (1−α) × b.

  When the image data is recorded, the JPEG processing unit 110 reads the image data from the SDRAM 106, compresses the read image data according to the JPEG compression method, and temporarily stores the compressed JPEG image data in the SDRAM 106. The microcomputer 115 creates a JPEG file by adding a JPEG header necessary for constructing a JPEG file to the JPEG image data stored in the SDRAM 106, and sends the created JPEG file via the memory I / F 111. To the recording medium 112. The recording medium 112 is, for example, a recording medium including a memory card that can be attached to and detached from the camera body 1, but is not limited thereto.

  The LCD driver 113 displays an image on the LCD 114. Display of images includes display of moving images such as REC view display that displays image data immediately after shooting for a short time, playback display of a JPEG file recorded on the recording medium 112, and live view display. When playing back a JPEG file recorded on the recording medium 112, the JPEG processing unit 110 reads the JPEG file recorded on the recording medium 112, performs decompression processing, and temporarily stores the decompressed image data in the SDRAM 106. Let The LCD driver 113 reads the decompressed image data from the SDRAM 106, converts the read image data into a video signal, and then outputs it to the LCD 114 to display an image.

  The microcomputer 115 having a function as a control unit comprehensively controls various sequences of the digital camera body 1. An operation unit 116 and a flash memory 117 are connected to the microcomputer 115.

  The operation unit 116 is an operation member such as a power button, a release button, and various input keys. When one of the operation members of the operation unit 116 is operated by the user, the microcomputer 115 executes various sequences according to the user's operation. The power button is an operation member for instructing power on / off of the digital camera. When the power button is pressed, the microcomputer 115 turns the digital camera on or off. The release button has a two-stage switch of a first release switch and a second release switch. When the release button is pressed halfway and the first release switch is turned on, the microcomputer 115 performs a shooting preparation sequence such as AE processing and AF processing. Further, when the release button is fully pressed and the second release switch is turned on, the microcomputer 115 performs shooting by executing a shooting sequence.

  The flash memory 117 stores various parameters necessary for the operation of the digital camera, such as a white balance gain and a low-pass filter coefficient corresponding to the white balance mode, a manufacturing number for specifying the digital still camera, and the like. The flash memory 117 also stores various programs executed by the microcomputer 115. The microcomputer 115 reads parameters necessary for various sequences from the flash memory 117 according to a program stored in the flash memory 117, and executes each process.

  FIG. 2 is a flowchart illustrating a main processing flow performed by a digital still camera that is an imaging apparatus according to an embodiment. When the user presses the power button to turn on the digital still camera, the microcomputer 115 starts the process of step S201.

  In step S201, it is determined whether or not to perform shooting, that is, whether or not the release button has been pressed. If it is determined that the release button has been pressed, the process proceeds to step S202.

  In step S202, it is determined whether or not the shooting mode is set to the still image shooting mode. The shooting modes include a still image shooting mode and a moving image shooting mode, which can be set by the user operating the operation member of the operation unit 116. If it is determined that the shooting mode is set to the still image shooting mode, the process proceeds to step S203, and if it is determined that the moving image shooting mode is set, the process proceeds to step S204.

  In step S203, a still image shooting / recording operation is performed. The still image shooting / recording operation will be described later with reference to the flowchart shown in FIG.

  In step S204, a moving image shooting / recording operation is performed. The moving image shooting / recording operation will be described later with reference to the flowchart shown in FIG.

  On the other hand, if it is determined in step S201 that the release button has not been pressed, the process proceeds to step S205. In step S205, live view display is performed. Detailed processing of live view display will be described later with reference to a flowchart shown in FIG.

  FIG. 3 is a flowchart showing detailed processing contents of the still image shooting / recording operation performed in step S203 of the flowchart shown in FIG. The processing from step S301 to step S303 is processing performed at the first release, and the processing after step S304 is processing performed at the second release.

  In step S301, the AF processing unit 109 calculates a focus evaluation value. The microcomputer 115 issues a command for driving the lens 1010 to the driver 1013 based on the focus evaluation value.

  In step S302, the AE processing unit 108 calculates subject brightness. In step S303, the aperture and shutter speed are calculated by referring to the aperture value and shutter speed determination table stored in the flash memory 117 based on the subject brightness.

  In step S304, it is determined whether or not the user has fully pressed the release button to turn on the second release switch. If it is determined that the second release switch is not turned on, the process proceeds to step S305. In step S305, it is determined whether or not the release button has been returned. If it is determined that the release button has been returned from the half-pressed state, the photographing / recording operation is terminated. If it is determined that the release button has not been returned, the process returns to step S304.

  On the other hand, if it is determined in step S304 that the second release switch is turned on, the process proceeds to step S306. In step S306, shooting is performed. The shooting is the same as the conventionally used method. The driver 1013 drives the aperture 1014 so as to achieve a set aperture value based on an instruction from the microcomputer 1012. Based on the calculated shutter speed, shooting is performed by controlling the mechanical shutter 101 to obtain image data.

  In step S307, the OB subtraction unit 1071 performs OB subtraction processing for subtracting the OB value obtained at the time of imaging from the image data obtained by imaging.

  In step S308, the WB correction unit 1072 performs a process of correcting the white balance by multiplying the image data subjected to the OB subtraction process by a white balance gain according to the white balance mode. Note that the white balance mode can be set for each shooting by the user operating an input key included in the operation unit 116. The microcomputer 115 sets the white balance mode based on the operation of the operation unit 116 by the user. When the digital still camera has an auto white balance function that automatically adjusts the white balance, the microcomputer 115 automatically sets a white balance mode in accordance with the light source at the time of shooting.

  In step S309, the synchronization processing unit 1073 performs synchronization processing on the image data on which the white balance correction processing has been performed. In step S310, the color matrix calculation unit 1075 performs a color matrix calculation for multiplying the image data subjected to the synchronization processing by a color matrix corresponding to the white balance mode.

  In step S311, the gamma / color reproduction processing unit 1074 performs gamma correction processing and color reproduction processing for changing the color of the image on the image data on which the color matrix calculation has been performed.

  In step S312, the edge enhancement processing unit 1076 performs edge enhancement processing on the image data that has been subjected to gamma correction processing and color reproduction processing.

  In step S313, the NR processing unit 1077 performs noise reduction processing on the image data on which the edge enhancement processing has been performed. In the noise reduction process, a coring process based on the coring parameter or a process using a filter that reduces high frequency is performed based on the noise reduction parameter (hereinafter referred to as NR parameter).

  In step S314, the low-pass filter unit 118 performs low-pass filter processing on the image data subjected to noise reduction processing in step S313, thereby generating an entirely blurred image.

  In step S315, image data that has been subjected to noise reduction processing in step S313 by the image composition unit 119, that is, image data that has not undergone low-pass filter processing, and image data that has undergone low-pass filter processing in step S314, Are synthesized at a synthesis ratio α to generate a synthesized image.

  In step S316, the JPEG processing unit 110 performs JPEG compression on the composite image data generated in step S315. In step S317, shooting information such as an image recording mode and exposure conditions is created as fill header information, and the created file header information is added to JPEG-compressed image data and recorded via the memory I / F 111. Recording on the medium 112.

  FIG. 4 is a flowchart showing detailed processing contents of the moving image shooting / recording operation performed in step S204 of the flowchart shown in FIG.

  In step S401, the AE processing unit 108 calculates subject brightness. In step S402, the aperture and shutter speed are calculated by referring to the aperture value and shutter speed determination table stored in the flash memory 117 based on the subject brightness.

  In step S403, shooting is performed. Here, shooting with a so-called electronic shutter is performed for moving image shooting. Since the photographing using the electronic shutter is the same as the method used conventionally, detailed explanation is omitted here.

  In step S404, the OB subtraction unit 1071 performs OB subtraction processing for subtracting the OB value obtained at the time of imaging from the image data obtained by imaging. In step S405, the WB correction unit 1072 performs a process of correcting the white balance by multiplying the image data subjected to the OB subtraction process by a white balance gain according to the white balance mode.

  In step S406, the synchronization processing unit 1073 performs synchronization processing on the image data that has been subjected to white balance correction processing. In step S407, the color matrix calculation unit 1075 performs color matrix calculation by multiplying the image data subjected to the synchronization processing by a color matrix corresponding to the white balance mode.

  In step S408, the gamma / color reproduction processing unit 1074 performs gamma correction processing and color reproduction processing for changing the color of the image on the image data on which the color matrix calculation has been performed. In step S409, the edge enhancement processing unit 1076 performs edge enhancement processing on the image data that has been subjected to gamma correction processing and color reproduction processing.

  In step S410, the NR processing unit 1077 performs noise reduction processing on the image data on which the edge enhancement processing has been performed. In the noise reduction process, a coring process based on the coring parameter or a process using a filter that reduces high frequency is performed based on the noise reduction parameter (hereinafter referred to as NR parameter).

  In step S <b> 411, it is determined whether the image taken immediately before is stored in the SDRAM 106. During moving image shooting, the processing from step S401 to step S417 is repeated. The image captured immediately before is an image stored in step S416, which will be described later, at the time of the previous shooting. Therefore, at the time of the first shooting process immediately after the start of moving image shooting, the image shot immediately before is not stored in the SDRAM 106. If it is determined that the image taken immediately before is not stored in the SDRAM 106, the process proceeds to step S415. If it is determined that the image is stored, the process proceeds to step S412.

  In step S412, the image synthesizing unit 119 synthesizes the image data subjected to the noise reduction process in step S410 and the image data stored in the SDRAM 106 in step S416 at a synthesis ratio α to generate a synthesized image. The image data stored in the SDRAM 106 in step S416 is image data that has been subjected to low-pass filter processing as will be described later.

  In step S413, the composite image generated in step S412 is displayed on the LCD 114 by the LCD driver 113.

  In step S414, the composite image generated in step S412 is recorded on the recording medium 112.

  In step S415, the low-pass filter unit 118 performs low-pass filter processing on the image data subjected to noise reduction processing in step S410, thereby generating an entirely blurred image.

  In step S416, the image data that has been subjected to the low-pass filter processing in step S415 is stored in the SDRAM 106.

  In step S417, it is determined whether or not to end the process. When the release button is pressed again by the user or the power button is pressed, the moving image shooting / recording process ends. On the other hand, if it determines with not complete | finishing a process, it will return to step S401.

  In the above-described processing, the first frame is not displayed on the LCD 114, but the second and subsequent frames are displayed. However, since the shooting interval is extremely short (for example, 1/30 second), the display is not performed. There is no problem.

  FIG. 5 is a flowchart showing the detailed processing contents of the live view display performed in step S205 of the flowchart shown in FIG. Of the processes in the flowchart shown in FIG. 5, steps that perform the same processes as those in the flowchart shown in FIG. 4 are given the same reference numerals. In the process of the flowchart shown in FIG. 5, the processes in steps S414 and S417 are omitted from the process of the flowchart shown in FIG. That is, when the composite image is displayed on the LCD 114 in step S413, the process proceeds to step S415 without performing the process of recording the composite image data on the recording medium 112. When the process of step S416 is finished, the process of the flowchart shown in FIG. 5 is finished. Since the processing of Step S414 and Step S417 is the same as the processing of the flowchart shown in FIG. 4 except that the processing of Step S414 is omitted, detailed description of the processing of the flowchart shown in FIG.

  FIG. 6A is a diagram illustrating a time chart during live view display and moving image shooting performed by the imaging apparatus according to the embodiment, and FIG. 6B is a view during live view display and moving image shooting according to the related art. It is a figure which shows a time chart.

  First, a time chart according to the prior art will be described. As shown in FIG. 6B, in the conventional technique, synchronized image data is generated based on RAW image data obtained by photographing, and low-pass filter processing is performed on the synchronized image data. Apply. Then, image processing for synthesizing the image data before being subjected to the low-pass filter processing and the image data subjected to the low-pass filter processing is performed, the synthesized image is displayed on the LCD, and a moving image is recorded during moving image shooting. With this method according to the prior art, the photographing interval Δt2 cannot be further shortened, and the frame rate cannot be increased.

  Then, the time chart by the imaging device in one Embodiment is demonstrated. 6A, when the processing from step S404 to step S410 in FIG. 4 or FIG. 5 (YC processing 62b in FIG. 6A) is performed on the RAW image data obtained by photographing 61b, YC A process 63b for synthesizing the image data obtained by the process 62b and the image data subjected to the low-pass filter process 65a after the previous photographing 61a is performed. The combined image data is displayed on the LCD 114 and is recorded as a moving image when displaying a moving image (display / recording process 64b).

  Thereafter, low-pass filter processing 65 b is applied to the image data after the YC processing 62 b and stored in the SDRAM 106. Then, using the image data that has been subjected to the low-pass filter process 65b, a process 63c for combining the RAW image data obtained by the next photographing 61c with the image data that has been subjected to the YC process 62c is performed. Since the image data subjected to the low-pass filter processing 65b is combined with the image data after the next shooting 61c, it can be performed in parallel with the display processing 64b after the current shooting 61b and the next shooting processing 61c.

  In general, since the correlation between frames is very high, even if the synthesis process is performed using image data obtained by the previous shooting, the synthesized image does not become so unnatural. .

  That is, in the imaging apparatus according to the embodiment, the image data after YC processing is combined with image data that has been subjected to low-pass filter processing after the previous shooting, and in parallel with the display processing and shooting processing. The low-pass filter processing is performed on the image data after YC processing. Thereby, the processing time from photographing to composition and display can be shortened. As shown in FIGS. 6A and 6B, the imaging interval Δt1 of the imaging apparatus in one embodiment is shorter than the imaging interval Δt2 of the conventional technique. The length (processing time) of the “image processing” block shown in FIG. 6B is the sum of the “YC”, “composite”, and “LP” blocks shown in FIG. This is the same as the processing time.

  As described above, according to the imaging apparatus of the embodiment, the RAW image data is continuously input, and the first RAW image data obtained at least at the first time is subjected to the first image processing. And obtaining second image data by performing second image processing on the second RAW image data obtained at a second time before the first time, The first image data and the second image data are combined to generate combined image data. Generation, display or recording of composite image data from shooting by generating second image data used for composition based on second RAW image data obtained at a second time before the first time Can be shortened.

  The second image processing is to perform the third image processing after the first image processing, and at least a part of the third image processing is performed at the time after the display processing and the first time. Since this is performed in parallel with at least one of the imaging processes performed in step S1, the time from shooting to generation, display, or recording of composite image data can be shortened.

  Further, the second image data is RAW image data having a higher correlation by being RAW image data obtained immediately before the first RAW image data among the RAW image data continuously input. Thus, more natural composite image data can be obtained.

  In the description of the embodiment described above, hardware processing is assumed as processing performed by the imaging apparatus, but it is not necessary to be limited to such a configuration. For example, a configuration in which processing is performed separately by software is possible. In this case, the imaging apparatus includes a main storage device such as a CPU and a RAM, and a computer-readable storage medium in which a program for realizing all or part of the above processing is stored. Here, this program is called an image processing program. Then, the CPU reads out the image processing program stored in the storage medium and executes information processing / calculation processing, thereby realizing the same processing as that of the imaging apparatus described above.

  Here, the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and the like. Alternatively, the image processing program may be distributed to a computer via a communication line, and the computer that has received the distribution may execute the image processing program.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention. For example, in order to generate composite image data, image data subjected to low-pass filter processing is used, but the present invention can also be applied to those using image data subjected to image processing other than low-pass filter processing. .

  The composite image data includes first image data generated based on the first RAW image data obtained at the first time, and second RAW image data obtained immediately before the first RAW image data. And the second image data generated based on the above. However, the second RAW image data is not limited to the data obtained immediately before the first RAW image data, and may be any data obtained at a time before the first time.

  In addition, when the interval from the last frame of the live view display to the still image shooting is short, the image data obtained by applying the low pass filter processing to the last frame image of the live view display and the image obtained by the still image shooting Composite image data can also be generated by combining the data. As a result, even during still image shooting, the time from shooting to generation of composite image data can be shortened.

  Here, a plurality of special effect processing is performed on the live view image data to generate a plurality of special effect image data corresponding to each special effect processing, and the generated plurality of special effect image data is time-sequentially. When automatically switching and displaying on the LCD 114, a technique for generating and displaying the composite image data of the special effect image data to be switched to display a plurality of special effect image data so as to gradually switch is conceivable. . Even in such a display technique, the first special effect process is performed on the first RAW image data obtained at the first time to obtain the first image data, and before the first time. The second special effect processing is performed on the second RAW image data obtained at the second time to obtain the second image data, and the first image data and the second image data are synthesized. Thus, the composite image data can be generated and displayed. Thereby, when display / photographing and image processing can be performed in parallel, the processing time can be shortened.

DESCRIPTION OF SYMBOLS 1 ... Camera body 2 ... Interchangeable lens 101 ... Mechanical shutter 102 ... Imaging element 103 ... Analog processing part 104 ... Analog / digital conversion part 106 ... SDRAM
107 ... Image processing unit 108 ... AE processing unit 109 ... AF processing unit 110 ... JPEG processing unit 112 ... Recording medium 114 ... LCD
DESCRIPTION OF SYMBOLS 115 ... Microcomputer 116 ... Operation part 117 ... Flash memory 118 ... Low pass filter part 119 ... Image composition part 1010 ... Lens 1012 ... Microcomputer 1071 ... OB subtraction part 1072 ... White balance correction part 1073 ... Synchronization processing part 1074 ... Gamma Color reproduction processing unit 1075 ... Color matrix calculation unit 1076 ... Edge enhancement processing unit 1077 ... Noise reduction processing unit

Claims (7)

An imaging unit for imaging a subject to obtain RAW image data;
The RAW image data is continuously input, and first image data is obtained by performing first image processing on the first RAW image data obtained at least at the first time, and the first image data is obtained. An image processing unit that obtains the second image data by performing the second image processing on the second RAW image data obtained at the second time before the time;
An image combining unit that combines the first image data and the second image data to generate combined image data;
A display unit for displaying the composite image data;
An imaging apparatus comprising: A recording unit for recording the composite image data;
The imaging apparatus according to claim 1, further comprising:   The second image processing is to perform third image processing after the first image processing, and the second image data is obtained by performing the third image processing on the first image data. The imaging apparatus according to claim 1, wherein the imaging apparatus is a device.   The image processing unit performs at least one of the third image processing by at least one of display processing by the display unit and imaging processing by the imaging unit at a time later than the first time. The imaging apparatus according to claim 3, wherein the imaging apparatus is performed in parallel.   The second image data is RAW image data obtained immediately before the first RAW image data among continuously input RAW image data. The imaging device according to any one of the above. Inputting RAW image data obtained by imaging a subject;
The RAW image data is continuously input, and first image data is obtained by performing first image processing on the first RAW image data obtained at least at the first time, and the first image data is obtained. Obtaining second image data by performing second image processing on the second RAW image data obtained at a second time before the time;
Combining the first image data and the second image data to generate combined image data;
An image processing method comprising: Inputting RAW image data obtained by imaging a subject;
The RAW image data is continuously input, and first image data is obtained by performing first image processing on the first RAW image data obtained at least at the first time, and the first image data is obtained. Obtaining second image data by performing second image processing on the second RAW image data obtained at a second time before the time;
Combining the first image data and the second image data to generate combined image data;
An image processing program for causing a computer to execute.

Images