JP2005323411A - Image synthesizing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image synthesizing apparatus in which an image equal to multiple exposure of a silver salt camera can be obtained through extremely simple operation by setting an image pickup device in a consecutive photographing state of continuously capturing an image in a fixed exposure cycle, and defining only an image in which a pixel equal to or higher than a fixed level is present in a captured image, as an image to be synthesized. <P>SOLUTION: During an automatic multi-exposure mode, an image captured by a CCD 1 is sent to an image memory 4 via a YUV processor 3. At the same time, an image captured for through-image display is also sent to a video encoder 6. A CPU 8 determines whether or not a pixel equal to or higher than a determination level is present in the captured image and if even one such pixel is present, the captured image is defined as a target of image synthesis. The image that becomes a target of synthesis since the pixel equal to or higher than the determination level is present, is sequentially synthesized on the image memory 4. When the automatic multi-exposure mode is ended, synthesized image data are written and stored on a recording medium 9. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image synthesizing apparatus that obtains a similar effect by synthesizing an image obtained by continuously taking an effect of multiple exposure photographing, which is a photographing method of a silver salt camera, at a constant exposure period in an electronic imaging apparatus.

  As a conventional silver halide camera photographing method, there is a method called multiple exposure in which a plurality of images are obtained by exposing a plurality of times on the same film. To perform multiple exposure, set the valve (long exposure because the shutter is open) with the front of the lens covered with a black curtain, and then perform exposure only while the black curtain in front of the lens is removed. After exposure, cover the front of the lens again with a black screen. Multiple exposure images can be obtained by repeating this operation a plurality of times.

  A digital camera is known as an electronic imaging device. This digital camera can set a recording mode and a reproduction mode, and in the recording mode state, an image pickup device such as a CCD arranged behind the lens outputs a photoelectric conversion output for one screen every fixed period. The photoelectric conversion output is sampled and held by a sample and hold circuit, converted to digital data by an A / D converter, color processed by a color process circuit, and digital luminance and color difference multiplexed signal (YUV data) are converted into DMA ( Direct Memory Access) is output to the controller. The DMA controller once writes the YUV data output of the color process circuit into the buffer inside the DMA controller using the synchronization signal, memory write enable, and clock output of the color process circuit, and DMA-transfers it to the DRAM via the DRAM I / F.

  After completing the DMA transfer of the YUV data to the DRAM, the CPU reads this YUV data from the DRAM via the DRAM I / F and writes it to the VRAM via the VRAM controller. A digital video encoder (hereinafter simply referred to as a video encoder) periodically reads the YUV data from the VRAM via the VRAM controller, generates a video signal based on the data, and outputs the video signal to the display device. As a result, an image based on the image information captured from the current CCD is displayed on the display device.

  In this way, when the shutter key is operated at the timing at which recording and storage are desired while the image is currently displayed on the display device, a signal is generated. Then, the CPU immediately stops the path from the CCD to the DRAM after the DMA transfer of the YUV data for one screen captured from the current CCD to the DRAM in response to this signal, and shifts to the record storage state. In this recording and storage state, the YUV data for one frame written in the DRAM by the CPU is called a basic block of 8 pixels vertically by 8 pixels horizontally for each component of Y, Cb, and Cr via the DRAM I / F. The data is read in units and written to the JPEG circuit, and the code data compressed by the JPEG circuit is read from the JPEG circuit and written to the flash memory, which is a nonvolatile memory. Then, along with the compression processing of one frame of YUV data and the completion of writing all the compressed data to the flash memory, the CPU activates the path from the CCD to the DRAM again.

  In the playback mode, the CPU stops the path from the CCD to the DRAM, and when the image selection key or the like is pressed, the CPU reads a specific frame of code data from the flash memory, writes it to the JPEG circuit, and decompresses it with the JPEG circuit. One frame of YUV data is developed into VRAM via the VRAM controller in units of 8 × 8 basic blocks obtained by processing. Then, the video encoder generates a video signal based on the YUV data for one frame developed in the VRAM and displays it on the display device.

  Since the digital camera as described above can handle captured images as image data, an effect like multiple exposure has been realized by combining all of a plurality of images obtained by a plurality of shootings.

  However, in order to perform multiple exposure in such a conventional silver salt camera, the front of the lens is covered with a black curtain or the like (the shutter is in an open state because the shutter is open), and the front of the lens is set. Repeating the operation to remove or cover the black curtain multiple times is an extremely time-consuming task, such as obtaining multiple exposure images. In digital cameras, the shutter is a valve (like the silver salt camera, because the shutter is open). It was difficult to set the time exposure state) to shoot and to superimpose an image on the CCD because of the discharge of electric charges. In addition, the operation of combining all the captured images after shooting a plurality of times is very complicated and troublesome.

  The present invention has been made in view of such a conventional problem, and is an image in which the imaging device is in a continuous shooting state in which an image is continuously taken at a constant exposure cycle, and a pixel having a certain level or more is present in the captured image. Therefore, it is an object of the present invention to provide an image composition apparatus capable of obtaining an image equivalent to the multiple exposure of a silver salt camera by a very simple operation.

  In order to solve the above-described problems, the invention according to claim 1 is an imaging apparatus including an imaging unit that images a subject and a recording unit that records a subject image generated by the imaging unit. A generation unit that generates image data of the subject image based on a signal generated by the imaging unit, a detection unit that detects a level of each pixel of the image data generated by the generation unit, and a predetermined level by the detection unit It is characterized in that there is provided discrimination means for discriminating the image data in which the above pixels are detected as an image to be synthesized, and synthesis means for sequentially synthesizing the image data discriminated by the discrimination means.

  As described above, according to the first aspect of the present invention, the imaging apparatus is kept in a continuous shooting state in which an image is continuously taken at a constant exposure cycle, the level of each pixel in the captured image is detected, and a pixel above a certain level is detected. Since only the images in which the image is present are discriminated and sequentially combined, it is possible to obtain an image equivalent to the multiple exposure of the silver halide camera by a very simple operation.

  The invention according to claim 2 is characterized in that the composition means composes an image by adding the levels of the corresponding pixels.

  Thus, according to the second aspect of the present invention, it is possible to obtain a composite image by performing simple level addition on the level of each pixel.

  According to a third aspect of the present invention, the composition means compares the levels of the corresponding pixels and selects the larger one to compose an image.

As described above, according to the third aspect of the present invention, as a result of comparing the levels for each pixel, it is possible to obtain a composite image by selecting the larger pixel level.
According to a fourth aspect of the present invention, the detection means includes a determination means for determining the image data detected when the average value of the levels of all detected pixels is equal to or higher than a predetermined level as an image to be combined. It is said.

  Thus, according to the fourth aspect of the present invention, when the average value of the levels of all the pixels detected by the detecting means is equal to or higher than a predetermined level, the image can be determined as an image to be synthesized.

  The invention according to claim 5 is characterized in that the detection means includes a discrimination means for discriminating the detected image data as an image to be combined when a predetermined number or more of pixels are detected. It is said.

  Thus, according to the fifth aspect of the present invention, when the number of pixels detected by the detecting means is not less than a predetermined level and not less than a predetermined number, the image can be determined as an image to be synthesized.

  According to a sixth aspect of the present invention, when the level of each corresponding pixel is equal to or lower than a predetermined level in the synthesizing unit, the image is synthesized with an average value of the pixels, and the level of each corresponding pixel is When either one is equal to or higher than a predetermined level, the level of each corresponding pixel is added to synthesize an image.

  Thus, according to the sixth aspect of the present invention, when the level for each pixel is equal to or lower than the predetermined level, the image is synthesized with the average value of each pixel, and either one of the levels for each corresponding pixel is combined. Since the image is synthesized by adding the level of each corresponding pixel when the level is equal to or higher than the predetermined level, it is possible to prevent the dark portion of the image from becoming bright due to repeated image synthesis, and a high contrast composite image can be obtained. Can be obtained.

  According to a seventh aspect of the present invention, when the level of each corresponding pixel is equal to or lower than a predetermined level in the synthesizing unit, the smaller one is selected to synthesize an image, and the level of each corresponding pixel is selected. When one of the levels is equal to or higher than a predetermined level, the image is synthesized by adding the levels of the corresponding pixels.

  Thus, according to the seventh aspect of the present invention, when both the levels for each pixel are equal to or lower than the predetermined level, the smaller one is selected and the image is synthesized, and one of the levels for the corresponding pixels is equal to or higher than the predetermined level. In the case of, the image is synthesized by adding the levels for each corresponding pixel, so repeated image composition can prevent the dark part of the image from becoming brighter and it is possible to obtain a composite image with high contrast It becomes.

  Further, in the invention according to claim 8, the image data generated in the generation means is composed of RGB components, and in the detection means, each pixel level detected is a value obtained by adding RGB values, The synthesizing means adds RGB for each corresponding pixel for each component to synthesize an image.

  Thus, according to the eighth aspect of the present invention, even for a color image in which each pixel is composed of three types of RGB, each pixel level obtained by adding RGB values is detected, and RGB for each pixel is obtained. The components can be added for each component to synthesize an image, and an image equivalent to the multiple exposure shooting of the silver salt camera can be obtained.

  As described above, according to the first aspect of the present invention, the image pickup apparatus keeps taking an image with a constant exposure cycle, detects the level of each pixel in the captured image, and there is a pixel that exceeds the predetermined level. Since only the images are discriminated and sequentially combined, an image equivalent to the multiple exposure shooting of the silver halide camera can be easily obtained, and shooting of fireworks, celestial bodies (meteors) and the like can be performed.

  According to the second aspect of the present invention, the image can be synthesized by simply adding the levels for each pixel of the image to be synthesized, which is discriminated by the discriminating means.

  According to the third aspect of the present invention, the level of each pixel of the synthesis target image determined by the determination unit is compared, and as a result of the comparison, the larger pixel level can be selected to combine the images.

  According to the fourth aspect of the present invention, when the average value of the levels of all the pixels of the image data detected by the detecting means is equal to or higher than a predetermined level, the image can be determined as an image to be synthesized.

  According to the fifth aspect of the present invention, when the level of the pixel of the image data detected by the detecting means is a predetermined level or more and a predetermined number or more, the image can be determined as a synthesis target image.

  According to the sixth aspect of the present invention, when the level of each pixel of the synthesis target image determined by the determination unit is not more than a predetermined level, the image is synthesized with the average value of the levels of the pixels. If either of them is above the specified level, the image can be synthesized by simple addition, so that it is possible to prevent the dark part of the image from becoming bright due to repeated image synthesis, and to obtain a composite image with high contrast. Can do.

  According to the seventh aspect of the present invention, when the level of each pixel of the synthesis target image determined by the determination unit is both equal to or lower than the predetermined level, the level of the smaller pixel is selected to synthesize the image, When either one is above the predetermined level, the image can be synthesized by simple addition, so that it is possible to prevent the dark part of the image from becoming bright due to repeated image synthesis, and a composite image with high contrast is obtained. be able to.

  According to the eighth aspect of the invention, even in a color image in which each pixel is composed of three types of RGB, each pixel level obtained by adding RGB values is detected, and RGB for each pixel is determined for each component. Since the images are added and synthesized, an image equivalent to the multiple exposure shooting of the silver halide camera can be easily obtained.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of the apparatus. In the recording mode, an image captured by the CCD 1 is sent to the YUV processor 3 by the CCD controller 2 that controls the CCD. Image data that has undergone color process processing by the YUV processor 3 is sent to the image memory 4 via the memory controller 5. At the same time, the captured image data for displaying the through image is also sent to the video encoder 6, which generates a video signal based on the image data sent from the memory controller 5 and outputs it to the video output unit 7. . The key processing unit 10 processes key input.

  Here, when the automatic multiple exposure mode is set, the CPU 8 determines whether or not there is a pixel of a determination level or higher in the captured image, and if there is even one such pixel, the captured image. Is the target of image composition. As described above, since the pixels having the determination level or higher exist, the images to be combined are sequentially combined on the image memory 4. When the automatic multiple exposure mode ends, the combined image data is written and stored in the recording medium 9. In this way, since the image synthesis is performed by discriminating pixels above a certain level from the captured image, a synthesized image can be easily obtained.

  FIG. 2 shows the timing of exposure of the CCD 1 of this apparatus, transfer of the captured image, and discrimination of the synthesis target image. The exposure of the CCD 1 and the image transfer to the image memory 4 are performed with the frame pulse F output every frame period as a reference. In this embodiment, it is set to obtain one image with an exposure time of 4 frames. When the exposure for four frame times is completed, the image is immediately transferred from the CCD 1 to the image memory 4. Of the images transferred every four frames, an image including a pixel having an image combination determination level or higher is an object to be combined. Instead of synthesizing all images, only the necessary images are selected and synthesized, so that efficient image synthesis can be performed without complications. Further, although the exposure time is fixed to a 4-frame cycle, the exposure time is not limited to 4 frames.

  FIG. 3 shows a method for determining an image to be synthesized as to whether or not there is a pixel of a certain level or higher in the image captured from the CCD 1. For the sake of clarity, each pixel is assumed to have monochrome levels of 0 to 255 (8 bits). The level of each pixel corresponds to the luminance or shading of the pixel. The captured image 11 is composed of 4 to 13 level pixels. The determination level is set to 100, and the image 11 is an image that is not an object of image synthesis because there are no pixels of level 100 or higher. The captured image 12 is composed of pixels of levels 2 to 156. The image 12 has five pixels of level 100 or higher. Accordingly, since there is one or more pixels of level 100 or higher, the image 12 is a target for image synthesis.

  As another determination method, the determination may be made based on the average value of the levels of all the pixels in the image, or may be determined by the number of pixels such as how many pixels of a certain level or more exist in the image. In addition, as described above, FIG. 3 illustrates an example of a monochrome image (when each pixel has one level). However, a similar image is applied to a color image in which each pixel is composed of three types of RGB. The synthesis method can be used. In this case, each pixel level for discriminating the image to be synthesized uses a value obtained by adding three values of RGB for each pixel, and in composition of the image, each RGB value is set for two corresponding pixels. What is necessary is just to add separately and to obtain | require the RGB value after a synthesis | combination.

  FIG. 4 shows an image composition method. A method of obtaining a composite image by simple addition of levels will be described. The image 13 is an image that is an image synthesis target and is stored in the image memory 4 in advance, and the image 14 is a new image synthesis target image. Pixels corresponding to images 13 and 14 are added in order to obtain a composite image. For example, in the upper left pixel, image 13 has level 11 and the corresponding upper left pixel of image 14 has level 8. When these are added, it becomes level 19 as in the upper left pixel of the image 15. Similarly, looking at the pixel on the right side, the level is 9 in the image 13 and the level 7 in the image 14, so the level is 16 as shown in the image 15. Similarly, when the process is repeated for each pixel, an image 15 synthesized from the image 13 and the image 14 is obtained.

  Although the pixel level does not exist in FIG. 4 and the pixel level becomes 255 or more as a result of the addition, the pixel of the image 13 is level 181 and the pixel level of the image 14 is 177, and the addition result is 258. In this case, the portion 3 which is 255 or more of the added result is trimmed to 255. As described above, by determining the image to be combined, it is possible to perform image combining by simple pixel level addition. Further, as a method of combining images, the levels of the corresponding pixels in the two images to be combined may be compared and the level of the larger pixel may be selected.

  FIG. 5 is a flowchart showing the processing operation of the CPU. First, the CPU counts the frame counter to determine whether or not four frames of exposure time have elapsed (S1). If four frames have passed, the image data from the CCD 1 is transferred to the image memory 4 (S2). The image data fetched into the image memory 4 is sent to the video encoder 6 for through image display (S3). Next, as described with reference to FIG. 3, it is determined whether or not there is a pixel of a determination level or higher in the captured image (S4). As a result, if even one pixel above the judgment level exists, that image is set as an image to be combined, and is added for each pixel as described in FIG. (S5). As described above, images are taken continuously at a constant exposure cycle, and only images with pixels that are above the discrimination level in the captured image are targeted for synthesis. Equivalent images can be obtained. Especially effective for shooting fireworks and celestial bodies (meteors).

  FIG. 6 shows a method for obtaining a composite image with high contrast. Since the simple addition of levels, which is the synthesis method shown in FIG. 4, is performed even for the level of a low-level pixel, if the image synthesis is repeated many times, the dark part of the image (the part with the low value of the pixel level) ) Will gradually become brighter. Even if the pixel level is low, a constant reference level is set, and if the levels of the corresponding pixels in the two images to be combined are both below a certain level, simple addition is not performed, but the average value of the levels of both pixels How to take.

  A constant reference level 16 is assumed. The level of the upper left pixel of the image 16 is 11, and the level of the upper left pixel of the corresponding image 17 is 8. Here, since both pixels are below the reference level 16, (11 + 8) /2=9.5, and the fractional part is rounded down, the average value of both pixels is 9, and the level of the pixel of the synthesized image 18 is 9. . Similarly, since both pixels on the right side are also below the reference level 16, (9 + 7) / 2 = 8, the average value of both pixels is 8, and the level of the pixel of the synthesized image 18 is 8. If at least one of the two pixels is at the reference level 16 or higher, simple addition is performed as shown in FIG. In the pixel in the second row from the top of the image 16 and in the third column from the left, the pixel level is 167 and the corresponding pixel level of the image 17 is 5. Here, since one pixel is above the reference level 16, simple addition is performed, and the level of the pixel of the image 18 synthesized by 167 + 5 = 172 becomes 172.

  By changing the compositing method by providing a fixed reference level for the low pixel level in this way, it is possible to prevent the dark part of the image from becoming bright due to repeated image compositing, and an image with high contrast can be obtained. Can be obtained. In the above embodiment, the average value of the levels of both pixels is used if the levels of the corresponding pixels in the two images to be combined are both below a certain level. However, the levels of the corresponding pixels are both constant. In the following cases, a method of selecting a level of a pixel having a smaller level may be used.

It is a block diagram of this invention apparatus. It is a figure which shows the timing of an automatic multiple exposure process. It is a figure which shows the discrimination method of the image of a synthetic | combination object. It is a figure which shows the synthetic | combination method of an image. It is the flow which showed the process of CPU. It is a figure which shows the synthetic | combination method which obtains an image with high contrast.

Explanation of symbols

1 CCD
2 CCD control unit 3 YUV processor 4 Image memory 5 Memory controller 6 Video encoder 7 Video output unit 8 CPU
9 Recording medium 10 Key processing part

Claims (8)

In an imaging apparatus having an imaging means for imaging a subject and a recording means for recording a subject image generated by the imaging means, the image data of the subject image is exposed based on a signal generated by the imaging means after exposure at regular intervals. Generating means for generating
Detecting means for detecting the level of each pixel of the image data generated by the generating means;
Discriminating means for discriminating as an image to be combined with image data in which pixels of a predetermined level or more are detected by the detecting means;
An image synthesizing apparatus comprising: synthesizing means for sequentially synthesizing the image data determined by the determining means.   2. The image synthesizing apparatus according to claim 1, wherein said synthesizing means synthesizes an image by adding the levels for each corresponding pixel.   2. The image synthesizing apparatus according to claim 1, wherein the synthesizing unit compares the levels of the corresponding pixels and selects the larger one to synthesize an image.   2. The image synthesizing apparatus according to claim 1, further comprising: a discriminating unit that discriminates the detected image data as an image to be synthesized when an average value of all detected pixel levels is equal to or higher than a predetermined level. .   2. The image synthesizing apparatus according to claim 1, further comprising a discriminating unit that discriminates the detected image data as an image to be synthesized when a predetermined number or more of pixels are detected in the detecting unit. .   In the combining means, when the levels of the corresponding pixels are both equal to or lower than the predetermined level, the image is combined with the average value of the pixels, and when one of the levels of the corresponding pixels is equal to or higher than the predetermined level. 2. The image synthesizing apparatus according to claim 1, wherein the image is synthesized by adding the levels of corresponding pixels.   In the combining means, if the level of each corresponding pixel is both equal to or lower than the predetermined level, the smaller one is selected and the image is combined, and if one of the levels for each corresponding pixel is equal to or higher than the predetermined level, it corresponds. The image synthesizing apparatus according to claim 1, wherein the image is synthesized by adding levels for each pixel.   In the generating means, the generated image data is composed of RGB components, and in the detecting means, each pixel level to be detected is a value obtained by adding each RGB value. In the synthesizing means, for each corresponding pixel The image synthesizing apparatus according to claim 1, wherein RGB are added for each component to synthesize an image.

Images