JP4989524B2 - Electronic camera - Google Patents

Description

  The present invention relates to an electronic camera that creates a composite image from which unnecessary moving objects are eliminated by combining a plurality of object scene images.

  The present invention also relates to an electronic camera that creates a moving image from which fixed pattern noise has been removed.

  An example of a camera that creates a composite image from which unnecessary moving objects are excluded is disclosed in Patent Document 1. According to Patent Document 1, a person with a landscape background is photographed continuously three times at predetermined time intervals. In the image composition, a matching part is extracted from two frames out of three taken frames. As a result, even if a third person who happens to pass through the scene is removed, the image of the moving third party is removed, and a composite image in which only the user and a desired landscape are photographed is obtained.

An example of an electronic camera that creates a moving image from which fixed pattern noise has been removed is disclosed in Patent Document 2. According to Patent Document 2, a random number signal is added to noise that appears fixedly in an imaging signal. The noise disturbed by this is removed using adjacent pixel correlation or frame correlation.
Japanese Patent Laid-Open No. 2005-12660 JP 2004-289241 A

  However, in Patent Document 1, since the shooting time interval is fixed, noise caused by the third party image may appear in the composite image depending on the moving speed of the third party in motion. Further, in Patent Document 2, since it is necessary to disturb fixed pattern noise before using adjacent pixel correlation or frame correlation, the circuit configuration may be complicated.

  Therefore, a main object of the present invention is to provide an electronic camera capable of improving the quality of a composite image from which moving objects are excluded.

  Another object of the present invention is to provide an electronic camera capable of easily creating a moving image from which fixed pattern noise has been removed.

An electronic camera according to the present invention (10: reference numeral corresponding to the embodiment; hereinafter the same) has an imaging surface on which an optical image of the object scene is irradiated, and imaging means for periodically generating the object scene image ( 16) combining means (36) for combining a plurality of object scene images generated for each specified period by the image pickup means, and a plurality of objects to be watched by the combining means for the movement of the imaging surface in the direction orthogonal to the optical axis. Detection means (S23, S27, S65) for detecting corresponding to each of the field images, and adjustment means (S75) for adjusting the image synthesis position by the synthesis means with reference to the motion detected by the detection means , means selects a plurality of pixels that are common to a plurality of object scene image, you and outputting a pixel having a predetermined th magnitude of the pixel values of the plurality of pixels.

The imaging means has an imaging surface on which an optical image of the object scene is irradiated, and periodically generates the object scene image. The synthesizing unit synthesizes a plurality of object scene images generated for each designated period by the imaging unit. In addition, a changing unit that changes the length of the designated cycle in response to the change instruction is provided, and the length of the designated cycle is changed by the changing unit in response to the change instruction. The movement of the imaging surface in the direction orthogonal to the optical axis is detected by the detecting unit corresponding to each of the plurality of object scene images noted by the combining unit. The adjustment means adjusts the image composition position by the composition means with reference to the motion detected by the detection means.

  By changing the designated period in response to the change instruction, the designated period can be extended as the movement of the moving object is slower, and thus the moving object can be reliably excluded from the synthesized image. Further, by adjusting the image composition position in consideration of the shake of the imaging surface in the direction orthogonal to the optical axis, it is possible to reduce the influence of the shake of the imaging surface due to the extension of the designated period on the image composition. As a result, it is possible to improve the quality of the synthesized image from which moving objects are excluded.

  Preferably, the synthesizing unit extracts the partial scene image belonging to the designated area from each of the plurality of scene images, and synthesizes the plurality of partial scene images extracted by the extracting unit. And a partial scene image synthesizing unit (100 to 112) that adjusts the position of the designated area so that the movement of the imaging surface is compensated.

More preferably, fraction containing the output means comprises a median filter or rank filter.

  Preferably, output means (40, 42) for outputting a moving image based on the object scene image generated by the image pickup means, and the composition means is activated after a plurality of object scene images noted by the composition means are generated The starting means (S67) is further provided.

  An imaging control program according to the present invention has an imaging surface on which an optical image of an object scene is irradiated, and is generated at every specified period by an imaging means (16) that periodically generates an object scene image, and the imaging means. A change step (S35) for changing the length of the specified period in response to the change instruction to the processor (28) of the electronic camera (10) provided with the combining means (36) for combining a plurality of object scene images; Detection step (S23, S27, S65) for detecting the movement of the imaging surface in the direction perpendicular to the axis corresponding to each of the plurality of object scene images noted by the combining means, and detecting the image combining position by the combining means It is an imaging control program for executing an adjustment step (S75) in which adjustment is performed with reference to the motion detected in the step.

  The imaging control method according to the present invention has an imaging surface on which an optical image of the object scene is irradiated, an imaging means (16) that periodically generates the object scene image, and an imaging means that is generated for each specified period. An imaging control method executed by an electronic camera (10) provided with a synthesizing means (36) for synthesizing a plurality of object scene images, wherein a change step of changing the length of a specified cycle in response to a change instruction S35), a detection step (S23, S27, S65) for detecting the movement of the imaging surface in a direction orthogonal to the optical axis corresponding to each of a plurality of object scene images noted by the combining means, and an image by the combining means An adjustment step (S75) for adjusting the synthesized position with reference to the movement detected by the detection step is provided.

  An electronic camera (10) according to another invention has an imaging surface on which an optical image of an object scene is irradiated, an image pickup means (16) for generating an object scene image at a specified period, and a plurality of images generated by the image pickup means Selection means (S13) for selecting N (N: integer greater than or equal to 2) object scene images from among the object scene images, and changing means (S15) for selecting the selection mode of the selection means at a specified cycle, selection By combining means (36) for combining N object scene images selected by the means, output means (40, 42) for outputting a moving image based on the object scene image created by the combining means, and combining means Adjustment means (S27, S29) for adjusting the image composition position with reference to the movement of the imaging surface in the direction orthogonal to the optical axis is provided.

  The imaging means has an imaging surface on which an optical image of the object scene is irradiated, and generates the object scene image at a specified period. The selection unit selects N (N: integer greater than or equal to 2) object scene images from the plurality of object scene images generated by the imaging unit. The selection mode of the selection unit is changed by the changing unit at a designated cycle. The synthesizing unit synthesizes the N object scene images selected by the selecting unit. A moving image based on the object scene image created by the synthesizing unit is output by the output unit. The adjusting unit adjusts the image combining position by the combining unit with reference to the movement of the imaging surface in the direction orthogonal to the optical axis.

  As described above, the imaging unit generates a scene image at a specified cycle, and the selection unit changes the selection mode of the N scene images at the specified cycle. The object scene image forming the moving image is obtained by combining the N object scene images thus selected. Here, the combined position of the N object scene images is adjusted with reference to the movement of the imaging surface in the direction orthogonal to the optical axis. For this reason, when camera shake occurs on the imaging surface, the relative position of the fixed pattern noise differs among the N object scene images. As a result, the fixed pattern noise can be removed during image synthesis (that is, simply).

  Preferably, the synthesizing means extracts an extraction means (96) for extracting a partial scene image belonging to the designated area from each of the N object scene images, and the N partial scene images extracted by the extraction means. And a partial field image composition unit (100 to 112) for synthesizing the image.

  More preferably, the partial scene image synthesizing unit includes a pixel selection unit (102 to 110) that selects a plurality of pixels common to the N partial scene images, and a plurality of pixels selected by the pixel selection unit. The pixel output means (112) for outputting one pixel based on the above is included. More preferably, the pixel output means outputs a pixel having a pixel value of a predetermined size among the plurality of pixels.

  According to the present invention, by changing the designated cycle in response to the change instruction, the designated cycle can be extended as the moving body moves slower, and thereby the moving body can be reliably excluded from the synthesized image. Further, by adjusting the image composition position in consideration of the shake of the imaging surface in the direction orthogonal to the optical axis, it is possible to reduce the influence of the shake of the imaging surface due to the extension of the designated period on the image composition. As a result, it is possible to improve the quality of the synthesized image from which moving objects are excluded.

  According to another invention, the imaging unit generates a scene image at a specified cycle, and the selection unit changes a selection mode of N field images at a specified cycle. The object scene image forming the moving image is obtained by combining the N object scene images thus selected. Here, the combined position of the N object scene images is adjusted with reference to the movement of the imaging surface in the direction orthogonal to the optical axis. For this reason, when camera shake occurs on the imaging surface, the relative position of the fixed pattern noise differs among the N object scene images. As a result, the fixed pattern noise can be removed during image synthesis (that is, simply).

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a digital camera 10 of this embodiment includes an optical lens 12 and an aperture unit 14. The optical image of the object scene is irradiated on the imaging surface of the CMOS image sensor 16 through these members. The imaging surface is covered by a primary color Bayer array color filter (not shown). Therefore, in each pixel, a charge having color information of any one of R (Red), G (Green), and B (Blue) is generated by photoelectric conversion.

  When the power is turned on, the CPU 28 activates the driver 18 to execute through image processing. In response to the vertical synchronization signal Vsync generated every 1/60 seconds, the driver 18 exposes the imaging surface and reads out the charges generated on the imaging surface in a raster scanning manner. From the image sensor 16, raw image data representing the object scene is output at a frame rate of 60 fps.

  The preprocessing circuit 20 performs processing such as digital clamping, pixel defect correction, gain control, and median filter on the raw image data from the image sensor 16. The raw image data generated thereby is written into the raw image area 34a (see FIG. 2) of the SDRAM 34 through the memory control circuit 32.

  The median filter is a process of detecting the median level of the target pixel and the same color pixel existing around the target pixel and setting the detected median level to the level of the target pixel. Further, the median filter of the preprocessing circuit 20 focuses on peripheral pixels in the planar direction.

  Therefore, when the target pixel is the R pixel at the center of the matrix shown in FIG. 3A, the median value of the nine pixels indicated by hatching in the same FIG. 3A is the level of the target pixel. Further, when the target pixel is the G pixel at the center of the matrix shown in FIG. 3B, the median value of the level of 13 pixels indicated by hatching in FIG. 3B is set as the level of the target pixel. Further, when the target pixel is the central B pixel of the matrix shown in FIG. 3C, the median value of the nine pixels indicated by hatching in FIG. 3C is set as the target pixel level.

  The raw image area 34a shown in FIG. 2 is formed by six banks. The CPU 28 cyclically changes the write-destination bank every time the vertical synchronization signal Vsync is generated. The write destination bank is specified in the order of bank 1 → bank 2 → bank 3 → bank 4 → bank 5 → bank 6 → bank 1 →. The raw image data output from the preprocessing circuit 20 is written in the bank thus specified.

  The preprocessing circuit 20 executes a simple Y generation process and a simple RGB generation process in addition to the above-described processes. The raw image data is converted into Y data by a simple Y conversion process, and converted into RGB data (data in which each pixel has all color information of R, G, and B) by a simple RGB conversion process. The Y data generated by the simple Y conversion process is supplied to the motion detection circuit 22 and the AF evaluation circuit 26, and the RGB data generated by the simple RGB conversion process is supplied to the AE / AWB evaluation circuit 24.

  Referring to FIG. 5, nine motion detection areas MD1 to MD9 and one extraction area EX are allocated to the imaging surface. The motion detection areas MD1 to MD3 are arranged horizontally in the upper stage of the imaging surface, the motion detection areas MD4 to MD6 are arranged horizontally in the interruption of the imaging surface, and the motion detection areas MD7 to MD9 are arranged horizontally in the lower stage of the imaging surface. .

  The motion detection circuit 22 detects a partial motion vector representing the motion of the object scene in each of the motion detection areas MD1 to MD9 based on Y data given from the preprocessing circuit 20 every 1/60 seconds. The motion detection circuit 22 further synthesizes the detected partial motion vectors to create an overall motion vector. The entire motion vector is output from the motion detection circuit 22 every 1/60 seconds.

  The CPU 28 determines whether the movement of the imaging surface in the direction orthogonal to the optical axis is caused by hand movement or pan / tilt operation based on the entire motion vector output from the motion detection circuit 22, and the movement of the imaging surface is determined. When it is caused by camera shake, the extraction area EX is moved along the entire motion vector. The position of the extraction area EX is changed so that the movement of the imaging surface due to camera shake is compensated (cancelled).

  Thereafter, the CPU 28 sets the current position of the extraction area EX in the register RGST1 provided in the filter circuit 36. As shown in FIG. 6, the register RGST1 has six columns corresponding to the banks 1 to 6 on the raw image area 34a. The position of the extraction area EX is set to a column corresponding to the current write destination bank among the six columns.

  After the five frames of raw image data are secured in the raw image area 34a, the CPU 28 cyclically changes the read start bank each time the vertical synchronization signal Vsync is generated. The read start bank is also designated in the order of bank 1 → bank 2 → bank 3 → bank 4 → bank 5 → bank 6 → bank 1 →.

  The filter circuit 36 identifies five columns starting from the column corresponding to the read start bank from the register RGST1, detects five positions from the five identified columns, and extracts five according to the detected five positions. The area EX is allocated to five banks on the raw image area 34a (five banks starting from the read start bank).

  The filter circuit 36 further reads out the five frames of partial raw image data belonging to the five extraction areas in parallel through the memory control circuit 32, and sets the five pixels existing at a common position on the basis of the upper left coordinates of the extraction area EX. Execute the median filter in the time axis direction. The readout process is executed for each designated pixel block, and readout of the partial raw image data of 5 frames is completed within 1/60 second.

  When the five pixel blocks MB1 to MB5 respectively belonging to the five frames of partial raw image data shown in FIG. 4 are read and attention is paid to five of these pixels P1 to P5, the median value of the level of these five pixels is the pixel P3. Level. Thus, one frame of partial raw image data from which noise has been removed is generated every 1/60 seconds. The generated partial raw image data is written into the filter image area 34b (see FIG. 2) of the SDRAM 34 through the memory control circuit 32.

  Therefore, when the fixed pattern noise NS is generated as shown in FIGS. 8A to 8E, and the extraction area EX is moved as shown in FIGS. 8A to 8E, As shown in FIG. 8F, partial raw image data from which the fixed pattern noise NS is removed is obtained in the filter image area 34b.

  The post-processing circuit 38 reads the partial raw image data stored in the filter image area 34b every 1/60 seconds through the memory control circuit 32, and performs color separation, white balance adjustment, and YUV conversion to the read partial raw image data. , Processing such as reduction zoom. As a result, low-resolution image data corresponding to the YUV format is created every 1/60 seconds. The created image data is output from the moving image output terminal M_OUT and is written in the YUV image area 34c (see FIG. 2) of the SDRAM 34 through the memory control circuit 32.

  The LCD driver 40 repeatedly reads the image data stored in the YUV image area 34c, and drives the LCD monitor 42 based on the read image data. As a result, a real-time moving image (through image) representing the scene is displayed on the monitor screen.

  The AE / AWB evaluation circuit 24 integrates each of the RGB data output from the preprocessing circuit 20 every 1/60 seconds. The CPU 28 performs through-image AE / AWB processing (simple AE / AWB processing) in order to calculate an appropriate EV value and an appropriate white balance adjustment gain based on these integrated values output from the AE / AWB evaluation circuit 24. ). The aperture amount and exposure time that define the calculated appropriate EV value are set in the aperture unit 14 and the driver 18, respectively, and the calculated appropriate white balance adjustment gain is set in the post-processing circuit 38. As a result, the brightness and white balance of the moving image output from the LCD monitor 42 are adjusted appropriately.

  When the shutter button 30s on the key input device 30 is half-pressed, a strict recording AE is performed so as to calculate the optimum EV value and the optimum white balance adjustment gain based on the integral value output from the AE / AWB evaluation circuit 24. / AWB processing is executed. The aperture amount and the exposure time that define the calculated optimum EV value are set in the aperture unit 14 and the driver 18, respectively, as described above.

  When the recording AE / AWB process is completed, the AF process based on the output of the AF evaluation circuit 26 is executed. The AF evaluation circuit 26 integrates the high-frequency component of the Y data output from the preprocessing circuit 20 every 1/60 seconds. The CPU 28 refers to the integrated value output from the AF evaluation circuit 26 and searches for the focal point by so-called hill climbing processing. The focus lens 12 is disposed at the found focal point.

  When the shutter button 30s is fully pressed, it is determined whether the current still image capturing mode is the normal mode or the special mode. The CPU 28 executes a normal recording process when the determination result indicates the normal mode, and executes a composite recording process when the determination result indicates the special mode.

  In connection with the normal recording process, the CPU 28 instructs the post-processing circuit 38 to extract one frame of image data, and instructs the I / F 44 to record the extracted image data. The post-processing circuit 38 extracts one frame of image data (format: YUV, resolution: high) at the time when the shutter button 30s is fully pressed, and outputs the extracted image data from the still image output terminal S_OUT. The output image data is written into the still image area 34 d of the SDRAM 34 through the memory control circuit 32. The I / F 44 reads the image data stored in the still image area 34d through the memory control circuit 32, and records the read image data on the recording medium 46 in a file format.

  In relation to the composite recording process, the CPU 28 executes five still image capturing processes at an imaging interval designated by an interval changing operation on the key input device 30. If the imaging interval is set to 2 seconds, the still image capturing process is executed every 2 seconds. If the imaging interval is set to 3 seconds, the still image capturing process is executed every 3 seconds. The five still image capturing processes are executed in parallel with the through image processing. In addition, a total of five frames of raw image data is saved in the work area 34e (see FIG. 2) by five still image capturing processes. The work area 34e has five banks, and the writing destination bank is changed every time the still image capturing process is executed. For this reason, 5 frames of raw image data are stored in order in these five banks.

  The CPU 28 also specifies the position of the extraction area EX at the time when the still image capturing process is performed, and sets the specified position in the register RGST2 shown in FIG. The register RGST2 has five columns respectively corresponding to the banks 1 to 5, and the position of the specified extraction area EX is set in order to these five columns.

  When the setting process to the register RGST2 is completed, the through image process is stopped. Thereafter, the CPU 28 changes the setting of the filter circuit 36 so that the read destination is the work area 34e, the reference register is the register RGST2, and the bank 1 is the read start bank. The CPU 28 also sets the zoom magnification of the post-processing circuit 38 to “1.0”.

  The filter circuit 36 identifies the five columns starting from the column corresponding to the read start bank from the register RGST2, detects the five positions set in the five identified columns, and follows the detected five positions. The five extraction areas EX are allocated to five banks (5 banks starting from the read start bank) on the work area 34e.

  Further, the filter circuit 36 reads out the five frames of partial raw image data belonging to the five extraction areas in parallel through the memory control circuit 32, and executes the above-described median filter in the time axis direction. One frame of partial raw image data from which noise has been removed by the median filter processing is written into the filter image area 34b (see FIG. 2) of the SDRAM 34 through the memory control circuit 32.

  When the raw image data of 5 frames saved in the work area 34e includes unnecessary moving body MO as noise as shown in FIGS. 9A to 9E, the median filter processing in the time axis direction by the filter circuit 36 is performed. Thus, as shown in FIG. 9F, partial image data representing the object scene from which the moving body MO is excluded is obtained.

  The post-processing circuit 38 reads the partial raw image data stored in the filter image area 34b through the memory control circuit 32, and performs a series of processes of color separation, white balance adjustment, and YUV conversion on the read partial raw image data. . As a result, high-resolution image data corresponding to the YUV format is created every 1/60 seconds. The created image data is output from the moving image output terminal M_OUT and is written in the YUV image area 34c (see FIG. 2) of the SDRAM 34 through the memory control circuit 32.

  When the image data is stored in the YUV image area, a recording command is given to the I / F 44. The I / F 44 reads the image data stored in the YUV image area 34c through the memory control circuit 32, and records the read image data on the recording medium 46 in a file format. When the recording process is completed, the zoom magnification of the post-processing circuit 38 is restored and the through image process is resumed.

  The preprocessing circuit 20 is configured as shown in FIG. The raw image data output from the imaging device 16 is subjected to digital clamp processing by the digital clamp circuit 50 and further subjected to pixel defect correction processing by the pixel defect correction circuit 52. The raw image data in which the pixel defect is corrected is amplified by the gain control circuit 54, and the amplified raw image data is given to the median filter 56. The median filter 56 pays attention to the peripheral pixels in the planar direction shown in FIGS. 3A to 3C and removes noise from the raw image data.

  Each of the Y generation circuit 62 and the RGB generation circuit 64 simply generates Y data and RGB data based on the raw image data output from the median filter 56. The raw image data output from the median filter 56 is also temporarily stored in the SRAM 60 and output to the memory control circuit 32 together with a write request issued by the controller 58. The write request is a request for writing to a bank designated as a write destination by the CPU 28. The output raw image data is written into a designated bank of the raw image area 34a by the memory control circuit 32.

  The motion detection circuit 22 is configured as shown in FIG. The Y data is given to the frame memory 66 and the distributor 68. The frame memory 66 is formed by two banks each having a capacity corresponding to one frame, and the supplied Y data is alternately written into the two banks.

  Distributor 68 gives Y data belonging to motion detection areas MD1, MD4 and MD7 to distributor 70, Y data belonging to motion detection areas MD2, MD5 and MD8 to distributor 72, and motion detection areas MD3, MD6. Y data belonging to MD9 is supplied to the distributor 74.

  The distributor 70 provides Y data belonging to the motion detection area MD1 to the partial motion vector detection circuit 76, provides Y data belonging to the motion detection area MD4 to the partial motion vector detection circuit 82, and Y data belonging to the motion detection area MD7. Is supplied to the partial motion vector detection circuit 88. The distributor 72 gives Y data belonging to the motion detection area MD2 to the partial motion vector detection circuit 78, gives Y data belonging to the motion detection area MD5 to the partial motion vector detection circuit 84, and Y data belonging to the motion detection area MD8. Is given to the partial motion vector detection circuit 90. The distributor 74 gives Y data belonging to the motion detection area MD3 to the partial motion vector detection circuit 80, gives Y data belonging to the motion detection area MD6 to the partial motion vector detection circuit 86, and Y data belonging to the motion detection area MD9. Is supplied to the partial motion vector detection circuit 92.

  Each of the partial motion vector detection circuits 76 to 92 compares the Y data supplied from the distributor 70, 72 or 74 with the Y data of the previous frame stored in the frame memory 66, and compares the Y data supplied from the distributor 70, 72 or 74 with the target motion detection area. A partial motion vector representing the motion of the scene is detected. As a result, nine partial motion vectors respectively corresponding to the motion detection areas MD1 to MD9 are obtained. The combined motion vector generation circuit 94 combines the nine partial motion vectors detected by the partial motion vector detection circuits 76 to 92, and generates an entire motion vector that represents the motion of the scene over the entire imaging surface.

  The filter circuit 36 is configured as shown in FIG. When executing the through image processing, the controller 96 sets the raw image area 34a as a reading destination and sets the register RGST1 as a reference register. On the other hand, when combining the five frames of raw image data saved in the work area 34e, the controller 96 sets the work area 34e as a read destination and the register RGST2 as a reference register.

  The controller 96 further issues a read request to the memory control circuit 32 to read in parallel the five frames of partial raw image data belonging to the five extraction areas EX respectively assigned to the five banks starting from the read start bank designated by the CPU 28. Issuing repeatedly toward The partial raw image data of each frame is read for each designated pixel block, and the read partial raw image data is given to the selector 100 via the SRAM 98.

  The selector 100 writes the partial raw image data of the first frame into the SRAM 102, writes the partial raw image data of the second frame into the SRAM 104, writes the partial raw image data of the third frame into the SRAM 106, and the partial raw image of the fourth frame. The data is written into the SRAM 108, and the partial raw image data of the fifth frame is written into the SRAM 110.

  The median filter 112 reads out common pixel data of five pixels from the SRAMs 102 to 110 on the basis of the upper left coordinates of the extraction area EX, and selects one pixel having a median value from the read out five pixels. The partial raw image data formed by the pixels thus selected is temporarily stored in the SRAM 116 and output to the memory control circuit 32 together with a write request issued by the controller 114. The output partial raw image data is written into the filter image area 34 b of the SDRAM 34 by the memory control circuit 32.

  The post-processing circuit 38 is configured as shown in FIG. The controller 118 repeatedly issues a read request to the memory control circuit 32 in order to read the raw image data stored in the filter image area 34 b of the SDRAM 34. The raw image data read in response to this is given to the color separation circuit 122 via the SRAM 120. The color separation circuit 122 generates RGB data in which each pixel has all the R, G, and B color information based on the given raw image data.

  The generated RGB data is subjected to white balance adjustment processing by the white balance adjustment circuit 124 and then converted to YUV format image data by the YUV conversion circuit 126. The converted image data is written into the SRAM 134 through the zoom circuit 128. The controller 132 outputs the image data stored in the SRAM 134 to the memory control circuit 32 together with a write request. The output image data is written into the YUV image area 34 c of the SDRAM 34 by the memory control circuit 32. The zoom magnification of the zoom circuit 128 is set to “1.0” when the five frames of raw image data saved in the work area 34e are combined.

  When the shutter button 30s under the normal mode is fully pressed, one frame of image data output from the YUV conversion circuit 126 at that time is extracted by the still image extraction circuit 130. The extracted image data is temporarily stored in the SRAM 138 and output to the memory control circuit 32 together with a write request issued by the controller 136. The output image data is written into the still image area 34 d of the SDRAM 34 by the memory control circuit 32.

  The CPU 28 executes in parallel a plurality of tasks including a bank control task shown in FIG. 14, a camera shake correction task shown in FIG. 15, a cycle setting task shown in FIG. 16, and an imaging task shown in FIGS. Note that control programs corresponding to these tasks are stored in the flash memory 48.

  Referring to FIG. 14, in step S1, settings of controller 96 provided in filter circuit 36 are initialized. Thus, the raw image area 34a is set as an access destination, and the register RGST1 is set as a reference register. In step S3, variables W and R for identifying the bank are set to “1”. When the vertical synchronization signal Vsync is generated, YES is determined in step S5, and the bank W is set in the controller 58 of the preprocessing circuit 20 as a write destination bank in step S7. When the setting is completed, the variable W is incremented in step S9. However, the variable W circulates between “1” to “5” and is updated to “1” after “5”.

  In step S11, it is determined whether or not the five frame period has elapsed since the activation of the bank control task. If NO, the process directly returns to step S5. If YES, the process goes to steps S13 to S15 to step S5. Return. In step S13, the bank R is set as the reading start bank in the controller 96 of the filter circuit 36, and in step S15, the variable R is incremented. The variable R also circulates between “1” to “5”, and is updated to “1” after “5”.

  Referring to FIG. 15, in step S21, it is determined whether or not the vertical synchronization signal Vsync has been generated. If YES, the entire motion vector is fetched from the motion detection circuit 22 in step S23. In step S25, it is determined based on the entire motion vector whether or not the current movement of the imaging surface is due to the pan / tilt operation. If YES, the process proceeds directly to step S29. The movement of the imaging surface is considered to be caused by camera shake, and the process proceeds to step S29 through the process of step S27.

  In step S27, the extraction area EX is moved along the entire motion vector captured in step S23. In step S29, the current position of the extraction area is set in the column corresponding to the bank W of the register RGST1. When the process of step S29 is completed, the process returns to step S21.

  Referring to FIG. 16, in step S31, an imaging interval referred to under the special mode is set to an initial value. When the interval changing operation is performed, YES is determined in step S33, and the imaging interval is changed to a desired value in step S35.

  Referring to FIG. 17, in step S41, through image processing is started, and at the same time, a bank control task and a camera shake correction task are activated. As a result, a through image is output from the LCD monitor 42. In step S43, it is determined whether or not the shutter button 30s is half-pressed, and as long as the determination result is NO, the through image AE / AWB process in step S45 is repeated. As a result, the brightness and white balance of the through image are appropriately adjusted. When the shutter button 30s is half-pressed, the recording AE / AWB process is executed in step S47, and the AF process is executed in step S49. The brightness and white balance of the through image are adjusted to optimum values by the recording AE / AWB process, and the focus lens 12 is placed at the focal point by the AF process.

  In step S51, it is determined whether or not the shutter button 30s has been fully pressed. In step S53, it is determined whether or not the operation of the shutter button 30s has been released. If “YES” in the step S51, the current still image capturing mode is determined in a step S55. If the current still image capturing mode is the normal mode, “YES” is determined in the step S55, and the normal recording process is executed in a step S57. On the other hand, if the current still image capturing mode is the special mode, NO is determined in step S55, and the composite recording process is executed in step S59. When the process of step S57 or S59 is completed, the process returns to step S43. If YES is determined in the step S53, the process returns to the step S43 as it is.

  The composite recording process in step S59 is executed according to the subroutine shown in FIGS. In step S61, a variable N for identifying a bank is set to “1”. Next, in step S63, still image capturing processing is executed, and the raw image data of the latest frame is saved in the bank N of the work area 34e. In step S65, the current position of the extraction area EX is set in the column corresponding to the bank N of the register RGST2. In step S67, it is determined whether or not the variable N has reached “5”. If “NO” here, the process proceeds to a step S69 to wait for the arrival of the still image capturing timing based on the setting of the imaging interval. When the still image capture timing arrives, the variable N is incremented in step S71, and then the process returns to step S63.

  When the variable N reaches “5”, it is considered that 5 frames of raw image data are secured in the banks 1 to 5 of the work area 34e, and the process proceeds from step S67 to step S73. In step S73, the through image processing is stopped, and the bank control task and the camera shake correction task are also stopped. In step S75, the setting of the controller 96 provided in the filter circuit 36 is changed. As a result, the work area 34e is set as a read destination, the register RGST2 is set as a reference register, and the bank 1 is set as a read start bank. In step S77, the zoom magnification of the zoom circuit 128 provided in the post-processing circuit 38 is set to “1.0”. As a result, one frame of image data based on the five frames of raw image data stored in the work area 34e is obtained in the YUV image area 34c.

  When such image composition processing is completed, YES is determined in step S79, and recording processing is executed in step S81. The created image data is recorded on the recording medium 46 in a file format. When the recording process is completed, the setting of the post-processing circuit 38, that is, the zoom magnification is returned to the original in step S83. In step S85, the through image processing is restarted, and the bank control task and the camera shake correction task are restarted. When the process of step S85 is completed, the process returns to the upper-level routine.

  As can be seen from the above description, the image sensor 12 has an imaging surface on which an optical image of the object scene is irradiated, and generates the object scene image at a specified period (= 1/60 second). The CPU 28 selects an N-frame (N = 5) scene image from a plurality of scene images generated by the image sensor 12 (S13), and selects an N-frame scene image selection mode. Change at specified cycle (S15). The filter circuit 36 synthesizes the N-frame object scene images thus selected. A moving image based on the combined object scene image is output from the LCD monitor 42. The CPU 28 adjusts the image composition position by the filter circuit 36 with reference to the movement of the imaging surface in the direction orthogonal to the optical axis (S27, S29).

  In this way, the image sensor 12 generates a scene image at a specified cycle, and the CPU 28 changes the selection mode of the N frame scene image at the specified cycle. The scene image of each frame forming the moving image is obtained by synthesizing the N scene images thus selected. Here, the composite position of the N frame object scene image is adjusted with reference to the movement of the imaging surface in the direction orthogonal to the optical axis. For this reason, when camera shake occurs on the imaging surface, the relative position of the fixed pattern noise differs among the N-frame object scene images. As a result, the fixed pattern noise can be removed during image synthesis (that is, simply).

  When this embodiment is viewed from another viewpoint, the image sensor 12 has an imaging surface on which an optical image of the object scene is irradiated, and periodically generates the object scene image. When the shutter button 30s under the special mode is fully pressed, the filter circuit 36 synthesizes a plurality of object scene images generated by the image sensor 12 at a specified period (specified imaging interval). The length of the designated cycle is changed by the CPU 28 in response to the change interval update operation (S35). The movement of the imaging surface in the direction orthogonal to the optical axis is detected by the CPU 28 corresponding to each of the plurality of object scene images noted by the filter circuit 36 (S23, S27, S65). The image composition position by the filter circuit 36 is adjusted with reference to the detected movement (S75).

  By changing the length of the designated cycle in response to the interval change operation, the designated cycle can be extended as the moving body moves slower, and thus the moving body can be reliably excluded from the synthesized image. Further, by adjusting the image composition position in consideration of the shake of the imaging surface in the direction orthogonal to the optical axis, it is possible to reduce the influence of the shake of the imaging surface due to the extension of the designated period on the image composition. As a result, it is possible to improve the quality of the synthesized image from which moving objects are excluded.

  In this embodiment, the median filter process in the time axis direction is performed on the raw image data. However, the median filter process in the time axis direction may be performed on the image data in the YUV format.

  In this embodiment, the median filters 56 and 112 are provided in the preprocessing circuit 20 and the filter circuit 36, respectively, in order to remove noise contained in the raw image data. Both may be replaced with rank filters.

It is a block diagram which shows the structure of one Example of this invention. It is an illustration figure which shows an example of the mapping state of SDRAM applied to FIG. 1 Example. (A) is an illustrative view showing an example of a median filter process for R data, (B) is an illustrative view showing an example of a median filter process for G data, and (C) is an example of a median filter process for B data. FIG. It is an illustration figure which shows a part of operation | movement of the filter circuit applied to the FIG. 1 Example. It is an illustration figure which shows an example of the allocation state of the motion detection area and extraction area in an imaging surface. FIG. 2 is an illustrative view showing one example of a configuration of a register applied to the filter circuit shown in FIG. 1. FIG. 8 is an illustrative view showing one example of a configuration of another register applied to the filter circuit shown in FIG. 1. (A) is an illustrative view showing an example of a photographed image, (B) is an illustrative view showing another example of the photographed image, (C) is an illustrative view showing another example of the photographed image, ( (D) is an illustrative view showing still another example of the photographed image, (E) is an illustrative view showing another example of the photographed image, and (F) is an illustrative view showing an example of the composite image. (A) is an illustrative view showing an example of a photographed image, (B) is an illustrative view showing another example of the photographed image, (C) is an illustrative view showing another example of the photographed image, ( (D) is an illustrative view showing still another example of the photographed image, (E) is an illustrative view showing another example of the photographed image, and (F) is an illustrative view showing an example of the composite image. It is a block diagram which shows an example of a structure of the pre-processing circuit applied to FIG. 1 Example. It is a block diagram which shows an example of a structure of the motion detection circuit applied to the FIG. 1 Example. It is a block diagram which shows an example of a structure of the filter circuit applied to the FIG. 1 Example. It is a block diagram which shows an example of a structure of the post-processing circuit applied to FIG. 1 Example. It is a flowchart which shows a part of operation | movement of CPU applied to the FIG. 1 Example. It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 1 Example. It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 1 Example. FIG. 12 is a flowchart showing yet another portion of behavior of the CPU applied to the embodiment in FIG. 1. It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 1 Example. It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 1 Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Digital camera 16 ... Image sensor 20 ... Pre-processing circuit 22 ... Motion detection circuit 28 ... CPU
36 ... Filter circuit 38 ... Post-processing circuit 44 ... I / F
46 Recording medium

Claims (12)

An imaging unit having an imaging surface on which an optical image of the object scene is irradiated, and periodically generating the object scene image;
Synthesizing means for synthesizing a plurality of object scene images generated for each specified period by the imaging means;
Detection means for detecting movement of the imaging surface in a direction orthogonal to the optical axis corresponding to each of a plurality of object scene images noted by the combining means, and an image combining position by the combining means by the detecting means Adjusting means for adjusting by referring to the detected movement ;
The electronic camera is configured to select a plurality of pixels common to the plurality of scene images and output a pixel having a pixel value of a predetermined size among the plurality of pixels . The synthesizing means includes an extracting means for extracting a partial scene image belonging to a specified area from each of the plurality of scene images, and a partial object for synthesizing the plurality of partial scene images extracted by the extracting means. Including field image composition means,
The electronic camera according to claim 1, wherein the plurality of pixels are pixels included in the partial scene image. 3. The electronic camera according to claim 1, further comprising changing means for changing the length of the designated cycle in response to a change instruction. 4. The electronic camera according to claim 1, wherein the synthesizing unit includes a median filter or a rank filter. The electronic camera according to claim 1, wherein the adjustment unit adjusts the position of the designated area so that the movement of the imaging surface is compensated. Output means for outputting a moving image based on the object scene image generated by the imaging means; and
The electronic camera according to claim 1, further comprising an activation unit that activates the synthesizing unit after a plurality of object scene images of interest are generated by the synthesizing unit. An imaging unit having an imaging surface on which an optical image of the object scene is irradiated, and periodically generating the object scene image, and a plurality of object scene images generated for each specified period by the imaging unit are synthesized In the processor of the electronic camera provided with the synthesis means ,
A detection step of detecting movement of the imaging surface in a direction orthogonal to the optical axis corresponding to each of a plurality of object scene images noted by the combining means ;
Adjusting step of adjusting an image synthesis position according to prior Symbol combining means with reference to the motion detected by said detection step,
A selection step of selecting a plurality of pixels common to the plurality of scene images; and
An imaging control program for executing an output step of outputting a pixel having a pixel value of a predetermined size among the plurality of pixels . An imaging unit having an imaging surface on which an optical image of the object scene is irradiated, and periodically generating the object scene image, and a plurality of object scene images generated for each specified period by the imaging unit are synthesized An imaging control method executed by an electronic camera provided with a synthesis means ,
A detection step of detecting movement of the imaging surface in a direction orthogonal to the optical axis corresponding to each of a plurality of object scene images noted by the combining means ;
Adjusting step of adjusting an image synthesis position according to prior Symbol combining means with reference to the motion detected by said detection step,
A selection step of selecting a plurality of pixels common to the plurality of scene images; and
An imaging control method comprising: an output step of outputting a pixel having a pixel value of a predetermined size among the plurality of pixels . An imaging means having an imaging surface on which an optical image of the object scene is irradiated, and generating an object scene image at a specified period;
Selecting means for selecting N (N: integer greater than or equal to 2) object scene images from a plurality of object scene images generated by the imaging means;
Combining means for combining N object scene images selected by the selecting means;
Output means for outputting a moving image based on the object scene image created by the synthesizing means, and adjusting means for adjusting the image synthesis position by the synthesizing means with reference to the movement of the imaging surface in the direction orthogonal to the optical axis equipped with a,
The synthesizing unit is an electronic camera that selects a plurality of pixels common to the N object scene images and outputs a pixel having a predetermined pixel value among the plurality of pixels . The synthesizing unit extracts an extraction unit that extracts a partial scene image belonging to a specified area from each of the N object scene images, and combines the N partial scene images extracted by the extraction unit. Including partial scene image synthesis means,
The electronic camera according to claim 9, wherein the adjustment unit adjusts the position of the designated area so that the movement of the imaging surface is compensated.   The partial object scene image synthesizing means includes a pixel selecting means for selecting a plurality of pixels common to the N partial object scene images, and one pixel based on the plurality of pixels selected by the pixel selecting means. The electronic camera according to claim 10, further comprising pixel output means for outputting The electronic camera according to claim 11 , further comprising a changing unit that changes a selection mode of the selection unit at the designated cycle .

Images