JP2006005536A - Digital imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital imaging apparatus capable of photographing a still image from which a moving object is removed by an inexpensive arrangement. <P>SOLUTION: Images of a plurality of frames are picked up continuously by a CCD imaging element 1. An accumulating section 4 accumulates image signals being outputted from the CCD imaging element 1 in units of frame. Output from the accumulating section 4 is normalized by dividing the accumulated image signal based on the number of accumulation times at the accumulating section. Consequently, the effect of image of a moving object is removed, and a still image from which the moving object is removed can be photographed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a digital imaging device suitable for use in a digital camera, and more particularly to a device capable of erasing moving objects such as passers-by and passing vehicles.

  When photographing a landscape or the like, there is a demand for photographing so that a passerby or a passing vehicle does not enter the photographing range. In order to shoot so that passers-by and passing vehicles do not appear, it is necessary to wait until the passers-by and passing vehicles are interrupted, regardless of whether a silver halide camera or a digital camera is used.

  However, in places where there is a relatively large amount of traffic, such as sightseeing spots, it is difficult to photograph only scenery and the like because passersby are rarely interrupted.

As a method for solving such a problem, as shown in Patent Document 1, an image signal repeatedly input is stored in an image memory, and whether each image signal is the same is compared for each pixel, As a result of the comparison, the pixel signal determined to be the same is determined as the main subject, the image signals repeatedly shot are compared sequentially, and the moving object is removed by sequentially determining the unchanged part. A camera that enables still image shooting has been proposed.
Japanese Patent Laid-Open No. 10-341395

  However, in the one disclosed in Patent Document 1, a monochrome object moving at a low speed or an object that starts moving after being stopped for a while is clearly recorded as a main subject.

  In order to avoid such erroneous recognition of the main subject, Patent Document 1 proposes a method of canceling the location where the erroneous recognition has occurred by allowing the user to designate it using an interface such as a tablet. . However, a dedicated input device is required in addition to the camera body, which is disadvantageous in terms of cost.

  In order to avoid misrecognition and reliably detect the main subject, it is sufficient to increase the number of image frames to be compared. In this case, however, the amount of memory used will increase, and memory writing, reading, etc. Data management becomes complicated. In addition, as the number of frames to be compared increases, the probability that non-matching pixels exist increases. Therefore, it may happen that shooting is not completed indefinitely.

  In view of the above-described conventional technology, an object of the present invention is to provide a digital imaging device that can capture a still image from which a moving object is removed with an inexpensive configuration.

  In order to solve the above-described problem, the invention according to claim 1 is directed to a cumulative addition unit that cumulatively adds image signals output from the image pickup device in units of frames, and an output of the cumulative addition unit to perform cumulative addition in the cumulative addition unit. A normalization unit that normalizes based on the number of times, and a control unit that controls the number of times of cumulative addition by the cumulative addition unit and controls so that normalization by the normalization unit is performed after the cumulative addition.

  The invention according to claim 2 further includes an operation unit for setting a desired operation, and the control unit performs control according to the operation set by the operation unit.

  The invention according to claim 3 is characterized in that the operation unit can set the stop of photographing as a desired operation.

  The invention according to claim 4 is characterized in that the operation unit can set the number of times of cumulative addition in the cumulative addition unit as a desired operation.

  The invention according to claim 5 is characterized in that the operation unit can set an imaging interval of each frame as a desired operation.

  The invention according to claim 6 further includes a display unit for displaying an image signal, and the control unit displays the image signal normalized by the normalization unit on the display unit at a predetermined timing.

  The invention according to claim 7 further includes a storage unit for storing the image signal normalized by the normalization unit, and the control unit displays the image signal stored in the storage unit on the display unit at a predetermined interval. It is characterized by that.

  The invention according to claim 8 further includes a recording device for storing the image signal stored in the storage unit, and the control unit is stored in the storage unit when recording is set as an operation of the operation unit. The image signal is recorded on a recording device.

  According to the first aspect of the present invention, the output image signal of the image sensor is cumulatively added for a plurality of frames by the cumulative adder. When a plurality of image frames are cumulatively added, an image component with a large number of times of imaging becomes dominant, and an image component with a small number of times of imaging is thinned. Focusing on a certain point in the image, a moving object passing through that point has a smaller number of times of imaging than the main subject behind that point, so that information fades in the process of cumulative addition. When the predetermined number of cumulative additions are completed, the control unit stops the cumulative addition operation of the cumulative addition unit and starts the normalization unit. The normalization unit divides the cumulatively added image signal by the cumulative number of additions to normalize, thereby returning the word length of the cumulatively added image signal to the word length of the image output from the image sensor. By returning only the word length of the image signal without changing the ratio of the main subject and the moving object at the time of cumulative addition, the influence of the moving object can be made inconspicuous. As described above, according to the first aspect of the present invention, a still image from which a moving object is removed can be taken without greatly changing the configuration of a general digital imaging apparatus or adding many processing circuits. .

  According to the second aspect of the present invention, when a certain operation is set in the operation unit, control is performed by the control unit in accordance with the set operation. Thereby, it is possible to easily perform an appropriate imaging operation corresponding to the imaging situation.

  According to the invention of claim 3, when the stop of the photographing operation is set in the operation unit, the control unit performs the control related to the stop of the imaging operation. As a result, it is possible to control the shooting time according to the shooting situation based on the user's judgment, and it is possible to more accurately remove the moving object.

  According to the invention of claim 4, when the number of times of cumulative addition in the cumulative addition unit is set in the operation unit, the cumulative addition unit is controlled to perform the cumulative addition number and cumulative addition set by the control unit. . As a result, it is possible to control the shooting time according to the shooting situation, so it is possible to more accurately remove the moving object. In addition, when the set cumulative addition number and cumulative addition are performed, the photographing operation is stopped, so that it is not necessary to instruct the stop again.

  According to the fifth aspect of the present invention, when the imaging interval is set in the operation unit, the imaging element is controlled so that imaging is performed at the imaging interval set by the control unit. This makes it possible to remove the moving object more accurately by controlling the imaging interval in accordance with the shooting situation.

  According to the invention of claim 6, the image data normalized by the normalization unit is displayed on the display unit by the control unit. Thereby, it is possible to confirm at any time on the display section how a plurality of image frames are cumulatively added.

  According to the invention of claim 7, the image data normalized by the normalization unit is stored in the storage unit, and the image stored in the storage unit by the control unit is displayed on the display unit. Accordingly, it is possible to confirm at any time on the display unit the state in which a plurality of image frames are cumulatively added based on the image data stored in the storage unit.

  According to the eighth aspect of the present invention, when a recording operation is set in the operation unit, the image data stored in the storage unit by the control unit is stored in the recording device. As a result, the image data displayed on the display unit when the recording operation is set on the operation unit can be stored in the storage device.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First Embodiment FIG. 1 shows a first embodiment of a digital camera to which the present invention is applied. In FIG. 1, reference numeral 1 denotes an image sensor. As the image sensor 1, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor is used.

  A subject image is formed on the light receiving surface of the image sensor 1 via the optical system 2. The optical system 2 includes a lens system that forms an image of subject image light on the light receiving surface of the image sensor 1, an iris that controls exposure, and the like.

  R (red), G (green), and B (blue) primary color filters, or Cy (cyan), Mg (magenta), and Ye (yellow) complementary color filters are arranged on the light receiving surface of the image sensor 1. Has been. The subject image is photoelectrically converted by the image sensor 1.

  The output signal of the image sensor 1 is supplied to the A / D converter 3. The imaging signal is digitized by the A / D converter 3.

  The output of the A / D converter 3 is supplied to the cumulative adder 4. The cumulative adder 4 includes an adder 5 and a memory 6. The memory 6 stores an image signal for one frame corresponding to the position of each pixel. The adder 5 adds the image signal read from the image sensor 1 and the image signal held in the memory 6, and writes the added image signal in the memory 6. In this way, the memory 6 holds the image signal input in the past and cumulatively added over a plurality of frames.

  The memory 6 is cleared to zero at the start of the photographing operation. At the time of addition, the adder 5 controls the memory address of the memory 6 so that cumulative addition at the same pixel position is performed.

  The output of the memory 6 is supplied to the normalization unit 7. The normalizing unit 7 normalizes the pixels of the cumulative image signal held in the memory 6 by dividing the pixels by the cumulative number of additions. For example, when 256 frames of images are cumulatively added by the cumulative addition unit 4, the normalization unit 7 normalizes the image signal from the cumulative addition unit 4 by dividing by 256.

  The output of the normalization unit 7 is supplied to the image processing unit 8. The image processing unit 8 performs digital image processing such as interpolation processing and contour enhancement. The output of the image processing unit 8 is supplied to the recording device 9 and recorded on the recording medium of the recording device 9. When the image signal is recorded in the recording device 9, the image signal may be compressed by JPEG (Joint Photographic Experts Group) or the like.

  The control unit 10 controls the operation of the entire digital camera. The control unit 10 is provided with an operation unit 11. The operation of each unit is set according to the input given to the operation unit 11. The operation unit 11 is provided with a shooting start key 12.

  FIG. 2 is a flowchart showing a process for capturing a captured image. In FIG. 2, when the start of shooting is instructed (step S 1), shooting is performed (step S 2), and every time shooting is performed, the cumulative addition frame and the cumulative addition unit 4 perform cumulative addition. (Step S3). For example, when 256 frames of image signals are cumulatively added to the cumulative addition unit 4 (step S4), the 256 frames of cumulatively added image signals are sent to the normalization unit 7. The image signal that has been cumulatively added by the normalization unit 7 is divided by the cumulative addition number 256 and normalized (step S5). Then, this image signal is recorded on the recording medium of the recording device 9 (step S6).

  As described above, in this embodiment, at the time of shooting, images of a plurality of frames are continuously shot, the image signals of the plurality of frames are cumulatively added by the cumulative adder 4, and the cumulatively added image signal is normalized by the normalizer 7. Normalized by. By performing such processing, when there is a moving object in the captured image, the influence of the moving object portion can be reduced. This will be described with reference to FIG.

  Now, it is assumed that there is a moving object in front of the subject in the captured image, and the gradation (luminance) of a line in the image signal changes as shown in FIG. In FIG. 3A, the original gradation of the subject to be photographed is assumed to be a dotted line portion 110 in FIG. Here, it is assumed that there is a moving object crossing the front of the subject, and as a result, the waveform 111a of the image of the moving object is generated in the figure with the gradation of this portion.

  A waveform 111b in FIG. 3B is a waveform due to the influence of a moving object when an image of a subject is captured 256 times and the results are cumulatively added. A waveform 111c in FIG. 3C is a waveform caused by the influence of a moving object when the subject image is captured 256 times, the results are cumulatively added, and divided by 256 and normalized.

  As can be seen by comparing the waveform 111a in FIG. 3A and the waveform 111c in FIG. 3C, images of a plurality of frames are cumulatively added, and each pixel of this cumulative image signal is divided by the cumulative number of additions. By normalizing, the influence of the moving object can be considerably reduced.

  Actual shooting is not as simple as this, but the important point is that if you shoot multiple frames and cumulatively add the images, the number of shots becomes dominant, and the number of shots What is less is that it fades according to the ratio of the number of times of imaging. Here, the object that has been imaged a large number is a main subject such as a landscape, and the object that has been imaged a small number is a moving object that crosses in front of the object. In this way, it is possible to photograph so that the moving object crossing the front of the main subject is hardly recorded as a result.

  In FIG. 1, since the image signals of a plurality of frames are cumulatively added to the memory 6 and supplied, the word length of the memory 6 is sufficiently larger than the word length of the image signal input from the image sensor 1. There is a need to. For example, if the word length of the image signal from the A / D converter 3 is 8 bits, the memory 6 having a word length of 16 bits is used. For example, if the image signal of the image sensor 1 is 8 bits and the word length of the memory 6 is 16 bits, the cumulative addition can be performed at least 256 times before the data in the memory 6 overflows. After normalization, the original number of bits is output.

  In the above example, the cumulative number of additions is set to a power of 2, for example 256. This is because when the cumulative number is raised to a power of 2, division by the normalization unit 7 can be performed by bit shift. The control unit 10 issues a control signal when, for example, 256 times of imaging and cumulative addition are completed, and stops the imaging operation and the cumulative addition operation. Then, 256 divisions can be executed by the normalization unit 7 performing bit shift to the right by 8 bits. Thus, when the cumulative number of additions is a power of 2, such as 256, a large-scale divider is not necessary, and division can be executed with a simple bit shift. After normalization, the original number of bits is output.

  In the above-described embodiment, since the number of times of imaging is 256, the bit shift is used for the division in the normalization unit 7. However, division by a normal divider may be performed if there is a margin in the circuit scale. . In that case, accumulation with an arbitrary number of shots becomes possible.

  In the above example, the number of bits of the input image is 8 bits, the word length of the memory 6 is 16 bits, and the memory 6 has a word length twice the number of bits of the input image. It is not limited to such a configuration.

  In recent years, with improvement in performance of an image sensor, an image signal from the image sensor is often quantized with a bit number of 8 bits or more. Even when such an image sensor is used, an image signal is often stored in the memory in units of 16 bits in order to simplify processing. In this case, for example, when a 10-bit image sensor is used, 6 unused bits are generated per pixel. This unused bit can be used as a cumulative addition bit. However, the cumulative number of additions in this case is 64.

  The cumulative addition may be controlled using AE (Auto Exposure) information mounted on the camera system. The AE information is information obtained from the entire level of the imaging signal. If there is a moving object, the AE signal changes. Therefore, for example, frames that have a AE value greatly different from the previous frames are not cumulatively added. As a result, it is possible to avoid accumulation of images where it is clear that a moving object exists, and it is possible to efficiently remove the moving object.

  In the above example, when shooting is started by pressing the shooting start key 12, images of a plurality of frames are cumulatively added, normalized and recorded on the recording medium by the recording device 9, and moving objects are always removed. However, it is needless to say that such a moving object removing shooting mode and a normal shooting mode may be selected.

Second Embodiment FIG. 4 shows a second embodiment of the present invention. In the second embodiment shown in FIG. 4, the same reference numerals are given to those having the same functions as those in the first embodiment, and the description thereof is omitted.

  In the first embodiment described above, the cumulative number of additions is fixed to, for example, 256, and the control unit 10 is configured to stop the imaging operation after 256 imaging operations. It is the maximum value calculated from 6 free bits. Therefore, 256 cumulative additions are not always optimal. Also, if the cumulative number is increased, the influence of moving objects can be reduced, but on the other hand, there is a high possibility that the image will be blurred. On the other hand, if the amount of the moving object is small, an image from which the influence of the moving object is removed can be obtained with a small number of shots. Therefore, in this embodiment, the cumulative number input key 22 is arranged on the operation unit 11 so that the user can change the cumulative number of additions.

  Also, if there is something in front of the subject, such as a car waiting for a signal, that does not move immediately from that location, the effect will be significant if the shooting interval is too short. Even in such a case, the user can efficiently remove the moving object by changing the imaging interval of the imaging device. Therefore, in this embodiment, an imaging interval input key 23 is provided on the operation unit 11 so that the user can change the imaging interval.

  In this embodiment, when the cumulative number is set with the cumulative number input key 22, imaging and cumulative addition are performed for the set number of frames. When the imaging interval is set with the imaging interval input key 23, an image is captured at the set imaging interval.

  Note that it is not necessary to provide physically separate keys as the cumulative number input key 22 and the imaging interval input key 23. In addition, the cumulative number and the photographing interval may be input using a pointing device.

Third Embodiment FIG. 5 shows a third embodiment of the present invention. In the third embodiment shown in FIG. 5, the same reference numerals are given to those having the same functions as those in the first and second embodiments, and the description thereof is omitted.

  In this embodiment, a display device 31 is provided, and the image signal cumulatively added by the cumulative addition unit 4 and normalized by the normalization unit 7 is supplied to the image processing unit 8 and also to the display device 31. The The display device 31 allows the user to check the image while shooting. In this embodiment, a shooting interruption key 25 for interrupting shooting is provided on the operation unit 11.

  In this embodiment, the image signal output from the image sensor 1 is cumulatively added by a cumulative addition unit 4 for a plurality of frames, and is held in the memory 6. When images of a predetermined number of frames are taken and the images of the predetermined number of frames are cumulatively added, the normalization unit 7 is activated and the image normalized by the normalization unit 7 is displayed on the display device 31. In the first and second embodiments, the normalization unit 7 is activated after completion of the cumulative addition of all the image signals. However, in this embodiment, after the cumulative addition of a predetermined number of frames is completed. It will be activated each time. The image signal normalized by the normalization unit 7 is subjected to image processing in the image processing unit 8, but the result is not recorded in the recording device 9 until an instruction from the control unit 10 is given.

  The user looks at the processing result image displayed on the display device 31 to determine whether the image is right or wrong. When a desired image is obtained, the user operates the shooting interruption key 25 of the operation unit 11 to capture the image. Instruct to interrupt operation.

  When the control unit 10 issues a shooting completion control signal to each unit, the imaging device 1 stops the imaging operation, and the normalization unit 7 performs a final normalization operation. The image signal that has been normalized and further subjected to image processing by the image processing unit 8 is stored in the recording medium by the recording device 9.

  Note that if an instruction to stop shooting is not issued from the user within a predetermined time, the control unit 10 may manage the imaging operation so that the predetermined upper limit of the number of shots is not exceeded because an overflow of the memory 6 may occur. There is a need to.

  Increasing the cumulative number can reduce the effect of moving objects, but also increases the possibility of a blurred image. Therefore, it is best to stop the shooting when the influence of the moving object is reduced to some extent.

  According to the present embodiment, an optimum image can be easily obtained by the user himself / herself judging whether the image is appropriate while viewing the processing result image that is updated every predetermined interval.

  Digital cameras often have a liquid crystal display. If the display device 31 of the present embodiment is also used as the liquid crystal display, the cost can be reduced.

Fourth Embodiment FIG. 6 shows a fourth embodiment of the present invention. Components having the same functions as those of the first, second, and third embodiments are given the same reference numerals, and descriptions thereof are omitted.

  In the present embodiment, a memory 41 for storing an image is provided in front of the display device 31 and the image processing unit 8. The image signal cumulatively added by the cumulative addition unit 4 and normalized by the normalization unit 7 is accumulated in the memory 41. The output of the memory 41 is recorded on the recording medium by the recording device 9 via the image processing unit 8 and displayed on the display device 31. Therefore, the image displayed on the display device 31 and the image recorded on the recording medium of the recording device 9 can be the same.

  In this embodiment, the number of cumulative additions to the image signal is set to a power of 2.

  In FIG. 6, the image signal output from the image sensor 1 is cumulatively added by a cumulative addition unit 4 for a plurality of frames and is held in the memory 6. The image signal stored in the memory 6 is normalized by the normalization unit 7. At this time, the control unit 10 activates the normalization unit 7 after the cumulative addition of powers of 2 is performed. The number of cumulative additions controlled by the control unit 10 corresponds to the number of times the normalization unit 7 is activated. Further, the normalization unit 7 performs a right shift operation of an amount corresponding to the number of cumulative additions.

  FIG. 7 shows the relationship between the cumulative addition number in the cumulative addition unit 4, the activation number of the normalization unit 7, and the bit shift amount. As shown in FIG. 7, if the cumulative number of additions is 2, the number of activations of the normalization unit 7 is 1 and the bit shift in the normalization unit 7 is 1 bit. If the cumulative number of additions is 4, the number of activations of the normalizing unit 7 is 2 and the bit shift in the normalizing unit 7 is 2 bits.

  The image signal normalized by the normalization unit 7 is stored in the memory 41. The memory 41 only needs to have the same word length (for example, 8 bits) as the word length of the final image signal. The image signal stored in the memory 41 is subjected to image processing by the image processing unit 8 and input to the display device 31.

  The user looks at the processing result image displayed on the display device 31 to determine whether the image is right. If the user determines that the desired image has been obtained, the user stops the imaging operation through the imaging interruption key 25 of the operation unit 11. Instruct. Upon receiving an instruction to stop imaging from the user, the control unit 10 controls the recording device 9 to write the image signal stored in the memory 41 to the recording medium.

  In the third embodiment, since the cumulative addition process is always executed during the photographing operation, there is a time lag between the image finally recorded on the recording medium and the image displayed on the display device 31. , It will be a little different. On the other hand, the user makes a decision to stop photographing with respect to the image displayed on the display device 31. Therefore, there is a slight difference between the image finally recorded on the recording medium and the image desired by the user.

  Also in the present embodiment, the cumulative addition process is always executed during the shooting operation. However, if the shooting stop determination is made before the image update of the display device 31, the image accumulated after the previous image update is the final result. It is not reflected in. Since what is finally recorded on the recording medium is the image displayed on the display device 31, the same image as that obtained when the user makes the shooting stop determination is obtained.

  Further, in this embodiment, the normalization unit 7 only performs a simple bit shift operation, and a complicated division circuit is not necessary.

  Note that, since the display image is updated in response to a power of 2, the display screen update interval increases in proportion to the shooting time. However, as shown in FIG. 8, the ratio of the image signal for one frame to the total accumulated image decreases as the number of accumulated frames increases. That is, as the number of accumulated frames increases, the influence of one frame image on the final result is reduced, so that an abrupt change is less likely to occur in the image. For this reason, there is no practical problem even if the display screen update interval is increased in proportion to the shooting time.

  In the present embodiment, the output of the normalization unit 7 is stored in the memory 41. However, the image processing by the image processing unit 8 may be stored in the memory 41.

It is a block diagram of a 1st embodiment of the present invention. It is a flowchart used for description of the 1st Embodiment of this invention. It is a graph used for operation | movement description of the 1st Embodiment of this invention. It is a block diagram of the 2nd Embodiment of this invention. It is a block diagram of the 3rd Embodiment of this invention. It is a block diagram of the 4th Embodiment of this invention. It is a figure used for description of the 4th Embodiment of this invention. It is a graph used for description of the 4th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up element 2 Optical system 3 A / D converter 4 Cumulative addition part 5 Adder 6 Memory 7 Normalization part 8 Image processing part 9 Recording apparatus 10 Control part 11 Operation part 12 Shooting start key 22 Cumulative count input key 23 Imaging interval input Key 25 Shooting interruption key 31 Display device 41 Memory

Claims (8)

A cumulative addition unit that cumulatively adds image signals output from the image sensor in units of frames;
A normalization unit that normalizes the output of the cumulative addition unit based on the number of cumulative additions in the cumulative addition unit;
A digital imaging apparatus comprising: a control unit that controls the number of times of cumulative addition by the cumulative addition unit, and that controls normalization by the normalization unit after cumulative addition.   The digital imaging apparatus according to claim 1, further comprising an operation unit configured to set a desired operation, wherein the control unit performs control according to an operation set by the operation unit.   The digital imaging apparatus according to claim 2, wherein the operation unit can set a stop of photographing as the desired operation.   The digital imaging apparatus according to claim 2, wherein the operation unit can set a cumulative addition number in the cumulative addition unit as the desired operation.   The digital imaging apparatus according to claim 2, wherein the operation unit can set an imaging interval of each frame as the desired operation.   The display unit according to claim 1, further comprising a display unit that displays an image signal, wherein the control unit displays the image signal normalized by the normalization unit on the display unit at a predetermined timing. Digital imaging device.   A storage unit that stores the image signal normalized by the normalization unit; and the control unit displays the image signal stored in the storage unit on the display unit at a predetermined interval. The digital imaging apparatus according to claim 6, wherein the digital imaging apparatus is characterized. The image processing apparatus further includes a recording device that stores the image signal stored in the storage unit, and the control unit stores the image signal stored in the storage unit when recording is set as an operation of the operation unit. The digital imaging apparatus according to claim 7, wherein recording is performed by the recording apparatus.

Images