US20070139547A1 - Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device - Google Patents

Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device Download PDF

Info

Publication number
US20070139547A1
US20070139547A1 US11678400 US67840007A US2007139547A1 US 20070139547 A1 US20070139547 A1 US 20070139547A1 US 11678400 US11678400 US 11678400 US 67840007 A US67840007 A US 67840007A US 2007139547 A1 US2007139547 A1 US 2007139547A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
image
up
signal
pixel
picking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11678400
Inventor
Kazuhito Horiuchi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Olympus Corp
Original Assignee
Olympus Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/232Devices for controlling television cameras, e.g. remote control; Control of cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in, e.g. mobile phones, computers or vehicles
    • H04N5/23229Devices for controlling television cameras, e.g. remote control; Control of cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in, e.g. mobile phones, computers or vehicles comprising further processing of the captured image without influencing the image pickup process
    • H04N5/23232Devices for controlling television cameras, e.g. remote control; Control of cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in, e.g. mobile phones, computers or vehicles comprising further processing of the captured image without influencing the image pickup process by using more than one image in order to influence resolution, frame rate or aspect ratio
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/235Circuitry or methods for compensating for variation in the brightness of the object, e.g. based on electric image signals provided by an electronic image sensor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/235Circuitry or methods for compensating for variation in the brightness of the object, e.g. based on electric image signals provided by an electronic image sensor
    • H04N5/2355Circuitry or methods for compensating for variation in the brightness of the object, e.g. based on electric image signals provided by an electronic image sensor by increasing the dynamic range of the final image compared to the dynamic range of the electronic image sensor, e.g. by adding correct exposed portions of short and long exposed images
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/235Circuitry or methods for compensating for variation in the brightness of the object, e.g. based on electric image signals provided by an electronic image sensor
    • H04N5/243Circuitry or methods for compensating for variation in the brightness of the object, e.g. based on electric image signals provided by an electronic image sensor by influencing the picture signal, e.g. signal amplitude gain control

Abstract

An image pick-up device for producing a synthesized image signal having a wide dynamic range by partially exchanging a plurality of image signals from a solid state image sensing element in an image synthesizing unit (41), said image signals being obtained by picking-up a subject moving on a bright background with different exposure amounts, including a synthetic unsuitable portion detecting unit (42, 44) for detecting a synthetic unsuitable portion by comparing said plurality of image signals picked-up with different exposure amounts, a synthetic unsuitable portion correcting unit (46, 48) for correcting a pixel signal of said synthesizing unsuitable portion to derived a corrected synthesized image signal, and a low pass filter (47) for combining said corrected synthesized image signal from said synthetic unsuitable portion correcting unit with said synthesized image signal from said image synthesizing unit to derive a corrected synthesized image signal. An image obtained from the corrected synthesized image signal is not influenced by the movement of the subject and does not cause any feeling of strangeness as compared with a conventional photographic image.

Description

  • This is a divisional of application Ser. No. 10/835,316, filed Apr. 30, 2004, which is a continuation of application Ser. No. 09/358,538, filed Jul. 22, 1999, now U.S. Pat. No. 6,801,248, the priority of which is claimed under 35 USC § 120.
  • BACKGROUND OF THE INVENTION
  • 1. Field of the Invention
  • The present invention relates to an image pick-up device such as electronic still camera and digital camera including a solid state image sensing element, and more particularly to an image pick-up device which can produce an image signal having a wider dynamic range than that of a solid state image sensing element. The present invention also relates to a record medium having recorded thereon a computer readable program for controlling an imaging operation of the above mentioned image pick-up device.
  • 2. Description of the Related Art
  • In the image pick-up device of the kind mentioned above, since the solid state image sensing element has a relatively narrow dynamic range compared with a conventional photographic film, when a subject situating on a relatively bright background is to be picked-up under such an exposure time that the subject is picked-up with a proper exposure amount, the background might be picked-up with an overexposure amount and an image signal corresponding to the background might be saturated, On the other hand, when the subject is picked-up under such an exposure time that the bright background is picked-up with a proper exposure amount, the subject might be picked-up with an underexposure amount and might become too dark. In order to increase a dynamic range of the solid state image sensing element, it has been proposed a known image pick-up device, in which an object is picked-up twice with different exposure amounts, and the thus obtained two image signals are synthesized by simply adding these image signals each other or by partially exchanging or combining these image signals in a mosaic fashion.
  • In the former solution in which the two image signals obtained with different exposure amounts are synthesized by simply adding them, when a subject moves or a camera moves to produce a relative positional shift between successively picked-up two image signals, S/N is decreased at a portion which is subjected to the movement and pseudo-color or pseudo-edge might be produced in a synthesized image signal.
  • In order to remove such a problem, in Japanese Patent Application Laid-open Publication Kokai Hei 2-280585, it has been proposed to derive a synthesized image signal in such a manner that a difference between the two image signals is derived and an absolute value of said difference is subtracted from the synthesized image signal obtained by merely adding the two image signals. However, in this case, since the image shift portion in the synthesized image signal is removed by the subtractive correction, when a subject moves, a configuration of the subject within a synthesized image might be deformed, and therefore the synthesized image has a feeling of strangeness as compared with an image formed on a conventional photographic film.
  • The above problem also occurs in the later solution, in which the two image signals are synthesized by combining or exchanging them in a mosaic fashion. That is to say, as shown in FIG. 13, when a too bright portion B (e.g. background) in a first image A picked-up with a large exposure amount is replaced by a corresponding portion D in a second image C picked-up with a small exposure amount after adjusting a gain of said portion D to derive a synthesized image E, if a subject F moves from right to left in FIG. 13 during a time interval between successive image picking-up operations, a part of the subject F in the second image C picked-up with small exposure amount corresponding to the movement is also synthesized as a part G in the synthesized image E. Then, the subject is partially seen double, and therefore a feeling of strangeness remains.
  • SUMMARY OF THE INVENTION
  • A first object of the present invention is to provide a novel and useful image pick-up device, in which a synthesized image having a wider dynamic range than that of a solid state image sensing element can be obtained by partially combining or exchanging a plurality of images picked-up with different exposure amounts in a mosaic fashion, while said synthesized image be seen without causing a feeling of strangeness.
  • A second object of the invention is to provide a record medium having recorded thereon a computer readable program which is to make a computer to execute procedures for controlling the image pick-up device in such a manner that a synthesized image with a wider dynamic range than that of a solid state image sensing element can be seen without causing a feeling of strangeness.
  • According to the invention for attaining the above mentioned first object, an image pick-up device for producing a synthesized image signal having a wide dynamic range by partially combining a plurality of images picked-up with different exposure amounts comprises:
  • an image picking-up means including a solid state image sensing element and picking-up an object by a plurality of times with different exposure amounts to derive a plurality of image signals;
  • a memory means for storing at least a part of said plurality of image signals derived successively from said image picking-up means;
  • an image synthesizing means for synthesizing said plurality of image signals a part of which is read out of said memory means to derive a synthesized image signal;
  • a detecting means for detecting a synthetic unsuitable portion in said plurality of image signals which is not suitable for synthesizing in accordance with a comparison of the plurality of image signals;
  • a correcting means for correcting pixel signals of said synthetic unsuitable portion detected by said detecting means to derive a corrected pixel signal; and
  • a combining means for combining said corrected pixel signal from said correcting means with said synthesized image signal from said synthesizing means to derive a corrected synthesized image signal.
  • In the image pick-up device according to the invention, a synthetic unsuitable portion is detected on the basis of a comparison of a plurality of image signals picked-up with different exposure amounts, and the pixel signal of the thus detected synthetic unsuitable portion is corrected. Therefore, even if a subject moves on a bright background during successive image picking-up operations, it is possible to minimize any undesired synthesis due to the movement of the subject, and thus a corrected synthesized image having a wide dynamic range can be obtained without causing a feeling of strangeness upon comparing with the conventional photographic film.
  • In an embodiment of the image pick-up device according to the invention, said synthetic unsuitable portion detecting means is constructed such that a ratio in level between a plurality of image signals picked-up with different exposure amounts is calculated, and a synthetic unsuitable portion is detected on the basis of a comparison of said ratio with a ratio in exposure amounts with which said plurality of image signals are picked-up. In this case, a synthetic unsuitable portion can be detected by a simple comparing operation using the plurality of image signals picked-up with different exposure amounts and data from a hardware controlling an exposing operation. Therefore, a cost can be reduced and an operation time can be shortened.
  • In another embodiment of the image pick-up device according to the invention, said synthetic unsuitable portion detecting means is constructed such that a difference between a plurality of image signals picked-up with different exposure amounts is calculated, and a synthetic unsuitable portion is detected on the basis of said difference. Also in this case, a synthetic unsuitable portion can be detected by a simple comparing operation using the plurality of image signals taken with different exposure amounts and data from a hardware controlling an exposing operation, and therefore a cost can be reduced and an operation time can be shortened.
  • In an embodiment of the image pick-up device according to the invention, said synthetic unsuitable portion detecting means is constructed such that a motional vector is calculated on the basis of a comparison of the plurality of image signals picked-up with different exposure amounts, and a synthetic unsuitable portion is detected on the basis of a magnitude of said motional vector. In this embodiment, since a moving amount and a moving direction of a subject can be derived, it is possible to detect the synthetic unsuitable portion much more precisely.
  • In an embodiment of the image pick-up device according to the invention, said synthetic unsuitable portion detecting means is constructed such that a difference between said plurality of image signals picked-up with different exposure amounts is calculated, a motional vector is derived on the basis of a comparison between said plurality of image signals only for a pixel region in which an absolute value of said difference is not less than a predetermined value, and a synthetic unsuitable portion is detected on the basis of a magnitude of said motional vector. When a synthetic unsuitable portion is detected by calculating a motional vector only for a pixel region in which an absolute value of a difference between a plurality of image signals picked-up with different exposure amounts in the manner mentioned above, the operation time of detecting a synthetic unsuitable portion can be materially shortened.
  • In an embodiment of the image pick-up device according to the invention, said synthetic unsuitable portion correcting means is constructed such that a pixel signal within a synthetic unsuitable portion is corrected in accordance with a synthesized pixel signal of a portion which situates in a portion which is not detected as the synthetic unsuitable portion, but is in a vicinity of the synthetic unsuitable portion. When the pixel signal within the synthetic unsuitable portion is corrected in accordance with the synthesized pixel signal of a portion which situates near the synthetic unsuitable portion, it is possible to obtain a corrected synthesized image signal which does not produce a feeling of strangeness with respect to surroundings of the subject as well as a whole synthesized image signal.
  • In another embodiment of the image pick-up device according to the invention, said synthetic unsuitable portion detecting means is constructed to detect a synthetic unsuitable portion on the basis of the motional vector, and said synthetic unsuitable portion correcting means is constructed such that a pixel signal in the synthetic unsuitable portion including a start point of the motional vector is corrected by a synthesized pixel signal in a portion including an end point of said motional vector and being excluded from said synthetic unsuitable portion, and a pixel signal of the synthetic unsuitable portion in a remaining region within the pixel block including the end point of the motional vector is corrected in accordance with an image signal of a corresponding region of one of said plurality of image signals. In this manner, when the pixel signal in the synthetic unsuitable portion including the start point of the motional vector is corrected by the synthesized pixel signal in the pixel block including the end point of said motional vector and being excluded from said synthetic unsuitable portion, the pixel signal in the synthetic unsuitable portion including the start point of the motional vector can be corrected using a lot of pixels without the synthetic suitable portion. Therefore, the precision of the correction can be improved, and it is possible to attain a corrected synthesized image which does not cause a feeling of strangeness with respect to a whole image.
  • In an embodiment of the image pick-up device according to the invention, said synthetic unsuitable portion detecting means is constructed such that a synthetic unsuitable portion is detected by calculating a motional vector, and said synthetic unsuitable portion correcting means is constructed such that a pixel signal of the synthetic unsuitable portion within a pixel block which has a center at a start point of said motional vector and contains an end point of said motional vector is corrected in accordance with a synthesized image signal of a synthetic suitable portion within a pixel block which has a center at the end point of said motional vector and has a same size as said pixel block having a center at the start point of the motional vector. Then, it is no more necessary for correcting the pixel signal of the synthetic unsuitable portion to perform a calculation at a pixel unit, and thus the structure can be simplified and the operating time can be shortened.
  • In order to achieve the second object of the invention, a computer readable record medium, having a program recorded thereon, wherein said program is to make a computer execute the following procedures for controlling an operation of an image pick-up device having a solid state image sensing element:
  • to derive a plurality of image signals with different exposure amounts from said solid state image sensing element;
  • to detect a synthetic unsuitable portion on the basis of a comparison of said plurality of image signals;
  • to correct a pixel signal of said detected synthetic unsuitable portion to produce a corrected pixel signal;
  • to synthesize said plurality of image signals with each other except for said synthetic unsuitable portion to derive a synthesized image signal; and
  • to combine said synthesized image signal with said corrected pixel signal of the synthetic unsuitable portion to derive a corrected synthesized image signal having a dynamic range which is wider than that of said solid state image sensing element.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a block diagram showing a principal structure of the image pick-up device according to the invention;
  • FIG. 2 is a block diagram illustrating a camera signal processing circuit shown in FIG. 1;
  • FIG. 3 is a block diagram depicting an image synthesizing circuit shown in FIG. 2 of a first embodiment of the image pick-up device according to the invention;
  • FIG. 4 is a flow chart representing an operation of major parts of the first embodiment;
  • FIG. 5 is a block diagram showing an embodiment of the image synthesizing circuit shown in FIG. 2 in a second embodiment of the image pick-up device according to the invention;
  • FIG. 6 is a schematic diagram explaining the operation of the image synthesizing circuit shown in FIG. 5;
  • FIG. 7 is a flow chart denoting the operation of major parts of the image synthesizing circuit illustrated in FIG. 5;
  • FIGS. 8, 9 and 10 are schematic diagrams explaining the operation of the image synthesizing circuit of FIG. 5;
  • FIG. 11 is a block diagram showing an embodiment of the image synthesizing circuit shown in FIG. 2 in a third embodiment of the image pick-up device according to the invention;
  • FIG. 12 is a flow chart representing the operation of major parts of the image synthesizing circuit shown in FIG. 11; and
  • FIG. 13 is a schematic view showing the known method of deriving a synthesized image signal having a wide dynamic range by partially combining two image signals picked-up with different exposure amounts.
  • DESCRIPTION OF THE PREFERRED EMBODIMENT
  • FIG. 1 is a block diagram showing a principal structure of the image pick-up device according to the invention which is constructed as an electronic still camera. The electronic still camera comprises a single plate type color CCD image sensing element 1 having an electronic shutter function. On to the CCD image sensing element 1 is projected an image of an object by means of a lens unit 2 and a stop-shutter mechanism 3. The object image is then photo-electrically converted into an image signal. After removing noise in the image signal by means of a correlation double sampling circuit not shown, the image signal is amplified by an amplifier 4 and is converted into a digital signal by an A/D converter 5. Then, the thus obtained digital image data is supplied to a camera signal processing circuit 6 and is processed thereby.
  • The output signal of the A/D converter 5 is also supplied to an AF, AE, AWB detection circuit 7, in which an AF detection processing for deriving AF information for automatically controlling a focus condition, an AE processing for deriving AE information for automatically controlling an exposure, and an AWB processing for deriving AWB information for automatically controlling a white balance. The AF information, AE information and AWB information are supplied to the lens unit 2, stop-shutter mechanism 3 and camera signal processing circuit 6, respectively through CPU 8.
  • The camera signal processing circuit 6 and CPU 8 are connected to a bus line 9, to which are also connected a DRAM 11 through a memory controller 10, said DRAM being used as a working memory upon processing color of the image data, and an image compressing circuit (JPEG) 12 for compressing the image data supplied from the camera signal processing circuit 6. To the bus line 9, there are further connected a memory card I/F 14 for storing a compressed image signal data into a memory card 13, a liquid crystal display (LCD) 16 via a display circuit 15, and a PC I/F 18 for transferring the image data stored in the memory card 13 to a personal computer (PC) 17. Said LCD 16 displays an image stored in the memory card 13 and various image picking-up conditions.
  • To the CPU 8 are connected a strobe flash 19 which is controlled in accordance with the AE information supplied from the AF, AE and AWB detection circuit 7, and an input key 20 for setting various image pick-up modes and driving a trigger switch. The CCD image sensing element 1 is driven by a timing pulse from a timing generator (TG) 21 under the control of the CPU 8.
  • In the image pick-up device shown in FIG. 1, any one of a normal pick-up mode without image synthesis and a wide dynamic range pick-up mode with image synthesis can be selected manually or can be automatically selected by detecting a too bright portion (white large area) in an image signal from the CCD image sensing element 1 under the control of the CPU 8. When the normal pick-up mode is selected, an image signal corresponding to a single picture is derived from the CCD image sensing element 1 by a single picking-up operation. When the wide dynamic mode is selected, the object image is repeatedly picked-up by the CCD element 1 with different exposure amounts to derive a plurality of image signals corresponding to a plurality of pictures by a single picking-up operation. This may be carried out in a known manner by the electronic shutter function of the CCD image sensing element 1 or the stop-shutter mechanism 3 or a combination of the electronic shutter function of the CCD image sensing element 1 and the stop-shutter mechanism 3. Here, for the sake of explanation, the CCD image sensing element 1 produces successively two image signals with different exposure amounts by a single image picking-up operation. The thus obtained single image signal or plural image signals are processed by the camera signal processing circuit 6 in accordance with the selected pick-up mode.
  • FIG. 2 is a block diagram illustrating an embodiment of the camera signal processing circuit 6 shown in FIG. 1. The camera signal processing circuit 6 comprises an input side exchange switch 25 and an output side exchange switch 26. One contact of the input side exchange switch 25 is connected to one contact of the output side exchange switch 26 by means of a camera signal image processing circuit 27. The other contact of the input side exchange switch 25 is connected to a switching arm of an image exchange switch 28, whose one contact is connected to one input of an image synthesis processing circuit 32 via image buffer 29, on-off switch 30 and process circuit 31. The other contact of the image exchange switch 28 is connected to the other input of the image synthesis processing circuit 32 through image buffer 33, on-off switch 34 and process circuit 35. An output of the image synthesis processing circuit 32 is connected to an input of an image compression processing circuit 36, whose output is connected to the other input of the output side exchange switch 26.
  • The input and output side exchange switches 25 and 26 are controlled by a pick-up mode signal ii from the CPU 8, and the image exchange switch 28 and on-off switches 30 and 34 are controlled by an image synthesis process control signal jj from the CPU 8 via a switching control circuit 37. When the wide dynamic range pick-up mode is selected, the CPU 8 supplies, to the image synthesis processing circuit 32, a data signal representing a ratio of exposure amounts with which the object is pick-up successively.
  • In the normal pick-up mode, the input and output side exchange switches 25 and 26 are set to the position of the camera signal image processing circuit 27 by the pick-up mode signal ii. Then, the image signal aa from the A/D converter 5 is supplied to the camera signal image processing circuit 27 by means of the input side exchange switch 25, and is subjected to conventional image processes such as AWB process in accordance with the AWB information from the CPU 8, interpolating process for color information on the basis of a color filter structure of the CCD image sensing element 1, and contrast enhancing process. Then, an image signal hh thus processed is supplied to the bus line 9 as a final image signal kk through the output side exchange switch 26.
  • Contrary to this, when the wide dynamic range pick-up mode is selected, the input side exchange switch 25 is connected to the image exchange switch 28 and at the same time, the output side exchange switch 26 is connected to the image compression processing circuit 36 under the control of the pick-up mode signal ii from the CPU 8. The image exchange switch 28 and on-off switches 30, 34 are controlled by the image synthesis process control signal jj supplied from the CPU 8 via the switching control circuit 37, such that two image signals picked-up with different exposure amounts are successively stored in the image data buffers 29 and 33, respectively.
  • That is to say, at first the on-off switches 30 and 34 are set to the open-state, and the image exchange switch 28 is driven in synchronism with the image picking-up operations for driving the two image signals picked-up with different exposure amounts, such that the image signal aa picked-up with a smaller exposure amount is stored in the image data buffer 29 and the image signal aa picked-up with a larger exposure amount is stored in the image data buffer 33. After that, the on-off switches 30, 34 are simultaneously driven into the closed-condition, an image signal bb picked-up with a smaller exposure amount is supplied from the image data buffer 29 to the process circuit 31 through the on-off switch 30 and an image signal cc picked-up with a larger exposure amount is supplied from the image data buffer 33 to the process circuit 35 through the on-off switch 34.
  • In the process circuits 31 and 35, the image signals are subjected to similar image processes to those of the camera signal image processing circuit 27. An image signal dd picked-up with a smaller exposure amount and an image signal ee picked-up with a larger exposure amount are supplied to the image synthesis processing circuit 32 in synchronism with each other. The image synthesis processing circuit 32 compose the image signals dd and ee by partially exchanging these image signals in accordance with the exposure amount ratio signal mm from the CPU 8 to derive a synthesized image signal ff having a wider dynamic range than the image signals dd and ee. The thus obtained synthesized image signal ff is supplied to the image compression processing circuit 36, in which the synthesized image signal ff is compressed such that the dynamic range is matched with a proper exposure level to derive a compressed synthesized image signal gg. This compressed synthesized image signal gg is supplied to the bus line 9 by means of the output side exchange switch 26 as a final output image signal kk.
  • In the embodiment shown in FIG. 2, there are provided two sets of the image data buffers 29, 33 and on-off switches 30, 34, but according to the invention, the image data buffer 33 and on-off switch 34 may be dispensed with. In such a case, after storing the image signal aa picked-up with a smaller exposure amount into the image data buffer 29, the on-off switch 30 is closed in synchronism with the switching operation of the image exchange switch 28 such that the image signal bb picked-up with a smaller exposure amount from the image data buffer 29 and the image signal aa picked-up with a larger exposure amount from the A/D converter 5 may be supplied to the process circuits 31 and 35, respectively in a synchronous manner.
  • In a first embodiment of the image pick-up device according to the invention, in the image synthesis processing circuit 32, upon synthesizing the image signals dd, ee picked-up with different exposure amounts with each other, a ratio of level between these image signals is calculated pixel by pixel, and when the calculated level ratio is equal to or larger than the exposure amount ratio data signal mm from the CPU 8 by a predetermined value, a relevant pixel is judged to be a synthetic unsuitable pixel, and this pixel is corrected by performing interpolation with the synthesized image signal of a pixel which is in a vicinity of the relevant synthetic unsuitable pixel, but is not detected as a synthetic unsuitable pixel.
  • FIG. 3 is a block diagram depicting an embodiment of the image synthesis processing circuit 32 of the first embodiment. In this image synthesis processing circuit 32, the image signal ee picked-up with a larger exposure amount is supplied to one input of an image synthesizing unit 41 as well as to one input of an image data comparing unit 42. The image signal dd picked-up with a smaller exposure amount is supplied to the other input of the image data comparing unit 42 as well as to a multiplier 43. In the multiplier 43, a gain of the image signal dd is adjusted by an amount corresponding to the exposure amount ratio on the basis of the exposure amount ratio data signal mm from the CPU 8. Then the thus adjusted image signal pp picked-up a smaller exposure amount is supplied to the other input of the image synthesizing unit 41.
  • In the image data comparing unit 42, the level ratio of the image signals dd and ee are compared with each other pixel by pixel to derive an image data comparison signal nn. The thus derived signal nn is supplied to a comparison calculating unit 44. In this comparison calculating unit 44, the image data comparison signal nn is compared with a reference value defined by the exposure ratio data signal mm from the CPU 8 to derive a comparison result logic data signal oo, which is supplied to the image synthesizing unit 41 and an synthetic unsuitable pixel region extracting unit 45. In this embodiment, the comparison result logic data signal oo is a logic “1”, when the image data comparison signal nn of a pixel is larger than the reference value and the relevant pixel is judged to be the synthetic unsuitable pixel. When the image data comparison signal nn is less than the reference value, the comparison result logic data signal oo is a logic “0”.
  • In the image synthesizing unit 41, when the comparison result data logic signal oo of a pixel is “0”, i.e. the relevant pixel belongs to the synthetic suitable pixel, the gain adjusted image signal pp picked-up with a smaller exposure amount and the image signal ee picked-up with a larger exposure amount are synthesized as explained above with reference to FIG. 13. When the comparison result data logic signal oo of a relevant pixel is “1”, i.e. the relevant pixel is judged to be the synthetic unsuitable pixel, the relevant pixel is not used in the image synthesis. Then, the image synthesizing unit 41 supplies a synthesized image signal qq in which synthetic unsuitable pixels are not synthesized. The thus obtained synthesized image signal qq is supplied to an interpolation pixel detecting unit 46 and a low pass filter (LPF) 47.
  • In the synthetic unsuitable pixel region extracting unit 45, a synthetic unsuitable pixel region is extracted for respective lines (horizontal direction) on the basis of the logic value of the comparison result logic data signal oo, i.e. a region in which logic “1” continues, to derive a region signal rr, which is supplied to the interpolation pixel detecting unit 46 and synthetic unsuitable pixel interpolating unit 48. In the interpolation pixel detecting unit 46, an interpolation pixel (correction reference pixel) for interpolating a pixel signal of the synthetic unsuitable pixel region in accordance with the synthesized image signal qq from the image synthesizing unit 41 and the region signal rr from the synthetic unsuitable pixel region extracting unit 45 to derive a correction reference pixel signal ss (including the information of the synthesized image signal qq). The thus derived correction reference pixel signal ss is supplied to a synthetic unsuitable pixel interpolating unit 48.
  • In the synthetic unsuitable pixel interpolating unit 48, a pixel signal of the synthetic unsuitable pixels corresponding to the region signal rr from the synthetic unsuitable pixel region extracting unit 45 is corrected by the interpolation on the basis of the correction reference pixel signal ss from the interpolation pixel detecting unit 46 to derive an interpolated synthesized correct pixel signal tt, which is then supplied to the low pass filter 47. In the low pass filter 47, the synthesized correct pixel signal tt is subjected to the low pass filtering treatment at least in the vertical direction by means of the synthesized image signal qq from the image synthesizing unit 41 to derive a synthesized image signal which is supplied to the image compression processing circuit 36 shown in FIG. 2 as the synthesized image signal ff.
  • FIG. 4 is a flow chart representing the correcting operation for the synthetic unsuitable pixel by means of the synthetic unsuitable pixel region extracting unit 45, interpolation pixel detecting unit 46, synthetic unsuitable pixel interpolating unit 48 and low pass filter 47 shown in FIG. 3. At first, the synthetic unsuitable pixel region extracting unit 45 judges whether or not a reference pixel belongs to the comparison unsuitable pixel on the basis of the comparison result logic data signal oo from the comparison calculating unit 44 (step S1). When the pixel is judged as the synthetic unsuitable one, i.e. logic “1”, a plurality of synthetic unsuitable pixels aligned in the horizontal direction are unified into a single region (step S2), and a length (S) of this synthetic unsuitable pixel region (step S3). The synthetic unsuitable pixels defined by this length constitutes the region signal rr.
  • After that, in the interpolation pixel extracting unit 46, pixels situating at positions which are separated from respective ends of the synthetic unsuitable pixel region by a distance S×β (here,β is an arbitrary adjusting parameter; β>0) are detected as correction reference pixels (LPx, RPx) in a step S4. Then, luminance signals (Y−LPx, Y−RPx) of the thus detected correction reference pixels are calculated in a step S5, and the thus calculated luminance signals are compared with each other in a step S6. When Y−LPx=Y−RPx, color difference signals (Cr−LPx, Cr−RPx) are calculated in a step S7. When Y−LPx>Y−RPx, after a correction region is expanded from the left end of the synthetic unsuitable pixel region by d pixels (d≦S×β) in a step S8, the treatment in the step S7 is executed. When Y−LPx<Y−RPx, after a correction region is expanded from the right end of the synthetic unsuitable pixel region by d pixels in a step S9, the treatment in the step S7 is executed.
  • In the synthetic unsuitable pixel interpolating unit 48, the pixel signal within the correction region (synthetic unsuitable pixel region+expanded portion) is interpolated in accordance with the correction reference pixel signal by smoothing the pixel signal for respective luminance signal and color difference signal (step S10). After completing the interpolation for all the correction region, the luminance and color difference signals of the correction region are subjected to the low pass filtering treatment (step S11). Then, the interpolated luminance and color difference signals are converted into RGB signals (step S12), and the thus converted signals are supplied as the synthesized image signal ff.
  • In the first embodiment, the synthetic unsuitable pixel region is calculated and corrected only in the horizontal direction. According to the invention, the synthetic unsuitable pixel region may be also calculated in the vertical direction, and the pixel signal of this region may be corrected in both the horizontal and vertical directions in accordance with the correction reference pixel signal. Then, the low pass filtering treatment may be conducted in a two-dimensional manner in the horizontal and vertical directions. In this case, it is possible to obtain the synthesized image from which a feeling of strangeness is removed much more effectively with respect to the surroundings and whole synthesized image.
  • Furthermore, in the first embodiment, the synthetic unsuitable region is corrected pixel by pixel, but according to the invention, the synthetic unsuitable region may be detected and corrected for a block including a given number of pixels.
  • In a second embodiment of the image pick-up device according to the invention, in the image synthesis processing circuit 32 shown in FIG. 2, upon synthesizing the image signals dd and ee picked-up with different exposure amounts, a vector representing a movement of a subject (here this vector is called motional vector) is calculated by comparing these image signals. When the motional vector of pixels exceeds a predetermined value, the relevant pixels are detected as the synthetic unsuitable pixel. Then, a pixel signal at a start point of the motional vector is corrected by performing the interpolation in accordance with a synthesized image signal of a synthetic suitable part within a small block including an end point of the motional vector, and a pixel signal of˜the remaining synthetic unsuitable region within the small block including the end point of the motional vector is corrected by conducting the interpolation in accordance with a pixel signal of a corresponding region of the image signal dd.
  • FIG. 5 is a block diagram showing an embodiment of the image synthesis processing circuit 32 in the second embodiment of the image pick-up device according to the invention. In this image synthesis processing circuit 32, the image signal ee picked-up with a larger exposure amount is supplied to one input of an image synthesizing unit 51. The image signal dd picked-up with a smaller exposure amount is supplied to a multiplier 52, in which a gain of the image signal is adjusted on the basis of the exposure amount ratio data signal mm from the CPU 8 by an amount corresponding to the exposure amount ratio. Then, the amplified image signal pp is supplied to the other input of the image synthesizing unit 51. The image signals pp and ee are supplied to edge detecting units 53 and 54, respectively to derive edge detection signals uu and vv. These edge detection signals are supplied to block setting units 55 and 56, respectively to derive block signals ww and xx, which are supplied to a motional vector calculating unit 57. In the motional vector calculating unit 57, a motional vector representing the movement of the subject is calculated in accordance with the block signals ww and xx to derive a motional vector data signal yy. The thus derived motional vector data signal yy is supplied to the image synthesizing unit 51, small block setting unit 58, small block average calculating unit 59 and vector pixel interpolating unit 60.
  • In the image synthesizing unit 51, the motional vector data signal yy is compared with a predetermined threshold value. When the motional vector data signal yy is less than the threshold value, a relevant pixel is judged as a synthetic suitable pixel and a pixel signal of the gain adjusted image signal pp picked-up with a smaller exposure amount and a corresponding pixel signal of the image signal ee picked-up with a larger exposure amount are synthesized by the partial exchanging in the manner explained with reference to FIG. 13. When, the motional vector data signal yy is larger than the threshold value, a pixel is judged as a synthetic unsuitable pixel and the image synthesis by the partial exchange is not executed. In this manner, the image synthesizing unit 51 produce the synthesized image signal qq in which pixels except for the synthetic unsuitable pixels are synthesized. The thus derived image synthesized signal qq is supplied to the small block setting unit 58 and low pass filter (LPF) 61.
  • In the small block setting unit 58, a magnitude of the motional vector data signal yy having a start point of a relevant pixel is compared with a predetermined threshold value. Only when the motional vector data signal yy exceeds the threshold value, the relevant pixel is judged as a synthetic unsuitable pixel. Then, as illustrated by (A) in FIG. 6, a small block (here 3×3 pixel block) is set from the synthesized image signal qq such that a center of the block corresponds to the end point of the motional vector to derive a small block set signal zz. This signal zz (including the information of the synthesized image signal qq) is then supplied to the small block average calculating unit 59, in which average values (Rv, Gv, Bv) of respective color signals by using the synthesized image signal within the small block as shown by (B) in FIG. 6. Then, a small block average value signal aaa (including the information of the synthesized image signal qq) is supplied to the vector pixel interpolating unit 60.
  • To the vector pixel interpolating unit 60 is also supplied the gain adjusted image signal pp picked-up with a smaller exposure amount, and as depicted by (C) in FIG. 6, in order to remove a moving object into an original position, a pixel signal of the synthetic unsuitable pixel at the start point of the motional vector is interpolated by the small block average signal aaa. The pixel signal of the synthetic unsuitable region including the end point of the motional vector is interpolated by the pixel signal of a corresponding region of the gain adjusted image signal pp. A synthesized corrected image signal bbb is supplied to the low pass filter 61. In the low pass filter 61, the synthesized corrected image signal bbb is subjected to the low pass filtering treatment by using the synthesized image signal qq from the image synthesizing unit 51 like as the first embodiment to derive the synthesized image signal ff.
  • FIG. 7 is a flow chart representing the interpolating operation for a pixel signal of synthetic unsuitable pixels by means of the small block setting unit 58, small block average calculating unit 59, vector pixel interpolating unit 60 and low pass filter 61 shown in FIG. 5. At first, in the small block setting unit 58, it is judged whether or not the motional vector having a start point corresponding to a relevant pixel exceeds the predetermined threshold value (step S21). When the vector exceeds the threshold value, the relevant pixel is judged to be a synthetic unsuitable pixel, and a small block having a size of 3×3 pixels such that said small has a center corresponding to an end point of the motional vector in accordance with the synthesized image signal qq from the image synthesizing unit 51 (step S22). Then, the small block set signal zz is supplied to the small block average calculating unit 59.
  • In the small block average calculating unit 59, in response to the small block set signal zz, variables (RB, GB, BB) of color signals and a count value (Count) of an inside counter counting the number of adding operations are initialized (step S23), and while the small block is scanned (step S24), it is judged whether or not the synthesized pixel signals (R, G, B) are existent within the small block (step S25). When the synthesized pixel signals are existent, the relevant synthesized pixel signals are added to corresponding variables and the count values are incremented by one (step S26). After completing the scanning for all the small block (step S27), averages (RV, GV, BV) of the variables (RB, GB, BB) are calculated by using the count values (Count) (step S28), and the thus calculated average values are supplied to the vector pixel interpolating unit 60 as the small block average signal aaa.
  • In the vector pixel interpolating unit 60, a pixel signal of a synthetic unsuitable pixel at the start point of the motional vector is interpolated by the small block average signal aaa (step S29), and a pixel signal of the remaining synthetic unsuitable pixel region including the end point of the motional vector is interpolated by a pixel signal of a corresponding region of the gain adjusted image signal pp (step S30). Then, the thus obtained pixel signal is supplied to the low pass filter 61 as the synthesized corrected image signal bbb. In the low pass filter 61, the pixel signal on the motional vector interpolated in accordance with the synthesized image signal qq is subjected to the low pass filtering treatment (step S31) to derive the synthesized image signal ff.
  • In the second embodiment, in the wide dynamic range picking-up mode, when the image signal dd picked-up with a smaller exposure amount shown by (A) in FIG. 8 and the image signal ee picked-up with a larger exposure amount denoted by (B) in FIG. 8 are derived from the process circuits 31 and 35 (see FIG. 2), respectively, a gain of the image signal picked-up with a shorter exposure time is adjusted such that a level of this image signal is matched with a level of the image signal picked-up with a longer exposure time as illustrated by (C) in FIG. 8. The motional vector is detected from the gain adjusted image signal picked-up with a shorter exposure time and the image signal picked-up with a longer exposure time, and these image signals are synthesized in a partially exchanging manner except for a region in which the detected motional vector exceeds the predetermined threshold value to obtain the synthesized image signal as depicted by (D) in FIG. 8. It should be noted that at this time, the start point of the motional vector shown by (D) in FIG. 8 has not been replaced yet. For respective motional vectors, a pixel signal of a pixel corresponding to a start point of a motional vector is interpolated by an average value of the synthesized image signals within a small block having a center corresponding to an end point of the motional vector as explained above with reference to FIG. 6. By this operation, the start point of the motional vector is replaced by the average value of the synthesized image signals within the small block as shown by (D) in FIG. 8. The pixel signal of the remaining region containing the end point of the motional vector is interpolated by the pixel signal of a corresponding region of the gain adjusted image signal illustrated by a white frame in (C) in FIG. 8. In this manner, it is possible to obtain the corrected synthesized image signal shown in (E) in FIG. 8. Therefore, when the above operation is performed for a plurality of motional vectors shown by (A) in FIG. 9, a corrected synthesized image can be finally obtained as depicted by (B) in FIG. 9.
  • It should be noted that in the above explanation, the pixel signal at a boundary between before and after the movement of the picked-up object is corrected (interpolated), a background may be equally corrected (interpolated). Also in this case, at first, an average value of synthesized image signals within a small block set at an end point of a motional vector is calculated, and a pixel signal at a start point of the motional vector is interpolated in accordance with said average value. In this case, the information of a moving object is dominant in the average value. Next, a pixel signal except for the start point of the motional vector is interpolated by a pixel signal of a corresponding region of the image signal picked-up with a shorter exposure time (corresponding to a smaller exposure amount). Since the information to be interpolated in a background, the gain of the image signal picked-up with a shorter exposure time is multiplied by 1 in order that the background becomes too bright. By performing such an interpolation, only the start point of the motional vector is represented by the moving object and the remaining portion is interpolated by the background. This operation is shown by (A)-(E) in FIG. 10. Since details of this interpolating operation are apparent from the explanations with reference to (A)-(E) in FIG. 8 as well as (A) and (B) in FIG. 9, its explanation is dispensed with.
  • It should be noted that in the above explained second embodiment, the pixel signal at the start point of the motional vector is interpolated by the average value of the image signals within a small block having a center corresponding to the end point of the motional vector. According to the invention, it is also possible to replace the pixel signal at the start point of the motional vector merely by the pixel signal at the end point of the motional vector.
  • In a third embodiment of the image pick-up device according to the invention, in the image synthesis processing circuit 32 shown in FIG. 2, upon synthesizing the image signals dd and ee picked-up with different exposure amounts, a difference between these image signals is calculated, and for a pixel region in which an absolute value of said difference exceeds a predetermined threshold value, a motional vector is calculated. When the motional vector exceeds a predetermined value, a relevant pixel is judged to be a synthetic unsuitable pixel. Then a pixel block having a center corresponding to a start point of the motional vector and including an end point of the motional vector is set, relevant synthetic unsuitable pixel, and a pixel signal of a synthetic unsuitable region within the pixel block is replaced by a synthesized image signal of a synthetic suitable region within a pixel block having a center corresponding to the end point of the motional vector and a size identical with said pixel block having a center at the start point of the motional vector. A pixel signal which is lost by the replacement is corrected by a pixel signal of a corresponding region of the gain adjusted image signal picked-up with a small exposure amount.
  • FIG. 11 is a block diagram showing an embodiment of the image synthesis processing circuit 32 of the third embodiment. In this image synthesis processing circuit 32, the image signal ee picked-up with a larger exposure amount is supplied to one input of an image synthesizing unit 70, and the image signal dd picked-up with a smaller exposure amount is supplied to a multiplier 71, in which a gain of the image signal is amplified by the exposure amount ratio in accordance with the exposure amount ratio data signal mm from the CPU 8. Then, the gain adjusted image signal pp is supplied to the other input of the image synthesizing unit 70. The image signals pp and ee are supplied to AND gates 72 and 73, respectively as well as to an image difference calculating unit 74. In the image difference calculating unit 74, differences between the image signals pp and ee are calculated on a pixel by pixel basis, i.e. at corresponding pixel positions, and absolute values of the differences are compared with a predetermined threshold value. When an absolute value of a difference exceeds the threshold value, a logic “1” is derived, and when an absolute value of a difference is not larger than the threshold value, a logic “0” is produced as a comparison result logic data signal ccc. This comparison result logic data signal ccc is supplied to the other inputs of the AND gates 72 and 73. Then, the AND gate 72 generates an image signal ddd of a region in which the absolute value of the difference between the image signals pp and ee exceeds the threshold value among the gain adjusted image signal pp, and the AND gate 73 generates an image signal eee of a region in which the absolute value of the difference between the image signals pp and ee exceeds the threshold value among the image signal ee.
  • The image signals ddd and eee generated from the AND gates 72 and 73 are supplied to edge detecting units 75 and 76, respectively and edge detection signals fff and ggg are supplied to block setting units 77 and 78, respectively to produce block signals ww and xx. These block signals ww and xx are supplied to a motional vector calculating unit 79, in which a motional vector is calculated on the basis of the block signals ww and xx to derive a motional vector data signal yy. The thus obtained motional vector data signal yy is supplied to the image synthesizing unit 70, corrected block setting unit 80, corrected block replacement pixel determining unit 81 and data correcting unit 82.
  • In the image synthesizing unit 70, like as the second embodiment, the motional vector data signal yy is compared with a predetermined threshold value, and the motional vector data signal yy is not larger than the threshold value, a relevant pixel is judged to be a synthetic suitable pixel. Then, corresponding pixels of the gain adjusted image signal pp picked-up with a smaller exposure amount and the image signal ee picked-up with a larger exposure amount are combined by performing the exchange in a mosaic manner explained with reference to FIG. 13. When the motional vector data signal yy exceeds the predetermined threshold value, a relevant pixel is judged as a synthetic unsuitable pixel, and the image synthesis by the exchange is not carried out. In this manner, the image synthesizing unit 70 derives the synthesized image signal qq, in which pixels other than synthetic unsuitable pixels are synthesized in a mosaic manner, and this synthesized image signal qq is supplied to the corrected block setting unit 80 and low pass filter (LPF) 83.
  • In the corrected block setting unit 80, also represented by a flow chart shown in FIG. 12, a motional vector data signal yy having a start point corresponding to a pixel under consideration is compared with a predetermined threshold value Th (step S41). Only when the motional vector data signal yy exceeds the threshold value Th, the relevant pixel is judged as a synthetic unsuitable pixel, and a corrected block is set by a similar calculation for deriving the motional vector such that said block has a center corresponding to a start point of the motional vector and includes an end point of the motional vector (step S42). A corrected block set signal hhh (including the information of the synthesized image signal qq) is supplied to the corrected block replacement pixel determining unit 81.
  • In this corrected block replacement pixel determining unit 81, a correction block having the same size as the corrected block and having a center corresponding to the end point of the motional vector in accordance with the corrected block set signal hhh and motional vector data signal yy (step S43). A correction block set signal iii (including the information of the synthesized image signal qq) is supplied to the data correcting unit 82.
  • To the data correcting unit 82 is also supplied the gain adjusted image signal pp, and a pixel signal of a synthetic unsuitable pixel is replaced by the synthesized image signal within the correction block in accordance with the correction block set signal iii and motional vector data signal yy (step S44). A pixel signal of the relevant region which has been lost by the replacement is obtained by performing the interpolation in accordance with a pixel signal of a corresponding region of the gain adjusted image signal pp picked-up with a smaller exposure amount (step S45). In this manner, a synthesized corrected image signal jjj is obtained and is supplied to the low pass filter 83. In the low pass filter 83, the pixel signal within the corrected block as well as the pixel signal in a region surrounding the corrected block are subjected to the low pass filtering treatment for respective color signals (step S46) to derive the synthesized image signal ff.
  • In the third embodiment, the pixel signal of the synthetic unsuitable region within the corrected block is replaced by the synthesized image signal of the synthetic suitable region within the correcting block, but according to the invention, the interpolation may be utilized instead of the replacement or substitution. Then, no pixel signal is lost within the correcting block and it is no more necessary to perform the above mentioned interpolation for the lost pixel signal. In this manner, the process can be simplified.
  • The present invention also provide a computer readable record medium, having a program recorded thereon, wherein said program is to make a computer execute the procedure for controlling an operation of the image pick-up device in the dynamic range image picking-up mode in the manner explained above in the various embodiments and possible alternations thereof. By setting such a record medium on a suitable driver installed in or connected to the image pick-up device and reading the program into the CPU 8 (FIG. 1) in the image pick-up device, the above explained process can be performed.
  • As explained above, according to the present invention, upon synthesizing a plurality of image signals picked-up with different exposure amounts in a partially exchanging manner to derive a synthesized image signal having a wide dynamic range, a synthetic unsuitable region in the image signals is detected and a pixel signal of such a synthetic unsuitable region is corrected. Therefore, the influence of the movement of a subject on a bright background during successive picking-up operations upon the synthesizing of the image signals can be minimized, and it is possible to obtain an image having a wide dynamic range which does not cause any feeling of strangeness as compared with a conventional photographic image.

Claims (6)

  1. 1. An image pick-up device comprising:
    an image picking-up means including a solid state image sensing element and picking-up an object by a plurality of times with different exposure amounts to derive a plurality of image signals;
    an image synthesizing means for synthesizing said plurality of image signals, which are successively derived from said image picking-up means, into a synthesized image signal;
    a selector means for selecting either one of a first picking-up mode without image synthesis by said image synthesizing means, and a second picking-up mode with image synthesis by said image synthesizing means; and
    a controller means for deriving an image signal corresponding to a single picture by a single picking-up operation and performing a normal image processing, when said first picking-up mode without image synthesis is selected by said selector means, and deriving a plurality of image signals corresponding to a plurality of pictures by a single picking-up operation and performing said image synthesis, when said second picking-up mode is selected by said selector means.
  2. 2. An image pick-up device as claimed in claim 1, wherein said selector means comprises an input key for manually selecting either one of said first and second picking-up modes.
  3. 3. An image pick-up device as claimed in claim 1, wherein said selector means is adapted to automatically select either one of said first and second picking-up modes, based on the image signal derived from said image picking-up means.
  4. 4. An image pick-up device as claimed in claim 1, wherein said image picking-up means is configured so that image signals of an object with different exposure amounts are derived from said solid state image sensing element, and wherein said image synthesis means is configured so that said image signals with different exposure amounts derived from said image picking-up means are added and synthesized to generate a single synthesized image signal.
  5. 5. An image pick-up device as claimed in claim 4, wherein said image synthesis means is configured so that said image signals with different exposure amounts derived from said image picking-up means are added and synthesized to generate a single synthesized image signal with a wide dynamic range.
  6. 6. An image pick-up device as claimed in claim 5, wherein said selector means is configured so as to automatically select said second picking-up mode with image synthesis, by detecting an excessively bright portion in the image signal derived from said picking-up means.
US11678400 1998-07-24 2007-02-23 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device Abandoned US20070139547A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP10-209450 1998-07-24
JP20945098A JP4282113B2 (en) 1998-07-24 1998-07-24 Imaging apparatus and an imaging method, and a recording medium recording an image pickup program
US09358538 US6801248B1 (en) 1998-07-24 1999-07-22 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device
US10835316 US7372487B2 (en) 1998-07-24 2004-04-30 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device
US11678400 US20070139547A1 (en) 1998-07-24 2007-02-23 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US11678400 US20070139547A1 (en) 1998-07-24 2007-02-23 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10835316 Division US7372487B2 (en) 1998-07-24 2004-04-30 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device

Publications (1)

Publication Number Publication Date
US20070139547A1 true true US20070139547A1 (en) 2007-06-21

Family

ID=16573080

Family Applications (3)

Application Number Title Priority Date Filing Date
US09358538 Expired - Fee Related US6801248B1 (en) 1998-07-24 1999-07-22 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device
US10835316 Expired - Fee Related US7372487B2 (en) 1998-07-24 2004-04-30 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device
US11678400 Abandoned US20070139547A1 (en) 1998-07-24 2007-02-23 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US09358538 Expired - Fee Related US6801248B1 (en) 1998-07-24 1999-07-22 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device
US10835316 Expired - Fee Related US7372487B2 (en) 1998-07-24 2004-04-30 Image pick-up device and record medium having recorded thereon computer readable program for controlling the image pick-up device

Country Status (3)

Country Link
US (3) US6801248B1 (en)
EP (1) EP0975154A1 (en)
JP (1) JP4282113B2 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090059039A1 (en) * 2007-08-31 2009-03-05 Micron Technology, Inc. Method and apparatus for combining multi-exposure image data
US20090278990A1 (en) * 2008-05-07 2009-11-12 Nec Electronics Corporation Error detection method and circuit, and error reduction method and circuit
US20100091119A1 (en) * 2008-10-10 2010-04-15 Lee Kang-Eui Method and apparatus for creating high dynamic range image
WO2010053507A2 (en) 2008-11-07 2010-05-14 Eastman Kodak Company Modifying color and panchromatic channel cfa image
US20100295932A1 (en) * 2009-05-25 2010-11-25 Olympus Corporation Image obtaining apparatus, image synthesis method and microscope system
US20110019014A1 (en) * 2009-07-23 2011-01-27 Samsung Electronics Co., Ltd. Apparatus and method for obtaining motion adaptive high dynamic range image
US20110069200A1 (en) * 2009-09-22 2011-03-24 Samsung Electronics Co., Ltd. High dynamic range image generating apparatus and method
US20110141228A1 (en) * 2009-12-15 2011-06-16 Sony Corporation Image capturing apparatus and image capturing method
US20110169980A1 (en) * 2010-01-11 2011-07-14 Samsung Electronics Co., Ltd. Apparatus and method for obtaining high dynamic range image
US20140063294A1 (en) * 2012-09-03 2014-03-06 Kabushiki Kaisha Toshiba Image processing device, image processing method, and solid-state imaging device
US9068831B2 (en) 2011-05-27 2015-06-30 Panasonic Intellectual Property Management Co., Ltd. Image processing apparatus and image processing method

Families Citing this family (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4282113B2 (en) * 1998-07-24 2009-06-17 オリンパス株式会社 Imaging apparatus and an imaging method, and a recording medium recording an image pickup program
US7428011B1 (en) * 1999-09-02 2008-09-23 Fujifilm Corporation Wide dynamic range electronic image recording and reproducing system
EP1237363B1 (en) * 1999-11-22 2006-01-18 Matsushita Electric Industrial Co., Ltd. Solid-state imaging device
US7050094B2 (en) * 2000-10-26 2006-05-23 Micron Technology, Inc. Wide dynamic range operation for CMOS sensor with freeze-frame shutter
JP2002218328A (en) * 2001-01-19 2002-08-02 Ricoh Co Ltd Image input device, image input method, and computer- readable/writable recording medium having stored program for implementing the method
JP4511066B2 (en) * 2001-03-12 2010-07-28 オリンパス株式会社 Imaging device
JP4028694B2 (en) * 2001-04-27 2007-12-26 松下電器産業株式会社 Electronic device having a camera device and the camera device
JP3624859B2 (en) * 2001-06-19 2005-03-02 カシオ計算機株式会社 Imaging apparatus, soft-focus image capturing method
JP4742453B2 (en) * 2001-06-20 2011-08-10 ソニー株式会社 Imaging method and apparatus
DE60225981D1 (en) 2001-08-20 2008-05-21 Photographic Art Res Lab A method for generating a digital photo
JP4154157B2 (en) * 2002-02-25 2008-09-24 株式会社東芝 Imaging device
WO2003083773A3 (en) * 2002-03-27 2004-03-11 Terrance Boult Imaging method and system
US7508421B2 (en) * 2002-06-24 2009-03-24 Fujifilm Corporation Image pickup apparatus and image processing method
JP4025207B2 (en) * 2003-01-17 2007-12-19 富士フイルム株式会社 A solid-state imaging device and a digital camera
US7889275B2 (en) * 2003-01-28 2011-02-15 Microsoft Corp. System and method for continuous flash
WO2004114677A1 (en) * 2003-06-23 2004-12-29 Sony Corporation Image processing method and device, and program
JP4392583B2 (en) * 2003-06-27 2010-01-06 ソニー株式会社 Signal processing device and signal processing method, and program and recording medium
FI20045445A0 (en) * 2004-11-18 2004-11-18 Nokia Corp The method, system, and software arrangement of the image data to modify the
US7616231B2 (en) * 2005-01-06 2009-11-10 Goodrich Corporation CMOS active pixel sensor with improved dynamic range and method of operation for object motion detection
JP4731953B2 (en) * 2005-03-02 2011-07-27 富士フイルム株式会社 Imaging apparatus, an imaging method, and an imaging program
JP4687265B2 (en) * 2005-06-14 2011-05-25 富士ゼロックス株式会社 Image Analyzer
JP2007124609A (en) * 2005-09-28 2007-05-17 Nissan Motor Co Ltd Apparatus for providing vehicle periphery image
JP4509925B2 (en) * 2005-12-27 2010-07-21 株式会社メガチップス The image processing apparatus and a camera system and image processing method and the moving picture display method
WO2007077719A8 (en) * 2005-12-27 2008-08-28 Kyocera Corp Imaging device and its image processing method
JP2007300463A (en) * 2006-05-01 2007-11-15 Opt Kk Camera system and image processing method
JP5011814B2 (en) * 2006-05-15 2012-08-29 ソニー株式会社 Imaging apparatus, image processing method, and computer program
DE102006046720A1 (en) 2006-10-02 2008-04-03 Arnold & Richter Cine Technik Gmbh & Co. Betriebs Kg Digital motion-picture camera for taking motion pictures, has opto-electronic sensor device, which has multiple sensor elements for producing of received signals as function of exposure time
US8072507B2 (en) * 2006-10-20 2011-12-06 Primax Electronics Ltd. Method and system of generating high dynamic range image corresponding to specific scene
JP4806329B2 (en) * 2006-10-23 2011-11-02 三洋電機株式会社 An imaging apparatus and an imaging method
KR20080037965A (en) * 2006-10-27 2008-05-02 삼성테크윈 주식회사 Method for controlling moving picture photographing apparatus, and moving picture photographing apparatus adopting the method
KR20080076004A (en) * 2007-02-14 2008-08-20 삼성전자주식회사 Image pickup device and method of extending dynamic range thereof
JP4905187B2 (en) * 2007-03-09 2012-03-28 ソニー株式会社 The image processing apparatus, an imaging apparatus, image processing method, and computer program
JP5188101B2 (en) * 2007-06-01 2013-04-24 株式会社キーエンス Magnifying observation apparatus, enlarged image capturing method, an enlarged image capturing program and a computer-readable recording medium
JP4872830B2 (en) * 2007-06-28 2012-02-08 セイコーエプソン株式会社 Imaging apparatus, imaging method, image processing apparatus, image processing program and an image processing method
JP5083046B2 (en) * 2008-06-03 2012-11-28 ソニー株式会社 An imaging apparatus and an imaging method
KR101534317B1 (en) * 2008-10-10 2015-07-06 삼성전자주식회사 Method and apparatus of generating high dynamic range image
EP2343904A4 (en) * 2008-11-04 2012-09-05 Konica Minolta Opto Inc Image inputting apparatus
US20100157079A1 (en) 2008-12-19 2010-06-24 Qualcomm Incorporated System and method to selectively combine images
JP5589446B2 (en) * 2009-09-18 2014-09-17 ソニー株式会社 The image processing apparatus, an imaging apparatus, image processing method, and program
JP5458865B2 (en) * 2009-09-18 2014-04-02 ソニー株式会社 The image processing apparatus, an imaging apparatus, image processing method, and program
JP5617445B2 (en) * 2010-08-31 2014-11-05 ソニー株式会社 Imaging apparatus, signal processing method, and program
JP5772044B2 (en) * 2011-02-17 2015-09-02 カシオ計算機株式会社 Imaging device, imaging method and program
US8965120B2 (en) 2012-02-02 2015-02-24 Canon Kabushiki Kaisha Image processing apparatus and method of controlling the same
JP6046966B2 (en) 2012-04-19 2016-12-21 キヤノン株式会社 Image processing apparatus and image processing method, program, and storage medium
JP6061488B2 (en) * 2012-04-26 2017-01-18 キヤノン株式会社 Image processing apparatus and image processing method
JP2014039169A (en) * 2012-08-16 2014-02-27 Sony Corp Image processor, image processing method, and program
WO2015072117A1 (en) * 2013-11-12 2015-05-21 Canon Kabushiki Kaisha Imaging apparatus, external apparatus, imaging system, control method for imaging apparatus, control method for external apparatus, control method for imaging system, and program
EP3091731A4 (en) * 2014-03-18 2017-08-30 Sony Corporation Image processing device, image processing method and program

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5309243A (en) * 1992-06-10 1994-05-03 Eastman Kodak Company Method and apparatus for extending the dynamic range of an electronic imaging system
US5365269A (en) * 1992-10-22 1994-11-15 Santa Barbara Instrument Group, Inc. Electronic camera with automatic image tracking and multi-frame registration and accumulation
US5396288A (en) * 1992-08-21 1995-03-07 Fuji Photo Film Co., Ltd. Image processing apparatus and method, and video camera
US5436656A (en) * 1992-09-14 1995-07-25 Fuji Photo Film Co., Ltd. Digital electronic still-video camera and method of controlling same
US5455621A (en) * 1992-10-27 1995-10-03 Matsushita Electric Industrial Co., Ltd. Imaging method for a wide dynamic range and an imaging device for a wide dynamic range
US5929908A (en) * 1995-02-03 1999-07-27 Canon Kabushiki Kaisha Image sensing apparatus which performs dynamic range expansion and image sensing method for dynamic range expansion
US6002433A (en) * 1995-08-29 1999-12-14 Sanyo Electric Co., Ltd. Defective pixel detecting circuit of a solid state image pick-up device capable of detecting defective pixels with low power consumption and high precision, and image pick-up device having such detecting circuit
US6204881B1 (en) * 1993-10-10 2001-03-20 Canon Kabushiki Kaisha Image data processing apparatus which can combine a plurality of images at different exposures into an image with a wider dynamic range
US6243136B1 (en) * 1996-10-14 2001-06-05 Ricoh Company, Ltd. Image input device and image input system which controls timing of successive imaging operations
US6278490B1 (en) * 1996-05-23 2001-08-21 Olympus Optical Co., Ltd. Exposure control for an image pickup apparatus that uses an electronic flash
US20020154829A1 (en) * 2001-03-12 2002-10-24 Taketo Tsukioka Image pickup apparatus
US6529640B1 (en) * 1998-06-09 2003-03-04 Nikon Corporation Image processing apparatus
US6995790B2 (en) * 1996-11-08 2006-02-07 Olympus Optical Co., Ltd. Image processing apparatus for joining a plurality of images

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2804782B2 (en) 1989-04-21 1998-09-30 オリンパス光学工業株式会社 Imaging device
US5764287A (en) * 1992-08-31 1998-06-09 Canon Kabushiki Kaisha Image pickup apparatus with automatic selection of gamma correction valve
JP3817301B2 (en) 1996-05-28 2006-09-06 キヤノン株式会社 Electronic camera
JP3834130B2 (en) * 1996-07-19 2006-10-18 株式会社リコー Digital camera
US7057653B1 (en) * 1997-06-19 2006-06-06 Minolta Co., Ltd. Apparatus capable of image capturing
JPH11166279A (en) 1997-12-05 1999-06-22 Kokuyo Co Ltd Partition
JP4282113B2 (en) * 1998-07-24 2009-06-17 オリンパス株式会社 Imaging apparatus and an imaging method, and a recording medium recording an image pickup program

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5309243A (en) * 1992-06-10 1994-05-03 Eastman Kodak Company Method and apparatus for extending the dynamic range of an electronic imaging system
US5396288A (en) * 1992-08-21 1995-03-07 Fuji Photo Film Co., Ltd. Image processing apparatus and method, and video camera
US5436656A (en) * 1992-09-14 1995-07-25 Fuji Photo Film Co., Ltd. Digital electronic still-video camera and method of controlling same
US5365269A (en) * 1992-10-22 1994-11-15 Santa Barbara Instrument Group, Inc. Electronic camera with automatic image tracking and multi-frame registration and accumulation
US5455621A (en) * 1992-10-27 1995-10-03 Matsushita Electric Industrial Co., Ltd. Imaging method for a wide dynamic range and an imaging device for a wide dynamic range
US6204881B1 (en) * 1993-10-10 2001-03-20 Canon Kabushiki Kaisha Image data processing apparatus which can combine a plurality of images at different exposures into an image with a wider dynamic range
US5929908A (en) * 1995-02-03 1999-07-27 Canon Kabushiki Kaisha Image sensing apparatus which performs dynamic range expansion and image sensing method for dynamic range expansion
US6002433A (en) * 1995-08-29 1999-12-14 Sanyo Electric Co., Ltd. Defective pixel detecting circuit of a solid state image pick-up device capable of detecting defective pixels with low power consumption and high precision, and image pick-up device having such detecting circuit
US6278490B1 (en) * 1996-05-23 2001-08-21 Olympus Optical Co., Ltd. Exposure control for an image pickup apparatus that uses an electronic flash
US6243136B1 (en) * 1996-10-14 2001-06-05 Ricoh Company, Ltd. Image input device and image input system which controls timing of successive imaging operations
US6995790B2 (en) * 1996-11-08 2006-02-07 Olympus Optical Co., Ltd. Image processing apparatus for joining a plurality of images
US6529640B1 (en) * 1998-06-09 2003-03-04 Nikon Corporation Image processing apparatus
US20020154829A1 (en) * 2001-03-12 2002-10-24 Taketo Tsukioka Image pickup apparatus

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090059039A1 (en) * 2007-08-31 2009-03-05 Micron Technology, Inc. Method and apparatus for combining multi-exposure image data
US8269896B2 (en) * 2008-05-07 2012-09-18 Renesas Electronics Corporation Error detection method and circuit, and error reduction method and circuit
US20090278990A1 (en) * 2008-05-07 2009-11-12 Nec Electronics Corporation Error detection method and circuit, and error reduction method and circuit
US20100091119A1 (en) * 2008-10-10 2010-04-15 Lee Kang-Eui Method and apparatus for creating high dynamic range image
EP2359604B1 (en) * 2008-11-07 2013-12-11 Omnivision Technologies, Inc. Modifying color and panchromatic channel cfa image
WO2010053507A2 (en) 2008-11-07 2010-05-14 Eastman Kodak Company Modifying color and panchromatic channel cfa image
US20100295932A1 (en) * 2009-05-25 2010-11-25 Olympus Corporation Image obtaining apparatus, image synthesis method and microscope system
US20110019014A1 (en) * 2009-07-23 2011-01-27 Samsung Electronics Co., Ltd. Apparatus and method for obtaining motion adaptive high dynamic range image
US8462214B2 (en) 2009-07-23 2013-06-11 Samsung Electronics Co., Ltd. Apparatus and method for obtaining motion adaptive high dynamic range image
US8379094B2 (en) * 2009-07-23 2013-02-19 Samsung Electronics Co., Ltd. Apparatus and method for obtaining motion adaptive high dynamic range image
US20120236169A1 (en) * 2009-07-23 2012-09-20 Oh Hyun-Hwa Apparatus and method for obtaining motion adaptive high dynamic range image
US8508619B2 (en) 2009-09-22 2013-08-13 Samsung Electronics Co., Ltd. High dynamic range image generating apparatus and method
US20110069200A1 (en) * 2009-09-22 2011-03-24 Samsung Electronics Co., Ltd. High dynamic range image generating apparatus and method
US8723918B2 (en) * 2009-12-15 2014-05-13 Sony Corporation Image capturing apparatus and image capturing method
US20110141228A1 (en) * 2009-12-15 2011-06-16 Sony Corporation Image capturing apparatus and image capturing method
US8917313B2 (en) 2009-12-15 2014-12-23 Sony Corporation Image capturing apparatus and image capturing method
KR20110082421A (en) * 2010-01-11 2011-07-19 삼성전자주식회사 Apparatus and method for obtaining high dynamic range image
US8547451B2 (en) * 2010-01-11 2013-10-01 Samsung Electronic Co., Ltd. Apparatus and method for obtaining high dynamic range image
US20110169980A1 (en) * 2010-01-11 2011-07-14 Samsung Electronics Co., Ltd. Apparatus and method for obtaining high dynamic range image
KR101643319B1 (en) * 2010-01-11 2016-07-27 삼성전자주식회사 Apparatus and Method for obtaining high dynamic range image
US9068831B2 (en) 2011-05-27 2015-06-30 Panasonic Intellectual Property Management Co., Ltd. Image processing apparatus and image processing method
US20140063294A1 (en) * 2012-09-03 2014-03-06 Kabushiki Kaisha Toshiba Image processing device, image processing method, and solid-state imaging device

Also Published As

Publication number Publication date Type
EP0975154A1 (en) 2000-01-26 application
JP4282113B2 (en) 2009-06-17 grant
US7372487B2 (en) 2008-05-13 grant
JP2000050173A (en) 2000-02-18 application
US6801248B1 (en) 2004-10-05 grant
US20040201759A1 (en) 2004-10-14 application

Similar Documents

Publication Publication Date Title
US7227576B2 (en) Electronic camera
US7129980B1 (en) Image capturing apparatus and automatic exposure control correcting method
US7453506B2 (en) Digital camera having a specified portion preview section
US20100053349A1 (en) Imaging Apparatus and Imaging Method
US7292280B2 (en) Camera, aperture controlling method and apparatus, lens controlling method and apparatus, and edging amount controlling method and apparatus
US6853401B2 (en) Digital camera having specifiable tracking focusing point
US6192162B1 (en) Edge enhancing colored digital images
US20060115297A1 (en) Imaging device and imaging method
US20040061796A1 (en) Image capturing apparatus
US20080112644A1 (en) Imaging device
US20040095472A1 (en) Electronic still imaging apparatus and method having function for acquiring synthesis image having wide-dynamic range
US20040105015A1 (en) Image processing device and image processing program
US7349119B2 (en) Image storage and control device for camera to generate synthesized image with wide dynamic range
US20050093886A1 (en) Image processing device
US20030189663A1 (en) Solid-state video camera and method for brightness control
US20060290796A1 (en) Digital image processing
US20090160992A1 (en) Image pickup apparatus, color noise reduction method, and color noise reduction program
US20100066868A1 (en) Image processing apparatus and method of processing image
US6882754B2 (en) Image signal processor with adaptive noise reduction and an image signal processing method therefor
US20050157188A1 (en) Image capturing apparatus and method of performing noise process on moving picture
US7301563B1 (en) Image pickup apparatus
US20010055064A1 (en) Digital still camera
US20060274173A1 (en) Digital camera comprising smear removal function
US20020145667A1 (en) Imaging device and recording medium storing and imaging program
US20100295953A1 (en) Image processing apparatus and method thereof