JP2008124728A - Imaging device - Google Patents
Imaging device Download PDFInfo
- Publication number
- JP2008124728A JP2008124728A JP2006305522A JP2006305522A JP2008124728A JP 2008124728 A JP2008124728 A JP 2008124728A JP 2006305522 A JP2006305522 A JP 2006305522A JP 2006305522 A JP2006305522 A JP 2006305522A JP 2008124728 A JP2008124728 A JP 2008124728A
- Authority
- JP
- Japan
- Prior art keywords
- blur
- unit
- correction
- optical system
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
Description
本発明は、撮影時におけるカメラのブレを検出して、静止画のブレを補正する撮像装置に関し、特に、光学式ブレ補正に、画像復元式ブレ補正を組み合わせた補正機能を備える撮像装置に関する。 The present invention relates to an imaging apparatus that detects camera shake at the time of shooting and corrects still picture blur, and more particularly, to an imaging apparatus that includes a correction function that combines optical blur correction and image restoration blur correction.
従来から、撮影時の手ブレによって生じる撮影画像の劣化を防ぐために、手ブレ補正機能を備えたカメラが知られている。手ブレ補正機能は、主に、手ブレを検出する要素と、検出結果に応じて手ブレを補正する要素により実現される。静止画の手ブレ補正機能は、近年ではデジタルカメラの差別化技術として重要になっている。 2. Description of the Related Art Conventionally, a camera having a camera shake correction function is known in order to prevent deterioration of a captured image caused by camera shake during shooting. The camera shake correction function is mainly realized by an element that detects camera shake and an element that corrects camera shake according to the detection result. In recent years, the camera shake correction function for still images has become important as a differentiating technique for digital cameras.
一般的に、手ブレを検出する要素としては、角速度センサや加速度センサなどの振動センサによりカメラの手ブレ量（振動量）を検出する方法と、撮影された画像の画像処理により手ブレ量を演算する方法がある。また、手ブレを補正する要素としては、光学系の駆動により補正を行う光学式手ブレ補正と、画像処理により補正を行う電子式手ブレ補正（例えば、特許文献１〜３参照）とがある。 In general, as an element for detecting a camera shake, a camera shake amount (vibration amount) is detected by a vibration sensor such as an angular velocity sensor or an acceleration sensor, and a camera shake amount is determined by image processing of a photographed image. There is a way to calculate. As elements for correcting camera shake, there are optical camera shake correction that performs correction by driving an optical system, and electronic camera shake correction that performs correction by image processing (for example, see Patent Documents 1 to 3). .
静止画の手ブレを補正する手法としては、手ブレ検出をセンサで行って手ブレ補正を光学系駆動で行なう手法と、手ブレ検出をセンサで行って手ブレ補正を画像処理で行なう手法と、手ブレ検出を画像処理で行って手ブレ補正を画像処理で行なう手法、の３通りの組み合わせが実用化されている。 There are two methods for correcting camera shake in still images: a method in which camera shake detection is performed by a sensor and camera shake correction is performed by an optical system drive, and a method in which camera shake detection is performed by a sensor and camera shake correction is performed by image processing. In addition, three combinations of a method of performing camera shake detection by image processing and performing camera shake correction by image processing have been put into practical use.
光学式手ブレ補正では、撮影レンズや可変頂角プリズムなどの撮影光学系や撮像素子を駆動して手ブレを補正する。これにより、撮影された画像に特別な後処理を加えること無く高い補正性能を実現できるが、光学系や撮像素子の機構的制限により追従できる手ブレ量に限界がある。一方、電子式手ブレ補正の１つに、手ブレによる画像劣化の関数であるブレ軌跡（Point Spread Function）から復元フィルタＨ−１を求め、ブレ画像Ｐ’に畳み込む事で手ブレ画像を改善した補正画像Ｐを計算する画像復元式手ブレ補正がある。この画像復元式手ブレ補正によれば、撮影光学系や撮像素子を駆動する駆動メカニズムが無くても手ブレを軽減することができる。 In the optical camera shake correction, camera shake is corrected by driving a shooting optical system such as a shooting lens or a variable apex angle prism or an image pickup device. This makes it possible to achieve high correction performance without adding special post-processing to the photographed image, but there is a limit to the amount of camera shake that can be tracked due to mechanical limitations of the optical system and the image sensor. On the other hand, as one of the electronic image stabilization, a restoration filter H- 1 is obtained from a blur locus (Point Spread Function) that is a function of image degradation caused by camera shake, and the image is improved by convolution with the blur image P '. There is an image restoration type camera shake correction for calculating the corrected image P. According to this image restoration type camera shake correction, it is possible to reduce camera shake even without a driving mechanism for driving a photographing optical system and an image sensor.
そこで、手ブレ補正機能を向上すべく、光学式手ブレ補正と電子式手ブレ補正とを組み合わせた補正手法が幾つか提案されている（例えば、特許文献４〜６参照）。例えば、特許文献４に開示された発明では、光学式手ブレ補正と画像復元式手ブレ補正とを組み合わせた補正手法として、手ブレを補正するためのレンズの駆動目標位置と実際に当該レンズを駆動した実駆動位置との差（制御位置誤差）を求め、撮像された画像に対して制御位置誤差を加味した画像処理による画像復元を行なっている。 In order to improve the camera shake correction function, several correction methods combining optical camera shake correction and electronic camera shake correction have been proposed (see, for example, Patent Documents 4 to 6). For example, in the invention disclosed in Patent Document 4, as a correction method that combines optical camera shake correction and image restoration type camera shake correction, a lens drive target position for correcting camera shake and an actual lens are used. A difference (control position error) from the driven actual drive position is obtained, and image restoration is performed by image processing in which the control position error is added to the captured image.
特許文献４に開示された補正手法では、光学式手ブレ補正が狙い通りに行なわれない場合も含めて、常にブレ補正効果を高め、確実に像ブレを補正することを解決課題としている。通常、手ブレを検出するセンサの出力にはドリフト成分が含まれ、このドリフトの影響により手ブレ検出値に誤差が生じる。特許文献４では、この誤差を光学式手ブレ補正における制御位置誤差として、画像復元式手ブレ補正により補完している。 In the correction method disclosed in Patent Document 4, it is a problem to always improve the blur correction effect and reliably correct the image blur, including the case where the optical camera shake correction is not performed as intended. Normally, a drift component is included in the output of a sensor that detects camera shake, and an error occurs in the camera shake detection value due to the influence of this drift. In Patent Document 4, this error is complemented by image restoration type camera shake correction as a control position error in optical camera shake correction.
しかし、特許文献４に開示された発明は、撮影光学系や撮像素子の駆動可能な範囲を超える手ブレが生じた場合にも対応できるものではない。通常、光学式手ブレ補正において、レンズや撮像素子などを駆動できる距離や領域は予め限定され、それを超えて駆動できないように設定されている。露光期間中にこの駆動距離や領域で対応できる手ブレ量を超える手ブレが発生してしまった場合、特許文献４に開示された発明では、手ブレ全体量のうち撮影光学系の駆動範囲を超える部分については、光学式手ブレ補正又は画像復元式手ブレ補正の何れの方法によっても手ブレを補正することはできない。 However, the invention disclosed in Patent Document 4 cannot cope with camera shake that exceeds the driveable range of the imaging optical system and the image sensor. Usually, in optical camera shake correction, a distance and a region where a lens, an image sensor, and the like can be driven are limited in advance, and are set so as not to be able to be driven beyond that. When a camera shake exceeding the amount of camera shake that can be dealt with in this driving distance or region occurs during the exposure period, the invention disclosed in Patent Document 4 sets the driving range of the photographing optical system out of the total amount of camera shake. For the excess part, the camera shake cannot be corrected by either the optical camera shake correction or the image restoration type camera shake correction.
本発明は、上記問題点を解決するために成されたものであり、その目的は、撮影光学系や撮像部の駆動可能な範囲を超える手ブレが生じた場合も含めて、ブレ補正効果を高めて像ブレを補正する撮像装置を提供することである。 The present invention has been made to solve the above-mentioned problems, and its purpose is to provide a blur correction effect including a case where a camera shake exceeding the driveable range of the imaging optical system and the imaging unit occurs. An object of the present invention is to provide an imaging device that enhances and corrects image blur.
本発明の第１の特徴は、撮像面に被写体像を結像させる撮影光学系と、予め設定された露光期間にわたって撮像面に結像された被写体像を撮像する撮像部とを備える撮像装置において、露光期間中における撮像装置のブレを検出するブレ検出部と、撮像部又は撮影光学系の一部（補正光学系）を駆動する補正光学系駆動部と、ブレ検出部が検出する撮像装置のブレに基づいて補正光学系駆動部を制御することにより、撮像部又は撮影光学系の一部を撮像装置のブレに追従させる駆動制御部と、補正光学系が駆動限界に達したか否かを判断する駆動限界判断部と、補正光学系が駆動限界に達している場合、駆動限界に達している状態で補正光学系の駆動を停止する駆動停止部と、撮像装置のブレのうち駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理することによりブレを補正する画像復元部とを備える撮像装置であることを要旨とする。 According to a first aspect of the present invention, there is provided an imaging apparatus including: a photographing optical system that forms a subject image on an imaging surface; and an imaging unit that images the subject image formed on the imaging surface over a preset exposure period. A blur detection unit that detects a blur of the imaging device during the exposure period, a correction optical system drive unit that drives a part of the imaging unit or the photographing optical system (correction optical system), and an imaging device that the blur detection unit detects By controlling the correction optical system driving unit based on the blur, a drive control unit that causes the imaging unit or a part of the photographing optical system to follow the blur of the imaging device, and whether the correction optical system has reached the driving limit. A drive limit determination unit for determining, a drive stop unit for stopping the drive of the correction optical system when the correction optical system has reached the drive limit, and a drive limit among the blurs of the imaging device. Subject image And summarized in that an imaging apparatus and an image restoring section for correcting the blur by image processing of the object image on the basis of the trajectory of blur.
補正光学系が駆動限界に達している場合、駆動停止部が駆動限界において補正光学系の駆動を停止し、画像復元部が撮像装置のブレのうち駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理することによりブレを電子的に補正する。つまり、光学式の補正限界を超えたら光学式ブレ補正を一旦停止して、以降のブレを計測して画像復元式で補正する。ブレが駆動限界内に戻ったら補正光学系の駆動を再開し、駆動制御部によって同様な処理を継続することで、駆動限界を超える部分について画像復元式で補正する。 When the correction optical system has reached the drive limit, the drive stop unit stops driving the correction optical system at the drive limit, and the image restoration unit causes blurring of the subject image for the portion of the image pickup device blur that exceeds the drive limit. Blur is corrected electronically by subjecting the subject image to image processing based on the trajectory. That is, when the optical correction limit is exceeded, the optical blur correction is temporarily stopped, and the subsequent blur is measured and corrected by the image restoration method. When the blur returns to within the drive limit, the drive of the correction optical system is resumed, and the same process is continued by the drive control unit, so that the portion exceeding the drive limit is corrected by the image restoration formula.
光学式手ブレ補正単体では、機構的制限により補正できる手ブレ量に限界が生じる。一般に露光時間の長い長秒露光になると発生する手ブレ量が大きくなり、手ブレが補正限界を超えるとそれ以上の補正は出来なかった。第１の特徴によれば、光学式の補正限界を超えるブレを補正でき、従来の補正方法では難しかった長秒露光時の手ブレ補正が可能になる。 The optical camera shake correction unit alone has a limit in the amount of camera shake that can be corrected due to mechanical limitations. In general, when the exposure time is long and the exposure time is long, the amount of camera shake generated increases, and when the camera shake exceeds the correction limit, further correction cannot be performed. According to the first feature, it is possible to correct the blur exceeding the optical correction limit, and it is possible to perform the camera shake correction at the time of long-second exposure which is difficult with the conventional correction method.
本発明の第２の特徴は、撮像面に被写体像を結像させる撮影光学系と、予め設定された露光期間にわたって撮像面に結像された被写体像を撮像する撮像部とを備える撮像装置において、露光期間中における撮像装置のブレを検出するブレ検出部と、撮像部又は撮影光学系の一部を駆動する補正光学系駆動部と、ブレ検出部が検出する撮像装置のブレの一部分のみに基づいて補正光学系駆動部を制御することにより、撮像部又は撮影光学系の一部を撮像装置のブレの一部分に限定して追従させる駆動制御部と、撮像装置のブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理することによりブレを補正する画像復元部とを備える撮像装置であることを要旨とする。 According to a second aspect of the present invention, there is provided an imaging apparatus including: a photographing optical system that forms a subject image on an imaging surface; and an imaging unit that images a subject image formed on the imaging surface over a preset exposure period. A blur detection unit that detects a blur of the imaging device during the exposure period, a correction optical system drive unit that drives a part of the imaging unit or the photographing optical system, and only a part of the blur of the imaging device that is detected by the blur detection unit. Based on the control of the correction optical system driving unit, a part of the imaging unit or the photographing optical system is limited to a part of the blur of the imaging device and the remaining part of the blur of the imaging device is subject. The gist of the present invention is an image pickup apparatus including an image restoration unit that corrects blurring by subjecting a subject image to image processing based on an image blurring trajectory.
駆動制御部が撮像装置のブレの一部分のみに基づいて補正光学系駆動部を制御することにより、補正光学系を撮像装置のブレの一部分に限定して追従させる。そして、画像復元部が撮像装置のブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理することによりブレの残り部分を補正する。つまり、光学式補正を行う際に補正を完全には行わず、計測されたブレの一定割合を補正せずに残しブレとする。露光終了後に、補正せずに残したブレを画像復元によりブレ補正する。 The drive control unit controls the correction optical system drive unit based only on a part of the shake of the image pickup device, thereby causing the correction optical system to follow only a part of the shake of the image pickup device. Then, the image restoration unit corrects the remaining blur portion by performing image processing on the subject image based on the blur locus of the subject image for the remaining blur portion of the imaging apparatus. That is, when optical correction is performed, the correction is not performed completely, and a fixed ratio of the measured blur is left as it is without correction. After the exposure is completed, the blur remaining without correction is corrected by image restoration.
これにより、光学補正量が小さくなり、機構的な補正限界を仮想的に拡大することができる。また、ＭＴＦや色収差など画質面の課題が顕著になる補正限界付近を利用する確率が減少する。 As a result, the optical correction amount is reduced, and the mechanical correction limit can be virtually expanded. In addition, the probability of using the vicinity of the correction limit where the problem of image quality such as MTF and chromatic aberration becomes significant decreases.
第２の特徴において、撮像装置のブレの一部分及び残り部分は所定の比率でそれぞれ設定しても構わない。 In the second feature, a part of the blur and the remaining part of the imaging device may be set at a predetermined ratio.
これにより、光学式補正の開始から残しブレを形成することができるので、光学式補正の負担が軽減され、機構的な補正限界を仮想的に拡大し、ＭＴＦや色収差など画質面の課題が顕著になる補正限界付近を利用する確率が減少する。 As a result, it is possible to form a blurring from the start of optical correction, thereby reducing the burden of optical correction, virtually expanding the mechanical correction limit, and noticeable image quality issues such as MTF and chromatic aberration. The probability of using the vicinity of the correction limit is reduced.
第２の特徴において、撮像装置のブレの一部分は撮像装置のブレの等速度成分であり、撮像装置のブレの残り部分は撮像装置のブレの加速度成分であっても構わない。 In the second feature, a part of blur of the imaging device may be a constant velocity component of blur of the imaging device, and a remaining part of blur of the imaging device may be an acceleration component of blur of the imaging device.
画像復元式ブレ補正では、加速度的なブレの方が等速度的なブレよりも補正効果が高い。なぜなら、等速度的なブレでは、画像の中の一定の空間周波数成分が完全に欠落してしまうことにより、画像復元が出来なくなるからである。この現象を避けるため、等速的なブレを光学式ブレ補正により補正し、加速度的なブレについては、光学式ブレ補正を抑制して、「残しブレ」を作るように制御し、残しブレを画像復元式ブレ補正により補正する。 In image restoration type blur correction, acceleration blur has a higher correction effect than constant velocity blur. This is because constant speed blurring causes a certain spatial frequency component in the image to be completely lost, thereby making it impossible to restore the image. In order to avoid this phenomenon, constant-speed blur is corrected by optical blur correction, and acceleration blur is controlled to suppress the optical blur correction to create “remaining blur”. Correct by image restoration type image stabilization.
これにより、画像復元による補正がより効果的になる。また、光学補正量が小さくなり、機構的な補正限界を仮想的に拡大することができる。また、ＭＴＦや色収差など画質面の劣化が顕著になる補正限界付近を利用する確率が減少する。 Thereby, correction by image restoration becomes more effective. Further, the optical correction amount is reduced, and the mechanical correction limit can be virtually enlarged. In addition, the probability of using the vicinity of the correction limit where the deterioration of the image quality such as MTF and chromatic aberration becomes remarkable decreases.
第１及び第２の特徴において、撮像装置は、全露光時間を所定の分割数で分割する露光分割部と、撮像部が繰り返し撮像を行って生成した分割数の被写体像を合成する画像合成部とを更に備え、露光期間は全露光時間を所定の分割数で分割したうちの少なくとも１つであり、画像復元部は、分割数の被写体像についてブレを補正し、画像合成部は、画像復元部によりブレが補正された分割数の被写体像を合成しても構わない。 In the first and second features, the imaging apparatus includes an exposure division unit that divides the total exposure time by a predetermined number of divisions, and an image synthesis unit that synthesizes the subject images of the division number generated by the imaging unit repeatedly imaging. The exposure period is at least one of the total exposure times divided by a predetermined number of divisions, the image restoration unit corrects blurring for the number of subject images, and the image synthesis unit performs image restoration. The subject images of the number of divisions whose blurring has been corrected by the unit may be synthesized.
露光分割部は全露光時間を所定の分割数で分割し、撮像部は繰り返し撮像を行って分割数の被写体像を生成し、駆動制御部は、補正光学系を撮像装置のブレに追従させ、画像復元部は分割数の被写体像についてブレを補正し、画像合成部は、画像復元部によりブレが補正された分割数の被写体像を合成する。 The exposure dividing unit divides the total exposure time by a predetermined number of divisions, the imaging unit repeatedly captures images to generate the number of subject images, and the drive control unit causes the correction optical system to follow the blur of the imaging device, The image restoration unit corrects the blur for the number of divided subject images, and the image composition unit synthesizes the number of the subject images for which the shake is corrected by the image restoration unit.
これにより、光学式ブレ補正と画像復元式ブレ補正と画像加算式ブレ補正との総ての効果を足し合わせることができるため、光学式の補正限界を超えるブレ補正が可能となり、補正範囲及び補正性能を更に向上させることができる。また、画像加算部分でロール方向の手ぶれを補正できる。さらに、長秒露光時のセンサ計測の誤差の影響も補正する事が出来る。これにより、従来の補正方法では難しかった長秒露光時の手ブレ補正が可能になる。 As a result, it is possible to add all the effects of optical blur correction, image restoration blur correction, and image addition blur correction, so that blur correction exceeding the optical correction limit is possible, and the correction range and correction The performance can be further improved. Further, camera shake in the roll direction can be corrected at the image addition portion. Furthermore, it is possible to correct the influence of sensor measurement errors during long-second exposure. This makes it possible to correct camera shake during long-second exposure, which was difficult with conventional correction methods.
本発明によれば、撮影光学系や撮像部の駆動可能な範囲を超える手ブレが生じた場合も含めて、ブレ補正効果を高めて像ブレを補正する撮像装置を提供することができる。 According to the present invention, it is possible to provide an imaging apparatus that enhances the blur correction effect and corrects the image blur including the case where the camera shake exceeding the driveable range of the photographing optical system and the imaging unit occurs.
以下図面を参照して、本発明の実施の形態を説明する。図面の記載において同一又は類似な部分には同一又は類似な符号を付している。 Embodiments of the present invention will be described below with reference to the drawings. In the description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.
（第１の実施の形態）
図１を参照して、本発明の第１の実施の形態に係わる撮像装置１ａの構成を説明する。撮像装置１ａは、撮像面に被写体像を結像させる撮影光学系１１と、予め設定された露光期間にわたって撮像面に結像された被写体像を撮像する撮像部１２と、露光期間中における撮像装置１ａのブレを検出するブレ検出部の一例としての角速度センサ１３と、センサからの出力信号を補正するセンサ信号補正部１４と、撮像部１２又は撮影光学系１１の一部を駆動する補正光学系駆動部１５と、角速度センサ１３が検出する撮像装置１ａのブレに基づいて補正光学系駆動部１５を制御する駆動制御部１６と、撮像部１２又は撮影光学系１１の一部が駆動限界に達したか否かを判断する駆動限界判断部１７と、撮像部１２又は撮影光学系１１の一部が駆動限界に達している場合、駆動限界に達している状態で撮像部１２又は撮影光学系１１の一部の駆動を停止する駆動停止部１８と、撮像装置１ａのブレのうち撮像部１２又は撮影光学系１１の一部の駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する画像復元部１９とを備える。
(First embodiment)
With reference to FIG. 1, the structure of the imaging device 1a concerning the 1st Embodiment of this invention is demonstrated. The imaging device 1a includes a photographic optical system 11 that forms a subject image on an imaging surface, an imaging unit 12 that images a subject image formed on the imaging surface over a preset exposure period, and an imaging device during the exposure period. An angular velocity sensor 13 as an example of a blur detection unit that detects a blur of 1a, a sensor signal correction unit 14 that corrects an output signal from the sensor, and a correction optical system that drives a part of the imaging unit 12 or the imaging optical system 11 The drive unit 15, the drive control unit 16 that controls the correction optical system drive unit 15 based on the shake of the imaging device 1 a detected by the angular velocity sensor 13, and a part of the imaging unit 12 or the imaging optical system 11 reach the drive limit. When the drive limit determination unit 17 for determining whether or not the image pickup unit 12 or the photographic optical system 11 has reached the drive limit, the image pickup unit 12 or the photographic optical system 11 is in a state of reaching the drive limit. of A driving stop unit 18 that stops the driving of the imaging unit, and a portion of the blur of the imaging device 1a that exceeds the driving limit of a part of the imaging unit 12 or the imaging optical system 11, based on the blur trajectory of the subject image. And an image restoration unit 19 for image processing.
以後、補正光学系駆動部１５が駆動する撮像部１２又は撮影光学系１１の一部を「補正光学系」と呼ぶ。 Hereinafter, a part of the imaging unit 12 or the photographing optical system 11 driven by the correction optical system driving unit 15 is referred to as a “correction optical system”.
撮影光学系１１は、カメラの被写体と撮像部１２との間の光路上に配置された、光を集中・発散・反射・屈折させるためのレンズ・反射鏡・プリズムなどの組み合わせからなる。撮像部１２は、被写体像を結像させる撮像面を有し、デジタルカメラの場合、光信号を電気信号に変える撮像素子が例示される。撮像素子としては、電荷結合素子（ＣＣＤ）や、ＣＭＯＳイメージセンサを用いることができる。 The photographing optical system 11 includes a combination of a lens, a reflector, a prism, and the like, which are arranged on an optical path between a camera subject and the imaging unit 12 for concentrating / diverging / reflecting / refracting light. The imaging unit 12 has an imaging surface on which a subject image is formed. In the case of a digital camera, an imaging element that converts an optical signal into an electrical signal is exemplified. As the imaging device, a charge coupled device (CCD) or a CMOS image sensor can be used.
角速度センサ１３は、撮影光学系１１及び撮像部１２に対して固定され、撮像装置１ａのパン（横方向）、チルト（縦、うなずき方向）の振動を検出することにより手ブレ等を計測する。パン（横方向）、チルト（縦、うなずき方向）の振動をそれぞれ異なるセンサを用いて検出しても、１つのセンサにより両方向の振動を検出してもいずれでも構わない。なお、ブレ検出部としては、角速度センサ１３の他に、角加速度センサ、角度センサ、速度センサ、或いは加速度センサを用いても構わない。 The angular velocity sensor 13 is fixed with respect to the imaging optical system 11 and the imaging unit 12, and measures camera shake and the like by detecting vibrations in the pan (lateral direction) and tilt (vertical and nodding directions) of the imaging apparatus 1a. Either pan (horizontal direction) or tilt (vertical or nodding direction) vibrations may be detected using different sensors, or a single sensor may be used to detect bi-directional vibrations. In addition to the angular velocity sensor 13, an angular acceleration sensor, an angle sensor, a velocity sensor, or an acceleration sensor may be used as the shake detection unit.
角速度センサ１３の出力には、温度ドリフトがあり、温度に応じた補正が必要である。また、低周波の温度ドリフトを取り除くため、ＨＰＦ（高域通過フィルタ）を用いた場合は、角速度センサの出力に露光前・露光中の振動による残留、減衰が発生するため、振動に応じた補正が必要である。さらにズームの度合いに応じて手ブレ角に対する撮像素子上での手ブレ量が異なるため、レンズ焦点距離に応じて、ブレ量を補正する。 There is a temperature drift in the output of the angular velocity sensor 13, and correction according to the temperature is necessary. Also, when using an HPF (high-pass filter) to remove low-frequency temperature drift, the output of the angular velocity sensor remains and attenuates due to vibration before and during exposure, so correction according to vibration is required. Furthermore, since the amount of camera shake on the image sensor with respect to the camera shake angle varies depending on the degree of zoom, the amount of blur is corrected according to the lens focal length.
補正光学系駆動部１５は、撮像装置１ａのブレに対して補正光学系を追従させることによりブレを光学的に補正する光学式ブレ補正を実施するために、補正光学系を駆動する。補正光学系の動きは、駆動制御部１６により制御される。すなわち、駆動制御部１６は、撮像装置１ａのブレに基づいて補正光学系駆動部１５を制御することにより、補正光学系を撮像装置のブレに追従させる。 The correction optical system driving unit 15 drives the correction optical system in order to perform optical blur correction that optically corrects blur by causing the correction optical system to follow the blur of the imaging device 1a. The movement of the correction optical system is controlled by the drive control unit 16. That is, the drive control unit 16 controls the correction optical system driving unit 15 based on the shake of the imaging device 1a, thereby causing the correction optical system to follow the shake of the imaging device.
ここで、光学式ブレ補正についてその種類と特徴を説明する。光学式ブレ補正は、レンズあるいは撮像素子を撮像装置１ａのブレとは逆方向に駆動することでブレの影響を軽減する補正手法である。光学式ブレ補正方式を２つに大別すると、レンズシフト方式と、撮像素子（ＣＣＤ、ＣＭＯＳイメージセンサ）シフト方式とに分けられる。レンズシフト方式には、撮影光学系１１に含まれる一部のレンズを光軸に対して垂直方向に移動させて光軸を偏心する方式と、可変頂角プリズム方式とが含まれる。前者の場合、防振時に偏心収差が発生するため、画面内に解像度の差が現れ、防振のため連続して光軸を上下左右に偏心させた場合、特に高周波成分を多く含む被写体を撮影した場合に画面の周辺部で解像度が時間的に変化して見える。可変頂角プリズム方式（後者）の場合、構造上補正可能な画像ブレの角度が小さく、かつプリズムを用いているためその角度が増すにつれ、色収差が発生する。撮像素子シフト方式は、撮像素子自体をブレとは逆方向に駆動することによりブレを補正する方式であるが、レンズシフト方式と同様に、駆動量の増加と共に画質の劣化が生じてしまう。このように、光軸からのズレが大きくなる事により補正光学系の駆動量も多くなり、色収差の発生や空間周波数（ＭＴＦ）の劣化など、画質面での課題が駆動限界域付近で顕著になる。 Here, the types and characteristics of optical blur correction will be described. The optical blur correction is a correction method that reduces the influence of blur by driving a lens or an image sensor in a direction opposite to the blur of the imaging device 1a. The optical blur correction method is roughly classified into two types: a lens shift method and an image sensor (CCD, CMOS image sensor) shift method. The lens shift method includes a method of decentering the optical axis by moving some lenses included in the photographing optical system 11 in a direction perpendicular to the optical axis, and a variable apex angle prism method. In the former case, decentration aberrations occur during image stabilization, resulting in a difference in resolution within the screen. When the optical axis is continuously decentered vertically and horizontally for image stabilization, a subject that contains a lot of high-frequency components is taken. In this case, the resolution appears to change with time in the periphery of the screen. In the case of the variable apex angle prism type (the latter), the image blur angle that can be corrected structurally is small, and since the prism is used, chromatic aberration occurs as the angle increases. The image sensor shift method is a method of correcting blur by driving the image sensor itself in the direction opposite to the blur. However, as with the lens shift method, image quality deteriorates as the drive amount increases. As described above, the displacement from the optical axis increases, so that the driving amount of the correction optical system also increases, and problems in image quality such as the occurrence of chromatic aberration and the deterioration of spatial frequency (MTF) are prominent in the vicinity of the drive limit region. Become.
なお、レンズシフト方式の場合、補正光学系駆動部１５は撮影光学系１１の一部（レンズやプリズム）を補正光学系として駆動し、撮像素子シフト方式の場合、補正光学系駆動部１５は撮像部１２を補正光学系として駆動する。 In the case of the lens shift method, the correction optical system drive unit 15 drives a part (lens or prism) of the photographing optical system 11 as the correction optical system. In the case of the image pickup device shift method, the correction optical system drive unit 15 picks up an image. The unit 12 is driven as a correction optical system.
露光期間中における撮像装置１ａのブレが補正光学系の駆動限界を超えてしまう場合、この超える部分について、上記の光学式ブレ補正を適用することはできない。そこで、駆動限界判断部１７は、補正光学系の位置、補正光学系の駆動量、或いは補正光学系駆動部１５の制御量を検出する手段を備え、補正光学系の位置、補正光学系の駆動量、或いは補正光学系駆動部１５の制御量から、補正光学系が駆動限界に達したか否かを判断する。そして、駆動停止部１８は、補正光学系が駆動限界に達している場合、駆動限界に達している状態で補正光学系の駆動を停止する。これにより、補正光学系は駆動限界で固定される。そして、画像復元部１９が、撮像装置１ａのブレのうち補正光学系の駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する。被写体像のブレの軌跡に基づいて被写体像を画像処理することにより被写体像のブレを電子的に補正することができる。 When the blur of the imaging device 1a during the exposure period exceeds the drive limit of the correction optical system, the optical blur correction described above cannot be applied to this excess portion. Therefore, the drive limit determination unit 17 includes means for detecting the position of the correction optical system, the drive amount of the correction optical system, or the control amount of the correction optical system drive unit 15, and the position of the correction optical system and the drive of the correction optical system. It is determined from the amount or the control amount of the correction optical system drive unit 15 whether or not the correction optical system has reached the drive limit. Then, when the correction optical system has reached the drive limit, the drive stop unit 18 stops the drive of the correction optical system in a state where the drive limit has been reached. Thereby, the correction optical system is fixed at the drive limit. Then, the image restoration unit 19 performs image processing on the subject image based on the blur locus of the subject image for the portion of the blur of the imaging device 1a that exceeds the drive limit of the correction optical system. By subjecting the subject image to image processing based on the subject image blur locus, the subject image blur can be corrected electronically.
ここで、画像復元部１９が行なう画像復元式ブレ補正について説明する。画像復元式ブレ補正は、露光期間中のブレ軌跡を求めておき、ブレによる画像劣化の逆フィルタをフィルタリングすることでブレ画像を改善する補正手法である。具体的には、まず、光学的ブレ補正を停止した後、つまり補正光学系の位置が駆動限界に到達した後の被写体像の動きベクトルを求める。動きベクトルから撮像装置１ａのブレによる画像劣化の関数であるブレ軌跡（ＰＳＦ）を求める。ブレ軌跡から復元フィルタ（逆フィルタ）Ｈ−１を求める。そして、（１）式に従って、ブレ画像Ｐ’に復元フィルタＨ−１を畳み込む事でブレを改善した補正画像Ｐを計算する。復元フィルタＨ−１として、５１×５１のＦＩＲフィルタを用いることができる。 Here, the image restoration type blur correction performed by the image restoration unit 19 will be described. The image restoration type blur correction is a correction method for improving a blur image by obtaining a blur locus during an exposure period and filtering an inverse filter for image degradation due to blur. Specifically, first, after the optical blur correction is stopped, that is, the motion vector of the subject image after the position of the correction optical system reaches the drive limit is obtained. A blur locus (PSF), which is a function of image degradation due to the blur of the imaging device 1a, is obtained from the motion vector. A restoration filter (inverse filter) H −1 is obtained from the blur locus. Then, according to the equation (1), a correction image P in which the blur is improved by calculating the restoration filter H −1 on the blur image P ′ is calculated. As the restoration filter H- 1 , a 51 × 51 FIR filter can be used.
Ｐ ＝ Ｈ−１ ＊ Ｐ’ ・・・（１） P = H− 1 * P ′ (1)
図２を参照して、図１の撮像装置１ａにおけるブレ補正処理の流れを説明する。 With reference to FIG. 2, the flow of blur correction processing in the imaging apparatus 1a of FIG. 1 will be described.
（イ）先ず、予め定められた露光期間が開始すると（Ｓ１１０）、角速度センサ１３が、撮像装置１ａのブレ検出を開始する（Ｓ１２０）。このブレ検出は、露光期間中継続的に実施される。センサ信号補正部１４が、角速度センサ１３の出力信号（センサ信号）を外気温やズームの度合いに基づいて補正する（Ｓ１３０）。駆動制御部１６は、撮像装置１ａのブレに基づいて補正光学系駆動部１５を制御することにより、補正光学系を駆動して撮像装置１ａのブレに追従させる（Ｓ１４０）。このとき、駆動限界判断部１７は、補正光学系の位置、補正光学系の駆動量、或いは補正光学系駆動部１５の制御量を検出している。 (A) First, when a predetermined exposure period starts (S110), the angular velocity sensor 13 starts blur detection of the imaging device 1a (S120). This blur detection is continuously performed during the exposure period. The sensor signal correction unit 14 corrects the output signal (sensor signal) of the angular velocity sensor 13 based on the outside air temperature and the degree of zoom (S130). The drive control unit 16 controls the correction optical system drive unit 15 based on the shake of the imaging device 1a, thereby driving the correction optical system to follow the shake of the imaging device 1a (S140). At this time, the drive limit determination unit 17 detects the position of the correction optical system, the drive amount of the correction optical system, or the control amount of the correction optical system drive unit 15.
（ロ）駆動限界判断部１７は、補正光学系が駆動限界に達したか否かを判断する。達していなければ（Ｓ１５０にてＮｏ）、Ｓ１２０段階へ戻り、ブレ検出に基づく補正光学系の駆動を継続する。一方、駆動限界に達していれば（Ｓ１５０にてＹｅｓ）、Ｓ１６０段階に進み、駆動停止部１８は補正光学系の駆動を停止する。また、これと同時に、補正されたセンサ信号の記録を開始する。センサ信号を記録しつつ、露光期間中にブレが駆動限界内に戻るか否かを監視する。露光期間中にブレが駆動限界内に入った場合（Ｓ１６５にてＹｅｓ）、Ｓ１２０段階に戻り、ブレ検出に基づく補正光学系の駆動を再開する。露光期間中にブレが駆動限界内に戻らなければ（Ｓ１６５にてＮｏ）、センサ信号の記録を、露光期間が終了するまで継続する。 (B) The drive limit determination unit 17 determines whether or not the correction optical system has reached the drive limit. If not reached (No in S150), the process returns to Step S120, and the driving of the correction optical system based on the blur detection is continued. On the other hand, if the drive limit has been reached (Yes in S150), the process proceeds to step S160, and drive stop unit 18 stops driving the correction optical system. At the same time, recording of the corrected sensor signal is started. While recording the sensor signal, it is monitored whether or not the blur returns to the drive limit during the exposure period. If the blur enters the drive limit during the exposure period (Yes in S165), the process returns to step S120, and the drive of the correction optical system based on the blur detection is resumed. If the blur does not return to within the drive limit during the exposure period (No in S165), recording of the sensor signal is continued until the exposure period ends.
（ハ）露光期間が終了した後（Ｓ１７０）、画像復元部１９が撮像装置１ａのブレのうち補正光学系の駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する（Ｓ１８０）。 (C) After the exposure period ends (S170), the image restoration unit 19 performs image processing on the subject image based on the blur locus of the subject image for the portion of the blur of the imaging device 1a that exceeds the drive limit of the correction optical system. (S180).
なお、Ｓ１６０段階において、センサ信号の記録を開始した場合について説明したが、それ以前から、例えば露光期間開始から記録し始めても構わない。 In addition, although the case where the recording of the sensor signal is started in the step S160 has been described, the recording may be started from the beginning of the exposure period, for example, before that.
以上説明したように、本発明の第１の実施の形態によれば以下の作用効果が得られる。 As described above, according to the first embodiment of the present invention, the following operational effects can be obtained.
補正光学系が駆動限界に達している場合、駆動停止部１８が駆動限界において補正光学系の駆動を停止し、画像復元部１９が撮像装置１ａのブレのうち駆動限界を超える部分について、被写体像のブレの軌跡（ＰＳＦ）に基づいて被写体像を画像処理することによりブレを電子的に補正する。つまり、光学式の補正限界を超えたら光学式ブレ補正を停止して、以降のブレを計測して画像復元式で補正する。 When the correction optical system has reached the drive limit, the drive stop unit 18 stops driving the correction optical system at the drive limit, and the image restoration unit 19 detects the subject image for the portion of the blur of the imaging device 1a that exceeds the drive limit. The blur is electronically corrected by image processing the subject image based on the blur locus (PSF). That is, when the optical correction limit is exceeded, the optical blur correction is stopped, and the subsequent blur is measured and corrected by the image restoration method.
光学式手ブレ補正単体では、機構的制限により補正できる手ブレ量に限界が生じる。一般に露光時間の長い長秒露光になると発生する手ブレ量が大きくなり、手ブレが補正限界を超えるとそれ以上の補正は出来なかった。撮像装置１ａによれば、光学式の補正限界を超えるブレを補正でき、従来の補正方法では難しかった長秒露光時の手ブレ補正が可能になる。 The optical camera shake correction unit alone has a limit in the amount of camera shake that can be corrected due to mechanical limitations. In general, when the exposure time is long and the exposure time is long, the amount of camera shake generated increases, and when the camera shake exceeds the correction limit, further correction cannot be performed. According to the imaging apparatus 1a, it is possible to correct blur exceeding the optical correction limit, and it is possible to correct camera shake during long-second exposure, which is difficult with the conventional correction method.
（第２の実施の形態）
図３を参照して、本発明の第２の実施の形態に係わる撮像装置１ｂの構成を説明する。撮像装置１ｂは、図１の撮像装置１ａに比べて、以下の点が異なる。
(Second Embodiment)
With reference to FIG. 3, the structure of the imaging device 1b concerning the 2nd Embodiment of this invention is demonstrated. The imaging device 1b differs from the imaging device 1a of FIG. 1 in the following points.
光学式ブレ補正を行う際にブレを完全に補正するのではなく、計測されたブレの一定割合を補正せずに「残しブレ」とする。露光期間終了後に、残しブレを画像復元式ブレ補正により補正する。具体的には、駆動制御部１６は、角速度センサ１３が検出する撮像装置１ｂのブレの一部分のみに基づいて補正光学系駆動部１５を制御する。これにより、補正光学系を撮像装置１ｂのブレの一部分に限定して追従させている。そして、画像復元部１９は、撮像装置１ｂのブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する。また、撮像装置１ｂは駆動限界判断部１７及び駆動停止部１８を備えなくてもよい。 Rather than correcting the blur completely when performing optical blur correction, a fixed ratio of the measured blur is not corrected, and “remaining blur” is set. After the exposure period, the remaining blur is corrected by image restoration type blur correction. Specifically, the drive control unit 16 controls the correction optical system drive unit 15 based only on a part of the blur of the imaging device 1b detected by the angular velocity sensor 13. As a result, the correction optical system is made to follow only a part of the blur of the image pickup apparatus 1b. Then, the image restoration unit 19 performs image processing on the subject image based on the blur locus of the subject image with respect to the remaining blur of the imaging device 1b. Further, the imaging apparatus 1b may not include the drive limit determination unit 17 and the drive stop unit 18.
したがって、撮像装置１ｂは、図３に示すように、撮像面に被写体像を結像させる撮影光学系１１と、予め設定された露光期間にわたって撮像面に結像された被写体像を撮像する撮像部１２と、露光期間中における撮像装置１ｂのブレを検出するブレ検出部の一例としての角速度センサ１３と、センサからの出力信号を補正するセンサ信号補正部１４と、補正光学系を駆動する補正光学系駆動部１５と、角速度センサ１３が検出する撮像装置１ｂのブレの一部分のみに基づいて補正光学系駆動部１５を制御することにより、補正光学系を撮像装置１ｂのブレの一部分に限定して追従させる駆動制御部１６と、撮像装置１ｂのブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理することにより被写体像のブレを電子的に補正する画像復元部１９とを備える。 Therefore, as shown in FIG. 3, the imaging device 1b includes a photographing optical system 11 that forms a subject image on the imaging surface, and an imaging unit that captures the subject image formed on the imaging surface over a preset exposure period. 12, an angular velocity sensor 13 as an example of a blur detection unit that detects a blur of the imaging device 1b during the exposure period, a sensor signal correction unit 14 that corrects an output signal from the sensor, and correction optics that drives the correction optical system By controlling the correction optical system drive unit 15 based only on a part of the blur of the image pickup apparatus 1b detected by the system drive unit 15 and the angular velocity sensor 13, the correction optical system is limited to a part of the shake of the image pickup apparatus 1b. The subject image blur is electronically processed by subjecting the subject image to image processing based on the blur locus of the subject image with respect to the drive control unit 16 to be followed and the remaining blur of the imaging device 1b. And an image restoring section 19 for correcting the.
図９を参照して、第２の実施の形態における、撮像装置のブレ全体と光学式ブレ補正により補正されるブレ量との関係を説明する。図９の実線は、センサ信号補正部１４により補正された後の撮像装置のブレ全体量を示し、図９の点線が示す「光学式補正」は、駆動制御部１６が補正光学系を追従させることにより光学的に補正される撮像装置１ｂのブレ量を示す。光学的に補正されるブレの一部分と、電子的に補正されるブレの残り部分は、所定の比率でそれぞれ設定されている。 With reference to FIG. 9, the relationship between the overall blur of the imaging apparatus and the blur amount corrected by optical blur correction in the second embodiment will be described. The solid line in FIG. 9 indicates the total amount of shake of the imaging apparatus after correction by the sensor signal correction unit 14, and the “optical correction” indicated by the dotted line in FIG. 9 causes the drive control unit 16 to follow the correction optical system. This indicates the amount of blurring of the image pickup apparatus 1b that is optically corrected. A part of the blur that is optically corrected and a remaining part of the blur that is electronically corrected are set at a predetermined ratio.
その他の構成については、図１の撮像装置１ａと同じであるため、説明を省略する。 The other configuration is the same as that of the imaging device 1a in FIG.
図４を参照して、図３の撮像装置１ｂにおけるブレ補正処理の流れを説明する。 With reference to FIG. 4, the flow of blur correction processing in the imaging apparatus 1b of FIG. 3 will be described.
（イ）先ず、予め定められた露光時間が開始すると（Ｓ２１０）、角速度センサ１３が、撮像装置１ｂのブレ検出を開始する（Ｓ２２０）。このブレ検出は、露光期間中継続的に実施される。センサ信号補正部１４が、角速度センサ１３の出力信号（センサ信号）を外気温やズームの度合いに基づいて補正する（Ｓ２３０）。駆動制御部１６は、撮像装置１ｂのブレの一部分のみに基づいて補正光学系駆動部１５を制御することにより、補正光学系を駆動して撮像装置のブレの一部分に限定して追従させる（Ｓ２４０）。補正光学系の駆動と同時に、補正されたセンサ信号及び補正光学系の位置を記録する。なお、補正光学系の位置の代わりに、補正光学系の駆動量、或いは補正光学系駆動部１５の制御量を記録してもよい。 (A) First, when a predetermined exposure time starts (S210), the angular velocity sensor 13 starts blur detection of the imaging device 1b (S220). This blur detection is continuously performed during the exposure period. The sensor signal correction unit 14 corrects the output signal (sensor signal) of the angular velocity sensor 13 based on the outside air temperature and the degree of zoom (S230). The drive control unit 16 controls the correction optical system drive unit 15 based only on a part of the shake of the image pickup apparatus 1b, thereby driving the correction optical system so as to follow only a part of the shake of the image pickup apparatus (S240). ). Simultaneously with the driving of the correction optical system, the corrected sensor signal and the position of the correction optical system are recorded. Instead of the position of the correction optical system, the drive amount of the correction optical system or the control amount of the correction optical system drive unit 15 may be recorded.
（ロ）露光期間が終了したか否かを判断する。終了していなければ（Ｓ２５０にてＮｏ）、Ｓ２２０段階に戻る。一方、終了していれば（Ｓ２５０にてＹｅｓ）、Ｓ２６０段階に進み、画像復元部１９が撮像装置１ｂのブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する（Ｓ２６０）。 (B) It is determined whether or not the exposure period has ended. If not completed (No in S250), the process returns to S220. On the other hand, if completed (Yes in S250), the process proceeds to step S260, and the image restoration unit 19 performs image processing on the subject image of the remaining blur of the imaging device 1b based on the blur locus of the subject image ( S260).
ここで、補正光学系の駆動と同時に、補正されたセンサ信号及び補正光学系の位置を記録しているため、図９に示す実線と点線との差、即ち、「残しブレ」を求めることができる。したがって、画像復元部１９は、この残しブレを、撮像装置１ｂのブレの残り部分として電子的に補正することができる。 Here, since the corrected sensor signal and the position of the correction optical system are recorded simultaneously with the driving of the correction optical system, the difference between the solid line and the dotted line shown in FIG. it can. Therefore, the image restoration unit 19 can electronically correct this remaining blur as the remaining blur of the imaging device 1b.
以上説明したように、本発明の第２の実施の形態によれば以下の作用効果が得られる。 As described above, according to the second embodiment of the present invention, the following operational effects can be obtained.
駆動制御部１６が撮像装置１ｂのブレの一部分のみに基づいて補正光学系駆動部１５を制御することにより、補正光学系を撮像装置１ｂのブレの一部分に限定して追従させる。そして、画像復元部１９が撮像装置１ｂのブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理することによりブレの残り部分を補正する。つまり、光学式補正を行う際に補正を完全には行わず、計測されたブレの一定割合を補正せずに残しブレとする。露光終了後に、補正せずに残したブレを画像復元によりブレ補正する。 The drive control unit 16 controls the correction optical system drive unit 15 based only on a part of the shake of the image pickup apparatus 1b, so that the correction optical system is limited to a part of the shake of the image pickup apparatus 1b. Then, the image restoration unit 19 corrects the remaining blur portion by performing image processing on the subject image based on the blur locus of the subject image with respect to the remaining blur portion of the imaging device 1b. That is, when optical correction is performed, the correction is not performed completely, and a fixed ratio of the measured blur is left as it is without correction. After the exposure is completed, the blur remaining without correction is corrected by image restoration.
これにより、光学補正量が小さくなり、機構的な補正限界を仮想的に拡大することができる。また、ＭＴＦや色収差など画質面の課題が顕著になる補正限界付近を利用する確率が減少する。 As a result, the optical correction amount is reduced, and the mechanical correction limit can be virtually expanded. In addition, the probability of using the vicinity of the correction limit where the problem of image quality such as MTF and chromatic aberration becomes significant decreases.
図９に示すように、撮像装置１ｂのブレの一部分及び残り部分を所定の比率でそれぞれ設定する。これにより、光学式補正の開始から残しブレを形成することができるので、光学式補正の負担が軽減され、機構的な補正限界を仮想的に拡大し、ＭＴＦや色収差など画質面の課題が顕著になる補正限界付近を利用する確率が減少する。 As shown in FIG. 9, a part of the blur and the remaining part of the imaging device 1b are set at a predetermined ratio. As a result, it is possible to form a blurring from the start of optical correction, thereby reducing the burden of optical correction, virtually expanding the mechanical correction limit, and noticeable image quality issues such as MTF and chromatic aberration. The probability of using the vicinity of the correction limit is reduced.
（第２の実施の形態の変形例）
第２の実施の形態において、撮像装置１ｂのブレの一部分及び残り部分は、所定の比率でそれぞれ設定する場合に限定されない。変形例として、撮像装置１ｂのブレの一部分を撮像装置１ｂのブレの等速度成分とし、撮像装置１ｂのブレの残り部分を撮像装置１ｂのブレの加速度成分とする場合について説明する。
(Modification of the second embodiment)
In the second embodiment, a part of the blur and the remaining part of the imaging device 1b are not limited to being set at a predetermined ratio. As a modification, a case will be described in which a part of the blur of the imaging device 1b is the constant velocity component of the blur of the imaging device 1b and the remaining blur of the imaging device 1b is the acceleration component of the blur of the imaging device 1b.
図１０を参照して、第２の実施の形態の変形例における、撮像装置１ｂのブレ全体と光学式ブレ補正により補正されるブレ量との関係を説明する。図１０の実線は、センサ信号補正部１４により補正された後の撮像装置１ｂのブレ全体量を示し、図１０の点線が示す「光学式補正」は、駆動制御部１６が補正光学系を追従させることにより光学的に補正される撮像装置１ｂのブレ量を示す。光学式ブレ補正により補正されるブレの一部分は、ブレ全体量のうち等速度成分である。ブレ全体量が露光期間中において加速度成分をもって変化した場合、画像復元式ブレ補正により補正されるブレの残り部分は、ブレ全体量からブレの等速度成分を除いた、ブレの加速度成分に相当する。 With reference to FIG. 10, the relationship between the overall blur of the imaging device 1 b and the blur amount corrected by the optical blur correction in the modification of the second embodiment will be described. The solid line in FIG. 10 indicates the total blur amount of the image pickup apparatus 1b after correction by the sensor signal correction unit 14, and the “optical correction” indicated by the dotted line in FIG. 10 indicates that the drive control unit 16 follows the correction optical system. The amount of shake of the image pickup apparatus 1b that is optically corrected by performing the correction is shown. A part of the blur corrected by the optical blur correction is a constant velocity component of the total blur amount. When the overall blur amount changes with an acceleration component during the exposure period, the remaining blur corrected by the image restoration type blur correction corresponds to the blur acceleration component obtained by removing the constant velocity component of the blur from the total blur amount. .
その他、撮像装置の構成及びそのブレ補正処理の流れについては、図３及び図４と同じであるため、説明を省略する。 In addition, since the configuration of the imaging apparatus and the flow of the blur correction process are the same as those in FIGS. 3 and 4, the description thereof is omitted.
以上説明したように、本発明の第２の実施の形態の変形例によれば以下の作用効果が得られる。 As described above, according to the modification of the second embodiment of the present invention, the following operational effects can be obtained.
光学式ブレ補正により補正される撮像装置１ｂのブレの一部分は、撮像装置１ｂのブレの等速度成分とし、画像復元式ブレ補正により補正される撮像装置１ｂのブレの残り部分は、撮像装置１ｂのブレの加速度成分とする。画像復元式ブレ補正では、加速度的なブレの方が等速度的なブレよりも補正効果が高い。なぜなら、等速度的なブレでは、画像の中の一定の空間周波数成分が完全に欠落してしまうことにより、画像復元が出来なくなるからである。この現象を避けるため、等速的なブレを光学式ブレ補正により補正し、加速度的なブレについては、光学式ブレ補正を抑制して、「残しブレ」を作るように制御し、残しブレを画像復元式ブレ補正により補正する。このような補正を行うことで、画像復元による補正がより効果的になる。また、光学補正量が小さくなり、機構的な補正限界を仮想的に拡大することができる。また、ＭＴＦや色収差など画質面の劣化が顕著になる補正限界付近を利用する確率が減少する。 A part of the shake of the image pickup apparatus 1b corrected by the optical shake correction is the constant velocity component of the shake of the image pickup apparatus 1b, and the remaining part of the shake of the image pickup apparatus 1b corrected by the image restoration type shake correction is the image pickup apparatus 1b. This is the acceleration component of blurring. In image restoration type blur correction, acceleration blur has a higher correction effect than constant velocity blur. This is because constant speed blurring causes a certain spatial frequency component in the image to be completely lost, thereby making it impossible to restore the image. In order to avoid this phenomenon, constant-speed blur is corrected by optical blur correction, and acceleration blur is controlled to suppress the optical blur correction to create “remaining blur”. Correct by image restoration type image stabilization. By performing such correction, correction by image restoration becomes more effective. Further, the optical correction amount is reduced, and the mechanical correction limit can be virtually enlarged. In addition, the probability of using the vicinity of the correction limit where the deterioration of the image quality such as MTF and chromatic aberration becomes remarkable decreases.
（第３の実施の形態）
第１及び第２の実施の形態では、光学式ブレ補正と画像復元式ブレ補正とを組み合わせたブレ補正手法について説明したが、本発明はこれに限定されるものではない。上記のブレ補正手法に対して、更に、画像加算式ブレ補正を組み合わせても構わない。第３の実施の形態では、光学式ブレ補正、画像復元式ブレ補正、及び画像加算式ブレ補正を組み合わせたブレ補正手法について説明する。
(Third embodiment)
In the first and second embodiments, the blur correction method combining optical blur correction and image restoration type blur correction has been described, but the present invention is not limited to this. You may combine an image addition type | formula blurring correction | amendment further with said blurring correction method. In the third embodiment, a blur correction method that combines optical blur correction, image restoration blur correction, and image addition blur correction will be described.
図１１を参照して、光学式ブレ補正、画像復元式ブレ補正、及び画像加算式ブレ補正を組み合わせたブレ補正手法の概要を説明する。 With reference to FIG. 11, an outline of a blur correction method combining optical blur correction, image restoration blur correction, and image addition blur correction will be described.
電子式ブレ補正手法の１つとして、全露光時間Ｔを複数回（分割数）に分割し、ブレの小さい画像を複数枚撮影し、撮影後に複数枚の画像を重ね合わせることで、ブレを補正する画像加算式ブレ補正がある。一般に露光時間の長い長秒露光になると発生するブレ量が大きくなり、よく知られる光学式ブレ補正ではブレ量が補正機構の限界を超えてそれ以上のブレを補正できなくなる。そこで、ブレ量が光学式ブレ補正の補正限界に到達し難いように、全露光時間Ｔを複数の露光期間に分割し、各分割期間内での露光中に光学式ブレ補正を実施しつつ各露光期間が終了する度に補正光学系を初期位置に戻し、画像として必要な明るさが得られるまで露光を繰り返した後、複数の分割画像を加算することで、ブレが補正された画像を得ることができる。ここで、各分割期間中の光学式ブレ補正において、ブレ量が光学式ブレ補正の補正限界を超えた場合、光学式ブレ補正を停止し、以降のブレを記録する。露光終了後に復元補正した画像を加算する。全露光時間を複数回に分割することと、復元式手ブレ補正を分割画像に実施することで、補正光学系が駆動限界に到達する確率を減少させることができ、かつ、駆動限界を超えた場合でも画像復元式補正により補正範囲を拡大することができ、長時間露光に対応できる。 As one of the electronic blur correction methods, the total exposure time T is divided into multiple times (the number of divisions), multiple images with small blurring are shot, and the multiple images are superimposed after shooting to correct blurring. There is an image addition type blur correction. In general, when the exposure time is long and the exposure time is long, the amount of blurring that occurs increases, and with the well-known optical blur correction, the blur amount exceeds the limit of the correction mechanism, and further blurring cannot be corrected. Therefore, the total exposure time T is divided into a plurality of exposure periods so that the blur amount does not easily reach the correction limit of the optical blur correction, and the optical blur correction is performed during the exposure within each divided period. Each time the exposure period ends, the correction optical system is returned to the initial position, exposure is repeated until the necessary brightness is obtained as an image, and then a plurality of divided images are added to obtain an image in which blurring is corrected. be able to. Here, in the optical blur correction during each divided period, when the blur amount exceeds the correction limit of the optical blur correction, the optical blur correction is stopped and the subsequent blur is recorded. An image restored and corrected after the exposure is added. By dividing the total exposure time into multiple times and performing restoration type camera shake correction on the divided images, the probability that the correction optical system reaches the drive limit can be reduced, and the drive limit has been exceeded. Even in this case, the correction range can be expanded by image restoration type correction, and long exposure can be handled.
ここで、図１１に示すように、全露光時間Ｔをｎ個に分割した場合、ｎ個の分割期間ｔ１〜ｔｎが形成されるが、各分割期間が、第１及び第２の実施の形態における「露光期間」にそれぞれ対応している。したがって、第３の実施の形態では、各分割時間中における撮像装置のブレに対して光学式ブレ補正を実施して、ｎ個の分割画像を撮像する。各分割期間における復元フィルタを作成し、ｎ個の分割画像に対して、対応する復元フィルタを畳み込んで画像復元を行なう。その後、各分割画像を位置合わせした上で画像加算（画像合成）を行なう。 Here, as shown in FIG. 11, when the total exposure time T is divided into n, n divided periods t 1 to t n are formed, and each divided period corresponds to the first and second implementations. Respectively corresponding to the “exposure period”. Therefore, in the third embodiment, the optical shake correction is performed on the shake of the imaging device during each divided time, and n divided images are taken. A restoration filter in each divided period is created, and image restoration is performed by convolving a corresponding restoration filter with respect to n divided images. After that, the divided images are aligned and image addition (image synthesis) is performed.
図５を参照して、本発明の第３の実施の形態に係わる撮像装置１ｃの構成を説明する。撮像装置１ｃは、図１の撮像装置１ａに比べて、以下の点が異なる。 With reference to FIG. 5, the structure of the imaging device 1c concerning the 3rd Embodiment of this invention is demonstrated. The imaging device 1c is different from the imaging device 1a of FIG. 1 in the following points.
撮像装置１ｃは、図１の撮像装置１ａの構成要素（１１〜１９）の他に、更に、全露光時間Ｔを所定の分割数（ｎ）で分割する露光分割部２０と、撮像部１２が繰り返し撮像を行って生成した分割数（ｎ）個の被写体像を合成する画像合成部２１とを備える。図１の撮像装置１ａにおける「露光期間」は全露光時間Ｔを所定の分割数（ｎ）で分割した各分割期間に相当する。画像復元部１９は、分割数（ｎ）個の被写体像について像ブレを補正する。画像合成部２１は、画像復元部１９により像ブレが補正された分割数（ｎ）個の被写体像を合成する。 In addition to the components (11 to 19) of the imaging device 1a in FIG. 1, the imaging device 1c further includes an exposure dividing unit 20 that divides the total exposure time T by a predetermined division number (n), and an imaging unit 12 An image synthesis unit 21 that synthesizes (n) subject images generated by repeated imaging. The “exposure period” in the imaging apparatus 1a in FIG. 1 corresponds to each divided period obtained by dividing the total exposure time T by a predetermined number (n). The image restoration unit 19 corrects the image blur for the number of division (n) subject images. The image synthesis unit 21 synthesizes (n) subject images whose image blur has been corrected by the image restoration unit 19.
したがって、撮像装置１ｃは、図５に示すように、撮像面に被写体像を結像させる撮影光学系１１と、分割期間にわたって撮像面に結像された被写体像を撮像する撮像部１２と、分割期間中における撮像装置１ｃのブレを検出するブレ検出部の一例としての角速度センサ１３と、センサからの出力信号を補正するセンサ信号補正部１４と、補正光学系を駆動する補正光学系駆動部１５と、角速度センサ１３が検出する撮像装置１ｃのブレに基づいて補正光学系駆動部１５を制御することにより、補正光学系を撮像装置１ｃのブレに追従させる駆動制御部１６と、補正光学系が駆動限界に達したか否かを判断する駆動限界判断部１７と、補正光学系が駆動限界に達している場合、駆動限界に達している状態で補正光学系の駆動を停止する駆動停止部１８と、撮像装置１ｃのブレのうち補正光学系の駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する画像復元部１９と、露光分割部２０と、画像合成部２１とを備える。 Therefore, as shown in FIG. 5, the imaging device 1 c includes a photographing optical system 11 that forms a subject image on the imaging surface, an imaging unit 12 that images the subject image formed on the imaging surface over a division period, and a division. An angular velocity sensor 13 as an example of a shake detection unit that detects a shake of the imaging device 1c during the period, a sensor signal correction unit 14 that corrects an output signal from the sensor, and a correction optical system drive unit 15 that drives the correction optical system. The correction optical system drive unit 15 is controlled based on the shake of the image pickup apparatus 1c detected by the angular velocity sensor 13, so that the drive control unit 16 causes the correction optical system to follow the shake of the image pickup apparatus 1c, and the correction optical system includes A drive limit determination unit 17 that determines whether or not the drive limit has been reached, and a drive stop that stops the drive of the correction optical system when the correction optical system has reached the drive limit. An image restoration unit 19 that performs image processing on a subject image based on a blurring trajectory of the subject image, an exposure division unit 20, and an image And a combining unit 21.
分割期間中における撮像装置１ｃのブレが補正光学系の駆動限界を超えてしまう場合、この超える部分について、上記の光学式ブレ補正を適用することはできない。そこで、駆動停止部１８は、補正光学系が駆動限界に達している場合、駆動限界に達している状態で補正光学系の駆動を停止する。これにより、補正光学系は駆動限界で固定される。そして、画像復元部１９が、撮像装置１ｃのブレのうち補正光学系の駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像復元する。その後、画像合成部２１が、各被写体像を位置合わせした上で画像加算を行なう。 When the blurring of the imaging device 1c during the divided period exceeds the drive limit of the correction optical system, the above-described optical blur correction cannot be applied to the exceeding part. Therefore, when the correction optical system has reached the drive limit, the drive stop unit 18 stops driving the correction optical system in a state where the drive limit has been reached. Thereby, the correction optical system is fixed at the drive limit. Then, the image restoration unit 19 restores the subject image based on the blur trajectory of the subject image for a portion of the blur of the imaging device 1c that exceeds the drive limit of the correction optical system. Thereafter, the image composition unit 21 performs image addition after aligning each subject image.
図６を参照して、図５の撮像装置１ｃにおけるブレ補正処理の流れを説明する。 With reference to FIG. 6, the flow of blur correction processing in the imaging apparatus 1c of FIG. 5 will be described.
（イ）先ず、露光分割部２０が予め定められた全露光時間Ｔを所定の分割数（ｎ）で分割する（Ｓ３１０）。そして、最初の露光期間が開始すると（Ｓ３１５）、角速度センサ１３が、撮像装置１ｃのブレ検出を開始する（Ｓ３２０）。このブレ検出は、露光期間中継続的に実施される。センサ信号補正部１４が、角速度センサ１３の出力信号（センサ信号）を外気温やズームの度合いに基づいて補正する（Ｓ３２５）。駆動制御部１６は、撮像装置１ｃのブレに基づいて補正光学系駆動部１５を制御することにより、補正光学系を駆動して撮像装置１ｃのブレに追従させる（Ｓ３３０）。このとき、駆動限界判断部１７は、補正光学系の位置、補正光学系の駆動量、或いは補正光学系駆動部１５の制御量を検出している。 (A) First, the exposure dividing unit 20 divides the predetermined total exposure time T by a predetermined division number (n) (S310). When the first exposure period starts (S315), the angular velocity sensor 13 starts blur detection of the imaging device 1c (S320). This blur detection is continuously performed during the exposure period. The sensor signal correction unit 14 corrects the output signal (sensor signal) of the angular velocity sensor 13 based on the outside air temperature and the degree of zoom (S325). The drive control unit 16 controls the correction optical system driving unit 15 based on the shake of the imaging device 1c, thereby driving the correction optical system to follow the shake of the imaging device 1c (S330). At this time, the drive limit determination unit 17 detects the position of the correction optical system, the drive amount of the correction optical system, or the control amount of the correction optical system drive unit 15.
（ロ）駆動限界判断部１７は、補正光学系の位置、補正光学系の駆動量、或いは補正光学系駆動部１５の制御量から、補正光学系が駆動限界に達したか否かを判断する。達していなければ（Ｓ３３５にてＮｏ）、Ｓ３２０段階へ戻り、ブレ検出に基づく補正光学系の駆動を継続する。一方、駆動限界に達していれば（Ｓ３３５にてＹｅｓ）、Ｓ３４０段階に進み、駆動停止部１８は、駆動限界に達している状態で補正光学系の駆動を停止する。また、これと同時に、補正されたセンサ信号の記録を開始する。センサ信号を記録しつつ、露光期間中にブレが駆動限界内に戻るか否かを監視する。露光期間中にブレが駆動限界内に入った場合（Ｓ３４３にてＹｅｓ）、Ｓ３２０段階に戻り、ブレ検出に基づく補正光学系の駆動を再開する。露光期間中にブレが駆動限界内に戻らなければ（Ｓ３４３にてＮｏ）、センサ信号の記録を、露光期間が終了するまで継続する。 (B) The drive limit determination unit 17 determines whether the correction optical system has reached the drive limit from the position of the correction optical system, the drive amount of the correction optical system, or the control amount of the correction optical system drive unit 15. . If not reached (No in S335), the process returns to step S320, and the driving of the correction optical system based on the blur detection is continued. On the other hand, if the drive limit has been reached (Yes in S335), the process proceeds to step S340, and the drive stop unit 18 stops driving the correction optical system while reaching the drive limit. At the same time, recording of the corrected sensor signal is started. While recording the sensor signal, it is monitored whether or not the blur returns to the drive limit during the exposure period. If the blur enters the drive limit during the exposure period (Yes in S343), the process returns to step S320, and the driving of the correction optical system based on the blur detection is resumed. If the shake does not return to within the drive limit during the exposure period (No in S343), the sensor signal recording is continued until the exposure period ends.
（ハ）露光期間が終了した後（Ｓ３４５）、画像復元部１９が撮像装置１ｃのブレのうち補正光学系の駆動限界を超える部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する（Ｓ３５０）。 (C) After the exposure period ends (S345), the image restoration unit 19 performs image processing on the subject image of the blur of the imaging apparatus 1c based on the blur trajectory of the subject image for the portion exceeding the drive limit of the correction optical system. (S350).
（ニ）ｎ個に分割した露光を総て実施したか否かを判断する。総ての分割露光を実施していない場合（Ｓ３５５にてＮｏ）、Ｓ３１５段階に戻り、実施していない分割露光を上記の手順に従って実施する。総ての分割露光を実施した場合（Ｓ３５５にてＹｅｓ）、Ｓ３６０段階に進み、画像合成部２１は、撮像部１２が繰り返し撮像を行って生成した分割数（ｎ）個の被写体像を位置あわせした上で画像合成する。 (D) It is determined whether or not the exposure divided into n pieces has been carried out. When all the divided exposures are not performed (No in S355), the process returns to step S315, and the divided exposures that are not performed are performed according to the above procedure. When all the divided exposures have been performed (Yes in S355), the process proceeds to step S360, and the image composition unit 21 aligns (n) number of subject images generated by the imaging unit 12 repeatedly imaging. And then compose the image.
なお、Ｓ３４０段階において、センサ信号の記録を開始した場合について説明したが、それ以前から、例えば露光期間開始から記録し始めても構わない。 In addition, although the case where the recording of the sensor signal is started in the step S340 has been described, the recording may be started from the beginning of the exposure period, for example.
以上説明したように、本発明の第３の実施の形態によれば以下の作用効果が得られる。 As described above, according to the third embodiment of the present invention, the following operational effects can be obtained.
露光分割部２０は全露光時間Ｔを所定の分割数で分割し、撮像部１２は繰り返し撮像を行って分割数の被写体像を生成し、駆動制御部１６は、補正光学系を撮像装置１ｃのブレに追従させ、画像復元部１９は分割数の被写体像についてブレを補正し、画像合成部２１は、画像復元部１９によりブレが補正された分割数の被写体像を合成する。これにより、光学式ブレ補正と画像復元式ブレ補正と画像加算式ブレ補正との総ての効果を足し合わせることができるため、光学式の補正限界を超えるブレ補正が可能となり、補正範囲及び補正性能を更に向上させることができる。また、画像加算部分でロール方向の手ぶれを補正できる。さらに、長秒露光時のセンサ計測の誤差の影響も補正する事が出来る。これにより、従来の補正方法では難しかった長秒露光時のブレ補正が可能になる。 The exposure dividing unit 20 divides the total exposure time T by a predetermined number of divisions, the imaging unit 12 repeatedly captures images to generate subject images of the number of divisions, and the drive control unit 16 sets the correction optical system to the imaging device 1c. Following image blur, the image restoration unit 19 corrects the blur for the number of subject images, and the image composition unit 21 synthesizes the subject image for the number of divisions corrected by the image restoration unit 19. As a result, it is possible to add all the effects of optical blur correction, image restoration blur correction, and image addition blur correction, so that blur correction exceeding the optical correction limit is possible, and the correction range and correction The performance can be further improved. Further, camera shake in the roll direction can be corrected at the image addition portion. Furthermore, it is possible to correct the influence of sensor measurement errors during long-second exposure. This makes it possible to perform blur correction during long-second exposure, which was difficult with the conventional correction method.
（第３の実施の形態の変形例）
第３の実施の形態においては、光学式ブレ補正の補正限界まで補正光学系を駆動し、補正限界を超えるブレについては画像復元式ブレ補正でブレを復元している、図１及び図２に示すブレ補正方法に対して、画像加算式ブレ補正を組み合わせた実施例を説明した。しかし、本発明はこれに限定されるものではない。ブレの一部分のみに対して補正光学系を駆動し、ブレの残り部分を「残しブレ」として画像復元式ブレ補正で復元している、図３及び図４に示すブレ補正方法に対して、画像加算式ブレ補正を組み合わせることも可能である。そこで、第３の実施の形態の変形例では、図３及び図４に示すブレ補正方法に対して、画像加算式ブレ補正を組み合わせる変形例を説明する。
(Modification of the third embodiment)
In the third embodiment, the correction optical system is driven to the correction limit of the optical shake correction, and the shake that exceeds the correction limit is restored by the image restoration type shake correction. FIG. 1 and FIG. The embodiment in which the image addition type blur correction is combined with the illustrated blur correction method has been described. However, the present invention is not limited to this. In contrast to the blur correction method shown in FIGS. 3 and 4, the correction optical system is driven only for a part of the blur, and the remaining part of the blur is restored as the “remaining blur” by the image restoration type blur correction. It is also possible to combine additive blur correction. Therefore, in a modification of the third embodiment, a modification in which an image addition type blur correction is combined with the blur correction method shown in FIGS. 3 and 4 will be described.
図７を参照して、本発明の第３の実施の形態の変形例に係わる撮像装置１ｄの構成を説明する。撮像装置１ｄは、図３の撮像装置１ｂに比べて、以下の点が異なる。 With reference to FIG. 7, the configuration of an imaging apparatus 1d according to a modification of the third embodiment of the present invention will be described. The imaging device 1d differs from the imaging device 1b of FIG. 3 in the following points.
撮像装置１ｄは、図３の撮像装置１ｂの構成要素（１１〜１６、１９）の他に、更に、全露光時間Ｔを所定の分割数（ｎ）で分割する露光分割部２０と、撮像部１２が繰り返し撮像を行って生成した分割数（ｎ）個の被写体像を合成する画像合成部２１とを備える。図３の撮像装置１ｂにおける「露光期間」は全露光時間Ｔを所定の分割数（ｎ）で分割した各分割期間に相当する。画像復元部１９は、分割数（ｎ）個の被写体像について像ブレを補正する。画像合成部２１は、画像復元部１９により像ブレが補正された分割数（ｎ）個の被写体像を合成する。 In addition to the components (11 to 16 and 19) of the imaging device 1b in FIG. 3, the imaging device 1d further includes an exposure division unit 20 that divides the total exposure time T by a predetermined division number (n), and an imaging unit. 12 includes an image synthesizing unit 21 that synthesizes (n) subject images generated by repeated imaging. The “exposure period” in the imaging apparatus 1b in FIG. 3 corresponds to each divided period obtained by dividing the total exposure time T by a predetermined number (n). The image restoration unit 19 corrects the image blur for the number of division (n) subject images. The image synthesis unit 21 synthesizes (n) subject images whose image blur has been corrected by the image restoration unit 19.
したがって、撮像装置１ｄは、図７に示すように、撮像面に被写体像を結像させる撮影光学系１１と、分割期間にわたって撮像面に結像された被写体像を撮像する撮像部１２と、分割期間中における撮像装置１ｄのブレを検出するブレ検出部の一例としての角速度センサ１３と、センサからの出力信号を補正するセンサ信号補正部１４と、補正光学系を駆動する補正光学系駆動部１５と、角速度センサ１３が検出する撮像装置１ｄのブレの一部分のみに基づいて補正光学系駆動部１５を制御することにより、補正光学系を撮像装置１ｄのブレの一部分に限定して追従させる駆動制御部１６と、撮像装置１ｄのブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理することにより被写体像のブレを電子的に補正する画像復元部１９と、露光分割部２０と、画像合成部２１とを備える。 Therefore, as shown in FIG. 7, the imaging device 1 d includes a photographing optical system 11 that forms a subject image on the imaging surface, an imaging unit 12 that captures the subject image formed on the imaging surface over a divided period, and a division. An angular velocity sensor 13 as an example of a blur detection unit that detects the blur of the imaging device 1d during the period, a sensor signal correction unit 14 that corrects an output signal from the sensor, and a correction optical system drive unit 15 that drives the correction optical system. Then, by controlling the correction optical system driving unit 15 based only on a part of the blur of the image pickup apparatus 1d detected by the angular velocity sensor 13, the drive control is performed to limit the correction optical system to a part of the shake of the image pickup apparatus 1d. An image for electronically correcting the blur of the subject image by performing image processing on the subject image based on the blur locus of the subject image with respect to the blur of the unit 16 and the imaging device 1d. It includes a restoring section 19, the exposure division unit 20, and an image synthesis section 21.
その他の構成については、図３の撮像装置１ｂと同じであるため、説明を省略する。 The other configuration is the same as that of the imaging device 1b of FIG.
図８を参照して、図７の撮像装置１ｄにおけるブレ補正処理の流れを説明する。 With reference to FIG. 8, the flow of the blur correction process in the imaging device 1d of FIG. 7 will be described.
（イ）先ず、露光分割部２０が予め定められた全露光時間Ｔを所定の分割数（ｎ）で分割する（Ｓ４１０）。そして、最初の露光期間が開始すると（Ｓ４１５）、角速度センサ１３が、撮像装置１ｄのブレ検出を開始する（Ｓ４２０）。このブレ検出は、露光期間中継続的に実施される。センサ信号補正部１４が、角速度センサ１３の出力信号（センサ信号）を外気温やズームの度合いに基づいて補正する（Ｓ４２５）。駆動制御部１６は、撮像装置１ｄのブレの一部分のみに基づいて補正光学系駆動部１５を制御することにより、補正光学系を駆動して撮像装置１ｄのブレの一部分に限定して追従させる（Ｓ４３０）。このとき、位置検出部１７は、補正光学系の位置をモニターしている。また、補正光学系の駆動と同時に、補正されたセンサ信号及び補正光学系の位置を記録する。なお、補正光学系の位置の代わりに、補正光学系の駆動量、或いは補正光学系駆動部１５の制御量を記録してもよい。 (A) First, the exposure dividing unit 20 divides the predetermined total exposure time T by a predetermined division number (n) (S410). Then, when the first exposure period starts (S415), the angular velocity sensor 13 starts blur detection of the imaging device 1d (S420). This blur detection is continuously performed during the exposure period. The sensor signal correction unit 14 corrects the output signal (sensor signal) of the angular velocity sensor 13 based on the outside air temperature and the degree of zoom (S425). The drive control unit 16 controls the correction optical system drive unit 15 based only on a part of the shake of the image pickup apparatus 1d, thereby driving the correction optical system to follow only a part of the shake of the image pickup apparatus 1d ( S430). At this time, the position detection unit 17 monitors the position of the correction optical system. Simultaneously with the driving of the correction optical system, the corrected sensor signal and the position of the correction optical system are recorded. Instead of the position of the correction optical system, the drive amount of the correction optical system or the control amount of the correction optical system drive unit 15 may be recorded.
（ロ）露光期間が終了したか否かを判断する。終了していなければ（Ｓ４３５にてＮｏ）、Ｓ４２０段階に戻る。一方、終了していれば（Ｓ４３５にてＹｅｓ）、Ｓ４４０段階に進み、画像復元部１９が撮像装置１ｄのブレの残り部分について、被写体像のブレの軌跡に基づいて被写体像を画像処理する。ここで、補正光学系の駆動と同時に、補正されたセンサ信号及び補正光学系の位置を記録しているため、「残しブレ」を求めることができる。したがって、画像復元部１９は、この残しブレを、撮像装置１ｂのブレの残り部分として電子的に補正することができる。 (B) It is determined whether or not the exposure period has ended. If not completed (No in S435), the process returns to S420. On the other hand, if completed (Yes in S435), the process proceeds to step S440, and the image restoration unit 19 performs image processing on the subject image based on the blurring locus of the subject image for the remaining blurring of the imaging device 1d. Here, since the corrected sensor signal and the position of the correction optical system are recorded simultaneously with the driving of the correction optical system, “remaining blur” can be obtained. Therefore, the image restoration unit 19 can electronically correct this remaining blur as the remaining blur of the imaging device 1b.
（ハ）ｎ個に分割した露光を総て実施したか否かを判断する。総ての分割露光を実施していない場合（Ｓ４４５にてＮｏ）、Ｓ４１５段階に戻り、実施していない分割露光を上記の手順に従って実施する。総ての分割露光を実施した場合（Ｓ４４５にてＹｅｓ）、Ｓ４５０段階に進み、画像合成部２１は、撮像部１２が繰り返し撮像を行って生成した分割数（ｎ）個の被写体像を位置あわせした上で画像合成する。 (C) It is determined whether or not exposure divided into n pieces has been performed. When all the divided exposures are not performed (No in S445), the process returns to step S415, and the divided exposures that are not performed are performed according to the above procedure. If all the divided exposures have been performed (Yes in S445), the process proceeds to step S450, and the image composition unit 21 aligns the number of divided (n) subject images generated by the imaging unit 12 repeatedly imaging. And then compose the image.
上記のように、本発明は、第１乃至第３の実施の形態及びその変形例によって記載したが、この開示の一部をなす論述及び図面はこの発明を限定するものであると理解すべきではない。この開示から当業者には様々な代替実施の形態、実施例及び運用技術が明らかとなろう。即ち、本発明はここでは記載していない様々な実施の形態等を包含するということを理解すべきである。したがって、本発明はこの開示から妥当な特許請求の範囲に係る発明特定事項によってのみ限定されるものである。 As described above, the present invention has been described with reference to the first to third embodiments and modifications thereof. However, it should be understood that the description and drawings constituting a part of this disclosure limit the present invention. is not. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art. That is, it should be understood that the present invention includes various embodiments not described herein. Therefore, the present invention is limited only by the invention specifying matters according to the scope of claims reasonable from this disclosure.
１ａ〜１ｄ…撮像装置
１１…撮影光学系
１２…撮像部
１３…角速度センサ（ブレ検出部）
１４…センサ信号補正部
１５…補正光学系駆動部
１６…駆動制御部
１７…駆動限界判断部
１８…駆動停止部
１９…画像復元部
２０…露光分割部
２１…画像合成部
DESCRIPTION OF SYMBOLS 1a-1d ... Imaging device 11 ... Imaging optical system 12 ... Imaging part 13 ... Angular velocity sensor (blur detection part)
DESCRIPTION OF SYMBOLS 14 ... Sensor signal correction | amendment part 15 ... Correction | amendment optical system drive part 16 ... Drive control part 17 ... Drive limit judgment part 18 ... Drive stop part 19 ... Image restoration part 20 ... Exposure division part 21 ... Image composition part
Claims (5)
前記露光期間中における前記撮像装置のブレを検出するブレ検出部と、
前記撮像部又は前記撮影光学系の一部を駆動する補正光学系駆動部と、
前記ブレ検出部が検出する前記撮像装置のブレに基づいて前記補正光学系駆動部を制御することにより、前記撮像部又は前記撮影光学系の一部を前記撮像装置のブレに追従させる駆動制御部と、
前記撮像部又は前記撮影光学系の一部が駆動限界に達したか否かを判断する駆動限界判断部と、
前記撮像部又は前記撮影光学系の一部が前記駆動限界に達している場合、前記駆動限界に達している状態で前記撮像部又は前記撮影光学系の一部の駆動を停止する駆動停止部と、
前記撮像装置のブレのうち前記駆動限界を超える部分について、前記被写体像のブレの軌跡に基づいて前記被写体像を画像処理することにより前記ブレを補正する画像復元部と
を備えることを特徴とする撮像装置。 An imaging apparatus comprising: a photographing optical system that forms a subject image on an imaging surface; and an imaging unit that captures the subject image formed on the imaging surface over a preset exposure period.
A blur detection unit for detecting blur of the imaging device during the exposure period;
A correction optical system driving unit that drives a part of the imaging unit or the photographing optical system;
A drive control unit that controls the correction optical system driving unit based on the blur of the imaging device detected by the blur detection unit, thereby causing the imaging unit or a part of the photographing optical system to follow the blur of the imaging device. When,
A drive limit determination unit that determines whether a part of the imaging unit or the photographing optical system has reached a drive limit;
A drive stopping unit that stops driving of the imaging unit or the photographing optical system in a state where the driving limit is reached when a part of the imaging unit or the photographing optical system has reached the driving limit; ,
An image restoration unit that corrects the blur by performing image processing on the subject image based on a blur trajectory of the subject image for a portion of the blur of the imaging device that exceeds the drive limit. Imaging device.
前記露光期間中における前記撮像装置のブレを検出するブレ検出部と、
前記撮像部又は前記撮影光学系の一部を駆動する補正光学系駆動部と、
前記ブレ検出部が検出する前記撮像装置のブレの一部分のみに基づいて前記補正光学系駆動部を制御することにより、前記撮像部又は前記撮影光学系の一部を前記撮像装置のブレの前記一部分に限定して追従させる駆動制御部と、
前記撮像装置のブレの残り部分について、前記被写体像のブレの軌跡に基づいて前記被写体像を画像処理することにより前記ブレを補正する画像復元部と
を備えることを特徴とする撮像装置。 An imaging apparatus comprising: a photographing optical system that forms a subject image on an imaging surface; and an imaging unit that captures the subject image formed on the imaging surface over a preset exposure period.
A blur detection unit for detecting blur of the imaging device during the exposure period;
A correction optical system driving unit that drives a part of the imaging unit or the photographing optical system;
By controlling the correction optical system driving unit based only on a part of the blur of the imaging device detected by the blur detection unit, the imaging unit or a part of the photographing optical system is controlled by the part of the blur of the imaging device. A drive control unit for following the motor only,
An imaging apparatus comprising: an image restoration unit that corrects the blur by performing image processing on the subject image based on a blur locus of the subject image with respect to a remaining blur of the imaging device.
前記撮像部が繰り返し撮像を行って生成した前記分割数の被写体像を合成する画像合成部とを更に備え、
前記露光期間は前記全露光時間を所定の分割数で分割したうちの少なくとも１つであり、前記画像復元部は、前記分割数の被写体像について前記ブレを補正し、前記画像合成部は、前記画像復元部によりブレが補正された前記分割数の被写体像を合成することを特徴とする請求項１乃至４いずれか一項に記載の撮像装置。 An exposure division unit that divides the total exposure time by a predetermined number of divisions;
An image compositing unit that synthesizes the number of subject images generated by the image capturing unit repeatedly capturing images;
The exposure period is at least one of the total exposure times divided by a predetermined division number, the image restoration unit corrects the blur for the division number of subject images, and the image synthesis unit 5. The imaging apparatus according to claim 1, wherein the subject images of the number of divisions whose blurring has been corrected by an image restoration unit are synthesized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2006305522A JP4420921B2 (en) | 2006-11-10 | 2006-11-10 | Imaging device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2006305522A JP4420921B2 (en) | 2006-11-10 | 2006-11-10 | Imaging device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JP2008124728A true JP2008124728A (en) | 2008-05-29 |
| JP4420921B2 JP4420921B2 (en) | 2010-02-24 |
Family
ID=39509040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2006305522A Expired - Fee Related JP4420921B2 (en) | 2006-11-10 | 2006-11-10 | Imaging device |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP4420921B2 (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008245056A (en) * | 2007-03-28 | 2008-10-09 | Fujifilm Corp | Photographing device |
| JP2009251492A (en) * | 2008-04-10 | 2009-10-29 | Olympus Imaging Corp | Imaging apparatus and control method for imaging apparatus |
| JP2010004370A (en) * | 2008-06-20 | 2010-01-07 | Sony Corp | Imaging device and image blurring correction method, and program |
| JP2010078635A (en) * | 2008-09-24 | 2010-04-08 | Sanyo Electric Co Ltd | Blur correcting device and imaging apparatus |
| JP2010078765A (en) * | 2008-09-25 | 2010-04-08 | Sanyo Electric Co Ltd | Imaging apparatus |
| JP2011039295A (en) * | 2009-08-11 | 2011-02-24 | Nikon Corp | Shake correcting device and optical equipment |
| JP2015035001A (en) * | 2014-11-04 | 2015-02-19 | 株式会社ニコン | Shake correcting device and optical apparatus |
| JP2015201776A (en) * | 2014-04-09 | 2015-11-12 | キヤノン株式会社 | Imaging apparatus and control method of the same |
| JP2016118701A (en) * | 2014-12-22 | 2016-06-30 | キヤノン株式会社 | Imaging device and control method thereof |
| JP2017092749A (en) * | 2015-11-12 | 2017-05-25 | キヤノン株式会社 | Imaging apparatus, blur correction method, program, and storage medium |
| US11503210B2 (en) | 2019-02-20 | 2022-11-15 | Fujifilm Corporation | Blur correction device, imaging apparatus, monitoring system, and program |
-
2006
- 2006-11-10 JP JP2006305522A patent/JP4420921B2/en not_active Expired - Fee Related
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008245056A (en) * | 2007-03-28 | 2008-10-09 | Fujifilm Corp | Photographing device |
| JP2009251492A (en) * | 2008-04-10 | 2009-10-29 | Olympus Imaging Corp | Imaging apparatus and control method for imaging apparatus |
| JP2010004370A (en) * | 2008-06-20 | 2010-01-07 | Sony Corp | Imaging device and image blurring correction method, and program |
| JP4518197B2 (en) * | 2008-06-20 | 2010-08-04 | ソニー株式会社 | Imaging apparatus, image blur correction method, and program |
| JP2010078635A (en) * | 2008-09-24 | 2010-04-08 | Sanyo Electric Co Ltd | Blur correcting device and imaging apparatus |
| JP2010078765A (en) * | 2008-09-25 | 2010-04-08 | Sanyo Electric Co Ltd | Imaging apparatus |
| JP2011039295A (en) * | 2009-08-11 | 2011-02-24 | Nikon Corp | Shake correcting device and optical equipment |
| JP2015201776A (en) * | 2014-04-09 | 2015-11-12 | キヤノン株式会社 | Imaging apparatus and control method of the same |
| JP2015035001A (en) * | 2014-11-04 | 2015-02-19 | 株式会社ニコン | Shake correcting device and optical apparatus |
| JP2016118701A (en) * | 2014-12-22 | 2016-06-30 | キヤノン株式会社 | Imaging device and control method thereof |
| JP2017092749A (en) * | 2015-11-12 | 2017-05-25 | キヤノン株式会社 | Imaging apparatus, blur correction method, program, and storage medium |
| US11503210B2 (en) | 2019-02-20 | 2022-11-15 | Fujifilm Corporation | Blur correction device, imaging apparatus, monitoring system, and program |
| JP7356557B2 (en) | 2019-02-20 | 2023-10-04 | 富士フイルム株式会社 | Image stabilization device, imaging device, monitoring system, and program |
| US11889191B2 (en) | 2019-02-20 | 2024-01-30 | Fujifilm Corporation | Blur correction device, imaging apparatus, monitoring system, and program |
Also Published As
| Publication number | Publication date |
|---|---|
| JP4420921B2 (en) | 2010-02-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4420921B2 (en) | Imaging device | |
| KR101322291B1 (en) | Image capturing apparatus and method of controlling image capturing apparatus | |
| US8811809B2 (en) | Image stabilization apparatus, control method therefor, optical apparatus and imaging apparatus | |
| JP5328307B2 (en) | Image capturing apparatus having shake correction function and control method thereof | |
| JP4522207B2 (en) | Camera system, camera body and interchangeable lens | |
| JP5519077B2 (en) | Imaging device | |
| JP4401949B2 (en) | Moving picture imaging apparatus and moving picture imaging method | |
| US20100295961A1 (en) | Imaging apparatus and image composition method | |
| JP2006113264A (en) | Image blur correcting apparatus | |
| JP6500238B2 (en) | Imaging device, imaging method and imaging program | |
| JP2011023987A (en) | Image sensing apparatus, and image stabilization method | |
| JP6432038B2 (en) | Imaging device | |
| JP4200767B2 (en) | Blur correction device and camera | |
| JP2003274281A (en) | Image pickup device | |
| JP2006332809A (en) | Imaging apparatus | |
| JP2013218031A (en) | Imaging apparatus | |
| US20150294442A1 (en) | Camera system and imaging method | |
| JP6024031B2 (en) | Blur correction device and optical apparatus | |
| JP2019092036A (en) | Imaging apparatus and control method | |
| JP7324284B2 (en) | Imaging device, camera shake correction device, imaging method, and camera shake correction method | |
| JP5699806B2 (en) | Imaging device | |
| JP5241403B2 (en) | Imaging device | |
| WO2020012961A1 (en) | Imaging device | |
| JP6559354B2 (en) | Imaging device | |
| JP7132844B2 (en) | IMAGE STABILIZATION DEVICE, IMAGING DEVICE INCLUDING THE SAME, AND IMAGE STABILIZATION METHOD |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20090625 |
|
| A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20090707 |
|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20090828 |
|
| TRDD | Decision of grant or rejection written | ||
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20091104 |
|
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20091201 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20121211 Year of fee payment: 3 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20121211 Year of fee payment: 3 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20131211 Year of fee payment: 4 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20131211 Year of fee payment: 4 |
|
| S111 | Request for change of ownership or part of ownership |
Free format text: JAPANESE INTERMEDIATE CODE: R313113 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20131211 Year of fee payment: 4 |
|
| R350 | Written notification of registration of transfer |
Free format text: JAPANESE INTERMEDIATE CODE: R350 |
|
| LAPS | Cancellation because of no payment of annual fees |