JP6559354B2 - Imaging device - Google Patents

Imaging device Download PDF

Info

Publication number
JP6559354B2
JP6559354B2 JP2018531820A JP2018531820A JP6559354B2 JP 6559354 B2 JP6559354 B2 JP 6559354B2 JP 2018531820 A JP2018531820 A JP 2018531820A JP 2018531820 A JP2018531820 A JP 2018531820A JP 6559354 B2 JP6559354 B2 JP 6559354B2
Authority
JP
Japan
Prior art keywords
psf
subject
unit
image
traveling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2018531820A
Other languages
Japanese (ja)
Other versions
JPWO2018025633A1 (en
Inventor
敬太 望月
敬太 望月
河野 裕之
裕之 河野
一 長原
長原  一
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Publication of JPWO2018025633A1 publication Critical patent/JPWO2018025633A1/en
Application granted granted Critical
Publication of JP6559354B2 publication Critical patent/JP6559354B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/67Focus control based on electronic image sensor signals
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00Special procedures for taking photographs; Apparatus therefor
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B5/00Adjustment of optical system relative to image or object surface other than for focusing
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B7/00Control of exposure by setting shutters, diaphragms or filters, separately or conjointly
    • G03B7/08Control effected solely on the basis of the response, to the intensity of the light received by the camera, of a built-in light-sensitive device
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B9/00Exposure-making shutters; Diaphragms
    • G03B9/02Diaphragms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/73Deblurring; Sharpening

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Adjustment Of Camera Lenses (AREA)
  • Exposure Control For Cameras (AREA)
  • Stroboscope Apparatuses (AREA)
  • Diaphragms For Cameras (AREA)

Description

本発明は、走行車両に搭載する撮影装置に関し、特に撮影中の被写体との距離の変動に影響されず被写体の画像が得られる撮影装置に関する。   The present invention relates to an imaging device mounted on a traveling vehicle, and more particularly to an imaging device that can obtain an image of a subject without being affected by a change in distance to the subject being shot.

高度経済成長期に建造された社会インフラ設備の劣化とその修復が社会問題となっており、効率的な劣化の点検技術が求められている。この中で、トンネル覆工面コンクリートのように範囲の広い設備の点検方法として、車両上に撮影装置を実装し、走行撮影する方法が提案されている(例えば、特許文献1、2参照)。コンクリートの代表的な劣化指標として、ひび割れがあるが、その定量的な調査には、画像中の1画素がサブミリ角の被写体を映し出す程度の高解像度撮影が要求される。   Deterioration and restoration of social infrastructure equipment built during the period of high economic growth has become a social problem, and efficient inspection technology for deterioration is required. Among these, as a method for inspecting equipment with a wide range such as tunnel lining concrete, a method of mounting a photographing device on a vehicle and photographing while traveling has been proposed (for example, see Patent Documents 1 and 2). Cracks are typical degradation indicators for concrete, but for quantitative investigations, high-resolution photography is required to the extent that one pixel in an image projects a subject with a submillimeter angle.

走行車両上から静止する被写体を高解像度で撮影する場合、走行車両の走行方向の移動に起因するぶれによる解像度の低下を防止するために、撮影した画像を後処理により補正し、走行方向のぶれを除去した画像に復元する方法が有効である。例えば、点像分布関数(Point-Spread Function、以下において「PSF」と呼ぶ。)で表される撮像画像上のぶれ関数が既知であれば、デジタル処理の逆フィルタ演算によって画像を補正できる。   When shooting a stationary subject from a traveling vehicle with high resolution, the captured image is corrected by post-processing to prevent a reduction in resolution due to blurring caused by movement of the traveling vehicle in the traveling direction. A method for restoring the image from which the image is removed is effective. For example, if the blurring function on a captured image represented by a point spread function (Point-Spread Function, hereinafter referred to as “PSF”) is known, the image can be corrected by a digital inverse filter operation.

通常の撮影では、露光時間中の露光状態は一定であり、一方向に走行する車両上から撮影した場合、その撮影過程におけるPSFは被写体の動き方向に延びた線形状であり、一様な値をもつ。このPSFの周波数特性は高帯域の周波数成分が低く、所定の周波数間隔で大幅に減衰するものであり、復元性が低くなるという問題がある。撮影過程での画像情報の損失を防ぐ方法として、露光時間中の露光状態を変調制御する方法が提案されている。例えば、露光時間中にシャッタを不規則に開閉したり、照明を不規則に点灯制御する方法(例えば、特許文献3)、画像センサのゲインを変調制御する方法(例えば、特許文献4)を用いて、PSFが広帯域でフラットな周波数特性になるようにしている。   In normal shooting, the exposure state during the exposure time is constant, and when shooting from a vehicle traveling in one direction, the PSF in the shooting process is a line shape extending in the direction of movement of the subject and has a uniform value. It has. The frequency characteristic of this PSF has a problem that the frequency component in the high band is low and attenuates significantly at a predetermined frequency interval, so that the restoration property is low. As a method for preventing loss of image information during the photographing process, a method for modulating the exposure state during the exposure time has been proposed. For example, a method of irregularly opening and closing the shutter during the exposure time, a method of controlling lighting of the illumination irregularly (for example, Patent Document 3), and a method of controlling the gain of the image sensor (for example, Patent Document 4) are used. Thus, the PSF has a wide band and flat frequency characteristics.

特開平5−256633号公報JP-A-5-256633 特許第5149403号Patent No. 5149403 特許第4679662号Japanese Patent No. 4679662 特開2007−336064号公報JP 2007-336064 A

走行車両上から被写体を撮影する場合、走行方向のぶれに加えて、走行ラインがずれることで被写体と撮影装置との距離が変動し、ピンぼけが発生する。このため、撮影過程でのPSFは、被写体の動きによるぶれとピンぼけとが重畳した形状となる。ピンぼけが発生した場合、特許文献3や4に記載された方法では、ぶれの補正に要求される被写体の動き方向(走行方向と反対方向)に広帯域なPSFの周波数特性が維持できない。ピンぼけを防止する場合、一般にはレンズの開口を絞ることで被写界深度を確保することが考えられるが、露光時間を短縮した場合と同様に、入射光量が減少するという問題がある。   When a subject is photographed from the traveling vehicle, the distance between the subject and the photographing device is fluctuated due to a shift in the traveling line in addition to the blur in the traveling direction, resulting in blurring. For this reason, the PSF in the shooting process has a shape in which blur and blur due to the movement of the subject are superimposed. When out-of-focus occurs, the methods described in Patent Documents 3 and 4 cannot maintain a wide-band PSF frequency characteristic in the moving direction of the subject (direction opposite to the traveling direction) required for blur correction. In order to prevent defocusing, it is generally considered to secure the depth of field by narrowing the aperture of the lens, but there is a problem that the amount of incident light is reduced as in the case where the exposure time is shortened.

そこで、本発明は、入射光量の低下を抑えつつ、被写体の動き方向に広帯域のPSF特性を維持する撮影装置の提供を目的とする。   SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus that maintains a wideband PSF characteristic in the direction of movement of a subject while suppressing a decrease in the amount of incident light.

本発明は、
走行車両に搭載される撮影装置において、
被写体の画像を取得するカメラ部であって、
該被写体に光を照射する照明と、
該被写体で反射した光を通すレンズユニットであって、走行方向の開口幅は、走行方向と該カメラ部の撮影方向とに直交する方向の開口幅より狭い開口部を有する絞りと、レンズとを有するレンズユニットと、
該レンズユニットを通った光から画像データを取得する画像センサと、を含むカメラ部と、
該照明の点灯および該画像センサの露光を制御する制御部を含む処理部と、
該走行車両の速度を検出する車両速度検出部と、を含むことを特徴とする撮影装置である。
The present invention
In an imaging device mounted on a traveling vehicle,
A camera unit that acquires an image of a subject,
Illumination for illuminating the subject;
A lens unit for passing light reflected by the subject, wherein an aperture width in the traveling direction is smaller than an aperture width in a direction perpendicular to the traveling direction and the photographing direction of the camera unit, and a lens. A lens unit having
An image sensor that acquires image data from light that has passed through the lens unit; and a camera unit that includes:
A processing unit including a control unit that controls lighting of the illumination and exposure of the image sensor;
And a vehicle speed detector that detects the speed of the traveling vehicle.

本発明にかかる撮影装置では、走行方向に狭い開口部を有する絞りを用いることで、撮影中に被写体との距離が変動しても走行方向に広帯域なPSF周波数特性が維持され、復元画像からぶれとピンぼけの影響を効果的に除去できる。また、開口部は走行方向に狭ければ良いため、入射光量の低下を最小限に抑えることができる。   In the photographing apparatus according to the present invention, by using a diaphragm having a narrow opening in the traveling direction, a wide band PSF frequency characteristic is maintained in the traveling direction even when the distance to the subject fluctuates during photographing. And the effect of defocusing can be effectively removed. Moreover, since it is sufficient that the opening is narrow in the traveling direction, a decrease in the amount of incident light can be minimized.

本発明の実施の形態1にかかる撮影装置のブロック図である。It is a block diagram of the imaging device concerning Embodiment 1 of the present invention. 本発明の実施の形態1にかかる撮影装置のカメラ部のブロック図である。It is a block diagram of the camera part of the imaging device concerning Embodiment 1 of this invention. 走行方向に幅の狭い開口部を備えた絞りを適用した場合のピンぼけPSFを示す図である。It is a figure which shows defocus PSF at the time of applying the aperture_diaphragm | restriction provided with the narrow opening part in the running direction. 本発明の実施の形態1にかかる撮影装置の制御部から出力される各信号のタイミングチャートである。4 is a timing chart of each signal output from the control unit of the imaging apparatus according to the first embodiment of the present invention. 本発明の実施の形態1にかかるぶれPSFを示す図である。It is a figure which shows blurring PSF concerning Embodiment 1 of this invention. 本発明の実施の形態1にかかる絞りを用いて作成したPSFを示す図である。It is a figure which shows PSF produced using the aperture_diaphragm | restriction concerning Embodiment 1 of this invention. 従来の絞りを用いて作成したPSFを示す図である。It is a figure which shows PSF produced using the conventional aperture_diaphragm | restriction. ピンぼけ発生時におけるPSF:hの走行方向D1の周波数特性を示す図である。It is a figure which shows the frequency characteristic of the running direction D1 of PSF: h at the time of defocusing generation | occurrence | production. 本発明の実施の形態1にかかる他の撮影装置のブロック図である。It is a block diagram of the other imaging device concerning Embodiment 1 of this invention. 本発明の実施の形態2にかかる絞りを用いて作成したPSFを示す図である。It is a figure which shows PSF produced using the aperture_diaphragm | restriction concerning Embodiment 2 of this invention. ピンぼけ発生時におけるPSF:hの直交方向D2の周波数特性を示す図である。It is a figure which shows the frequency characteristic of orthogonal direction D2 of PSF: h at the time of defocusing generation | occurrence | production.

実施の形態1.
図1は、全体が100で表される、本発明の実施の形態1にかかる撮影装置の構成を示すブロック図である。撮影装置100は、撮影面となる被写体1と平行して走行する走行車両に実装される。例えば、撮影装置100は、車両に搭載してトンネル内を通り、トンネル覆工面コンクリートの撮影を行う。
Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of the photographing apparatus according to the first embodiment of the present invention, the whole of which is represented by 100. The photographing apparatus 100 is mounted on a traveling vehicle that travels in parallel with the subject 1 serving as a photographing surface. For example, the photographing apparatus 100 is mounted on a vehicle, passes through a tunnel, and photographs tunnel lining concrete.

図1において、撮影装置100は、被写体1を撮影するカメラ部10と、車両の速度を検出するための車両速度検出部20と、取得したデータを処理するための処理部40とを有する。   In FIG. 1, the photographing apparatus 100 includes a camera unit 10 that photographs a subject 1, a vehicle speed detection unit 20 that detects a vehicle speed, and a processing unit 40 that processes acquired data.

図2は、カメラ部10の構成を示す図である。カメラ部10は、被写体1を照らすための照明11と、被写体1から得られた光信号を集光するレンズユニット13と、レンズユニット13を通った光信号を検出する画像センサ12とを有する。また、図2中、D1は、カメラ部10を搭載する移動車両の移動方向であり、Zは、被写体1に直交する方向、D2は、これら2軸に直交する上下方向を示す。   FIG. 2 is a diagram illustrating the configuration of the camera unit 10. The camera unit 10 includes an illumination 11 for illuminating the subject 1, a lens unit 13 that collects an optical signal obtained from the subject 1, and an image sensor 12 that detects an optical signal that has passed through the lens unit 13. 2, D1 is the moving direction of the moving vehicle on which the camera unit 10 is mounted, Z is the direction orthogonal to the subject 1, and D2 is the vertical direction orthogonal to these two axes.

照明11は、外部からの点灯制御信号によって点灯および消灯が制御され、点灯時に被写体1に光を照射する。レンズユニット13により、被写体1から反射された光の像Iが画像センサ12の上に結像する。像Iは、被写体1の各点からの反射光が、画像センサ12上に結像した点像Pの総和からなる。点像Pは、カメラ部10と被写体1との距離Lによって異なる形状となる。レンズユニット13の被写界深度を超えて被写体1が離れるにつれて、点像Pはより広がった形状となる。   The lighting 11 is controlled to be turned on and off by a lighting control signal from the outside, and emits light to the subject 1 when turned on. An image I of light reflected from the subject 1 is formed on the image sensor 12 by the lens unit 13. The image I is composed of the sum of point images P formed on the image sensor 12 by the reflected light from each point of the subject 1. The point image P has a different shape depending on the distance L between the camera unit 10 and the subject 1. As the subject 1 moves beyond the depth of field of the lens unit 13, the point image P has a wider shape.

画像センサ12は、例えばアレイ状に配置されたCCDやCMOSのような複数の光電変換素子からなり、図3に示すように、その露光のタイミングおよび露光時間Teは、露光トリガー信号S2によって制御される。画像センサ12は、露光時間中の像Iの積算パターンを、画像データgとして処理部40に出力する。画像データg上において、所望の分解能X(mm/画素)で被写体1を撮影するように、予め画像センサ12の各画素サイズおよびレンズユニット13の焦点距離が設定される。画像データg上での点像Pは、走行車両が静止した状態で撮影した場合の点像分布関数(PSF)に相当する。このときのPSFをピンぼけPSF:h1とする。   The image sensor 12 includes a plurality of photoelectric conversion elements such as CCDs and CMOSs arranged in an array, for example. As shown in FIG. 3, the exposure timing and the exposure time Te are controlled by an exposure trigger signal S2. The The image sensor 12 outputs the integrated pattern of the image I during the exposure time to the processing unit 40 as image data g. On the image data g, the pixel size of the image sensor 12 and the focal length of the lens unit 13 are set in advance so as to photograph the subject 1 with a desired resolution X (mm / pixel). The point image P on the image data g corresponds to a point image distribution function (PSF) when the moving vehicle is photographed in a stationary state. The PSF at this time is defocused PSF: h1.

被写体1の像Iは、走行車両の移動により、画像センサ12上を相対的に移動する(逆方向に移動する)。従って、画像データgは、照明11の点灯時の像Iを移動方向にずらしながら積算したパターンとなる。露光時間中において、照明11の点灯時に像Iが画像データg上を移動した軌跡を、ぶれPSF:h2とする。走行車両が走行中に撮影した場合におけるPSF:hは、次式(1)で得られる。   The image I of the subject 1 relatively moves on the image sensor 12 (moves in the reverse direction) by the movement of the traveling vehicle. Therefore, the image data g is a pattern obtained by integrating the image I when the illumination 11 is turned on while shifting in the moving direction. A trajectory of the image I moving on the image data g when the illumination 11 is turned on during the exposure time is defined as a blur PSF: h2. PSF: h when the traveling vehicle is photographed while traveling is obtained by the following equation (1).

Figure 0006559354
Figure 0006559354

式(1)中の*は畳み込み演算を示す。   * In the formula (1) indicates a convolution operation.

レンズユニット13は、例えば2つのレンズ130、132の間に挟まれた絞り131を有する。絞り131は、走行車両の走行方向D1が、それと直交する方向D2より幅が狭くなった開口部を有する。通常のレンズユニットの場合、被写体1と走行車両との間の距離(Z方向の距離)が変動し、合焦位置から被写体1が外れた場合(非合焦)、像のボケは大きくなる。絞り131は、被写体1の反射光のうち、レンズユニット13を通過し、画像センサ12に結像される光線束の量(画像の明るさに相当)と大きさ(一般にレンズの開口数NAと呼ばれる値)を制限するものである。絞り131は、レンズ130、132の近傍であって、被写体1の撮影範囲の各点からの各光線束が互いに重なる箇所に配置され、画像センサ12と間隔をあけて配置される。一般に絞り131の開口幅が狭いほど被写界深度が深くなるが、明るさが低下する。絞り131は、その開口形状によって明るさと被写界深度を制御するものであって、画像センサ上に配置されてカメラの視野又は画角といった撮影範囲を制限するマスクとは異なる。   The lens unit 13 has a diaphragm 131 sandwiched between two lenses 130 and 132, for example. The diaphragm 131 has an opening in which the traveling direction D1 of the traveling vehicle is narrower than the direction D2 orthogonal thereto. In the case of an ordinary lens unit, the distance between the subject 1 and the traveling vehicle (distance in the Z direction) fluctuates, and the blurring of the image increases when the subject 1 is out of focus (out of focus). The diaphragm 131 passes through the lens unit 13 out of the reflected light of the subject 1 and forms the amount of light bundle (corresponding to the brightness of the image) and the size (generally the numerical aperture NA of the lens). The value called). The diaphragm 131 is disposed in the vicinity of the lenses 130 and 132 where the beam bundles from the respective points in the photographing range of the subject 1 are overlapped with each other and spaced from the image sensor 12. In general, the narrower the aperture width of the stop 131, the deeper the depth of field, but the brightness decreases. The aperture 131 controls the brightness and depth of field according to the aperture shape, and is different from a mask that is arranged on the image sensor and restricts an imaging range such as a field of view or a field angle of the camera.

これに対して、本発明の実施の形態1にかかるレンズユニット13では、走行方向D1に、幅W1の狭い矩形の開口部を有するため(W1<W2)、走行方向D1の被写界深度が深くなる。図3は、PSF:h1の形状を輝度値で表現した画像と、走行方向D1および走行方向D1と光軸Zに直交する方向D2のPSFのプロファイルを示す。   On the other hand, since the lens unit 13 according to the first embodiment of the present invention has a rectangular opening with a narrow width W1 in the traveling direction D1 (W1 <W2), the depth of field in the traveling direction D1 is small. Deepen. FIG. 3 shows an image in which the shape of PSF: h1 is expressed by a luminance value, and a PSF profile in the traveling direction D1 and the direction D2 orthogonal to the traveling direction D1 and the optical axis Z.

図3は合焦位置の場合(a)と、距離が変動して非合焦位置となった場合(b)とをそれぞれ示している。PSF:h1は、静止する被写体上の点像が画像上でどのような広がりを持つかを表す。理想的なレンズでは合焦位置で撮像した場合、被写体上の点像はレンズを介して画像センサの特定の1画素に結像される。このときのPSFは結像する1画素のみ明るく、それ以外の画素が暗い画像で表現される。   FIG. 3 shows the case (a) of the in-focus position and the case (b) of the in-focus position when the distance varies. PSF: h1 represents how the point image on the stationary subject has a spread on the image. When an image is captured at an in-focus position with an ideal lens, a point image on the subject is formed on a specific pixel of the image sensor via the lens. The PSF at this time is expressed as a bright image with only one pixel to be imaged bright and the other pixels dark.

本発明の実施の形態1にかかるレンズユニット13は、図3(a)に示すように、合焦位置で少なくともPSFの広がりが、D1方向、D2方向ともに2画素以下となるよう設計されている。   As shown in FIG. 3A, the lens unit 13 according to the first embodiment of the present invention is designed so that at least the PSF spread at the in-focus position is 2 pixels or less in both the D1 direction and the D2 direction. .

本発明の形態の形態1にかかるレンズユニット13の絞り131は、走行方向D1に幅の狭い開口部を有するため、非合焦位置の場合(b)も走行方向D1のPSFの広がりを抑えられる。走行方向D1の開口幅W1は、所定の範囲で被写体1と走行車両との間の距離が変動した場合においても、走行方向D1にピンぼけが発生しないように設定される。すなわち、図3(b)に示すように、幅W1は、被写体1と走行車両との間の距離が所定の範囲内の場合、走行方向D1での被写体1からのピンボケPSF:h1の広がりが、2画素以下となるよう設定される(D1プロファイル)。一方、走行方向D1と光軸Zに直交する方向D2に関する絞り131の幅W2は、被写体1が画像データg上で十分な明るさの階調値をもつよう設定される(D2プロファイル)。   Since the diaphragm 131 of the lens unit 13 according to the first embodiment of the present invention has a narrow opening in the traveling direction D1, the spread of the PSF in the traveling direction D1 can be suppressed even in the out-of-focus position (b). . The opening width W1 in the traveling direction D1 is set so that no blur occurs in the traveling direction D1 even when the distance between the subject 1 and the traveling vehicle varies within a predetermined range. That is, as shown in FIG. 3B, when the distance between the subject 1 and the traveling vehicle is within a predetermined range, the width W1 is the extent of the out-of-focus PSF: h1 from the subject 1 in the traveling direction D1. It is set to be 2 pixels or less (D1 profile). On the other hand, the width W2 of the stop 131 with respect to the traveling direction D1 and the direction D2 orthogonal to the optical axis Z is set so that the subject 1 has a gradation value with sufficient brightness on the image data g (D2 profile).

また、車両速度検出部20は、走行車両に搭載したオドメータなどを介して、走行車両の車速Vを検出する。   Moreover, the vehicle speed detection part 20 detects the vehicle speed V of a traveling vehicle via the odometer etc. which were mounted in the traveling vehicle.

また、被写体距離検出部30は、カメラ部10と同じ方向、即ち図2のZ方向に配向され、被写体1までの距離Lを検出する。   Further, the subject distance detection unit 30 is oriented in the same direction as the camera unit 10, that is, the Z direction in FIG. 2, and detects the distance L to the subject 1.

更に、処理部40は、制御部41、ピンぼけPSFテーブル記録部42、ピンぼけPSF算出部43、ぶれPSF算出部44、PSF算出部45、画像復元演算部46、および画像記録部47を有する。   The processing unit 40 further includes a control unit 41, a defocused PSF table recording unit 42, a defocused PSF calculation unit 43, a blur PSF calculation unit 44, a PSF calculation unit 45, an image restoration calculation unit 46, and an image recording unit 47.

ここで、図4は、本発明の実施の形態1にかかる撮影装置の制御部から出力される各信号のタイミングチャートである。制御部41は、画像センサ12に対して、図4に示すような露光トリガー信号S2を出力すると共に、画像センサ12の露光のタイミングと露光時間とを制御する。また、制御部41は、照明11に対して、図4に示すような点灯制御信号S1を出力する。このとき、点灯制御信号S1は、所定の符号化系列Cに基づいて決定される。このようにして、制御部41は露光時間中の露光状態を符号化変調する。   Here, FIG. 4 is a timing chart of each signal output from the control unit of the photographing apparatus according to the first embodiment of the present invention. The control unit 41 outputs an exposure trigger signal S2 as shown in FIG. 4 to the image sensor 12, and controls the exposure timing and exposure time of the image sensor 12. Further, the control unit 41 outputs a lighting control signal S1 as shown in FIG. At this time, the lighting control signal S1 is determined based on a predetermined coding sequence C. In this way, the control unit 41 encodes and modulates the exposure state during the exposure time.

ところで、前記絞り131の実装により、ピンぼけPSF:h1は走行方向D1に広がりがない一方で、走行方向D1と光軸Zに直交する方向D2にカメラ部10と被写体1との間の距離L1に応じてわずかに広がりをもつ。本実施の形態1における処理部40は、走行方向D1へのぶれの補正と併せて、方向D2のピンぼけPSF:h1の広がりの補正処理を行う。
ピンぼけPSFテーブル記録部42には、予め、距離L1に対応したピンぼけPSF:h1が記録される。
By the way, due to the mounting of the diaphragm 131, the defocused PSF: h1 does not spread in the traveling direction D1, while the distance L1 between the camera unit 10 and the subject 1 in the direction D2 orthogonal to the traveling direction D1 and the optical axis Z. Slightly broadens accordingly. The processing unit 40 according to the first embodiment performs a correction process of the spread of the defocused PSF: h1 in the direction D2 together with the correction of the shake in the traveling direction D1.
The out-of-focus PSF table recording unit 42 records in advance a out-of-focus PSF: h1 corresponding to the distance L1.

ピンぼけPSF選択部43は、制御部41により制御され、画像センサ12の露光のタイミングで、被写体距離検出部30から、走行車両と被写体1との距離L1を取得する。次に、ピンぼけPSF選択部43は、ピンぼけPSFテーブル記録部42を参照し、距離L1に対応するピンぼけPSF:h1を取得する。   The out-of-focus PSF selection unit 43 is controlled by the control unit 41, and acquires the distance L1 between the traveling vehicle and the subject 1 from the subject distance detection unit 30 at the exposure timing of the image sensor 12. Next, the out-of-focus PSF selection unit 43 refers to the out-of-focus PSF table recording unit 42 and acquires the out-of-focus PSF: h1 corresponding to the distance L1.

ぶれPSF算出部44は、制御部41により制御され、画像センサ12の露光のタイミングで車両速度検出部20から車速Vを取得する。次に、ぶれPSF算出部44は、前述の照明11の点灯パターンと車速Vから、ぶれPSF:h2を算出する。画像上では、被写体1は、走行車両の走行方向D1と反対方向に相対的に移動する。図5はPSF:h2の形状を輝度値で表現した画像である。縦方向が走行方向D1、横方向が上下方向D2である。画像上での明暗は露光時間中の画像上での被写体1の点像の動きの軌跡に相当する。軌跡の幅P(画素)は、走行車両の走行速度Vおよび撮影分解能X、露光時間Tから次式(2)のように算出される。   The shake PSF calculation unit 44 is controlled by the control unit 41 and acquires the vehicle speed V from the vehicle speed detection unit 20 at the exposure timing of the image sensor 12. Next, the shake PSF calculation unit 44 calculates the shake PSF: h2 from the lighting pattern of the illumination 11 and the vehicle speed V described above. On the image, the subject 1 moves relatively in the direction opposite to the traveling direction D1 of the traveling vehicle. FIG. 5 is an image in which the shape of PSF: h2 is expressed by a luminance value. The vertical direction is the traveling direction D1, and the horizontal direction is the vertical direction D2. The contrast on the image corresponds to the movement trajectory of the point image of the subject 1 on the image during the exposure time. The trajectory width P (pixel) is calculated by the following equation (2) from the traveling speed V of the traveling vehicle, the imaging resolution X, and the exposure time T.

Figure 0006559354
Figure 0006559354

PSF:h2は、被写体1の動きに、露光時間中の照明の点灯消灯の状態を重畳して算出される。図5に示すPSF画像中の輝度値の高い画素は照明が点灯中に、輝度値の低い画素は消灯中に、それぞれ被写体1の点像が画像センサの画素間を通過したことを示す。具体的には、PSF:h2は、軌跡の幅Pに合わせて符号化系列Cを拡大または縮小し、補間処理することで得られる。   PSF: h2 is calculated by superimposing the lighting on / off state during the exposure time on the movement of the subject 1. A pixel with a high luminance value in the PSF image shown in FIG. 5 indicates that the point image of the subject 1 has passed between the pixels of the image sensor while the illumination is on and the pixel with a low luminance value is off. Specifically, PSF: h2 is obtained by enlarging or reducing the encoded sequence C according to the width P of the trajectory and performing an interpolation process.

PSF算出部45は、ピンぼけPSF選択部43よりピンぼけPSF:h1を、ぶれPSF算出部44よりぶれPSF:h2をそれぞれ取得し、式(1)に基づいてPSF:hを算出する。   The PSF calculation unit 45 acquires the out-of-focus PSF: h1 from the out-of-focus PSF selection unit 43 and the out-of-motion PSF: h2 from the out-of-focus PSF calculation unit 44, and calculates PSF: h based on Expression (1).

画像復元演算部46は、画像センサ12から画像データgを、またPSF算出部45からPSF:hをそれぞれ取得する。次に、画像復元演算部46は、画像データgおよびPSF:hをそれぞれフーリエ変換したG(u,v)、H(u,v)を算出する。u、vは画像データgの水平および垂直の各方向の周波数空間上での座標を表す。   The image restoration calculation unit 46 obtains image data g from the image sensor 12 and PSF: h from the PSF calculation unit 45. Next, the image restoration calculation unit 46 calculates G (u, v) and H (u, v) obtained by performing Fourier transform on the image data g and PSF: h, respectively. u and v represent the coordinates in the frequency space of the horizontal and vertical directions of the image data g.

また、画像復元演算部46は、次式(3)に示すウィナーフィルタ関数を用いた逆フィルタ演算による画像補正を実行する。   Further, the image restoration calculation unit 46 performs image correction by inverse filter calculation using a Wiener filter function expressed by the following equation (3).

Figure 0006559354
Figure 0006559354

ここで、HはHの複素共役、σはウィナーフィルタの安定化定数である。また、Fは補正画像データfがフーリエ変換されたものである。Here, H * is the complex conjugate of H, and σ is the stabilization constant of the Wiener filter. F represents the corrected image data f that has been subjected to Fourier transform.

さらに、画像復元演算部46は、前述のようにして算出したFに対して逆フーリエ変換を実行し、補正画像データfを算出し、画像記録部47に出力する。   Further, the image restoration calculation unit 46 performs inverse Fourier transform on F calculated as described above, calculates corrected image data f, and outputs it to the image recording unit 47.

画像記録部47は、画像復元演算部46より補正画像データfを取得し、これを随時記録する。   The image recording unit 47 acquires the corrected image data f from the image restoration calculation unit 46 and records it as needed.

図6は、走行方向D1に幅の狭い開口部を有する絞り131の概略図と、この絞り131を用いた場合の、ピンぼけPSF:h1、ぶれPSF:h2、およびPSF:hの例を示す。左の図が絞り131の概略図で、右の2列の図の、上段がピンぼけPSF:h1の合焦および非合焦、中段がぶれPSF:h2、下段が、それらを重畳したPSF:hの合焦および非合焦を示す。これらにおいて、左右方向が走行方向D1、上下方向がD2である。   FIG. 6 shows a schematic diagram of a diaphragm 131 having a narrow opening in the traveling direction D1, and an example of out-of-focus PSF: h1, blur PSF: h2, and PSF: h when this diaphragm 131 is used. The left figure is a schematic diagram of the diaphragm 131, and the upper two figures in the right column are in-focus and non-focused with a defocused PSF: h1, the middle part is blurred PSF: h2, and the lower part is a superposed PSF: h. In-focus and out-of-focus. In these, the left-right direction is the running direction D1, and the up-down direction is D2.

ぶれPSF:h2の幅は、走行車両の車速Vと露光時間Teによって決まる。例えば、車速V=100km/h、露光時間Te=1(ms)、分解能X=0.5(mm/画素)の場合、画像データg上での露光時間中の像Iの移動量、すなわち、ぶれPSF:h2の形状の幅Pは約56画素である。また、ぶれPSF:h2の形状は、照明11の点灯パターンとなる点灯制御信号S1によって決まる。点灯制御信号S1は、ぶれPSF:h2の周波数特性が広帯域かつフラットとなるように決定することが望ましい。ここでは、露光時間中の照明11の点灯パターンに、特許文献3に記載の以下の符号系列を適用して図示している。   The width of the blur PSF: h2 is determined by the vehicle speed V and the exposure time Te of the traveling vehicle. For example, when the vehicle speed V = 100 km / h, the exposure time Te = 1 (ms), and the resolution X = 0.5 (mm / pixel), the moving amount of the image I during the exposure time on the image data g, that is, The width P of the shape of blur PSF: h2 is about 56 pixels. The shape of the blur PSF: h2 is determined by a lighting control signal S1 that is a lighting pattern of the illumination 11. It is desirable that the lighting control signal S1 is determined so that the frequency characteristic of the blur PSF: h2 is wide and flat. Here, the following code sequence described in Patent Document 3 is applied to the lighting pattern of the illumination 11 during the exposure time.

1010000111000001010000110011110111010111001001100111   101000011100000101000011001111011110101110001001100111

走行車両が、被写体1に対してピントが合う距離を走行した場合(以下、「合焦条件」という。)、ピンぼけPSF:h1の広がりは小さい。図6では、広がりが1画素に収まるものとして図示している。   When the traveling vehicle travels a distance that is in focus with respect to the subject 1 (hereinafter referred to as “focusing condition”), the spread of the defocused PSF: h1 is small. In FIG. 6, the spread is illustrated as being within one pixel.

一方、走行車両が被写体1に対してピントが合わない距離を走行した場合(以下、「非合焦条件」という。)、通常の円形開口絞りを有するレンズユニットでは点像Pはピンぼけしたものとなる。   On the other hand, when the traveling vehicle travels a distance that is out of focus with respect to the subject 1 (hereinafter referred to as “out-of-focus condition”), the point image P is out of focus in a lens unit having a normal circular aperture stop. Become.

これに対して、本発明の実施の形態1にかかる撮影装置100では、結像するレンズユニット13は、走行方向D1に幅の狭い開口部を備えた絞り131を有する(W1<W2)。そのため、非合焦条件であっても、ピンぼけPSF:h1は直交方向D2へは広がった形状となるが、走行方向D1には広がりが抑えられる。図6では、ピンぼけPSF:h1の走行方向D1の広がりは、1画素に収まるものとし、直交方向D2の広がりは、ガウシアン分布で半値全幅5画素分を有するものとして図示している。このとき、PSF:hは走行方向D1の形状は、ぶれPSF:h2の形状に維持される。このため、非合焦条件時においても、PSF:hの走行方向D1の周波数特性は、広帯域かつフラットな周波数特性となるよう制御されたぶれPSF:h2の特性を引き継ぐ(PSF:h(非合焦)参照)。   On the other hand, in the photographing apparatus 100 according to the first embodiment of the present invention, the lens unit 13 that forms an image has the diaphragm 131 having a narrow opening in the traveling direction D1 (W1 <W2). Therefore, even under the out-of-focus condition, the out-of-focus PSF: h1 has a shape that spreads in the orthogonal direction D2, but the spread is suppressed in the traveling direction D1. In FIG. 6, the spread in the running direction D1 of the out-of-focus PSF: h1 is assumed to be within one pixel, and the spread in the orthogonal direction D2 is illustrated as having a full width at half maximum of 5 pixels in a Gaussian distribution. At this time, PSF: h maintains the shape of the running direction D1 in the shape of the shake PSF: h2. For this reason, even in the out-of-focus condition, the frequency characteristic in the running direction D1 of PSF: h inherits the characteristic of the blur PSF: h2 that is controlled so as to be a broadband and flat frequency characteristic (PSF: h (non-focusing). (See Jiao))).

図7は、比較例として、明るさを維持した円形開口絞りを適用した場合の、ピンぼけPSF:h1、ぶれPSF:h2、およびPSF:hを示す。左の図が円形開口絞りの概略図で、右の2列の図の、上段がピンぼけPSF:h1の合焦および非合焦、中段がぶれPSF:h2、下段が、それらを重畳したPSF:hの合焦および非合焦を示す。これらにおいて、左右方向が走行方向D1、上下方向がD2である。   FIG. 7 shows, as a comparative example, out-of-focus PSF: h1, blur PSF: h2, and PSF: h when a circular aperture stop that maintains brightness is applied. The figure on the left is a schematic diagram of a circular aperture stop, and in the two rows on the right, the upper stage is out-of-focus PSF: h1 in-focus and out-of-focus, the middle stage is blurred PSF: h2, and the lower stage is a PSF on which they are superimposed: h indicates in-focus and out-of-focus. In these, the left-right direction is the running direction D1, and the up-down direction is D2.

図7に示すように、非合焦条件において、ピンぼけPSF:h1の形状は、全方向に広がったものとなり、ここでは、ガウシアン分布で半値全幅5画素分を有するものとして図示している。このとき、ピンぼけPSF:h1は走行方向D1にも広がりをもつため、図7のように、PSF:hはぶれPSF:h2の走行方向D1の形状を引き継ぐことができない(PSF:h(非合焦)参照)。   As shown in FIG. 7, in the out-of-focus condition, the shape of the defocused PSF: h1 spreads in all directions, and is illustrated here as having a full width at half maximum of 5 pixels in a Gaussian distribution. At this time, the out-of-focus PSF: h1 also spreads in the traveling direction D1, and as shown in FIG. 7, PSF: h cannot take over the shape of the traveling direction D1 of the blurred PSF: h2 (PSF: h (See Jiao))).

図8に、ピンぼけ発生時におけるPSF:hの走行方向D1の周波数特性を示す。横軸が周波数、縦軸が輝度の対数を示す。また実線は、走行方向に幅の狭い開口部(図7)を備えた絞りを適用した場合(実施例)、破線は、円形開口絞り(図8)を適用した場合(比較例)を示す。   FIG. 8 shows the frequency characteristic of the running direction D1 of PSF: h when defocusing occurs. The horizontal axis represents frequency and the vertical axis represents the logarithm of luminance. A solid line indicates a case where an aperture having a narrow opening (FIG. 7) in the traveling direction is applied (Example), and a broken line indicates a case where a circular aperture stop (FIG. 8) is applied (Comparative Example).

図8では、ぶれPSF:h2の形状と点像Pの広がりは、図6、7の例に示した条件で算出している。図8によると、走行方向D1に幅の狭い開口の絞り131を組み合わせた本実施の形態にかかる例では、非合焦条件においても、PSF:hは広帯域かつフラットな走行方向D1の周波数特性を維持できている。一方、一般的な円形開口絞りを用いた比較例では、ぶれに対して十分小さなピンぼけであっても(図7ではぶれ56画素、ピンぼけ5画素としている)、PSF:hの走行方向D1の周波数特性は、高い周波数帯域で低くなってしまい、特許文献3や特許文献4で期待される復元性能や補正性能が得られないことが分かる。   In FIG. 8, the shape of the blur PSF: h2 and the spread of the point image P are calculated under the conditions shown in the examples of FIGS. According to FIG. 8, in the example according to the present embodiment in which the aperture 131 having a narrow opening is combined with the traveling direction D1, PSF: h has a broadband and flat frequency characteristic in the traveling direction D1 even under out-of-focus conditions. Maintained. On the other hand, in a comparative example using a general circular aperture stop, the frequency in the running direction D1 of PSF: h is sufficient even if the blur is sufficiently small with respect to blur (in FIG. 7, the blur is 56 pixels and the blur is 5 pixels). It can be seen that the characteristics are low in the high frequency band, and the restoration performance and correction performance expected in Patent Document 3 and Patent Document 4 cannot be obtained.

このように、本発明の実施の形態1にかかる撮影装置100は、走行方向D1に幅の狭い開口部を備えた絞り131を有するレンズユニット13を含むため、撮影中に、カメラ部10と被写体1との距離が変動して非合焦条件となっても、走行方向D1に広帯域かつフラットなPSF周波数特性が維持され、画像復元処理の際、ぶれとピンぼけとを効果的に除去できる。また、円形開口絞りを単純に絞った場合に比べて、絞り131では、走行方向D1にのみ幅を絞っており、画像データg上の明るさの低下を最小限に抑えることができる。   As described above, since the imaging apparatus 100 according to the first embodiment of the present invention includes the lens unit 13 having the aperture 131 having the narrow opening in the traveling direction D1, the camera unit 10 and the subject are captured during imaging. Even if the distance to 1 fluctuates and the in-focus condition is reached, a wide and flat PSF frequency characteristic is maintained in the traveling direction D1, and blur and defocus can be effectively removed during the image restoration process. Further, compared with a case where the circular aperture stop is simply reduced, the stop 131 reduces the width only in the traveling direction D1, and the reduction in brightness on the image data g can be minimized.

なお、本発明の実施の形態1にかかる撮影装置100では、絞り131の開口部の開口形状は、走行車両の走行方向D1が、それと直交する方向D2より幅の狭い矩形形状を有しているが、開口形状は、非合焦条件下でピンぼけPSF:h1が走行方向D1に広がりをもたない形状であればよい。絞り131の開口部の開口形状は、例えば、走行車両の走行方向D1が、それと直交する方向D2より幅の狭い長方形状であって、開口部のコーナがR面取り又はC面取りされた形状であってもよい。絞り131の開口部の開口形状は、例えば、走行車両の走行方向D1を長軸、それと直交する方向D2を短軸とする楕円形状であってもよい。   In the imaging device 100 according to the first embodiment of the present invention, the opening shape of the opening portion of the diaphragm 131 has a rectangular shape in which the traveling direction D1 of the traveling vehicle is narrower than the direction D2 perpendicular to the traveling direction. However, the opening shape may be a shape in which the out-of-focus PSF: h1 does not spread in the traveling direction D1. The opening shape of the opening of the diaphragm 131 is, for example, a shape in which the traveling direction D1 of the traveling vehicle is narrower than the direction D2 perpendicular to the traveling direction, and the corner of the opening is rounded or chamfered. May be. The opening shape of the opening of the diaphragm 131 may be, for example, an elliptical shape in which the traveling direction D1 of the traveling vehicle is the long axis and the direction D2 orthogonal thereto is the short axis.

なお、本発明の実施の形態1にかかる撮影装置100では、ピンぼけPSFテーブル記録部42に、カメラ部10と被写体1との間の距離Lに対応したピンぼけPSF:h1が予め記録され、ピンぼけPSF選択部43が、ピンぼけPSFテーブル記録部42を参照し、距離Lに対応するピンぼけPSF:h1を取得していたが、ピンぼけPSF:h1は、実際に画像センサ12で取得した画像データgから直接取得しても良い。   In the photographing apparatus 100 according to the first embodiment of the present invention, the defocus PSF: h1 corresponding to the distance L between the camera unit 10 and the subject 1 is recorded in advance in the defocus PSF table recording unit 42, and the defocus PSF. The selection unit 43 refers to the defocused PSF table recording unit 42 and acquires the defocused PSF: h1 corresponding to the distance L. However, the defocused PSF: h1 is directly from the image data g actually acquired by the image sensor 12. You may get it.

図9は、本発明の実施の形態1にかかる他の撮影装置200の撮影装置のブロック図であり、図9中、図1と同一符合は、同一または相当箇所を示す。撮影装置200では、撮影装置100に比べて、被写体距離検出部30、ピンぼけPSFテーブル記録部42、ピンぼけPSF選択部43、およびPSF算出部45が省略されている。   FIG. 9 is a block diagram of a photographing apparatus of another photographing apparatus 200 according to the first embodiment of the present invention. In FIG. 9, the same reference numerals as those in FIG. 1 indicate the same or corresponding parts. In the photographing apparatus 200, the subject distance detection unit 30, the defocused PSF table recording unit 42, the defocused PSF selection unit 43, and the PSF calculation unit 45 are omitted as compared with the photographing apparatus 100.

画像復元演算部46は、画像センサ12から画像データgを取得する。この時、画像データgから直接、合焦条件および非合焦状条件のピンぼけPSF:h1を取得する。   The image restoration calculation unit 46 acquires the image data g from the image sensor 12. At this time, the out-of-focus condition and the out-of-focus condition defocus PSF: h1 are acquired directly from the image data g.

一方、ぶれPSF算出部44は、制御部41により制御され、画像センサ12の露光のタイミングで車両速度検出部20から車速Vを取得し、前述の照明11の点灯パターンと車速Vから、ぶれPSF:h2を算出する。   On the other hand, the blur PSF calculation unit 44 is controlled by the control unit 41, acquires the vehicle speed V from the vehicle speed detection unit 20 at the exposure timing of the image sensor 12, and calculates the blur PSF from the lighting pattern of the illumination 11 and the vehicle speed V described above. : Calculate h2.

画像復元演算部46において、上述のようにPSF:h1とPSF:h2とをそれらを重畳してPSF:hを得る。   In the image restoration calculation unit 46, PSF: h is obtained by superimposing PSF: h1 and PSF: h2 as described above.

このように、撮影蔵置200では、より簡単な構成で、PSF:hの取得が可能となる。   As described above, the photographing storage 200 can acquire PSF: h with a simpler configuration.

実施の形態2.
本発明の実施の形態2にかかる撮影装置では、実施の形態1の走行方向D1に幅の狭い開口部を有する絞り131に代えて、走行方向D1に幅が狭く、かつ直交方向D2に符号化された開口部を備えた絞りを用いる。他の構造は撮影装置100と同様である。
Embodiment 2. FIG.
In the photographing apparatus according to the second embodiment of the present invention, instead of the diaphragm 131 having a narrow opening in the traveling direction D1 of the first embodiment, the width is narrow in the traveling direction D1 and the encoding is performed in the orthogonal direction D2. A diaphragm with an aperture is used. Other structures are the same as those of the photographing apparatus 100.

図10は、走行方向D1に幅が狭く、かつ直交方向D2に符号化された開口絞りを適用した場合の、ピンぼけPSF:h1、ぶれPSF:h2、およびPSF:hを示す。左の図がこのような開口の概略図で、右の2列の図の、上段がピンぼけPSF:h1の合焦および非合焦、中段がぶれPSF:h2、下段が、それらを重畳したPSF:hの合焦および非合焦を示す。これらにおいて、左右方向が走行方向D1、上下方向がD2である。   FIG. 10 shows out-of-focus PSF: h1, blur PSF: h2, and PSF: h when an aperture stop that is narrow in the traveling direction D1 and encoded in the orthogonal direction D2 is applied. The figure on the left is a schematic diagram of such an opening, and the upper row of the two rows on the right is in-focus and out-of-focus PSF: h1, the middle is blurred PSF: h2, and the lower is the PSF on which they are superimposed. : Indicates in-focus and out-of-focus in h. In these, the left-right direction is the running direction D1, and the up-down direction is D2.

図10では、非合焦条件において、ピンぼけPSF:h1の形状は、走行方向D1のみならず、符号化した直交方向D2でも拡がりが押さえられている。この結果、ぶれPSF:h2と重畳した場合、PSF:hは、ぶれPSF:h2の走行方向D1の形状を、D1、D2の両方向において引き継ぐことができる(PSF:h(非合焦)参照)。   In FIG. 10, in the out-of-focus condition, the spread of the defocused PSF: h1 is suppressed not only in the traveling direction D1 but also in the encoded orthogonal direction D2. As a result, when superimposed on the shake PSF: h2, the PSF: h can take over the shape of the running direction D1 of the shake PSF: h2 in both directions D1 and D2 (see PSF: h (unfocused)). .

図11は、ピンぼけ発生時におけるPSF:hの直交方向D2の周波数特性(PSF:h(非合焦)参照。)であり、横軸は周波数、縦軸は輝度の対数を示す。実施の形態1の図10に示したPSF:hの走行方向D1の周波数特性と同様に、図11に示す周波数特性は広帯域かつフラットである。   FIG. 11 shows frequency characteristics in the orthogonal direction D2 of PSF: h when defocusing occurs (see PSF: h (out-of-focus)). The horizontal axis represents frequency and the vertical axis represents logarithm of luminance. Similar to the frequency characteristic in the traveling direction D1 of PSF: h shown in FIG. 10 of the first embodiment, the frequency characteristic shown in FIG. 11 is a broadband and flat.

このように、走行方向D1に狭く、直交方向D2に符号化した開口部の絞りを適用することで、直交方向D2も広帯域かつフラットなPSF:hの周波数特性が得られ、画像復元処理の際、ぶれとピンぼけとを効果的に除去できる。   As described above, by applying the aperture stop narrow in the traveling direction D1 and encoded in the orthogonal direction D2, the frequency characteristic of the PSF: h that is wide and flat in the orthogonal direction D2 can be obtained. It is possible to effectively remove blurring and defocusing.

なお、本実施の形態2で用いた絞りは、上述の撮影装置200にも適用することができる。   The aperture used in the second embodiment can also be applied to the photographing apparatus 200 described above.

1 被写体、10 カメラ部、20 車両速度検出部、30 被写体距離検出部、40 処理部、11 照明、12 画像センサ、13 レンズユニット、130、132 レンズ、131 走行方向に幅の狭い開口の絞り、41 制御部、42 ピンぼけPSFテーブル記録部、43 ピンぼけPSF選択部、44 ぶれPSF算出部、45 PSF算出部、46 画像復元演算部、47 画像記録部、100、200 撮影装置。   DESCRIPTION OF SYMBOLS 1 Subject, 10 Camera part, 20 Vehicle speed detection part, 30 Subject distance detection part, 40 Processing part, 11 Illumination, 12 Image sensor, 13 Lens unit, 130, 132 Lens, 131 Aperture with a narrow opening in the running direction, 41 Control unit, 42 Out-of-focus PSF table recording unit, 43 Out-of-focus PSF selection unit, 44 Blur PSF calculation unit, 45 PSF calculation unit, 46 Image restoration operation unit, 47 Image recording unit, 100, 200

Claims (7)

走行車両に搭載される撮影装置において、
被写体の画像を取得するカメラ部であって、
該被写体に光を照射する照明と、
該被写体で反射した光を通すレンズユニットであって、走行方向の開口幅は、走行方向と該カメラ部の撮影方向とに直交する方向の開口幅より狭い開口部を有する絞りと、レンズとを有するレンズユニットと、
該レンズユニットを通った光から画像データを取得する画像センサと、を含むカメラ部と、
該照明の点灯および該画像センサの露光を制御する制御部を含む処理部と、
該走行車両の速度を検出する車両速度検出部と、を含むことを特徴とする撮影装置。
In an imaging device mounted on a traveling vehicle,
A camera unit that acquires an image of a subject,
Illumination for illuminating the subject;
A lens unit for passing light reflected by the subject, wherein an aperture width in the traveling direction is smaller than an aperture width in a direction perpendicular to the traveling direction and the photographing direction of the camera unit, and a lens. A lens unit having
An image sensor that acquires image data from light that has passed through the lens unit; and a camera unit that includes:
A processing unit including a control unit that controls lighting of the illumination and exposure of the image sensor;
And a vehicle speed detector for detecting the speed of the traveling vehicle.
上記絞りは、画像センサから間隔をおいて配置されることを特徴とする請求項1に記載の撮影装置。   The photographing apparatus according to claim 1, wherein the diaphragm is disposed at a distance from the image sensor. 上記絞りの開口部は、更に、走行方向に直交する方向に符号化されたことを特徴とする請求項1または2に記載の撮影装置。   The photographing apparatus according to claim 1, wherein the aperture of the diaphragm is further encoded in a direction orthogonal to the traveling direction. 更に、上記撮影装置と上記被写体との間の距離を測定する被写体距離検出部を含み、
上記処理部は、更に、ぶれPSF算出部と、画像復元演算部とを含むことを特徴とする請求項1〜3のいずれかに記載の撮影装置。
And a subject distance detecting unit for measuring a distance between the photographing apparatus and the subject.
The imaging apparatus according to claim 1, wherein the processing unit further includes a blur PSF calculation unit and an image restoration calculation unit.
上記処理部は、上記走行車両の走行速度および方向と、上記撮影装置と上記被写体との距離と、上記画像データとに基づいて、該画像データのPSFを算出し、算出した該PSFに基づいて画像復元処理を行って復元画像を得たことを特徴とする請求項1〜4のいずれかに記載の撮影装置。   The processing unit calculates a PSF of the image data based on the traveling speed and direction of the traveling vehicle, the distance between the imaging device and the subject, and the image data, and based on the calculated PSF. 5. The photographing apparatus according to claim 1, wherein a restored image is obtained by performing an image restoration process. 上記PSFは、ピンぼけPSFと、ぶれPSFとを、上記走行車両の走行方向に畳み込み演算して作成したことを特徴とする請求項5に記載の撮影装置。   6. The photographing apparatus according to claim 5, wherein the PSF is created by convolution calculation of a defocused PSF and a blur PSF in a traveling direction of the traveling vehicle. 上記ピンぼけPSFは、被写体距離検出部が検出した距離に応じて、予め記録されたピンぼけPSFが記録部から選択されることを特徴とする請求項6に記載の撮影装置。   7. The photographing apparatus according to claim 6, wherein a defocus PSF recorded in advance is selected from the recording unit according to the distance detected by the subject distance detection unit.
JP2018531820A 2016-08-03 2017-07-19 Imaging device Active JP6559354B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2016153102 2016-08-03
JP2016153102 2016-08-03
PCT/JP2017/026037 WO2018025633A1 (en) 2016-08-03 2017-07-19 Imaging device

Publications (2)

Publication Number Publication Date
JPWO2018025633A1 JPWO2018025633A1 (en) 2019-01-10
JP6559354B2 true JP6559354B2 (en) 2019-08-14

Family

ID=61073361

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2018531820A Active JP6559354B2 (en) 2016-08-03 2017-07-19 Imaging device

Country Status (2)

Country Link
JP (1) JP6559354B2 (en)
WO (1) WO2018025633A1 (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5366229A (en) * 1976-11-25 1978-06-13 Fuji Photo Film Co Ltd Photographic device for stereophotography
JPH03223616A (en) * 1989-12-20 1991-10-02 Fuji Electric Co Ltd Photoelectric range finder
JP6328447B2 (en) * 2014-03-07 2018-05-23 西日本高速道路エンジニアリング関西株式会社 Tunnel wall surface photographing device

Also Published As

Publication number Publication date
WO2018025633A1 (en) 2018-02-08
JPWO2018025633A1 (en) 2019-01-10

Similar Documents

Publication Publication Date Title
JP5388671B2 (en) Imaging apparatus and image conversion method
JP5132401B2 (en) Image processing apparatus and image processing method
US8849055B2 (en) Image processing apparatus and method
JP5898481B2 (en) Imaging apparatus and focus detection method
JP2009070384A (en) Video restoration device and method
JP4420921B2 (en) Imaging device
JP5237978B2 (en) Imaging apparatus and imaging method, and image processing method for the imaging apparatus
JP5947601B2 (en) FOCUS DETECTION DEVICE, ITS CONTROL METHOD, AND IMAGING DEVICE
JP2011043646A (en) Image capturing apparatus and method
JP6353233B2 (en) Image processing apparatus, imaging apparatus, and image processing method
JP2008033060A (en) Imaging apparatus and imaging method, and image processor
JP5712031B2 (en) Imaging device, photographing lens unit, and imaging unit
JP5487955B2 (en) Imaging apparatus and imaging method
JP2009260622A (en) Imaging apparatus
JP6559354B2 (en) Imaging device
JP5409588B2 (en) Focus adjustment method, focus adjustment program, and imaging apparatus
JP2015204566A (en) camera system
JP6436840B2 (en) Image processing apparatus, imaging apparatus, image processing method, image processing program, and storage medium
JP2009047735A (en) Imaging apparatus and image processing program
JP2007214662A (en) Solid-state imaging apparatus
JP6570230B2 (en) Imaging apparatus and imaging method
JP2018110299A (en) Image processing method, image processing device, and imaging device
JP6604737B2 (en) Image processing apparatus, imaging apparatus, image processing method, image processing program, and storage medium
JP6628510B2 (en) Focus adjustment device control device, imaging device, focus adjustment device control method, program, and storage medium
JP6671963B2 (en) Image processing apparatus, imaging apparatus, image processing method, image processing program, and recording medium

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20180802

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20190618

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20190716

R150 Certificate of patent or registration of utility model

Ref document number: 6559354

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250