JP4270776B2

JP4270776B2 - Method of and apparatus for emphasizing high frequency or middle high frequency in pixel shifted signal

Info

Publication number: JP4270776B2
Application number: JP2001222638A
Authority: JP
Inventors: 宏一浜田; 正幸菅原; 勝金澤; 文男岡野
Original assignee: Japan Broadcasting Corp
Current assignee: Japan Broadcasting Corp
Priority date: 2001-07-24
Filing date: 2001-07-24
Publication date: 2009-06-03
Anticipated expiration: 2021-07-24
Also published as: JP2003037753A

Description

【０００１】
【発明の属する技術分野】
本発明は、固体撮像装置に関し、特に、固体撮像素子を用いるテレビジョンカメラに適用して、カメラの出力信号の高域または中高域を強調する処理（エンハンス処理）を行い、画質を改善する技術、例えば２つの固体撮像素子の画素を１／２画素分ずらした位置に配置し、画素数を４倍として撮影する技術に適用して有効な技術に関するものである。
【０００２】
【従来の技術】
従来の固体撮像装置については、例えば、特開平９−３７１２６号公報に記載されている。前記公報の図１８には、１チャンネル当たり２枚の固体撮像素子を用い、空間画素ずらし法による位置に画素を配置した固体撮像装置の構成が開示されている。その固体撮像装置は、ハーフミラー１、固体撮像素子２及び３、プリプロセス回路４及び５、遅延素子６、加算器８、及びプロセス回路９から構成されている。
【０００３】
前記従来の固体撮像装置は、ハーフミラー１で入射される被写体の光学像を分割し、分割された光学像を固体撮像素子２、３で受光し、受光した光学像を電気信号に変換する。変換された電気信号は、プリプロセス回路４、５で、ノイズ除去及びそれぞれの出力レベルが等しくなるようにゲイン調整が行われ、ゲイン調整が行われた信号は、２つの固体撮像素子の出力信号を同位相にするため、遅延素子６で位相調整が行われる。位相調整が行われた２つの信号は、加算器で加算され、その加算された信号はプロセス回路で最終的な映像信号に必要な処理をされ、出力される。また、２つの固体撮像装置は、前記公報の図２０に示すように、光学的に水平方向にずらして配置されている。
【０００４】
前記従来の技術によれば、空間画素ずらし法において、空間画素ずらし誤差による解像度改善効果の劣化を補正する方法が開示されている。その手段として、前記公報の図１の構成において、固体撮像装置３に対し空間的にずらして配置された固体撮像素子２の出力信号に遅延素子６を接続して第１の位相調整を行い、さらに、第１の位相調整を行って得られる出力信号Ｇ２に対し、入力信号の周波数に応じた遅延量を与える周波数依存型遅延装置７によって第２の位相調節を行い、この第１及び第２の位相調整がなされた後の出力信号Ｇ２´と固体撮像装置３の出力信号Ｇ１を加算器８で加算することで解決している。
【０００５】
【発明が解決しようとする課題】
本発明者は、前記の従来技術において、１チャンネル当たり複数の固体撮像素子を用い、空間画素ずらし法を行った固体撮像装置の表示される画像の鮮鋭度を増すための技術について検討した。
【０００６】
以下は、公知とされた技術ではないが、本発明者によって検討された技術であり、その概要は次のとおりである。
【０００７】
固体撮像素子を用いるテレビジョンカメラにおいて、２つの固体撮像素子の画素を１／２画素分ずらして配列し、画素数を４倍として撮像された映像信号を、同じく画素ずらしを用いて表示する表示装置を利用して表示する場合に、各々の固体撮像素子の出力を、そのまま表示装置に表示することが望ましい。その際に、表示される画像の鮮鋭度を増すために、カメラの出力信号の高域を強調処理（エンハンス処理）することを考える。
【０００８】
しかし、画素ずらし配置された２つの固体撮像素子の出力信号の高域をそれぞれ強調し、それらを出力しても画像の鮮鋭度は高くならない。また、２つの固体撮像素子の出力信号を合成して４倍の画素数の信号にした後に、高域の強調処理を行い、再び表示用に信号を分離すればよいが、この方法は高域強調処理の処理速度が遅い点、回路規模が大きくなる点で不利である。これまで、画素ずらしされた２つの固体撮像素子の出力信号のままでの、エンハンス処理の方法は提案されていない。
【０００９】
本発明の目的は、画素ずらしされた２つの固体撮像素子のそれぞれの出力信号のままでの、高域または中高域を強調する処理（エンハンス処理）の方法を提供することにある。
【００１０】
本発明の前記並びにその他の目的と新規な特徴は、本明細書の記述及び添付図面によって明らかにする。
【００１１】
【課題を解決するための手段】
本願において開示される発明のうち代表的なものの概要を簡単に説明すれば下記のとおりである。
【００１２】
第１の発明は、入射光を複数の光学像に分割し、該分割された光学像を空間画素ずらし法によらない位置に配置された第１固体撮像素子及び前記空間画素ずらし法による位置に配置された第２固体撮像素子でそれぞれ受光し、該受光した光学像を電気信号に変換し、該変換された電気信号に所望の処理をし、外部に出力する固体撮像装置の信号処理方法において、前記所望の処理を施す過程は、前記第１固体撮像素子の第１出力信号の低域または高域の周波数成分を抽出する第１ステップと、前記第２固体撮像素子の第２出力信号の前記第１ステップにおける周波数成分と同じ周波数成分を抽出する第２ステップと、前記第１ステップで得られた第１周波数成分から前記第２ステップで得られた第２周波数成分を減算して差分を求める第３ステップと、前記第２ステップで得られた第２周波数成分から前記第１ステップで得られた第１周波数成分を減算して差分を求める第４ステップと、前記第１出力信号と第３ステップで得られた周波数成分の差分を加算する第５ステップと、前記第２出力信号と第４ステップで得られた周波数成分の差分を加算する第６ステップとを有する信号処理方法である。
【００１４】
第２の発明は、入射光を複数の光学像に分割し、該分割された光学像を空間画素ずらし法によらない位置に配置された第１固体撮像素子及び前記空間画素ずらし法による位置に配置された第２固体撮像素子でそれぞれ受光し、該受光した光学像を電気信号に変換し、該変換された電気信号に所望の処理をし、外部に出力する固体撮像装置の信号処理方法において、前記所望の処理を施す過程は、前記第１固体撮像素子の第１出力信号から前記第２固体撮像素子の第２出力信号を減算して差分を求める第１ステップと、前記第２固体撮像素子の第２出力信号から前記第１固体撮像素子の第１出力信号を減算して差分を求める第２ステップと、前記第１ステップで得られた差分の低域または高域の周波数成分を抽出する第３ステップと、前記第２ステップで得られた差分の前記第３ステップにおける周波数成分と同じ周波数成分を抽出する第４ステップと、前記第１出力信号と前記第３ステップで得られた周波数成分を加算する第５ステップと、前記第２出力信号と前記第４ステップで得られた周波数成分を加算する第６ステップとを有する信号処理方法である。
【００１７】
第３の発明は、入射光を複数の光学像に分割し、該分割された光学像を空間画素ずらし法によらない位置に配置された第１固体撮像素子及び前記空間画素ずらし法による位置に配置された第２固体撮像素子でそれぞれ受光し、該受光した光学像を電気信号に変換し、該変換された電気信号に所望の処理をし、外部に出力する固体撮像装置において、前記第１固体撮像素子の第１出力信号の低域または高域の周波数成分を抽出する第１出力信号特定周波数抽出手段と、前記第２固体撮像素子の第２出力信号の前記第１出力信号特定周波数抽出手段における周波数成分と同じ周波数成分を抽出する第２出力信号特定周波数抽出手段と、前記第１出力信号特定周波数抽出手段で得られた第１周波数成分から前記第２出力信号特定周波数抽出手段で得られた第２周波数成分を減算して差分を求める第１減算手段と、前記第２出力信号特定周波数抽出手段で得られた第２周波数成分から前記第１出力信号特定周波数抽出手段で得られた第１周波数成分を減算して差分を求める第２減算手段と、前記第１出力信号と前記第１減算手段で得られた周波数成分の差分を加算する第１加算手段と、前記第２出力信号と前記第２減算手段で得られた周波数成分の差分を加算する第２加算手段とを備えるものである。
【００１９】
第４の発明は、入射光を複数の光学像に分割し、該分割された光学像を空間画素ずらし法によらない位置に配置された第１固体撮像素子及び前記空間画素ずらし法による位置に配置された第２固体撮像素子でそれぞれ受光し、該受光した光学像を電気信号に変換し、該変換された電気信号に所望の処理をし、外部に出力する固体撮像装置において、前記第１固体撮像素子の第１出力信号から前記第２固体撮像素子の第２出力信号を減算して差分を求める第１減算手段と、前記第２固体撮像素子の第２出力信号から前記第１固体撮像素子の第１出力信号を減算して差分を求める第２減算手段と、前記第１減算手段で得られた差分の低域または高域の周波数成分を抽出する第１出力信号特定周波数抽出手段と、前記第２減算手段で得られた差分の前記第１出力信号特定周波数抽出手段における周波数成分と同じ周波数成分を抽出する第２出力信号特定周波数抽出手段と、前記第１出力信号と前記第１出力信号特定周波数抽出手段で得られた周波数成分を加算する第１加算手段と、前記第２出力信号と前記第１出力信号特定周波数抽出手段で得られた周波数成分を加算する第２加算手段とを有するものである。
【００２２】
第５の発明は、前記第３または４の固体撮像装置において、前記空間画素ずらし法による前記複数の固体撮像素子の配置は、水平方向もしくは斜め方向にずらした配置とするものである。
【００２３】
本発明のポイントは、画素ずらしされた２つの固体撮像素子の出力信号と、それぞれの出力信号における特定の周波数帯域を抽出するためにＬＰＦ（ローパスフィルタ）もしくはＥＱＡ（イコライザアンプ）をかけた信号を用いることで、２つの固体撮像素子の出力信号を合成すること無しに、出力される信号の高域の強調処理をした点である。
【００２４】
前記した手段によれば、空間画素ずらし法によらない位置に配置された複数の第１固体撮像素子及び前記空間画素ずらし法による位置に配置された複数の第２固体撮像素子の画素により得られる出力信号において、それらの出力信号の低周波数を抽出するためにＬＰＦを通過させ、そのＬＰＦ通過後の信号の差を取ることで出力される信号の高域成分のみを取り出し、これらの高域成分を、それぞれの固体撮像素子の出力信号に加算する。または、空間画素ずらし法によらない位置に配置された複数の第１固体撮像素子及び前記空間画素ずらし法による位置に配置された複数の第２固体撮像素子の画素により得られる出力信号の差分信号からＬＰＦを用いて、出力される信号の高域成分のみを取り出し、これらの高域成分を、それぞれの固体撮像素子の出力信号に加算する。以上により高域強調（エンハンス処理）を達成するものである。
【００２５】
以下に、本発明について、本発明による実施形態（実施例）とともに図面を参照して詳細に説明する。
なお、実施形態を説明するための全図において、同一機能を有するものは同一符号を付け、その繰り返しの説明は省略する。
【００２６】
【発明の実施の形態】
（実施例１）
図１は、本発明の実施例１の固体撮像素子の概略構成を示す図である。図１において、１０１は第１出力信号特定周波数抽出手段、１０２は第２出力信号特定周波数抽出手段、１０３は第１減算手段、１０４は第２減算手段、１０５は第１加算手段、１０６は第２加算手段である。また、Ｆ１（ｎ）、Ｆ２（ｎ）は空間画素ずらし法によらない位置に配置された複数の第１固体撮像素子及び前記空間画素ずらし法による位置に配置された複数の第２固体撮像素子の画素によって得られた画像信号、Ｆ´１（ｎ）、Ｆ´２（ｎ）はＦ１（ｎ）及びＦ２（ｎ）の高域が強調された画像信号である。
【００２７】
本実施例１の固体撮像装置は、図１に示すように、第１固体撮像素子の第１出力信号Ｆ１（ｎ）の特定の周波数成分を抽出する第１出力信号特定周波数抽出手段１０１と、第２固体撮像素子の第２出力信号Ｆ２（ｎ）の特定の周波数成分を抽出する第２出力信号特定周波数抽出手段１０２と、前記第１出力信号特定周波数抽出手段１０１で第１周波数成分と第２出力信号特定周波数抽出手段１０２で得られた第２周波数成分との差を求める第１減算手段１０３と、前記第１出力信号Ｆ１（ｎ）と第１減算手段１０３で得られた第１周波数成分差分を加算する第１加算手段１０５と、前記第２出力信号特定周波数抽出手段１０２で得られた第２周波数成分と第１出力信号特定周波数抽出手段１０１で得られた第１周波数成分との差を求める第２減算手段１０４と、前記第２出力信号Ｆ２（ｎ）と第２減算手段１０４で得られた第２周波数成分差分を加算する第２加算手段１０６とを備えている。
【００２８】
図７は、空間的にずらして配置された２つの固体撮像素子の位置関係を示す図であり、７０１及び７０２は固体撮像素子、７０３は画素である。
【００２９】
前記第１固体撮像素子と第２固体撮像素子は、例えば、図７に示すように互いに水平方向にずらして配置する。また、斜め方向にずらして配置してもよい。出力信号特定周波数抽出手段としては、例えば、ＬＰＦ（ローパスフィルタ）、ＥＱＡ（イコライザアンプ）、ＨＰＦ（ハイパスフィルタ）の使用が可能である。
【００３０】
図２は、本実施例１の固体撮像素子の概略構成図を示す図であり、２０１及び２０２はＬＰＦ、２０３及び２０４は減算器、２０５及び２０６は加算器である。
【００３１】
図３（ａ）は、本発明の実施例１の出力信号特定周波数抽出手段１０１への入力信号波形を示す図、図３（ｂ）は、本実施例１の出力信号特定周波数抽出手段１０２の出力信号波形を示す図である。
【００３２】
本実施例１では、図１において、第１出力信号特定周波数抽出手段１０１及び第２出力信号特定周波数抽出手段１０２にＬＰＦ、第１減算手段１０３及び第２減算手段１０４に減算器、第１加算手段１０５及び第２加算手段１０６に加算器を使用した固体撮像装置である。
【００３３】
第１固体撮像素子及び第２固体撮像素子により得られる画像信号Ｆ１（ｎ）、Ｆ２（ｎ）は、それぞれの固体撮像素子の出力信号においては、表示される画像に対してサンプリング周波数が半分になっているので、そのまま高域強調することはできない。そこで、第１固体撮像素子及び第２固体撮像素子により得られるそれぞれの出力信号からＬＰＦで低域成分のみを取り出す。この低域成分の中には実際に表示される信号の高域成分が、折り返しにより含まれている。また、画素ずらしによる隣接画素の低域成分は、ほぼ同じレベルであるので、ＬＰＦ通過後の信号の差を取ると、表示される信号の高域成分のみが取り出せる。
【００３４】
この高域成分を、それぞれの固体撮像素子の出力に加算することにより高域強調（エンハンス処理）することができる。
【００３５】
以上の説明に基づいて、本実施例１の固体撮像装置の動作を具体的に説明する。
固体撮像素子を用いるテレビジョンカメラにおいて、第１固体撮像素子及び第２固体撮像素子によって得られる出力信号Ｆ１（ｎ）、Ｆ２（ｎ）において、Ｆ１（ｎ）にＬＰＦ２０１をかけた信号は、減算器２０３によりＦ２（ｎ）にＬＰＦ２０２をかけた信号から、減算された後に加算器２０５によりＦ１（ｎ）に加算され、高域強調された信号Ｆ´１（ｎ）となる。
【００３６】
同様に、Ｆ２（ｎ）にＬＰＦ２０２をかけた信号は、減算器２０４により、Ｆ１（ｎ）にＬＰＦ２０１をかけた信号から減算された後に、加算器２０６によりＦ２（ｎ）に加算されて高域強調された信号Ｆ´２（ｎ）となる。
【００３７】
例えば、入力周波数をω、画素サンプリングの周波数をω_０、サンプリング点は奇数番目と偶数番目での画素とする。
【００３８】
ｎ番目のサンプリング点における出力信号Ｆ１（ｎ）、Ｆ２（ｎ）は、ｔ_０を定数としてそれぞれ数１の式、数２の式で表せる。
【００３９】
【数１】

【数２】

ここで、ωは、ω_０／２に帯域制限されているとする。
【００４０】
前記数１の式、数２の式にＬＰＦをかける。ＬＰＦの遮断周波数をω_１とすると、ナイキスト周波数がω_０／４となっていることから、数３の式及び数４の式が得られる。これらの関係により、図３に示すように、ＬＰＦにより低域周波数だけ取り出せる。
【００４１】
【数３】

【数４】

前記数３の式と数４の式との差をとると、数５の式となる。
【００４２】
【数５】

ここで、ω_１が小さい時、例えば、ω_０／８の場合では、数６の式となる。
【００４３】
【数６】

【００４４】
前記数５の式は、前記ω_０／８の場合では数６の式となるので、数７の式となり、周波数が高いときのみ、ＬＰＦ（Ｆ１（ｎ））−ＬＰＦ（Ｆ２（ｎ））の成分が現れる。
【００４５】
【数７】

【００４６】
したがって、次の数８の式及び数９の式により高域成分が強調された画像信号が得られる。
【数８】
Ｆ´１（ｎ）＝Ｆ１（ｎ）＋ＬＰＦ（Ｆ１（ｎ））−ＬＰＦ（Ｆ２（ｎ））
【数９】
Ｆ´２（ｎ）＝Ｆ２（ｎ）−ＬＰＦ（Ｆ１（ｎ））＋ＬＰＦ（Ｆ２（ｎ））
【００４７】
（実施例２）
図４は、本発明の実施例２の固体撮像素子の概略構成を示す図であり、４０１及び４０２はＥＱＡ（イコライザアンプ）である。
【００４８】
本実施例２では、出力信号特定周波数抽出手段にＥＱＡを用い、任意の周波数を選択して、その強度を調整する。
【００４９】
ＥＱＡを用いることで、強調する周波数帯域の選択が可能であり、所望の周波数成分が強調された画像信号が得られる。
【００５０】
（実施例３）
図５は、本発明の実施例３の固体撮像素子の概略構成を示す図であり、５０１及び５０２は減算器、５０３及び５０４はＥＱＡ（イコライザアンプ）、５０５及び５０６は加算器である。図６（ａ）は、本実施例２の出力信号特定周波数抽出手段への入力信号波形を示す図、図６（ｂ）は、本実施例３の出力信号特定周波数抽出手段にＬＰＦを用いた場合のＬＰＦの出力信号波形を示す図、図６（ｃ）は、本実施例３の出力信号特定周波数抽出手段にＨＰＦを用いた場合のＨＰＦの出力信号波形を示す図である。
【００５１】
本実施例３では、固体撮像素子を用いるテレビジョンカメラにおいて、第１固体撮像素子及び第２固体撮像素子によって得られた画像信号Ｆ１（ｎ）、Ｆ２（ｎ）において、Ｆ１（ｎ）は、減算器５０１でＦ２（ｎ）から減算された後に、ＥＱＡ５０３を通過し、通過した信号は加算器５０５によりＦ１（ｎ）に加算され、高域強調された信号Ｆ´１（ｎ）となる。
【００５２】
同様にＦ２（ｎ）は、減算器５０２でＦ１（ｎ）から減算された後に、ＥＱＡ５０４を通過し、通過した信号は加算器５０６により、Ｆ２（ｎ）に加算され、高域強調された信号Ｆ´２（ｎ）となる。
【００５３】
ＥＱＡを用いることにより、強調したい高域の周波数や、その帯域を調整することができる。つまり、ＥＱＡでＬＰＦを構成した場合は、実施例１の構成例と同じ動作をする。ＥＱＡでＨＰＦを構成した場合は、ナイキスト周波数がω_０／４となっているため、信号の中域から高域にかけての周波数を強調することができる。
【００５４】
図５のＡ点の信号は、数１０の式で表せる。
【数１０】

【００５５】
この数１０の式の周波数特性を図６（ａ）に示す。この周波数特性と、ＥＱＡの周波数特性との積が強調される周波数となりＥＱＡより出力される。ＥＱＡとしてＬＰＦを構成した場合、図６（ｂ）左図の斜線部分の帯域が抽出され、出力は右図のように高域成分が出力される。また、ＥＱＡとしてＨＰＦを構成した場合、図６（ｃ）左図の斜線部分の帯域が抽出され、出力は右図のように中高域成分が出力される。これらをＦ１（ｎ）、Ｆ２（ｎ）に加算することで、中高域成分が強調された画像信号が得られる。
【００５６】
ＥＱＡを所望の特性に選ぶことにより、信号の中域から高域にかけて強調することが可能となる。
以上、本発明者によってなされた発明を、実施例に基づき具体的に説明したが、本発明は、前記実施の形態に限定されるものではなく、その要旨を逸脱しない範囲において種々変更可能であることは勿論である。
【００５７】
例えば、本実施例では、第１出力信号特定周波数抽出手段１０１、第２出力信号特定周波数抽出手段１０２、第１減算手段１０３、第２減算手段１０４、第１加算手段１０５、第２加算手段１０６を別々に設ける場合について説明したが、各手段を組み合わせることにより新たな一つの手段として実現させても同様の効果を得ることができる。
【００５８】
【発明の効果】
本願において開示される発明によって得られる効果を簡潔に説明すれば、下記のとおりである。
すなわち、固体撮像素子を用いるテレビジョンカメラにおいて、例えば、２つの固体撮像素子の画素を１／２画素分ずらした位置に配置し、画素数を４倍として撮像された映像信号を、同じく画素ずらしを用いて表示する表示装置を利用して表示する場合に、画素ずらしされた２つの固体撮像素子の出力と、それにＬＰＦをかけた信号もしくはＥＱＡを通した信号を用いることにより、画素ずらし信号のまま、表示される信号の高域または中高域を強調することができる。
【図面の簡単な説明】
【図１】本発明の実施例１の固体撮像装置の概略構成を示す図である。
【図２】本明の実施例１の固体撮像装置の動作を説明するための図である。
【図３】本実施例１の出力信号特定周波数抽出手段の入力信号波形及び出力信号波形を示す図である。
【図４】本発明の実施例２の固体撮像装置の概略構成を示す図である。
【図５】本発明の実施例３の固体撮像装置の概略構成を示す図である。
【図６】本実施例３の出力信号特定周波数抽出手段の入力信号波形、ＬＰＦの出力信号波形、及びＨＰＦの出力信号波形を示す図である。
【図７】従来の空間的にずらして配置された２つの固体撮像素子の位相関係を示す図である
【符号の説明】
１０１…第１出力信号特定周波数抽出手段
１０２…第２出力信号特定周波数抽出手段
１０３…第１減算手段１０４…第２減算手段
１０５…第１加算手段１０６…第２加算手段
２０１、２０２…ＬＰＦ２０３、２０４…減算器
２０５、２０６…加算器４０１、４０２…ＥＱＡ
５０１、５０２…減算器５０３、５０４…ＥＱＡ
５０５、５０６…加算器７０１、７０２…固体撮像素子
７０３…画素[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a solid-state image pickup device, and more particularly to a technique for applying to a television camera using a solid-state image pickup device and performing processing (enhancement processing) for enhancing high or middle / high frequencies of an output signal of the camera to improve image quality. For example, the present invention relates to a technique that is effective when applied to a technique in which pixels of two solid-state imaging elements are arranged at positions shifted by ½ pixels and the number of pixels is four times.
[0002]
[Prior art]
A conventional solid-state imaging device is described in, for example, Japanese Patent Laid-Open No. 9-37126. FIG. 18 of the publication discloses a configuration of a solid-state imaging device in which two solid-state imaging elements are used per channel and pixels are arranged at positions based on a spatial pixel shifting method. The solid-state imaging device includes a half mirror 1, solid-

state imaging elements

2 and 3, preprocess

circuits

4 and 5, a delay element 6, an adder 8, and a process circuit 9.
[0003]
The conventional solid-state imaging device divides an optical image of a subject incident by the half mirror 1, receives the divided optical images by the solid-

state imaging devices

2 and 3, and converts the received optical image into an electrical signal. The converted electric signal is subjected to noise removal and gain adjustment in the

preprocess circuits

4 and 5 so that the respective output levels are equal, and the signal subjected to gain adjustment is an output signal of two solid-state image sensors. Are adjusted in phase by the delay element 6. The two signals that have undergone phase adjustment are added by an adder, and the added signal is processed by the process circuit for the final video signal and output. Further, the two solid-state imaging devices are optically shifted in the horizontal direction as shown in FIG.
[0004]
According to the prior art, a method for correcting deterioration of the resolution improvement effect due to a spatial pixel shift error in the spatial pixel shift method is disclosed. As the means, in the configuration of FIG. 1 of the above publication, the delay element 6 is connected to the output signal of the solid-state image sensor 2 arranged spatially shifted with respect to the solid-state image sensor 3, and the first phase adjustment is performed. Further, the output signal G2 obtained by performing the first phase adjustment is subjected to the second phase adjustment by the frequency dependent delay device 7 that gives a delay amount corresponding to the frequency of the input signal, and the first and second phase adjustments are performed. This is solved by adding the output signal G 2 ′ after the phase adjustment of the above and the output signal G 1 of the solid-state imaging device 3 by the adder 8.
[0005]
[Problems to be solved by the invention]
The present inventor examined a technique for increasing the sharpness of an image displayed by a solid-state imaging device using a plurality of solid-state imaging devices per channel and performing a spatial pixel shifting method in the above-described conventional technology.
[0006]
The following is not a publicly known technique, but is a technique examined by the present inventor, and its outline is as follows.
[0007]
In a television camera using a solid-state image pickup device, a display in which pixels of two solid-state image pickup devices are arranged with a shift of ½ pixel and a video signal picked up with four times the number of pixels is displayed using the same pixel shift When displaying using an apparatus, it is desirable to display the output of each solid-state image sensor on a display apparatus as it is. At that time, in order to increase the sharpness of the displayed image, it is considered that the high frequency of the output signal of the camera is enhanced (enhanced).
[0008]
However, even if the high frequencies of the output signals of the two solid-state imaging devices arranged so as to be shifted from each other are emphasized and outputted, the sharpness of the image does not increase. Further, after combining the output signals of the two solid-state imaging devices into a signal having four times the number of pixels, high-frequency emphasis processing may be performed to separate the signals for display again. This is disadvantageous in that the processing speed of the enhancement processing is slow and the circuit scale becomes large. So far, no enhancement processing method has been proposed in which the output signals of the two solid-state imaging devices shifted in pixels remain as they are.
[0009]
An object of the present invention is to provide a processing ( enhancement processing ) method for enhancing a high frequency range or a mid-high frequency range with the respective output signals of two solid-state imaging devices shifted in pixels.
[0010]
The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.
[0011]
[Means for Solving the Problems]
An outline of typical inventions disclosed in the present application will be briefly described as follows.
[0012]
A first aspect of the present invention is to divide the incident light into a plurality of optical images, the position of the divided first solid image pickup element and the spatial pixel shifting method is arranged an optical image at a position that does not depend on the spatial pixel shifting method In a signal processing method of a solid-state imaging device that receives light by each of the arranged second solid-state imaging devices, converts the received optical image into an electrical signal, performs a desired process on the converted electrical signal, and outputs the processed signal to the outside The process of performing the desired processing includes a first step of extracting a low frequency component or a high frequency component of the first output signal of the first solid-state image sensor, and a second output signal of the second solid-state image sensor. the second step of extracting the same frequency components as the frequency components in the first step, the second differencing frequency components by subtracting obtained in the second step from the first frequency component obtained in the first step 3rd seeking And-up, the fourth step and said first output signal and a third step of the first frequency component by subtracting obtains the difference obtained in the first step from the second frequency component obtained in the second step a fifth step and the signal processing method and a sixth step of adding the difference of the frequency components obtained by said second output signal and the fourth step of adding the difference of the obtained frequency components.
[0014]
According to a second aspect of the present invention, incident light is divided into a plurality of optical images, and the divided optical images are arranged at positions not based on the spatial pixel shifting method and at positions based on the spatial pixel shifting method. In a signal processing method of a solid-state imaging device that receives light by each of the arranged second solid-state imaging devices, converts the received optical image into an electrical signal, performs a desired process on the converted electrical signal, and outputs the processed signal to the outside The process of performing the desired processing includes a first step of subtracting a second output signal of the second solid-state image sensor from a first output signal of the first solid-state image sensor and obtaining a difference, and the second solid-state image sensor. A second step of subtracting the first output signal of the first solid-state imaging device from the second output signal of the element to obtain a difference, and extracting a low frequency or high frequency component of the difference obtained in the first step A third step of performing the second step A fourth step of extracting the same frequency component as the frequency component in the third step of the difference obtained in the step; a fifth step of adding the first output signal and the frequency component obtained in the third step; The signal processing method includes the second output signal and a sixth step of adding the frequency components obtained in the fourth step .
[0017]
A third invention is to divide the incident light into a plurality of optical images, the position of the divided first solid image pickup element and the spatial pixel shifting method is arranged an optical image at a position that does not depend on the spatial pixel shifting method In the solid-state imaging device that receives the light from each of the arranged second solid-state imaging devices, converts the received optical image into an electrical signal, performs a desired process on the converted electrical signal, and outputs the processed signal to the outside. a first output signal specific frequency extracting means for extracting a low-frequency or high-frequency of the frequency component of the first output signal of the solid-state imaging device, the first output signal specific frequency extraction of the second output signal of said second solid-state imaging element obtained in the second output signal specific frequency extracting means, wherein the first frequency component obtained by the first output signal specific frequency extracting means second output signal specific frequency extracting means for extracting the same frequency components as the frequency components in the unit A first subtraction means for obtaining a difference component by subtracting the second frequency component, obtained from the second frequency components obtained by said second output signal a particular frequency extracting means with said first output signal a particular frequency extracting means Second subtracting means for subtracting the first frequency component to obtain a difference; first adding means for adding the difference between the first output signal and the frequency component obtained by the first subtracting means; and the second output. And a second adding means for adding the difference between the signal and the frequency component obtained by the second subtracting means.
[0019]
According to a fourth aspect of the present invention, incident light is divided into a plurality of optical images, and the divided optical images are arranged at positions not based on the spatial pixel shifting method and at positions based on the spatial pixel shifting method. In the solid-state imaging device that receives the light from each of the arranged second solid-state imaging devices, converts the received optical image into an electrical signal, performs a desired process on the converted electrical signal, and outputs the processed signal to the outside. First subtraction means for subtracting the second output signal of the second solid-state imaging device from the first output signal of the solid-state imaging device to obtain a difference; and the first solid-state imaging from the second output signal of the second solid-state imaging device. Second subtracting means for subtracting the first output signal of the element to obtain a difference; first output signal specific frequency extracting means for extracting a low frequency or high frequency component of the difference obtained by the first subtracting means; , The difference obtained by the second subtraction means The second output signal specific frequency extracting means for extracting the same frequency component as the frequency component in the first output signal specific frequency extracting means, the first output signal and the frequency component obtained by the first output signal specific frequency extracting means a first adding means for adding is configured to have a second adder for adding the frequency components obtained by the second output signal and said first output signal a particular frequency extracting means.
[0022]
According to a fifth aspect of the present invention, in the third or fourth solid-state imaging device, the plurality of solid-state imaging elements arranged by the spatial pixel shifting method are arranged in a horizontal direction or an oblique direction.
[0023]
The point of the present invention is that an output signal of two solid-state imaging devices shifted by pixels and a signal obtained by applying an LPF (low pass filter) or an EQA (equalizer amplifier) in order to extract a specific frequency band in each output signal. By using it, the high-frequency emphasis processing of the output signals is performed without synthesizing the output signals of the two solid-state imaging devices.
[0024]
According to the above-described means, it is obtained by a plurality of first solid-state imaging devices arranged at positions not using the spatial pixel shifting method and a plurality of second solid-state imaging devices arranged at positions based on the spatial pixel shifting method. In the output signal, the LPF is passed through in order to extract the low frequency of those output signals, and only the high frequency components of the output signal are extracted by taking the difference between the signals after passing through the LPF, and these high frequency components are extracted. Is added to the output signal of each solid-state imaging device. Or the difference signal of the output signal obtained by the pixel of the some 1st solid-state image sensor arrange | positioned in the position which is not based on a spatial pixel shifting method, and the some 2nd solid-state image sensor arranged in the position by the said spatial pixel shifting method Using LPF, only the high frequency components of the output signal are extracted, and these high frequency components are added to the output signals of the respective solid-state imaging devices. Thus, high-frequency emphasis (enhancement processing) is achieved.
[0025]
Hereinafter, the present invention will be described in detail with reference to the drawings together with embodiments (examples) according to the present invention.
In all the drawings for explaining the embodiments, parts having the same function are given the same reference numerals, and repeated explanation thereof is omitted.
[0026]
DETAILED DESCRIPTION OF THE INVENTION
(Example 1)
FIG. 1 is a diagram illustrating a schematic configuration of a solid-state imaging element according to Embodiment 1 of the present invention. In FIG. 1, 101 is a first output signal specific frequency extraction means, 102 is a second output signal specific frequency extraction means, 103 is a first subtraction means, 104 is a second subtraction means, 105 is a first addition means, and 106 is a first output means. 2 addition means. F1 (n) and F2 (n) are a plurality of first solid-state image sensors arranged at positions not based on the spatial pixel shift method and a plurality of second solid-state image sensors arranged at positions based on the spatial pixel shift method. The image signals F′1 (n) and F′2 (n) obtained by the pixels are the image signals in which the high frequencies of F1 (n) and F2 (n) are emphasized.
[0027]
As shown in FIG. 1, the solid-state imaging device according to the first embodiment includes a first output signal specific frequency extraction unit 101 that extracts a specific frequency component of the first output signal F1 (n) of the first solid-state image sensor, The second output signal specific frequency extracting means 102 extracts a specific frequency component of the second output signal F2 (n) of the second solid-state imaging device, and the first output signal specific frequency extracting means 101 extracts the first frequency component and the first frequency component. A first subtraction unit 103 for obtaining a difference from the second frequency component obtained by the two-output signal specific frequency extraction unit 102; a first frequency obtained by the first output signal F1 (n) and the first subtraction unit 103; A first addition means 105 for adding a component difference; a second frequency component obtained by the second output signal specific frequency extraction means 102; and a first frequency component obtained by the first output signal specific frequency extraction means 101. Second to find the difference And calculation means 104, and a second adding means 106 for adding the second output signal F2 (n) and a second frequency component difference obtained by the second subtraction means 104.
[0028]
FIG. 7 is a diagram showing the positional relationship between two solid-state image sensors that are spatially shifted. 701 and 702 are solid-state image sensors, and 703 is a pixel.
[0029]
The first solid-state imaging device and the second solid-state imaging device are arranged so as to be shifted from each other in the horizontal direction as shown in FIG. 7, for example. Moreover, you may arrange | position by shifting in the diagonal direction. For example, LPF (low-pass filter), EQA (equalizer amplifier), and HPF (high-pass filter) can be used as the output signal specific frequency extracting means.
[0030]
FIG. 2 is a diagram illustrating a schematic configuration diagram of the solid-state imaging device according to the first embodiment. 201 and 202 are LPFs, 203 and 204 are subtractors, and 205 and 206 are adders.
[0031]
FIG. 3A is a diagram illustrating a waveform of an input signal to the output signal specific frequency extracting unit 101 according to the first embodiment of the present invention, and FIG. 3B is a diagram illustrating the output signal specific frequency extracting unit 102 according to the first embodiment. It is a figure which shows an output signal waveform.
[0032]
In the first embodiment, in FIG. 1, the first output signal specific frequency extracting means 101 and the second output signal specific frequency extracting means 102 are LPF, the first subtracting means 103 and the second subtracting means 104 are subtracters, and the first addition. The solid-state imaging device uses an adder for the means 105 and the second addition means 106.
[0033]
The image signals F1 (n) and F2 (n) obtained by the first solid-state image sensor and the second solid-state image sensor have a sampling frequency halved with respect to the displayed image in the output signals of the respective solid-state image sensors. Therefore, it is not possible to emphasize the high range as it is. Therefore, only the low frequency component is extracted by the LPF from the respective output signals obtained by the first solid-state image sensor and the second solid-state image sensor. Among the low frequency components, the high frequency components of the actually displayed signal are included by folding. Further, since the low-frequency components of the adjacent pixels due to pixel shifting are substantially the same level, if the difference between the signals after passing through the LPF is taken, only the high-frequency components of the displayed signal can be extracted.
[0034]
By adding this high-frequency component to the output of each solid-state image sensor, high-frequency enhancement (enhancement processing) can be performed.
[0035]
Based on the above description, the operation of the solid-state imaging device according to the first embodiment will be specifically described.
In a television camera using a solid-state image sensor, a signal obtained by multiplying F1 (n) by LPF 201 in the output signals F1 (n) and F2 (n) obtained by the first solid-state image sensor and the second solid-state image sensor is subtracted. The signal is subtracted from the signal obtained by multiplying the LPF 202 by F2 (n) by the unit 203 and then added to F1 (n) by the adder 205 to obtain a signal F′1 (n) with high frequency emphasis.
[0036]
Similarly, a signal obtained by multiplying F2 (n) by LPF 202 is subtracted from a signal obtained by multiplying F1 (n) by LPF 201 by a subtractor 204, and then added to F2 (n) by an adder 206. The emphasized signal F′2 (n) is obtained.
[0037]
For example, the input frequency is ω, the pixel sampling frequency is ω ₀ , and the sampling points are odd-numbered and even-numbered pixels.
[0038]
The output signals F1 (n) and F2 (n) at the n-th sampling point can be expressed by Equation 1 and Equation 2, respectively, where t ₀ is a constant.
[0039]
[Expression 1]

[Expression 2]

Here, omega is assumed to be band-limited to omega _0/2.
[0040]
The LPF is applied to the formulas (1) and (2). If the cutoff frequency of LPF and omega _1, since the Nyquist frequency is in the omega _0/4, equation (3) of the formula and the number 4 is obtained. Due to these relationships, as shown in FIG. 3, only the low frequency can be extracted by the LPF.
[0041]
[Equation 3]

[Expression 4]

When the difference between the formula 3 and the formula 4 is taken, the formula 5 is obtained.
[0042]
[Equation 5]

Here, when the omega ₁ is small, for example, in the case of omega _0/8, the formula 6.
[0043]
[Formula 6]

[0044]
The foregoing equation 5, since the number 6 expression in the case of the omega _0/8, becomes number 7 expression, only when the frequency is high, LPF (F1 (n)) - LPF (F2 (n)) The component of appears.
[0045]
[Expression 7]

[0046]
Therefore, an image signal in which high frequency components are emphasized by the following equations (8) and (9) is obtained.
[Equation 8]
F′1 (n) = F1 (n) + LPF (F1 (n)) − LPF (F2 (n))
[Equation 9]
F′2 (n) = F2 (n) −LPF (F1 (n)) + LPF (F2 (n))
[0047]
(Example 2)
FIG. 4 is a diagram showing a schematic configuration of the solid-state imaging device according to the second embodiment of the present invention, and 401 and 402 are EQAs (equalizer amplifiers).
[0048]
In the second embodiment, EQA is used as the output signal specific frequency extracting means, an arbitrary frequency is selected, and the intensity is adjusted.
[0049]
By using EQA, it is possible to select a frequency band to be emphasized, and an image signal in which a desired frequency component is enhanced is obtained.
[0050]
(Example 3)
FIG. 5 is a diagram illustrating a schematic configuration of the solid-state imaging device according to the third embodiment of the present invention, in which 501 and 502 are subtracters, 503 and 504 are EQA (equalizer amplifier), and 505 and 506 are adders. FIG. 6A is a diagram showing an input signal waveform to the output signal specific frequency extracting unit of the second embodiment, and FIG. 6B is an LPF used for the output signal specific frequency extracting unit of the third embodiment. FIG. 6C is a diagram showing the output signal waveform of the HPF when the HPF is used for the output signal specific frequency extracting means of the third embodiment.
[0051]
In Example 3, in a television camera using a solid-state image sensor, in the image signals F1 (n) and F2 (n) obtained by the first solid-state image sensor and the second solid-state image sensor, F1 (n) is After being subtracted from F2 (n) by the subtractor 501, the signal that has passed through the EQA 503 is added to F1 (n) by the adder 505, and becomes a high-frequency emphasized signal F′1 (n).
[0052]
Similarly, F2 (n) is subtracted from F1 (n) by the subtractor 502, and then passes through the EQA 504. The signal that has passed through is added to F2 (n) by the adder 506, and is subjected to high-frequency emphasis. F′2 (n).
[0053]
By using EQA, it is possible to adjust a high frequency to be emphasized and its band. That is, when the LPA is configured by EQA, the same operation as the configuration example of the first embodiment is performed. If you configured HPF in EQA, since the Nyquist frequency is in the omega _0/4, it is possible to emphasize the frequency of toward the high-frequency from the middle region of the signal.
[0054]
The signal at point A in FIG. 5 can be expressed by equation (10).
[Expression 10]

[0055]
FIG. 6A shows the frequency characteristics of the equation (10). The product of this frequency characteristic and the frequency characteristic of EQA becomes the emphasized frequency and is output from EQA. When an LPF is configured as EQA, the hatched portion of the band in the left diagram of FIG. 6B is extracted, and a high frequency component is output as shown in the right diagram. Further, when an HPF is configured as EQA, the hatched portion of the band in the left diagram of FIG. 6 (c) is extracted, and the middle and high frequency components are output as shown in the right diagram. By adding these to F1 (n) and F2 (n), an image signal in which the middle and high frequency components are emphasized is obtained.
[0056]
By selecting the EQA as a desired characteristic, it is possible to emphasize the signal from the middle range to the high range.
As mentioned above, the invention made by the present inventor has been specifically described based on the examples. However, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention. Of course.
[0057]
For example, in this embodiment, the first output signal specific frequency extracting unit 101, the second output signal specific frequency extracting unit 102, the first subtracting unit 103, the second subtracting unit 104, the first adding unit 105, and the second adding unit 106 are used. However, the same effect can be obtained even if it is realized as a new means by combining the means.
[0058]
【The invention's effect】
The effects obtained by the invention disclosed in the present application will be briefly described as follows.
That is, in a television camera using a solid-state image sensor, for example, the video signals obtained by arranging the pixels of two solid-state image sensors at a position shifted by ½ pixel and the number of pixels being four times are similarly shifted by pixels. In the case of displaying using a display device that displays using a pixel, by using the output of two pixel-shifted solid-state imaging devices and the signal applied with LPF or the signal passed through EQA, The high or middle high frequency range of the displayed signal can be emphasized.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a schematic configuration of a solid-state imaging apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram for explaining the operation of the solid-state imaging device according to Embodiment 1 of the present invention.
FIG. 3 is a diagram illustrating an input signal waveform and an output signal waveform of the output signal specific frequency extracting unit according to the first embodiment.
FIG. 4 is a diagram illustrating a schematic configuration of a solid-state imaging apparatus according to a second embodiment of the present invention.
FIG. 5 is a diagram illustrating a schematic configuration of a solid-state imaging apparatus according to Embodiment 3 of the present invention.
FIG. 6 is a diagram showing an input signal waveform, an LPF output signal waveform, and an HPF output signal waveform of the output signal specific frequency extracting means of the third embodiment.
FIG. 7 is a diagram showing a phase relationship between two conventional solid-state image pickup devices arranged in a spatially displaced manner.
DESCRIPTION OF SYMBOLS 101 ... 1st output signal specific frequency extraction means 102 ... 2nd output signal specific frequency extraction means 103 ... 1st subtraction means 104 ... 2nd subtraction means 105 ... 1st addition means 106 ... 2nd addition means 201, 202 ... LPF 203 204,

subtracters

205, 206 ... adders 401, 402 ... EQA
501, 502 ... Subtractor 503, 504 ... EQA
505, 506 ... Adders 701, 702 ... Solid-state image sensor 703 ... Pixel

Claims

Divides the incident light into a plurality of optical images, the divided second solid optical image is arranged at a position of the first solid-state imaging device and the spatial pixel shifting method is arranged at a position that does not depend on the spatial pixel shifting method In the signal processing method of the solid-state imaging device that receives each light by the image sensor, converts the received optical image into an electrical signal, performs a desired process on the converted electrical signal, and outputs the processed signal to the outside.
The process of performing the desired processing includes a first step of extracting a low frequency or high frequency component of the first output signal of the first solid-state imaging device;
A second step of extracting the same frequency component as the frequency component in the first step of the second output signal of the second solid-state imaging device;
A third step of obtaining a difference component by subtracting the second frequency component obtained in the second step from the first frequency component obtained in the first step,
A fourth step of subtracting the first frequency component obtained in the first step from the second frequency component obtained in the second step to obtain a difference;
A fifth step of adding the difference between the first output signal and the frequency component obtained in the third step;
A signal processing method for a solid-state imaging device, comprising: a sixth step of adding a difference between the second output signal and the frequency component obtained in the fourth step.

The incident light is divided into a plurality of optical images, and the divided optical image is arranged at a position not based on the spatial pixel shifting method and the second solid state positioned at the position based on the spatial pixel shifting method. In the signal processing method of the solid-state imaging device that receives each light by the image sensor, converts the received optical image into an electrical signal, performs a desired process on the converted electrical signal, and outputs the processed signal to the outside.
The process of performing the desired processing includes a first step of subtracting a second output signal of the second solid-state image sensor from a first output signal of the first solid-state image sensor to obtain a difference;
A second step of subtracting the first output signal of the first solid-state image sensor from the second output signal of the second solid-state image sensor to obtain a difference;
A third step of extracting a low frequency or high frequency component of the difference obtained in the first step;
A fourth step of extracting the same frequency component as the frequency component in the third step of the difference obtained in the second step;
A fifth step of adding the first output signal and the frequency component obtained in the third step;
A signal processing method for a solid-state imaging device, comprising: the second output signal; and a sixth step of adding the frequency components obtained in the fourth step .

Divides the incident light into a plurality of optical images, the divided second solid optical image is arranged at a position of the first solid-state imaging device and the spatial pixel shifting method is arranged at a position that does not depend on the spatial pixel shifting method respectively received by the image pickup element, converts the optical image receiving light into an electric signal, and the desired processing on the converted electric signal, in the solid-state imaging device that outputs to the outside,
First output signal specific frequency extracting means for extracting a low frequency or high frequency component of the first output signal of the first solid-state imaging device;
Second output signal specific frequency extracting means for extracting the same frequency component as the frequency component in the first output signal specific frequency extracting means of the second output signal of the second solid-state imaging device;
A first subtraction means for obtaining a difference component by subtracting the second frequency component obtained by the second output signal specific frequency extracting means from the first frequency component obtained by the first output signal specific frequency extracting means,
Second subtracting means for subtracting the first frequency component obtained by the first output signal specific frequency extracting means from the second frequency component obtained by the second output signal specific frequency extracting means to obtain a difference;
First addition means for adding a difference between the first output signal and the frequency component obtained by the first subtraction means;
A solid-state imaging device comprising: a second addition unit that adds a difference between the second output signal and the frequency component obtained by the second subtraction unit.

The incident light is divided into a plurality of optical images, and the divided optical image is arranged at a position not based on the spatial pixel shifting method and the second solid state positioned at the position based on the spatial pixel shifting method. In a solid-state imaging device that receives light with an imaging device, converts the received optical image into an electrical signal, performs a desired process on the converted electrical signal, and outputs the processed signal to the outside.
First subtraction means for subtracting the second output signal of the second solid-state image sensor from the first output signal of the first solid-state image sensor to obtain a difference;
Second subtracting means for subtracting the first output signal of the first solid-state image sensor from the second output signal of the second solid-state image sensor to obtain a difference;
First output signal specific frequency extracting means for extracting a low frequency or high frequency component of the difference obtained by the first subtracting means;
Second output signal specific frequency extracting means for extracting the same frequency component as the frequency component in the first output signal specific frequency extracting means of the difference obtained by the second subtracting means;
First addition means for adding the first output signal and the frequency component obtained by the first output signal specific frequency extraction means;
A solid-state imaging device comprising: the second output signal; and second addition means for adding the frequency components obtained by the first output signal specific frequency extraction means .

In the solid-state imaging device according to claim 3 or 4, before Symbol wherein the first solid-state imaging device arranged in the second solid-solid-state imaging device which is characterized in that the arrangement shifted in the horizontal direction or diagonal direction .