JP4857556B2 - In-vehicle camera video presentation device - Google Patents

In-vehicle camera video presentation device Download PDF

Info

Publication number
JP4857556B2
JP4857556B2 JP2004362521A JP2004362521A JP4857556B2 JP 4857556 B2 JP4857556 B2 JP 4857556B2 JP 2004362521 A JP2004362521 A JP 2004362521A JP 2004362521 A JP2004362521 A JP 2004362521A JP 4857556 B2 JP4857556 B2 JP 4857556B2
Authority
JP
Japan
Prior art keywords
brightness
vehicle surrounding
surrounding image
vehicle
ave
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2004362521A
Other languages
Japanese (ja)
Other versions
JP2006173988A (en
Inventor
健 赤塚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nissan Motor Co Ltd
Original Assignee
Nissan Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nissan Motor Co Ltd filed Critical Nissan Motor Co Ltd
Priority to JP2004362521A priority Critical patent/JP4857556B2/en
Publication of JP2006173988A publication Critical patent/JP2006173988A/en
Application granted granted Critical
Publication of JP4857556B2 publication Critical patent/JP4857556B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Closed-Circuit Television Systems (AREA)

Description

本発明は車載用カメラ映像提示装置に関する。 The present invention relates to an in-vehicle camera video presentation device .

下記特許文献１に記載されているように、車両のビュー装置において、左右の連続していない視界領域を１個のカメラにて撮像する場合、明るさの調整対象とする領域を限定して、必要な視界領域の明るさを最適にする仕組みが提案されている。以下、「カメラ」は、この特許文献１にも記載されているように、すべて電子式撮像手段を用いる電子式カメラを意味するものとする。   As described in Patent Document 1 below, in a vehicle view device, when imaging a left and right non-continuous field of view area with one camera, the area to be adjusted for brightness is limited, A mechanism for optimizing the brightness of the required visual field has been proposed. Hereinafter, the “camera” means an electronic camera using electronic imaging means as described in Patent Document 1.

特開平０９−０５８３４２号公報JP 09-058342 A

しかしながら、明るさの調整対象とする領域を限定して、必要な視界領域の明るさを最適にする仕組みでは、複数のカメラを用いた場合は、それぞれのカメラで明るさの調整を行ってから処理をすることとなり、複数のカメラでの連続した領域を撮影し、カメラごとに明るさを調整した映像を用いて、１つの提示映像を生成する場合の、複数のカメラ映像の持つ明るさの調整のための処理には適用できず、複数のカメラ映像ごとに明るさが大きく異なるような場合には合成するつなぎ目付近で明るさの不連続が起き、その明るさの段差によって、本来ないはずの段差があるように、提示映像としては見えてしまうという問題点があった。   However, in a mechanism that limits the brightness adjustment target area and optimizes the brightness of the required field of view, if multiple cameras are used, adjust the brightness with each camera. The brightness of multiple camera images when shooting a continuous area with multiple cameras and generating one presentation video using images with adjusted brightness for each camera. It cannot be applied to the adjustment process, and when the brightness differs greatly for each camera image, a discontinuity of brightness occurs near the joint where the images are to be combined. There was a problem that it was visible as a presentation video as if there was a step.

本発明は上記の問題に鑑みてなされたものであり、本発明が解決しようとする課題は、複数のカメラで撮影された映像の明るさが異なる場合でも、取得映像を合成するつなぎ目付近での明るさの不連続を防ぐことができる車載用カメラ映像提示装置を提供することにある。 The present invention has been made in view of the above-mentioned problems, and the problem to be solved by the present invention is that the image captured by a plurality of cameras has different brightness near the joint where the acquired images are combined even if the brightness is different. An object of the present invention is to provide an in- vehicle camera video presentation device that can prevent discontinuity in brightness.

この目的を達成するため、本発明においては、車両の２つ以上の相異なる位置に設置され、共通な視野領域を有する車両周囲映像を撮影する複数の撮影手段が、明るさ調整信号に応じて映像の明るさを調整して撮影し、撮影手段が撮影した車両周囲映像の視点変換処理を行い、視点変換された複数の車両周囲映像を１つの映像に合成して得られる車両周囲映像を提示することを特徴とする車載用カメラ映像提示装置を構成する。 In order to achieve this object, in the present invention, a plurality of photographing means that are installed at two or more different positions of the vehicle and photograph a vehicle surrounding image having a common visual field area are provided according to the brightness adjustment signal. Adjusts the brightness of the video and shoots, performs viewpoint conversion processing of the vehicle surrounding video shot by the shooting means, and presents the vehicle surrounding video obtained by combining the multiple vehicle surrounding videos converted from the viewpoint into one video An in-vehicle camera video presentation device is configured.

本発明の実施によって、複数のカメラで撮影された映像の明るさが異なる場合でも、取得映像を合成するつなぎ目付近での明るさの不連続を防ぐことができる車載用カメラ映像提示装置を提供することが可能となる。 By implementing the present invention, there is provided an in- vehicle camera video presentation device capable of preventing discontinuity of brightness in the vicinity of a joint for synthesizing acquired images even when the images captured by a plurality of cameras have different brightness. It becomes possible.

以下に、本発明を、実施の形態例によって、詳細に説明する。   Hereinafter, the present invention will be described in detail by way of embodiments.

（実施の形態例１）
図１に実施の形態例１である車載用カメラ映像提示装置の構成図を示す。この車載用カメラ映像提示装置は、２つの相異なる位置に設置され、車両周辺の共通な視野領域を有する車両周囲映像を撮影する撮影手段110a、110bと、画像処理装置100と、映像提示手段である液晶ディスプレイ150とから構成され、画像処理装置100には、映像を、仮想視点から撮影された映像に視点変換処理を行う視点変換処理手段120と、映像の明るさ調整を行う明るさ調整処理手段130と、複数の映像を１つの映像に合成する画像合成処理手段140とが具備されている。液晶ディスプレイ150は画像合成処理手段140による処理を経た車両周囲映像を提示する。 (Embodiment 1)
FIG. 1 shows a configuration diagram of an in-vehicle camera video presentation device according to the first embodiment. This in-vehicle camera video presentation device is installed at two different positions, and includes photographing means 110a and 110b for photographing a vehicle surrounding image having a common visual field around the vehicle, an image processing device 100, and a video presentation means. The image processing apparatus 100 includes a liquid crystal display 150. The image processing apparatus 100 includes a viewpoint conversion processing unit 120 that performs a viewpoint conversion process on a video captured from a virtual viewpoint, and a brightness adjustment process that adjusts the brightness of the video. Means 130 and image composition processing means 140 for synthesizing a plurality of videos into one video are provided. The liquid crystal display 150 presents a vehicle surrounding image that has been processed by the image composition processing means 140.

撮影手段110a、110bには、例えば、CCDカメラやCMOSカメラが用いられる。   For the photographing means 110a and 110b, for example, a CCD camera or a CMOS camera is used.

視点変換処理手段120や、明るさ調整処理手段130や、映像合成処理手段140には、例えば、マイクロコンピュータとメモリから構成される映像処理装置100上で動作するプログラムや、それぞれの処理を回路として組み込んだASIC、FPGAが用いられる。   The viewpoint conversion processing unit 120, the brightness adjustment processing unit 130, and the video composition processing unit 140 include, for example, a program that operates on the video processing device 100 including a microcomputer and a memory, and each process as a circuit. Embedded ASIC and FPGA are used.

以上の構成における装置の動作を説明する。図２に、本実施の形態例のフローチャートを示す。   The operation of the apparatus having the above configuration will be described. FIG. 2 shows a flowchart of this embodiment.

撮影手段110aと110bとを用いて映像を撮影し、その映像データを読み込む（S1001）。   A video is shot using the shooting means 110a and 110b, and the video data is read (S1001).

ここでは、撮影手段110aと110bとの映像を、視点変換処理手段120において、例えば、同一視点から見た映像に視点変換した映像を取得する（S1002）。   Here, the viewpoint conversion processing means 120 obtains, for example, a viewpoint-converted video from the images of the photographing means 110a and 110b (S1002).

撮影手段110aで取得した映像に視点変換処理を施して得た映像をIa、撮影手段110bで取得した映像に視点変換処理を施して得た映像をIbとし、この二つの映像の共通視野領域をそれぞれIa0、Ib0とする。   The video obtained by subjecting the video acquired by the photographing means 110a to the viewpoint conversion process is Ia, the video obtained by subjecting the video obtained by the photographing means 110b to the viewpoint conversion processing is Ib, and the common visual field area of these two videos is defined as Ib. Let them be Ia0 and Ib0, respectively.

IaとIbはそれぞれｋ個、ｍ個の画素から、Ia0とIb0はそれぞれｎ個の画素からなるものとし、Iaの各画素jの明るさをYaj、Ibの各画素jの明るさをYbjとし、Ia0の各画素jの明るさをYa0j、Ib0の各画素jの明るさをYb0jとすると、IaとIb及びIa0とIb0の明るさの平均値Ya_ave、Yb_ave、Ya0_ave、Yb0_aveを

Iaの明るさ Ya_ave ＝ (Ya1＋Ya2＋…＋Yak)/ｋ

Ibの明るさ Yb_ave ＝ (Yb1＋Yb2＋…＋Ybm)/ｍ

Ia0の明るさ Ya0_ave ＝ (Ya01＋Ya02＋…＋Ya0n)/ｎ

Ib0の明るさ Yb0_ave ＝ (Yb01＋Yb02＋…＋Yb0n)/ｎ

で定義する。 Ia and Ib are each composed of k and m pixels, Ia0 and Ib0 are each composed of n pixels, the brightness of each pixel j of Ia is Yaj, and the brightness of each pixel j of Ib is Ybj. When the brightness of each pixel j of Ia0 is Ya0j and the brightness of each pixel j of Ib0 is Yb0j, the average values Ya_ave, Yb_ave, Ya0_ave, Yb0_ave of the brightness of Ia and Ib and Ia0 and Ib0 are

Brightness of Ia Ya_ave = (Ya1 + Ya2 +… + Yak) / k

Ib brightness Yb_ave = (Yb1 + Yb2 + ... + Ybm) / m

Brightness of Ia0 Ya0_ave = (Ya01 + Ya02 +… + Ya0n) / n

Ib0 brightness Yb0_ave = (Yb01 + Yb02 + ... + Yb0n) / n

Define in.

明るさの調整処理を行う明るさ調整処理手段130において、撮影手段110aと110bで撮影した映像の視点変換を行った映像の明るさ平均と、共通視野領域の明るさ平均を求める（S1003）。   In the brightness adjustment processing means 130 that performs the brightness adjustment processing, the brightness average of the images that have undergone viewpoint conversion of the images shot by the shooting means 110a and 110b and the brightness average of the common visual field region are obtained (S1003).

次いで、撮影手段110aと110bで撮影した映像全体の明るさ平均の目標値をYtargetとすると、明るさの調整処理として、
撮影手段110aの映像を視点変換した映像の明るさを
(ｋ＋ｍ)×Yb0_ave×Ytarget/(Ya_ave×Yb0_ave×ｋ＋Yb_ave×Ya0_ave×ｍ）倍し、
撮影手段110bの映像を視点変換した映像の明るさを
(ｋ＋ｍ)×Ya0_ave×Ytarget/(Ya_ave×Yb0_ave×ｋ＋Yb_ave×Ya0_ave×ｍ)倍する（S1004）。この処理により共通視野領域のIa0とIb0の明るさが等しく、またIaとIbの映像を合計した範囲の明るさも目標値のYtargetとなる。このような映像の明るさ調整は、明るさ調整処理手段130の内部で行われる。 Next, assuming that Ytarget is the average brightness target value of the entire video imaged by the imaging means 110a and 110b,
The brightness of the image converted from the viewpoint of the image of the shooting means 110a
Multiply by (k + m) × Yb0_ave × Ytarget / (Ya_ave × Yb0_ave × k + Yb_ave × Ya0_ave × m)
The brightness of the image converted from the viewpoint of the image of the shooting means 110b
Multiply by (k + m) * Ya0_ave * Ytarget / (Ya_ave * Yb0_ave * k + Yb_ave * Ya0_ave * m) (S1004). By this processing, the brightness of the common visual field region Ia0 and Ib0 is equal, and the brightness of the range obtained by summing the images of Ia and Ib also becomes the target value Ytarget. Such image brightness adjustment is performed inside the brightness adjustment processing means 130.

このようにして、撮影手段110a、110bにより撮影された複数の車両周囲映像の共通な視野領域の明るさが等しくなり、しかも、該複数の車両周囲映像全体の明るさが予め決められた明るさ（Ytarget）となるように、撮影手段110a、110bにより得られた複数の車両周囲映像の明るさ調整処理を行うことができる。この場合には、時間が経過しても、一定の明るさの映像が提示される。   In this way, the brightness of the common visual field area of the plurality of vehicle surrounding images photographed by the photographing means 110a, 110b becomes equal, and the brightness of the entire plurality of vehicle surrounding images is predetermined brightness. The brightness adjustment processing of the plurality of vehicle surrounding images obtained by the photographing units 110a and 110b can be performed so as to be (Ytarget). In this case, a video with a certain brightness is presented even if time elapses.

次いで、明るさ調整を行った映像を画像合成処理手段140において合成し（S1005）、生成された映像データを液晶ディスプレイ150に提示し（S1006）、以降S1001から処理を繰り返す。   Next, the image whose brightness has been adjusted is synthesized by the image synthesis processing means 140 (S1005), and the generated video data is presented on the liquid crystal display 150 (S1006). Thereafter, the processing is repeated from S1001.

図７に映像データのイメージを示す。図において、上から１段目は被写体の明るさを示し、さらに、撮影手段であるカメラ110a、110bの撮影範囲（Ia、Ib）と、その範囲に共通な視野領域（Ia0、Ib0）を示している。上から２段目は、それぞれのカメラが、それぞれの標準明るさで撮影した映像を示している。上から３段目は、２段目に示した２つの映像を、視点変換処理手段120によって、同一視点から見た映像に視点変換し、そのまま、画像合成処理手段140によって、１つの映像に合成した結果を示していて、共通な視野領域（Ia0、Ib0）には、異なるカメラにより撮影された２つの映像の明るさの段差が現われている。   FIG. 7 shows an image of the video data. In the figure, the first row from the top shows the brightness of the subject, and further shows the shooting ranges (Ia, Ib) of the cameras 110a, 110b as the shooting means and the visual field areas (Ia0, Ib0) common to the ranges. ing. The second row from the top shows images taken by the respective cameras at the respective standard brightness. In the third row from the top, the two images shown in the second row are converted into the images viewed from the same viewpoint by the viewpoint conversion processing means 120, and are directly combined into one video by the image composition processing means 140. In the common visual field region (Ia0, Ib0), a difference in brightness between two images taken by different cameras appears.

これに対して、本実施の形態例においては、２段目に示した２つの映像を、視点変換処理手段120によって、同一視点から見た映像に視点変換し、さらに、明るさ調整処理手段130によって、共通な視野領域（Ia0、Ib0）における明るさが同じになるように、明るさ調整を行って４段目に示した映像とし、この２つの映像を、画像合成処理手段140によって、１つの映像に合成して、最下段に示した映像としている。画像合成前の、共通な視野領域（Ia0、Ib0）における明るさが同じになっているので、画像合成後の、最下段に示した映像には、異なるカメラにより撮影された２つの映像の明るさの段差が現われていない。すなわち、本発明の実施によって、このような明るさの段差を除去することができる。   On the other hand, in the present embodiment, the two images shown in the second row are converted into the images viewed from the same viewpoint by the viewpoint conversion processing unit 120, and the brightness adjustment processing unit 130 is further converted. Thus, the brightness is adjusted so that the brightness in the common visual field areas (Ia0, Ib0) is the same, so that the images shown in the fourth row are displayed. The images shown in the bottom row are combined into two images. Since the brightness in the common visual field area (Ia0, Ib0) before image composition is the same, the image shown in the bottom row after image composition is the brightness of two images taken by different cameras. There is no step. In other words, such brightness steps can be removed by implementing the present invention.

このように、複数の撮影手段により取得した複数の車両周囲映像の共通な視野領域の明るさが等しくなるようように、取得した映像全体の明るさが目標の明るさになるように、複数の撮影手段により得られた映像の映像処理を行ってから、処理した映像を合成して、生成した映像を提示するようにしたため、複数のカメラで撮影された映像の明るさが異なる場合でも、取得映像を合成するつなぎ目付近での明るさの不連続（明るさの段差）を防ぐことができる。   In this way, a plurality of images are acquired so that the brightness of the entire acquired image becomes the target brightness so that the brightness of the common visual field region of the plurality of vehicle surrounding images acquired by the plurality of photographing means becomes equal. After processing the video obtained by the shooting means, the processed video is combined and the generated video is presented, so even if the brightness of the video shot by multiple cameras is different It is possible to prevent discontinuity of brightness (brightness level difference) in the vicinity of the joint where the video is synthesized.

下記実施の形態例２と異なり、撮影した映像に対して明るさ調整を行うようにしているため、明るさの調整範囲は小さくなるが、撮影手段の明るさ調整を行うための制御信号（明るさ調整信号）を送る必要がなく、従来、車載カメラとして利用されているカメラをそのまま利用することが可能である。   Unlike the second embodiment described below, brightness adjustment is performed on a photographed image. Therefore, the brightness adjustment range is reduced, but a control signal (brightness) for adjusting the brightness of the photographing means is reduced. Therefore, it is possible to use a camera that has been conventionally used as an in-vehicle camera as it is.

なお、撮影手段110aと110bとの視差が大きくない場合等においては、視点変換処理手段120による視点変換処理を行わなくてもよい。すなわち、撮影手段110a、110bにより撮影された車両周囲映像を、視点変換処理なしに、明るさ調整処理手段130によって明るさ調整処理し、画像合成処理手段140によって画像合成処理し、液晶ディスプレイ150によって提示してもよい。   When the parallax between the photographing units 110a and 110b is not large, the viewpoint conversion process by the viewpoint conversion processing unit 120 may not be performed. That is, the vehicle surrounding image captured by the imaging units 110a and 110b is subjected to brightness adjustment processing by the brightness adjustment processing unit 130 without performing viewpoint conversion processing, image synthesis processing by the image synthesis processing unit 140, and the liquid crystal display 150. May be presented.

また、視点変換処理手段120による視点変換処理と明るさ調整処理手段130による明るさ調整処理との順序を反対にしてもよい。すなわち、撮影手段110a、110bにより撮影された車両周囲映像を、明るさ調整処理手段130によって明るさ調整処理した後に、視点変換処理手段120によって視点変換処理し、画像合成処理手段140によって画像合成処理し、液晶ディスプレイ150によって提示してもよい。   Further, the order of the viewpoint conversion processing by the viewpoint conversion processing unit 120 and the brightness adjustment processing by the brightness adjustment processing unit 130 may be reversed. That is, after the vehicle surrounding images captured by the imaging units 110a and 110b are subjected to brightness adjustment processing by the brightness adjustment processing unit 130, the viewpoint conversion processing unit 120 performs viewpoint conversion processing, and the image synthesis processing unit 140 performs image synthesis processing. However, it may be presented on the liquid crystal display 150.

（実施の形態例２）
図３に実施の形態例２の構成図を示す。この車載用カメラ映像提示装置は、２つの相異なる位置に設置され、車両周辺の共通な視野領域を有する車両周囲映像を撮影する、外部入力信号（明るさ調整信号）によって撮影映像の明るさ調整が可能な撮影手段である撮影手段210a、210bと、画像処理装置200と、映像提示手段である液晶ディスプレイ250とから構成され、画像処理装置200には、映像を、仮想視点から撮影された映像に視点変換処理を行う視点変換処理手段220と、映像の明るさ調整を行う明るさ調整処理手段230と、映像を１つの映像に合成する画像合成処理手段240とが具備されている。液晶ディスプレイ250は画像合成処理手段240による処理を経た車両周囲映像を提示する。 (Embodiment 2)
FIG. 3 shows a configuration diagram of the second embodiment. This in-vehicle camera image presentation device is installed at two different positions and shoots a vehicle surrounding image having a common visual field around the vehicle, and adjusts the brightness of the captured image by an external input signal (brightness adjustment signal). Imaging means 210a and 210b, which are imaging means capable of image processing, an image processing apparatus 200, and a liquid crystal display 250, which is a video presentation means. In the image processing apparatus 200, video is captured from a virtual viewpoint. Are further provided with viewpoint conversion processing means 220 for performing viewpoint conversion processing, brightness adjustment processing means 230 for adjusting the brightness of the video, and image composition processing means 240 for synthesizing the video into one video. The liquid crystal display 250 presents a vehicle surrounding image that has been processed by the image composition processing means 240.

撮影手段210a、210bには、例えば、CCDカメラやCMOSカメラが用いられる。   For the photographing means 210a and 210b, for example, a CCD camera or a CMOS camera is used.

視点変換処理手段220や、明るさ調整処理手段230や、画像合成処理手段240には、例えば、マイクロコンピュータとメモリから構成される映像処理装置200上で動作するプログラムや、それぞれの処理を回路として組み込んだASIC、FPGAが用いられる。   The viewpoint conversion processing unit 220, the brightness adjustment processing unit 230, and the image composition processing unit 240 include, for example, a program that operates on the video processing device 200 including a microcomputer and a memory, and each process as a circuit. Embedded ASIC and FPGA are used.

以上の構成における装置の動作を説明する。図４に本実施の形態例のフローチャートを示す。   The operation of the apparatus having the above configuration will be described. FIG. 4 shows a flowchart of this embodiment.

撮影手段210a、210bとを用いて映像を撮影し、その映像データを読み込む（S2001）。   A video is shot using the shooting means 210a and 210b, and the video data is read (S2001).

撮影手段210aで取得した映像をIa、撮影手段210bで取得した映像をIbとし、この二つの映像の共通視野領域をそれぞれIa0、Ib0とする。   The video acquired by the imaging means 210a is Ia, the video acquired by the imaging means 210b is Ib, and the common visual field areas of these two videos are Ia0 and Ib0, respectively.

IaとIbはそれぞれｍ個の画素から、Ia0とIb0はそれぞれｎ個の画素からなるものとし、Iaの各画素jの明るさをYaj、Ibの各画素jの明るさをYbjとし、Ia0の各画素jの明るさをYa0j、Ib0の各画素jの明るさをYb0jとすると、IaとIb及びIa0とIb0の明るさの平均値Ya_ave、Yb_ave、Ya0_ave、Yb0_aveを

Iaの明るさ Ya_ave ＝ (Ya1＋Ya2＋…＋Yam)/ｍ

Ibの明るさ Yb_ave ＝ (Yb1＋Yb2＋…＋Ybm)/ｍ

Ia0の明るさ Ya0_ave ＝ (Ya01＋Ya02＋…＋Ya0n)/ｎ

Ib0の明るさ Yb0_ave ＝ (Yb01＋Yb02＋…＋Yb0n)/ｎ

で定義する。 Ia and Ib are each composed of m pixels, Ia0 and Ib0 are each composed of n pixels, the brightness of each pixel j of Ia is Yaj, the brightness of each pixel j of Ib is Ybj, and Ia0 If the brightness of each pixel j is Ya0j and the brightness of each pixel j of Ib0 is Yb0j, the average values Ya_ave, Yb_ave, Ya0_ave, Yb0_ave of the brightness of Ia and Ib and Ia0 and Ib0 are

Brightness of Ia Ya_ave = (Ya1 + Ya2 +… + Yam) / m

Ib brightness Yb_ave = (Yb1 + Yb2 + ... + Ybm) / m

Brightness of Ia0 Ya0_ave = (Ya01 + Ya02 +… + Ya0n) / n

Ib0 brightness Yb0_ave = (Yb01 + Yb02 + ... + Yb0n) / n

Define in.

明るさの調整処理を行う明るさ調整処理手段230において、撮影手段210aと210bで撮影した映像の共通視野領域の明るさ平均を求める（S2002）。   In the brightness adjustment processing means 230 that performs the brightness adjustment processing, the brightness average of the common visual field area of the images photographed by the photographing means 210a and 210b is obtained (S2002).

次いで、撮影手段210aと210bで撮影した映像の明るさ平均の目標値をYtargetとすると、明るさの調整処理として、
撮影手段210aの映像を視点変換した映像の明るさを
２×Yb0_ave×Ytarget/(Ya_ave×Yb0_ave＋Yb_ave×Ya0_ave）倍し、
撮影手段210bの映像を視点変換した映像の明るさを
２×Ya0_ave×Ytarget/(Ya_ave×Yb0_ave＋Yb_ave×Ya0_ave）倍するように、撮影手段210aと210bの撮影照度の目標値を変更する（S2003）。撮影手段210aと210bにおける撮影照度の目標値の変更は、明るさ調整処理手段230が出力する照度調整のための信号（明るさ調整信号）を外部入力信号として行われる。すなわち、本実施の形態例においては、実施の形態例１とは異なり、映像の明るさ調整は、明るさ調整処理手段230からの信号（明るさ調整信号）によって、撮影手段210aと210bにおいて行われる。この明るさ調整処理により共通視野領域のIa0とIb0の明るさが等しく、またIaとIbの撮像範囲を合計した範囲の明るさも目標値のYtargetとなる。 Next, assuming that Ytarget is the average brightness target value of the images taken by the photographing means 210a and 210b,
Multiply the brightness of the image obtained by converting the viewpoint of the image of the photographing means 210a by 2 × Yb0_ave × Ytarget / (Ya_ave × Yb0_ave + Yb_ave × Ya0_ave)
The target values of the photographing illuminances of the photographing means 210a and 210b are changed so that the brightness of the picture obtained by converting the viewpoint of the picture of the photographing means 210b is multiplied by 2 × Ya0_ave × Ytarget / (Ya_ave × Yb0_ave + Yb_ave × Ya0_ave) (S2003). The change of the target value of the photographing illuminance in the photographing means 210a and 210b is performed by using an illuminance adjustment signal (brightness adjustment signal) output from the brightness adjustment processing means 230 as an external input signal. That is, in the present embodiment, unlike the first embodiment, image brightness adjustment is performed in the photographing means 210a and 210b by a signal (brightness adjustment signal) from the brightness adjustment processing means 230. Is called. By this brightness adjustment processing, the brightness of the common visual field area Ia0 and Ib0 is equal, and the brightness of the range obtained by summing the imaging ranges of Ia and Ib also becomes the target value Ytarget.

このようにして、撮影手段210a、210bにより撮影された複数の車両周囲映像の共通な視野領域の明るさが等しくなり、しかも、該複数の車両周囲映像全体の明るさが予め決められた明るさ（Ytarget）となるように、撮影手段210a、210bにより得られた複数の車両周囲映像の明るさ調整処理を行うことができる。この場合には、時間が経過しても、一定の明るさの映像が提示される。   In this way, the brightness of the common visual field area of the plurality of vehicle surrounding images photographed by the photographing means 210a and 210b is equal, and the brightness of the entire plurality of vehicle surrounding images is predetermined brightness. The brightness adjustment processing of a plurality of vehicle surrounding images obtained by the photographing units 210a and 210b can be performed so as to be (Ytarget). In this case, a video with a certain brightness is presented even if time elapses.

また、撮影手段210aと210bにおいて明るさ調整信号に応じて明るさを調整して撮影した映像を、視点変換処理手段220において、例えば、同一視点から見た映像に視点変換処理した映像を取得する（S2004）。   In addition, a video obtained by adjusting the brightness according to the brightness adjustment signal in the photographing units 210a and 210b and a video in which the viewpoint conversion processing unit 220 performs a viewpoint conversion process to a video viewed from the same viewpoint, for example, is acquired. (S2004).

次いで、視点変換処理手段240において視点変換を行った映像を合成し（S2005）、生成された映像データを液晶ディスプレイ250に提示し（S2006）、以降S2001から処理を繰り返す。   Next, the video whose viewpoint has been converted by the viewpoint conversion processing means 240 is synthesized (S2005), the generated video data is presented on the liquid crystal display 250 (S2006), and the processing from S2001 is repeated thereafter.

複数の撮影手段により取得した複数の車両周囲映像の共通な視野領域の明るさが等しくなるようように、取得した映像全体の明るさが目標の明るさになるように、複数の撮影手段により得られる映像の映像処理を行ってから、処理した映像を合成して、生成した映像を提示するようにしたため、複数のカメラで撮影された映像の明るさが異なる場合でも、取得映像を合成するつなぎ目付近での明るさの不連続を防ぐことができる。   Obtained by the plurality of photographing means so that the brightness of the entire acquired image becomes the target brightness so that the brightness of the common visual field area of the plurality of vehicle surrounding images obtained by the plurality of photographing means becomes equal. Since the processed video is processed and then the processed video is combined to present the generated video, even if the brightness of the video shot by multiple cameras is different, The discontinuity of brightness in the vicinity can be prevented.

また、実施の形態例１のように撮影した映像に対して明るさ調整を行うのではなく、撮影手段210a、210bにおいて、明るさ調整信号に応じて、撮影映像の明るさ調整を行うようにしたため、複数の撮影手段で撮影する映像の明るさの違いが大きい場合でも対応可能となる。   In addition, instead of adjusting the brightness of the captured image as in the first embodiment, the image capturing means 210a and 210b adjust the brightness of the captured image in accordance with the brightness adjustment signal. For this reason, it is possible to cope with a case where there is a large difference in the brightness of images taken by a plurality of photographing means.

なお、撮影手段210aと210bとの視差が大きくない場合等においては、視点変換処理手段220による視点変換処理を行わなくてもよい。すなわち、撮影手段210a、210bにより撮影された車両周囲映像（明るさ調整処理手段230からの明るさ調整信号によって明るさ調整されている）を、視点変換処理なしに、画像合成処理手段140によって画像合成処理し、液晶ディスプレイ250によって提示してもよい。   Note that when the parallax between the photographing units 210a and 210b is not large, the viewpoint conversion processing by the viewpoint conversion processing unit 220 may not be performed. That is, the vehicle surrounding image (brightness adjusted by the brightness adjustment signal from the brightness adjustment processing unit 230) captured by the imaging units 210a and 210b is imaged by the image composition processing unit 140 without the viewpoint conversion processing. It may be combined and presented on the liquid crystal display 250.

また、明るさ調整処理手段230による明るさ調整処理と視点変換処理手段220による視点変換処理との順序を反対にしてもよい。すなわち、撮影手段210a、210bにより撮影された車両周囲映像を、視点変換処理手段220によって視点変換処理した後に、明るさ調整処理手段230によって明るさ調整処理し、画像合成処理手段240によって画像合成処理し、液晶ディスプレイ250によって提示してもよい。ただし、この場合には、映像の明るさ調整は撮影手段210a、210bにおいて行われるので、明るさ調整後の映像は、視点変換処理手段220によって視点変換処理を施されてから画像合成処理手段240に入力される。   Further, the order of the brightness adjustment processing by the brightness adjustment processing means 230 and the viewpoint conversion processing by the viewpoint conversion processing means 220 may be reversed. That is, after the vehicle surroundings images shot by the photographing means 210a and 210b are subjected to viewpoint conversion processing by the viewpoint conversion processing means 220, brightness adjustment processing is performed by the brightness adjustment processing means 230, and image synthesis processing is performed by the image composition processing means 240. However, it may be presented on the liquid crystal display 250. However, in this case, since the image brightness adjustment is performed by the photographing means 210a and 210b, the image after the brightness adjustment is subjected to the viewpoint conversion processing by the viewpoint conversion processing means 220 and then the image composition processing means 240. Is input.

（実施の形態例３）
図５に実施の形態例３の構成図を示す。この車載用カメラ映像提示装置は、３つの相異なる位置に設置され、車両周辺の共通な視野領域を含む車両周囲映像を撮影する、外部入力信号（明るさ調整信号）によって撮影映像の明るさ調整が可能な撮影手段である撮影手段310a、310b、310cと、画像処理装置300と、映像提示手段である液晶ディスプレイ350とから構成され、画像処理装置300には、映像を、仮想視点から撮影された映像に視点変換処理を行う視点変換処理手段320と、映像の明るさ調整を行う明るさ調整処理手段330と、映像の合成を行う画像合成処理手段340とが具備されている。液晶ディスプレイ350は画像合成処理手段340による処理を経た車両周囲映像を提示する。 (Embodiment 3)
FIG. 5 shows a configuration diagram of the third embodiment. This in-vehicle camera video presentation device is installed at three different positions and shoots a vehicle surrounding image including a common visual field around the vehicle, and adjusts the brightness of the captured image by an external input signal (brightness adjustment signal). Image capturing means 310a, 310b, 310c, which are capable of capturing images, an image processing apparatus 300, and a liquid crystal display 350, which is a video presenting means. The image processing apparatus 300 captures images from a virtual viewpoint. Viewpoint conversion processing means 320 for performing viewpoint conversion processing on the video, brightness adjustment processing means 330 for adjusting the brightness of the video, and image composition processing means 340 for synthesizing the video. The liquid crystal display 350 presents a vehicle surrounding image that has been processed by the image composition processing means 340.

撮影手段310a、310b、310cには、例えば、CCDカメラやCMOSカメラが用いられる。   For the imaging means 310a, 310b, 310c, for example, a CCD camera or a CMOS camera is used.

視点変換処理手段320や、明るさ調整処理手段330や、画像合成処理手段340には、例えば、マイクロコンピュータとメモリから構成される映像処理装置300上で動作するプログラムや、それぞれの処理を回路として組み込んだASIC、FPGAが用いられる。   The viewpoint conversion processing unit 320, the brightness adjustment processing unit 330, and the image composition processing unit 340 include, for example, a program that operates on the video processing device 300 including a microcomputer and a memory, and each process as a circuit. Embedded ASIC and FPGA are used.

以上の構成において、撮影手段310aと310b、310bと310cがそれぞれ共通の視野領域を含む配置になっているものとして、その動作を説明する。図６に実施の形態例３のフローチャートを示す。   In the above configuration, the operation will be described on the assumption that the imaging units 310a and 310b and 310b and 310c are arranged to include a common visual field region. FIG. 6 shows a flowchart of the third embodiment.

撮影手段310a、310b、310cとを用いて映像を撮影し、その映像データを読み込む（S3001）。   Images are captured using the imaging means 310a, 310b, 310c, and the image data is read (S3001).

撮影手段310aで取得した映像をIa、撮影手段310bで取得した映像をIb、撮影手段310cで取得した映像をIcとし、この撮影手段310aと撮影手段310bの映像の共通視野領域をそれぞれIa0、 Ib0、撮影手段310bと撮影手段310cの映像の共通視野領域をそれぞれIb1、 Ic1とする。   The video acquired by the imaging means 310a is Ia, the video acquired by the imaging means 310b is Ib, the video acquired by the imaging means 310c is Ic, and the common visual field areas of the video of the imaging means 310a and the imaging means 310b are Ia0 and Ib0, respectively. The common visual field areas of the images of the photographing means 310b and the photographing means 310c are denoted by Ib1 and Ic1, respectively.

IaとIbとIcはそれぞれｋ個の画素から、Ia0とIb0はそれぞれｍ個の画素から、Ib1とIc1はそれぞれｎ個の画素からなるものとし、Iaの各画素jの明るさをYaj、Ibの各画素jの明るさをYbj、Icの各画素jの明るさをYcjとし、Ia0の各画素jの明るさをYa0j、Ib0の各画素jの明るさをYb0j、Ib1の各画素jの明るさをYb1j、Ic1の各画素jの明るさをYc1jとすると、IaとIbとIc及びIa0とIb0とIb1とIc1の明るさの平均値Ya_ave、Yb_ave、Yc_ave、Ya0_ave、Yb0_ave、Yb1_ave、Yc1_aveを

Iaの明るさ Ya_ave ＝ (Ya1＋Ya2＋…＋Yak)/ｋ

Ibの明るさ Yb_ave ＝ (Yb1＋Yb2＋…＋Ybk)/ｋ

Icの明るさ Yc_ave ＝ (Yc1＋Yc2＋…＋Yck)/ｋ

Ia0の明るさ Ya0_ave ＝ (Ya01＋Ya02＋…＋Ya0m)/ｍ

Ib0の明るさ Yb0_ave ＝ (Yb01＋Yb02＋…＋Yb0m)/ｍ

Ib1の明るさ Yb1_ave ＝ (Yb11＋Yb12＋…＋Yb1n)/ｎ

Ic1の明るさ Yc1_ave ＝ (Yc11＋Yc12＋…＋Yc1n)/ｎ

で定義する。 Ia, Ib, and Ic are each composed of k pixels, Ia0 and Ib0 are each composed of m pixels, Ib1 and Ic1 are each composed of n pixels, and the brightness of each pixel j of Ia is defined as Yaj, Ib The brightness of each pixel j of Ybj, the brightness of each pixel j of Ic is Ycj, the brightness of each pixel j of Ia0 is Ya0j, the brightness of each pixel j of Ib0 is the brightness of each pixel j of Yb0j, Ib1 If the brightness is Yb1j and the brightness of each pixel j of Ic1 is Yc1j, the average brightness Ya_ave, Yb_ave, Yc_ave, Ya0_ave, Yb0_ave, Yb1_ave, Yc1_ave of Ia, Ib, Ic, Ia0, Ib0, Ib1, and Ic1 The

Brightness of Ia Ya_ave = (Ya1 + Ya2 +… + Yak) / k

Ib brightness Yb_ave = (Yb1 + Yb2 + ... + Ybk) / k

Ic brightness Yc_ave = (Yc1 + Yc2 + ... + Yck) / k

Brightness of Ia0 Ya0_ave = (Ya01 + Ya02 +… + Ya0m) / m

Ib0 brightness Yb0_ave = (Yb01 + Yb02 +… + Yb0m) / m

Ib1 brightness Yb1_ave = (Yb11 + Yb12 + ... + Yb1n) / n

Ic1 brightness Yc1_ave = (Yc11 + Yc12 + ... + Yc1n) / n

Define in.

明るさの調整処理を行う明るさ調整処理手段330において、撮影手段310aと310bで撮影した映像の共通視野領域の明るさ平均を求める（S3002）。   In the brightness adjustment processing means 330 that performs brightness adjustment processing, the average brightness of the common visual field area of the images photographed by the photographing means 310a and 310b is obtained (S3002).

次いで、撮影手段310aと310bで撮影した映像の明るさ平均の目標値をYtargetとすると、明るさの調整処理として
撮影手段310aの映像を視点変換した映像の明るさを
３×Yb0_ave×Yc1_ave×Ytarget/(Ya_ave×Yb0_ave×Yc1_ave＋Yb_ave×Ya0_ave×Yc1_ave＋Yc_ave×Ya0_ave×Yb1_ave）倍し、
撮影手段310bの映像を視点変換した映像の明るさを
３×Ya0_ave×Yc1_ave×Ytarget/(Ya_ave×Yb0_ave×Yc1_ave＋Yb_ave×Ya0_ave×Yc1_ave＋Yc_ave×Ya0_ave×Yb1_ave）倍し、
撮影手段310cの映像を視点変換した映像の明るさを
３×Ya0_ave×Yb1_ave×Ytarget／（Ya_ave×Yb0_ave×Yc1_ave＋Yb_ave×Ya0_ave×Yc1_ave＋Yc_ave×Ya0_ave×Yb1_ave）倍するように、
撮影手段310aと310bと310cの撮影照度の目標値を変更する（S3003）。撮影手段310aと310bと310cにおける撮影照度の目標値の変更は、明るさ調整処理手段330が出力する照度調整のための信号（明るさ調整信号）を外部入力信号として行われる。すなわち、本実施の形態例においては、実施の形態例１とは異なり、映像の明るさ調整は、明るさ調整処理手段330からの信号（明るさ調整信号）によって、撮影手段310aと310bと310cにおいて行われる。 Next, if the target value of the average brightness of the images taken by the photographing means 310a and 310b is Ytarget, the brightness of the video obtained by converting the viewpoint of the video of the photographing means 310a as brightness adjustment processing is 3 × Yb0_ave × Yc1_ave × Ytarget / (Ya_ave x Yb0_ave x Yc1_ave + Yb_ave x Ya0_ave x Yc1_ave + Yc_ave x Ya0_ave x Yb1_ave)
Multiply the brightness of the image obtained by converting the image of the photographing means 310b by 3 × Ya0_ave × Yc1_ave × Ytarget / (Ya_ave × Yb0_ave × Yc1_ave + Yb_ave × Ya0_ave × Yc1_ave + Yc_ave × Ya0_ave × Yb1_ave)
The brightness of the image obtained by converting the viewpoint of the image of the photographing means 310c by 3 × Ya0_ave × Yb1_ave × Ytarget / (Ya_ave × Yb0_ave × Yc1_ave + Yb_ave × Ya0_ave × Yc1_ave + Yc_ave × Ya0_ave × Yb1_ave)
The target values of the illuminance for photographing of the photographing means 310a, 310b and 310c are changed (S3003). The change of the target value of the photographing illuminance in the photographing means 310a, 310b and 310c is performed using the signal for adjusting the illuminance (brightness adjustment signal) output from the brightness adjustment processing means 330 as an external input signal. That is, in the present embodiment, unlike the first embodiment, the image brightness adjustment is performed by the photographing means 310a, 310b, and 310c by the signal (brightness adjustment signal) from the brightness adjustment processing means 330. Done in

このようにして、撮影手段310a、310b、310cにより撮影された複数の車両周囲映像の共通な視野領域の明るさが等しくなり、しかも、該複数の車両周囲映像全体の明るさが予め決められた明るさ（Ytarget）となるように、撮影手段310a、310b、310cにより得られた複数の車両周囲映像の明るさ調整処理を行うことができる。この場合には、時間が経過しても、一定の明るさの映像が提示される。   In this way, the brightness of the common visual field area of the plurality of vehicle surrounding images photographed by the photographing means 310a, 310b, 310c becomes equal, and the brightness of the entire plurality of vehicle surrounding images is determined in advance. Brightness adjustment processing of a plurality of vehicle surrounding images obtained by the photographing units 310a, 310b, and 310c can be performed so that the brightness (Ytarget) is obtained. In this case, a video with a certain brightness is presented even if time elapses.

また、撮影手段310aと310bと310cにおいて明るさ調整信号に応じて明るさを調整して撮影した映像を、視点変換処理手段320において、例えば、同一視点から見た映像に視点変換処理した映像を取得する（S3004）。   Further, a video obtained by adjusting the brightness in accordance with the brightness adjustment signal in the photographing units 310a, 310b, and 310c, and a video in which the viewpoint conversion processing is performed in the viewpoint conversion processing unit 320, for example, a video viewed from the same viewpoint, are performed. Obtain (S3004).

次いで、画像合成手段340において、視点変換を行った複数の映像を１つの映像に合成し（S3005）、生成された映像データを液晶ディスプレイ350に提示し（S3006）、以降S3001から処理を繰り返す。   Next, the image synthesizing unit 340 synthesizes the plurality of videos subjected to viewpoint conversion into one video (S3005), presents the generated video data on the liquid crystal display 350 (S3006), and thereafter repeats the processing from S3001.

このようにして、複数の撮影手段により取得した複数の車両周囲映像の共通な視野領域の明るさが等しくなるようように、取得した映像全体の明るさが目標の明るさになるように、複数の撮影手段により得られた映像の映像処理を行ってから、処理した映像を合成して、生成した映像を提示するようにしたため、複数のカメラで撮影された映像の明るさが異なる場合でも、取得映像を合成するつなぎ目付近での明るさの不連続を防ぐことができる。   In this way, a plurality of images are acquired so that the brightness of the entire acquired image becomes the target brightness so that the brightness of the common visual field area of the plurality of vehicle surrounding images acquired by the plurality of photographing means becomes equal. After processing the video obtained by the shooting means, the processed video is synthesized and the generated video is presented, so even if the brightness of the video shot by multiple cameras is different, It is possible to prevent discontinuity of brightness near the joint where the acquired images are combined.

また、明るさ調整処理を、撮影された映像に対して行うのでなく、撮影手段310a、310b、310cにおいて行うようにしたため、複数の撮影手段で撮影する映像の明るさの違いが大きい場合でも対応可能となる。   In addition, the brightness adjustment processing is not performed on the captured video, but is performed on the imaging means 310a, 310b, 310c, so even if the difference in the brightness of the video captured by multiple imaging means is large It becomes possible.

なお、撮影手段310a、310b、310cの間の視差が大きくない場合等においては、視点変換処理手段320による視点変換処理を行わなくてもよい。すなわち、撮影手段310a、310b、310cにより撮影された車両周囲映像（明るさ調整処理手段330からの明るさ調整信号によって明るさ調整されている）を、視点変換処理なしに、画像合成処理手段340によって画像合成処理し、液晶ディスプレイ350によって提示してもよい。   Note that, when the parallax between the photographing units 310a, 310b, and 310c is not large, the viewpoint conversion processing by the viewpoint conversion processing unit 320 may not be performed. That is, the image composition processing means 340 without the viewpoint conversion processing is performed on the vehicle surrounding images (brightness adjusted by the brightness adjustment signal from the brightness adjustment processing means 330) taken by the photographing means 310a, 310b, 310c. The image composition processing may be performed by the liquid crystal display 350.

また、明るさ調整処理手段330による明るさ調整処理と視点変換処理手段320による視点変換処理との順序を反対にしてもよい。すなわち、撮影手段310a、310b、310cにより撮影された車両周囲映像を、視点変換処理手段320によって視点変換処理した後に、明るさ調整処理手段330によって明るさ調整処理し、画像合成処理手段340によって画像合成処理し、液晶ディスプレイ350によって提示してもよい。ただし、この場合には、映像の明るさ調整は撮影手段310a、310b、310cにおいて行われるので、明るさ調整後の映像は、視点変換処理手段320によって視点変換処理を施されてから画像合成処理手段340に入力される。   Further, the order of the brightness adjustment processing by the brightness adjustment processing means 330 and the viewpoint conversion processing by the viewpoint conversion processing means 320 may be reversed. In other words, the vehicle surrounding images captured by the imaging units 310a, 310b, and 310c are subjected to viewpoint conversion processing by the viewpoint conversion processing unit 320, and then brightness adjustment processing is performed by the brightness adjustment processing unit 330, and an image is combined by the image composition processing unit 340. It may be combined and presented on the liquid crystal display 350. However, in this case, since the brightness adjustment of the video is performed by the photographing means 310a, 310b, 310c, the video after the brightness adjustment is subjected to the viewpoint conversion processing by the viewpoint conversion processing means 320 and then the image composition processing Input to means 340.

実施の形態例１の構成図である。FIG. 2 is a configuration diagram of Embodiment 1; 実施の形態例１のフローチャートである。3 is a flowchart of the first embodiment. 実施の形態例２の構成図である。It is a block diagram of Example 2 of Embodiment. 実施の形態例２のフローチャートである。10 is a flowchart of Embodiment 2; 実施の形態例３の構成図である。FIG. 10 is a configuration diagram of a third embodiment. 実施の形態例３のフローチャートである。It is a flowchart of Embodiment 3. 実施の形態例１の映像データのイメージである。3 is an image of video data according to the first embodiment.

符号の説明Explanation of symbols

100：画像処理装置、110a、110b：撮影手段、120：視点変換処理手段、130：明るさ調整処理手段、140：画像合成処理手段、150：液晶ディスプレイ、200：画像処理装置、210a、210b：撮影手段、220：視点変換処理手段、230：明るさ調整処理手段、240：画像合成処理手段、250：液晶ディスプレイ、300：画像処理装置、310a、310b、310c：撮影手段、320：視点変換処理手段、330：明るさ調整処理手段、340：画像合成処理手段、350：液晶ディスプレイ。   100: Image processing device, 110a, 110b: Shooting means, 120: Viewpoint conversion processing means, 130: Brightness adjustment processing means, 140: Image composition processing means, 150: Liquid crystal display, 200: Image processing device, 210a, 210b: Shooting means, 220: viewpoint conversion processing means, 230: brightness adjustment processing means, 240: image composition processing means, 250: liquid crystal display, 300: image processing device, 310a, 310b, 310c: shooting means, 320: viewpoint conversion processing Means: 330: brightness adjustment processing means; 340: image composition processing means; 350: liquid crystal display.

Claims (2)

車載され、カメラにより車両周囲の映像を撮影して提示する車載用カメラ映像提示装置において、
少なくとも２つ以上の相異なる位置に設置され、共通な視野領域を有する車両周囲映像を撮影する複数の撮影手段と、
上記撮影手段における映像の明るさ調整用の明るさ調整信号を上記撮影手段へ向けて出力する明るさ調整処理手段と、
上記撮影手段が上記明るさ調整信号に応じて映像の明るさを調整して撮影した車両周囲映像の視点変換処理を行う視点変換処理手段と、
上記視点変換処理手段により視点変換された複数の車両周囲映像を１つの映像に合成する画像合成処理手段と、
上記画像合成処理手段により１つの映像に合成された車両周囲映像を提示する映像提示手段とを備え、
共通な視野領域の明るさが等しくなるように、しかも、複数の上記車両周囲映像全体の明るさが予め決められた明るさとなるように明るさ調整され、視点変換された複数の車両周囲映像を１つの映像に合成し、提示することにより、異なる上記撮影手段により撮影された車両周囲映像の共通な視野領域における明るさの段差が除去された車両周囲映像を提示し、
上記明るさ調整処理手段は、
共通の視野領域を有する少なくとも２つの車両周囲映像において、
一方の車両周囲映像の明るさを、
他方の車両周囲映像における共通の視野領域の明るさ平均値と、
所定の明るさ平均目標値と、
第１の値と、に基づいて調整すると共に、
上記他方の車両周囲映像の明るさを、
上記一方の車両周囲映像における共通の視野領域の明るさ平均値と、
上記所定の明るさ平均目標値と、
上記第１の値と、に基づいて調整し、
上記第１の値は、一の車両周囲映像の明るさ平均値に対して、他の車両周囲映像における共通の視野領域の明るさ平均値を乗じて得た第２の値を、全ての車両周囲映像について求めて、全ての上記第２の値を総和した値であることを特徴とする車載用カメラ映像提示装置。 In a vehicle-mounted camera video presentation device that is mounted on a vehicle and shoots and presents a video around the vehicle with a camera,
A plurality of photographing means installed at at least two different positions and photographing a vehicle surrounding image having a common visual field area;
A brightness adjustment processing means for outputting a brightness adjustment signal for adjusting the brightness of the image in the photographing means to the photographing means;
Viewpoint conversion processing means for performing viewpoint conversion processing of a vehicle surrounding image captured by the imaging means by adjusting the brightness of the video according to the brightness adjustment signal;
Image synthesis processing means for synthesizing a plurality of vehicle surrounding images whose viewpoints are converted by the viewpoint conversion processing means into one video;
Video presentation means for presenting a vehicle surrounding video synthesized into one video by the image synthesis processing means,
Brightness adjustment is performed so that the brightness of the common visual field area is equal , and the brightness of the entire plurality of the vehicle surrounding images is a predetermined brightness. By synthesizing and presenting in one video, the vehicle surrounding image in which the brightness step in the common visual field region of the vehicle surrounding image captured by the different imaging means is removed is presented ,
The brightness adjustment processing means includes:
In at least two vehicle surrounding images having a common field of view,
The brightness of the image around one vehicle
Brightness average value of the common visual field area in the other vehicle surrounding image,
A predetermined average brightness target value,
And adjusting based on the first value,
The brightness of the surrounding image of the other vehicle
Brightness average value of the common visual field area in the one vehicle surrounding image,
The predetermined brightness average target value;
Adjusting based on the first value,
The first value is a value obtained by multiplying the average brightness value of one vehicle surrounding image by the average brightness value of a common visual field area in other vehicle surrounding images. An in-vehicle camera image presentation device characterized in that it is a value obtained by calculating a surrounding image and summing all the second values . 請求項１に記載の車載用カメラ映像提示装置おいて、
上記明るさ調整処理手段は、複数の上記車両周囲映像の共通な視野領域の明るさが等しくなり、しかも、該複数の上記車両周囲映像全体の明るさが予め決められた明るさとなるように、映像の明るさ調整処理を行い、
上記車両周囲映像は、複数の画素を有し、
上記明るさ調整処理は、第１の演算又は第２の演算の一方に基づいて実行され、
上記第１の演算は、下記（１）及び（２）式を含み、
上記第２の演算は、下記（３）〜（５）式を含むことを特徴とする車載用カメラ映像提示装置。
Figure 0004857556
Figure 0004857556
Figure 0004857556
Figure 0004857556
Figure 0004857556
 但し、上記（１）〜（５）式において、
Yajは、複数の上記撮影手段に含まれる第１の撮影手段によって撮影された車両周囲映像の各画素における明るさであり、
Ybjは、複数の上記撮影手段に含まれ、上記第１の撮影手段と共通の視野領域を有する第２の撮影手段よって撮影された車両周囲映像の各画素における明るさであり、
Ycjは、複数の上記撮影手段に含まれ、上記第２の撮影手段と共通の視野領域を有する第３の撮影手段よって撮影された車両周囲映像の各画素における明るさであり、
Yaj_adjは、上記第１の撮影手段よって撮影され、明るさ調整が実行された車両周囲映像の各画素における明るさであり、
Ybj_adjは、上記第２の撮影手段よって撮影され、明るさ調整が実行された車両周囲映像の各画素における明るさであり、
Ycj_adjは、上記第３の撮影手段よって撮影され、明るさ調整が実行された車両周囲映像の各画素における明るさであり、
Ya_aveは、上記第１の撮影手段によって撮影された車両周囲映像における明るさ平均値であり、
Yb_aveは、上記第２の撮影手段によって撮影された車両周囲映像における明るさ平均値であり、
Yc_aveは、上記第３の撮影手段によって撮影された車両周囲映像における明るさ平均値であり、
Ya0_aveは、上記第１の撮影手段によって撮影された車両周囲映像において、上記第２の撮影手段によって撮影された車両周囲映像との共通の視野領域における明るさ平均値であり、
Yb0_aveは、上記第２の撮影手段によって撮影された車両周囲映像において、上記第１の撮影手段によって撮影された車両周囲映像との共通の視野領域における明るさ平均値であり、
Yb1_aveは、上記第２の撮影手段によって撮影された車両周囲映像において、上記第３の撮影手段によって撮影された車両周囲映像との共通の視野領域における明るさ平均値であり、
Yc1_aveは、上記第３の撮影手段によって撮影された車両周囲映像において、上記第２の撮影手段によって撮影された車両周囲映像との共通の視野領域における明るさ平均値であり、
Ytargetは、明るさ平均目標値である。 The in-vehicle camera video presentation device according to claim 1,
The brightness adjustment processing means is configured so that the brightness of a common visual field area of the plurality of vehicle surrounding images is equal, and the brightness of the plurality of vehicle surrounding images is a predetermined brightness. There line the brightness adjustment processing of the video,
The vehicle surrounding image has a plurality of pixels,
The brightness adjustment process is executed based on one of the first calculation or the second calculation,
The first calculation includes the following expressions (1) and (2):
The on-vehicle camera video presentation device , wherein the second calculation includes the following equations (3) to (5) .
Figure 0004857556
Figure 0004857556
Figure 0004857556
Figure 0004857556
Figure 0004857556
 However, in the above formulas (1) to (5),
Yaj is the brightness at each pixel of the vehicle surrounding image photographed by the first photographing means included in the plurality of photographing means,
Ybj is the brightness at each pixel of the vehicle surrounding image that is included in the plurality of imaging units and is captured by the second imaging unit having a visual field common to the first imaging unit,
Ycj is the brightness at each pixel of the vehicle surrounding image that is included in the plurality of imaging units and is captured by the third imaging unit having a common visual field area with the second imaging unit,
Yaj_adj is the brightness at each pixel of the vehicle surrounding image that has been photographed by the first photographing means and subjected to brightness adjustment,
Ybj_adj is the brightness at each pixel of the vehicle surrounding image that has been shot by the second shooting means and for which the brightness adjustment has been performed,
Ycj_adj is the brightness at each pixel of the vehicle surrounding image that has been photographed by the third photographing means and subjected to brightness adjustment,
Ya_ave is an average brightness value in the vehicle surrounding image captured by the first imaging means,
Yb_ave is the average brightness value in the vehicle surrounding image captured by the second imaging means,
Yc_ave is an average brightness value in the vehicle surrounding image captured by the third imaging means,
Ya0_ave is a brightness average value in a visual field region common to the vehicle surrounding image captured by the second imaging unit in the vehicle surrounding image captured by the first imaging unit,
Yb0_ave is a brightness average value in a visual field region common to the vehicle surrounding image captured by the first imaging unit in the vehicle surrounding image captured by the second imaging unit,
Yb1_ave is an average brightness value in a visual field region common to the vehicle surrounding image captured by the third imaging unit in the vehicle surrounding image captured by the second imaging unit,
Yc1_ave is an average brightness value in a visual field region common to the vehicle surrounding image captured by the second imaging unit in the vehicle surrounding image captured by the third imaging unit,
Ytarget is the brightness average target value.
JP2004362521A 2004-12-15 2004-12-15 In-vehicle camera video presentation device Active JP4857556B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2004362521A JP4857556B2 (en) 2004-12-15 2004-12-15 In-vehicle camera video presentation device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2004362521A JP4857556B2 (en) 2004-12-15 2004-12-15 In-vehicle camera video presentation device

Publications (2)

Publication Number Publication Date
JP2006173988A JP2006173988A (en) 2006-06-29
JP4857556B2 true JP4857556B2 (en) 2012-01-18

Family

ID=36674274

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2004362521A Active JP4857556B2 (en) 2004-12-15 2004-12-15 In-vehicle camera video presentation device

Country Status (1)

Country Link
JP (1) JP4857556B2 (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4801549B2 (en) * 2006-09-25 2011-10-26 パナソニック株式会社 Composite video display device and integrated circuit
JP4818862B2 (en) * 2006-09-25 2011-11-16 パナソニック株式会社 Composite video display device
US8791984B2 (en) 2007-11-16 2014-07-29 Scallop Imaging, Llc Digital security camera
WO2009064504A1 (en) 2007-11-16 2009-05-22 Tenebraex Corporation Systems and methods of creating a virtual window
WO2011093354A1 (en) * 2010-01-29 2011-08-04 富士通テン株式会社 Image processing device and image processing method
JP5413843B2 (en) * 2010-01-29 2014-02-12 富士通テン株式会社 Image processing apparatus and image processing method
EP2892230A4 (en) 2012-08-30 2015-09-02 Fujitsu Ltd Image processing apparatus, image processing method, and program
CN104071076A (en) * 2014-07-17 2014-10-01 贵州大学 Vehicle-mounted head-up display system and display method thereof
CN105007380B (en) * 2015-07-20 2019-02-01 深圳前海智云谷科技有限公司 The method and system of audio call vehicle secretarial service are utilized based on head-up-display system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE69324224T2 (en) * 1992-12-29 1999-10-28 Koninkl Philips Electronics Nv Image processing method and apparatus for generating an image from a plurality of adjacent images
JP2001320616A (en) * 2000-02-29 2001-11-16 Matsushita Electric Ind Co Ltd Image pickup system
JP2002251608A (en) * 2001-02-23 2002-09-06 Mixed Reality Systems Laboratory Inc Device and method for controlling imaging device, device and method for image processing, program code, and storage medium
JP3297040B1 (en) * 2001-04-24 2002-07-02 松下電器産業株式会社 Image composing and displaying method of vehicle-mounted camera and apparatus therefor

Also Published As

Publication number Publication date
JP2006173988A (en) 2006-06-29

Similar Documents

Publication Publication Date Title
JP4900343B2 (en) Video imaging apparatus and exposure guide display method
US9473698B2 (en) Imaging device and imaging method
US8885067B2 (en) Multocular image pickup apparatus and multocular image pickup method
US8854528B2 (en) Imaging apparatus
WO2012077464A1 (en) Image pickup device
KR20130112574A (en) Apparatus and method for improving quality of enlarged image
KR20110022534A (en) Display control apparatus, display control method and storage medium
JP6786311B2 (en) Image processing equipment, image processing methods, computer programs and storage media
JP4857556B2 (en) In-vehicle camera video presentation device
JP4586707B2 (en) Image processing apparatus, electronic camera, and image processing program
US20130083169A1 (en) Image capturing apparatus, image processing apparatus, image processing method and program
JP2010206521A (en) Image capturing apparatus, method of correcting image, and program
JP5569617B2 (en) Image processing apparatus and program
JP5245648B2 (en) Image processing apparatus and program
JP2003348422A (en) Imaging apparatus, imaging method, program and storage medium
JP2010213105A (en) Imaging apparatus
JP2022083147A (en) Imaging apparatus, imaging method, program, and recording medium
JP6961423B2 (en) Image processing equipment, imaging equipment, control methods for image processing equipment, programs and recording media
CN109906602B (en) Image matching method and device
US11165956B2 (en) Imaging apparatus
JP3368012B2 (en) Image data processing device
JP2014236244A (en) Image processing system, image processing program, and digital camera
JP2013046343A (en) Image pickup device
US11800234B2 (en) Image capturing apparatus, control method, and storage medium
US20240089597A1 (en) Image capturing apparatus for capturing and compositing images different in in-focus position, control method, and storage medium

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20071029

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20100618

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20100629

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20100727

RD02 Notification of acceptance of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7422

Effective date: 20101013

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20101028

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20110322

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20110608

A911 Transfer to examiner for re-examination before appeal (zenchi)

Free format text: JAPANESE INTERMEDIATE CODE: A911

Effective date: 20110616

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20111004

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20111017

R150 Certificate of patent or registration of utility model

Ref document number: 4857556

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20141111

Year of fee payment: 3