JP4486737B2 - Spatial information generation device for mobile mapping - Google Patents

Description

そして、単方向モデル生成処理部２３は、メモリ３２のレーザデータを入力する毎にキャリブレーションデータを用いて、地図座標における座標値（距離、角度、位置、姿勢より求められる）を定義して、空間座標におけるレーザデータモデルを生成する。これは、レーザの３方向について行っている。
【００７２】
つまり、図１０の（ｂ）のレーザデータＲＭ（ＲＭＤ）は、図１２に示すような測定点の座標群に変換して記憶される。そして、多方向モデル合成処理部は、これらの３方向のレーザデータを地理座標空間にて合成して、図１３に示すレーザモデルを得る。
【００７３】
サーフェースモデル生成処理部２５は、このレーザモデルを分割して図１４の（ａ）、（ｂ）、（ｃ）に示すようにサーフェース化する。
【００７４】
サーフェースを生成するに当たっての平面パッチの抽出（図１４の（ｂ））は、初めにレンジポイント（測定点）全体をあるルールによって分割する。分割されたものをレンジセルと称する。分割に当たっては、各レンジセルに注目して事前に設定したしきい値の範囲内で対応するレンジポイントが空間平面に回帰できるかどうかを判断する（回帰の残差の分散値がしきい値より小さいと一つの空間平面に回帰できると判断する）。
【００７５】
そこで、一つの空間平面に回帰されないレンジセルはさらに分割し、このプロセスを繰り返す。
【００７６】
分割が終わった後のレンジセルは平面パッチを抽出するプロセスにおける最小なユニットとなる。
【００７７】
このような初期化によりレンジポイントを幾つかのレンジセルに分割し（図１４の（ｂ））、各レンジセルは最小二乗法による空間に一意な平面パッチに対応しているので、空間物体の形状はこれらの平面パッチによって近似される。
【００７８】
しかし、この時点での近似は、平面パッチの数が多くて近似効率が悪い、近似効率を高めるために、同じ平面に対応するレンジセルをマージしポリゴンを生成する（図１４（ｃ）、図１５の（ａ）、（ｂ））。このプロセスは、対応する平面がほぼ同一平面上にある。すなわち、同じセンサ座標の原点から平面に下ろした垂線の足の位置が最も近い２つのレンジセルから始める。
【００７９】
レンジセルをマージした結果として近似効率は高まるが近似精度は悪くなる可能性もある。
【００８０】
そこで、２つのレンジセルのマージするかどうか、つまり、近似効率と近似精度のバランスは、ＭＤＬ（Ｍｉｎｉｍｕｍ Ｄｅｓｃｒｉｐｔｉｏｎ Ｌｅｎｇｈｔｈ）符号長が小さくなるかどうかによって決まる。
【００８１】
次に、画像マッピング処理部２６は、サーフェースモデルが生成されると、サーフェースモデルの各ポリゴンを読み、このポリゴンに対応するカメラデータを位置姿勢データとカメラキャリブレーションデータより引き当て、この位置姿勢データに対応する全ての画像データをポリゴンに割り当てた図１３の（ｂ）に示す３次元グラフィック画像を生成する。
【００８２】
つまり、この３次元グラフィック画像には、３次元位置とデータとが対応させられていることになるから、例えば地図座標に定義されたカーナビに本３次元画像データを記憶しておくと、カーナビ画像の道路上の所定の点を選択すると、その点の建物の３次元画像を得ることができる。
【００８３】
なお、以下に本発明に係る装置の代表的な用途例について説明する。
【００８４】
（１）都市防災
都市には道路、建物及び大小様々な施設が密集しており、上空からでは捉えられない地物も存在するが、本発明の装置は、都市構造物の空間的な配置や３次元形状を正確に再現させることが可能となるため、災害時の避難・誘導シミュレーションや被害予測システムを構築する上で極めて有効な情報を提供できる。
【００８５】
（２）施設管理
大都市では、電力、ガス、上下水道等の膨大なライフライン網が複雑に入り組んでいる。既に、公共企業が独自の管理システムを構築しているが、情報の相互運用を図るための基準情報や不足情報の新規取得に効力を発揮する。この情報を利用すれば、３次元地図画像で電柱と建物の位置関係を見て工事の場所を決めたり、既設設備の位置や劣化状況を正確に把握させることが可能となる。
【００８６】
＜実施の形態２＞
本発明によって得られた３次元グラフィック画像は、実映像のテクスチャー情報を持つため、３次元ＣＧを利用したリアルな景観シュミレーション、都市の中を動き回るウォークスルーやサイバーショッピングに利用できる。
【００８７】
例えば、タクシーは常に街を回っているが、その間は客が乗っていなければ無駄な走行であるが街の最新の様子を知ることになる。また、客を乗せて走ったとしても、料金は走行距離だけである。そこで、図１６に示すように、例えば実施の形態１で用いたマルチラインカメラとＧＰＳ、カメラ制御装置からなる画像取得装置をタクシーに搭載し、この画像データをセンターに送信する。
【００８８】
センターは携帯電話又はパソコンからの要望地点を取得し、この地点の画像をその携帯又はパソコンに送信して課金すると共に、タクシーの会社サーバには画像取得の履歴を配信しておき、後日その分を支払う。
【００８９】
前述の画像取得装置４０（レンタルが好ましい）は、携帯電話４１に接続され、センター側から携帯電話４１で「ｘｘ通りのｘｘｘビル当たりを取得」という指示があったときに、画像取得装置４０に対して撮影指示を入力して撮影させ、このマルチラインカメラの各ラインセンサの画像データを年月日とＧＰＳ時刻と対応させてメモリ（図示せず）に順次記憶する。このデータを総称してカメラデータという。
【００９０】
そして、タクシーのドライバーは撮影が終わると、携帯電話４１を用いてメモリのカメラデータをネットワークを介してセンターに送信する。センターの画像収集装置４２は、タクシーからカメラデータが送信される毎に、携帯電話４１の電話番号（タクシーＩＤとして用いる）とカメラデータとを対応させてデータベース（図示せず）に記憶する。
【００９１】
また、センターの配信・課金サーバ４３は、ユーザの携帯電話４４又はパソコン４５から例えば「渋谷」のエリア地図の配信要求を受けたとき、携帯電話４４の電話番号又はパソコンのネットワークＩＤをデータベース４６に記憶すると共に、渋谷の地図をデータベース４７から検索し、これを送信する。この地図は地理座標に定義付けられたものであり、この地図の所望の地点が選択されると、その地点の緯度経度が分かる。
【００９２】
そして、ユーザは、携帯電話４４又はパソコン４５の画面に表示された渋谷の地図の所望の地点を選択し、カメラデータの欲しい時間帯を入力する。
【００９３】
これによって、選択地点の緯度経度と時間帯がネットワークを介してセンターに送信され、配信・課金サーバ４３が受信する。
【００９４】
配信・課金サーバ４３は、ユーザからの緯度経度、時間帯を有するカメラデータがデータベース４２に存在するかどうかを判断し、存在する場合は、そのカメラデータをユーザ側の携帯電話４４又はパソコン４５に送信する。
【００９５】
これによって、携帯電話４４又はパソコン４５には、渋谷の所望の地点のビル前の様子等を画像で判断できるので、例えばこれから行きたい店が混んでいるか又は車が道路に複数停車しているかが分かる。
【００９６】
そして、配信・課金サーバ４３は、ユーザに配信した時間帯のカメラデータを取得したタクシーＩＤと取得画像配信日時刻とをデータベース４８に記憶し、後日これをタクシー会社のサーバ４９に配信しておく。タクシー会社はこのサーバの配信データの回数から予め定められた金額をセンターに請求する。
【００９７】
また、配信・課金サーバ４３は、要求のあったパソコン又は携帯電話４４のＩＤ又は電話番号、要求エリア、時間等をデータベースに４６に記憶しており、これらのデータに基づく所定の課金を行って、銀行のユーザの口座からセンターの口座に引き落とす。
【００９８】
さらに、ユーザからの緯度経度、時間帯を有するカメラデータがデータベース４２に存在しない場合は、タクシーの携帯電話４１に電話をかけて、「渋谷のｘｘビルをｘｘｘ時間帯で取得することを通報する。
【００９９】
この通報は、タクシーにＧＰＳを搭載しているので、タクシー側から現在位置を常に取得し、渋谷のｘｘビルをｘｘｘ時間帯で最も早く通過するタクシーを見つけて、通報する。
【０１００】
なお、上記のユーザは、報道関係、警察であってもよい。特に警察の場合は、通りの駐車違反及び車両番号を直ぐに分かることになると共に治安に役立つ。
【０１０１】
【発明の効果】
以上のように本発明によれば、２個のマルチラインカメラと、ＤＧＰＳ受信器と、３台のレーザスキャナと、慣性航法装置とを移動体上に固定して移動しながら得たカメラデータ、レーザデータ、ＩＮＳデータ、ＧＰＳ時刻を用いて３次元グラフィック画像を生成するとき、ＩＮＳデータである位置及び姿勢データ等をカメラデータ、レーザデータに結びつけた後に、これらのレーザデータを地理座標において合成し、この３次元のレーザデータを所定条件に従って分割した平面のポリゴンに、各距離値、角度に結びつけられる画像データをカメラデータのＩＮＳデータに基づいて検索する。そして、該検索した画像データをポリゴンに割り付けて３次元グラフィック画像を生成する。
【０１０２】
このため、都市空間等をより忠実に再現できる。また、カメラデータとレーザデータには位置姿勢データ、時刻が対応させられているから、各地点における所望の方向からの高精度画像を容易に得ることができるという効果が得られている。
【０１０３】
また、それぞれのカメラデータは、撮像方向の角度は４５度にされて取得したものであるから樹木があっても、その樹木を除いた建物の映像を生成することができるとうい効果が得られている。
【図面の簡単な説明】
【図１】本発明の実施の形態１のモービルマッピング用空間情報生成装置の概略構成図である。
【図２】マルチラインカメラの側面図である。
【図３】マルチラインカメラの正面図である。
【図４】架台上におけるマルチラインカメラとレーザ、ＩＮＳ、ＤＧＰＳの取り付け位置を示す平面図である。
【図５】画像取得装置の撮像、測距離の方向を説明する説明図である。
【図６】取得されたカメラデータ、レーザデータ、ＩＮＳデータを説明する説明図である。
【図７】正確な時刻にされたＩＮＳデータの説明図である。
【図８】カメラデータ、レーザデータ、ＩＮＳデータの同期、補間のタイミングチャートである。
【図９】同期、補間されたＩＮＳデータを説明する説明図である。
【図１０】同期、補間されてＩＮＳデータが対応づけられたカメラデータ、レーザデータを説明する説明図である。
【図１１】キャリブレーション処理を説明する説明図である。
【図１２】キャリブレーション結果を反映したレーザデータの説明図である。
【図１３】測距離データとサーフェース画像を説明する説明図である。
【図１４】レーザデータによる合成モデルと分割を説明する説明図である。
【図１５】本実施の形態による抽出画像を説明する説明図である。
【図１６】実施の形態２のモービルマッピング取得データの配信システムの概略構成図である。
【符号の説明】
１ 架台
２ レーザスキャナ
３ マルチラインカメラ
２０ データ取込処理部
２１ データ同期処理部
２２ センサキャリブレーション処理部
２３ 単方向モデル生成処理部
２４ 多方向モデル合成処理部
２５ サーフェースモデル生成処理部
２６ 画像マッピング処理部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a mobile mapping spatial information generation apparatus that easily and accurately matches laser distance measurement data and image data even for complex things such as urban spaces.
[0002]
[Prior art]
In the case of obtaining a three-dimensional image of a city, if it is performed by CAD, it must be created manually by using an existing map or photograph, which is very time consuming. Moreover, since an actual image and measurement data are not used, it lacks reality.
[0003]
Therefore, in recent years, three-dimensional information of buildings, roads, trees, and the like (hereinafter collectively referred to as features) is measured from stereo photographs using area sensors.
[0004]
[Problems to be solved by the invention]
In order to obtain the shape information of the entire object with high accuracy, range images (views) measured from a plurality of points and directions must be connected to the same global coordinate system.
[0005]
In addition, the features in the urban space are scattered, and the concealing part cannot be measured only from the same viewpoint.
[0006]
Therefore, there is a problem that it is extremely difficult to match the distance measurement data by the laser with the image data in an urban space or the like, to reduce the concealment portion, and to obtain a high resolution image.
[0007]
The present invention has been made to solve the above-described problems, and automatically obtains distance measurement data and image data up to a feature obtained directly by a laser even for a complicated matter such as an urban space. It is an object of the present invention to provide a mobile mapping spatial information generating apparatus capable of efficiently obtaining a spatial image matched on three-dimensional map coordinates with high accuracy.
[0008]
[Means for Solving the Problems]
The first aspect of the present invention is that two multi-line cameras in which three cameras having a fisheye lens and a line sensor are combined horizontally with a predetermined imaging angle and a DGPS receiver are emitted in different directions. 3D graphic images using camera data, laser data, INS data consisting of position and orientation, and GPS time obtained while moving the three laser scanners and the inertial navigation device fixed on the moving body. A mobile mapping spatial information generating device for generating the camera data and the laser data having the GPS time, the means for linking the INS data having the GPS time to the camera data having the GPS time, and reading the laser data of each laser scanner , The distance value to the measurement point indicated by each laser data, the angle of the laser direction with respect to the measurement point, Means for defining each three-dimensional geographic coordinate on the basis of data; means for synthesizing a unidirectional model of each laser defined in each three-dimensional geographic coordinate on the three-dimensional coordinate; and the synthesized model Is divided into planes according to predetermined conditions, and the divided area is converted into polygons, and image data linked to each distance value and angle of the polygon is searched based on the INS data of the camera data. And a means for allocating image data to the polygon to generate the three-dimensional graphic image.
[0009]
  The present inventionTwo multi-line cameras that have a horizontal combination of three fisheye lenses and line sensors with a predetermined imaging angle, a DGPS receiver, and a laser that emits laser beams in different directions 3 Laser scannerWhen,Inertial navigation device fixed on a moving objectAndA mobile mapping spatial information generating device that generates a three-dimensional graphic image using camera data, laser data, INS data including a position and orientation, and GPS time obtained while moving,
[0010]
  Means for linking the INS data having the GPS time to the camera data and laser data having the GPS time;
[0011]
  Means for reading the laser data of each laser scanner and defining the distance value to the measurement point indicated by each laser data and the angle of the laser direction with respect to the measurement point in each three-dimensional geographic coordinate based on the INS data When,
[0012]
  Means for synthesizing the unidirectional model of each laser defined in the respective three-dimensional geographic coordinates on the three-dimensional coordinates;
Means for dividing the composite model into planes according to predetermined conditions, and polygonizing the divided areas;
Means for searching image data linked to each distance value and angle of the polygon based on the INS data of camera data, and assigning the searched image data to the polygon to generate the three-dimensional graphic image;The main point is that
[0013]
DETAILED DESCRIPTION OF THE INVENTION
<Embodiment 1>
FIG. 1 is a schematic configuration diagram of a mobile mapping spatial information generating apparatus according to the first embodiment.
[0014]
The mobile mapping spatial information generating apparatus according to the present embodiment includes lasers 2A, 2B, 2C and multi-line cameras 3A, 3B (three fisheye lenses, respectively) of the image acquisition device 12 fixed behind the roof of the moving body 1. A line sensor), distance data Ai (A1, A2, A3) to the feature acquired by the hybrid inertial navigation system and DGPS, line sensor image data Di (D1, D2, D3), and position data Bi, attitude data θi, and GPS data Fi (GPS time data f1, position data f2) are taken in, and distance data Ah, image data Di, position data Bi, and attitude data θi are automatically connected with high accuracy. Spatial information is made into a graphic by generating a spatial model.
[0015]
Before describing the configuration of the mobile mapping spatial information generation apparatus according to the present embodiment, the spatial information acquisition apparatus will be briefly described.
[0016]
The multiline cameras 3A and 3B of the image acquisition device 12 will be described with reference to FIGS. FIG. 2 is a side view of the multiline camera, and FIG. 3 is a front view of the multiline camera.
[0017]
As shown in FIG. 2, the multi-line camera 3 has an optical system unit 8 provided with a line sensor 7 behind the fish-eye lens 6 so that the arrangement direction of the pixels becomes a vertical direction, and the line sensor 7 to drive the subject. As shown in FIG. 3, three line cameras 10 including the camera main body 9 that sends out the image pickup signal are arranged side by side in the horizontal direction and integrated (in the present embodiment, the first The first line camera 10a, the second line camera 10b, and the third line camera 10c are referred to.
[0018]
Further, the first line camera 10a, the second line camera 10b, and the third line camera 10c have the first line camera 10a and the third line on both sides of the second line camera 10b at the center. The camera 10c is integrated in the horizontal direction so that the imaging direction of the camera 10c is 45 degrees with respect to the imaging direction of the second line camera 10b.
[0019]
Such multi-line cameras 3A and 3B are fixed to a gantry 4 which is a flat plate as shown in FIG. The gantry 4 is attached to the rear of the moving body 1 as shown in FIG. 5, and the multiline cameras 3A and 3B are gantry so that the imaging direction of the respective second line cameras 10b is perpendicular to the traveling direction. 4 is attached to the center of both sides.
[0020]
In addition, three laser scanners 2A, 2B, and 2C are arranged on the gantry 4 while being shifted by a predetermined angle.
[0021]
  Also, RackA DGPS antenna 4 installed at a substantially central portion of the base 4, a DGPS receiver (in the vehicle),Sex voyageAnd a legal apparatus 11 (also referred to as INS).
[0022]
The laser scanners 2A, 2B, and 2C have scanner heads 2a, 2b, and 2c whose taper surfaces (inclined mirrors) are rotatable, respectively, at the corners on both sides of the gantry 4 on the rear part of the vehicle and the central part thereof. It is attached.
[0023]
For example, the laser scanner 2A is arranged at the rear center of the gantry 4 and is attached to the laser scanner 2A at an angle of 45 degrees horizontally. The three laser scanners 2A, 2B, and 2C irradiate infrared lasers having a very short pulse width from the scanner heads 2a, 2b, and 2c, and the lasers are reflected from the features such as buildings and trees and returned. The distance is calculated from the time until the laser beam is transmitted, and the cross-sectional profile measurement in the direction of 360 degrees is performed in combination with the rotation angle of the scanner heads 2a, 2b, and 2c when the laser is transmitted to obtain distance data and laser angle data.
[0024]
Such an image acquisition device 12 is mounted on the moving body 1 and travels to acquire surrounding images, distance data, position, posture, and angle as shown in FIG.
[0025]
In other words, if there is a tree in front of the second line camera in the center, an image of the building that includes the tree is taken, but since it is running, the second line camera when it passes the tree position The first line camera obtains an image of the back of the tree that could not be taken in step 1. Since each line sensor is attached in the vertical direction, the image obtained from the fisheye lens is a line image of 180 degrees or more in the vertical direction.
[0026]
At the same time, the distance from each point of the surrounding features to the car is acquired by three laser scanners.
[0027]
The distance data Ah, image data Di, position data Bi, and attitude data θi used in the first embodiment are premised on using data acquired by such a moving body 1.
[0028]
The distance data Ah is acquired at a frequency of 20 Hz, the image data Di is acquired at a frequency of 100 Hz, and the position and orientation data is acquired at a frequency of 50 Hz.
[0029]
Next, the configuration of FIG. 1 will be described. As shown in FIG. 1, the mobile mapping spatial information generating device 15 of the present embodiment includes a file 16 in which distance data Ah is stored, a file 17 in which position data Bi and posture data θi are stored, and image data. A file 18 in which Di is stored, a data capture processing unit 20, a data synchronization processing unit 21, a sensor calibration processing unit 22, a unidirectional model generation processing unit 23, a multidirectional model synthesis processing unit 24, A surface model generation processing unit 25, an image mapping processing unit 26, a database 27 for storing three-dimensional graphic data, and a multiline camera sensor calibration processing unit 33 (also referred to as a camera calibration processing unit) are provided. Yes.
[0030]
The image data Di, distance data Ah, position data Bi, and attitude data θi of each file will be described with reference to FIG. However, in this description, the image data Di is described as image data Di of a multi-line camera on one side. The distance data Ai will be described as distance data Ah from one laser scanner.
[0031]
The image data Di is acquired by the line camera of the multi-line camera, and the multi-line camera control device (not shown) measures the time by previously setting the internal timer with the GPS time, and the camera start pulse. Each time is input at 10 msec (100 Hz), the line sensor is driven to take an image, and the GPS time Tg at the time of shooting, the measurement time Dt of the camera start pulse from the PPS signal, and the camera start pulse are input The internal timer time Dts (referred to as shooting time) is stored in correspondence.
[0032]
  That is, in the file 18, as shown in FIG. 6A, the image data Di for three lines acquired at a cycle of 100 Hz (every 10 msec) corresponds to the GPS time Tg, the measurement time Dt, and the shooting time Dts.EtStored in units of recorded records. The date may be added to this record. Of course, you may add as a header of the file per run instead of the record unit.
[0033]
Furthermore, it is preferable that an order number, an image data number, a lens focal length, a lens aperture, a photographing width, a route name, a measurer, illuminance, an ordering party, and the like are added to this file (camera). These data are collectively referred to simply as camera data.
[0034]
The distance data Ah is distance data Ah to the feature point measured by the laser scanner, and the laser control device reads various commands and distance data Ah to the laser scanner.
[0035]
This laser control device (not shown) synchronizes the internal timer with a GPS time in advance, synchronizes the internal timer every time a PPS signal is input, fires a laser every 20 Hz, and acquires data. At this time, the scan head of the laser scanner is rotated (one rotation at 50 msec).
[0036]
And GPS time Tg when the laser was received (if GPS time was input when the laser was fired, the GPS time, or if the GPS time was not input when the laser was fired, the previous GPS time) and measurement Laser internal timer time At (distance measurement time), laser measurement angle Ar (scan head rotation angle: Ar = Arp1, Arp2,... 360) and distance Ah to the point where the laser hits are obtained and stored. Yes.
[0037]
  That is, in the file 16, as shown in FIG. 6B, the measurement angle Ar of the laser, the measurement distance Ah, the GPS time Tg at the time of measurement, and the distance measurement time are acquired at a cycle of 20 Hz (every 50 msec). Let At correspondEtStored in units of recorded records. The date may be added to this record. Of course, you may add as a header of the file per run instead of the record unit.
[0038]
Furthermore, it is preferable that weather, temperature, etc. are added to this file (laser).
[0039]
Such data is simply referred to as laser data.
[0040]
Further, the position data Bi and the attitude data θi are measured by INS, and the INS internal timer (the time measurement accuracy is coarser than that of the camera and laser internal timer) is input each time the PPS signal is input. Synchronize with the internal timer, send measurement pulses at a periodic interval of 50 Hz, obtain high-accuracy position data Bi from the reference position from GPS and the position of the device by dead reckoning calculation, and obtain the inclination (attitude Data θi) and the time (time tag) by the internal timer until the measurement pulse is input from the PPS signal, GPS time data is recorded, and this is added to the position data Bi and the attitude data θi.
[0041]
That is, as shown in FIG. 6C, the file 17 stores the position data Bi, the attitude data θi, the GPS time Tg, and the measurement time INT every 20 msec.
[0042]
Such data is simply referred to as INS data.
[0043]
When a date, a route, a measurer, and a photographing time zone are input, the data capture processing unit 20 captures these related data from the files 16, 17, and 18.
[0044]
For example, in the case of 2000, June 1, 3 o'clock to 4 o'clock, the date and GPS time Tg are read, and the data between 2000, June 1, 3 o'clock to 4 o'clock is sequentially stored in the internal memory 30. (For camera data), memory 31 (for INS data), and memory 32 (for laser data).
[0045]
  Each time data is fetched, the data synchronization processing unit 21(INS data)SynchronizationForInterpolate, position and orientation data into laser data and camera data(INS data)Tie.
[0046]
The sensor calibration processing unit 22 determines (calibration data) the relative relationship between the position, orientation, and distance measurement according to the direction of the laser and the respective attachment positions.
[0047]
The camera calibration processing unit 33 determines the orientation and the relative relationship between the image data (calibration data) according to the orientation of the camera and the respective attachment positions.
[0048]
The unidirectional model generation processing unit 23 obtains a three-dimensional model in geographic coordinates for each laser by using the calibration data for the laser data of each laser scanner.
[0049]
The multi-directional model synthesis processing unit 24 generates a highly accurate model by synthesizing a three-dimensional model of the laser in each direction.
[0050]
The surface model generation processing unit 25 extracts a plane from a set of measurement points of the combined model generated by the multi-directional model combining processing unit 24 based on a predetermined condition, and converts it into a surface.
[0051]
The image mapping processing unit 26 assigns camera data corresponding to each surface of the surface model based on position and orientation data to generate a three-dimensional graphic image.
[0052]
The operation of the mobile mapping spatial information generating apparatus configured as described above will be described below.
[0053]
When each data in the specified range is stored in the memories 30, 31, and 32 by the data capture processing unit 20, the data synchronization processing unit 21 is first acquired in the camera data at a cycle of 100 Hz (every 10 msec). The GPS time Tg of the image data Di of one line camera is read.
[0054]
For example, in FIG. 6A, since the GPS time is output at a cycle of 10 Hz, ten pieces of image data Di (D11, D21, D31,... D101) are allocated at the GPS time Tg1. Since one multi-line camera is composed of three line cameras, when the image data of three line sensors is read, (D11, D21, D31,... D101) and (D12, D22, D32,... D102) and (D13, D23, D33,... D103) are read.
[0055]
In addition, the data synchronization processing unit 21 reads the laser data acquired at a cycle of 20 Hz (every 50 msec) in the laser data.
[0056]
  For example, in FIG. 6B, GPS time is output at a 10 Hz cycle, and laser data is output every 50 msec. Therefore, at GPS time Tg1, distance measurement times At1, At2ReadExpected.
[0057]
Further, in the INS data, the data synchronization processing unit 21 reads the GPS time Tg at a cycle of 50 Hz (every 20 msec), and for each GPS time Tg, the time tag INT, the position data Bi, and the attitude data θi are delayed. An accurate INS measurement time ITh is obtained by adding the time tm and the GPS time Tg and sequentially stored in the memory 31.
[0058]
For example, in (c) of FIG. 6, GPS time is output at a cycle of 10 Hz, and INS data is output every 20 msec. Therefore, at GPS time Tg1, time tags INT1... INT5, position data B1. ... θ5 is read.
[0059]
Then, the time tag INT1, the delay time tm, and the GPS time Tg1 are added to obtain each accurate INS measurement time ITh, and as shown in FIG. 7, it is associated with the position data Bi and the attitude data θi. Stored in the memory 30.
[0060]
  Next, the data synchronization process 21 matches the camera data CMi of the shooting time Dts with reference to the accurate shooting time Dts of the camera data stored in the memory 30, for example.likeThe INS data NMi is interpolated and stored in the memory 30 as shown in FIG. That is, INS data corresponding to the camera data is obtained by interpolation.
[0061]
  For example, as shown in FIG. 8, an accurate shooting time Dts1 of the camera data CM1 every 10 msec is read and synchronized with the camera data CM1 of this Dts1.likeThe measurement time ITh1h (interpolated measurement time ITh1h) of the INS data NMD1 is interpolated.
[0062]
At the same time, as shown in FIG. 9, based on the measurement time ITh1h, the posture data θ1 is interpolated to θ1h, and the position data B1 is interpolated to B1h.
[0063]
  Similarly, the camera data CM2 is synchronized with CM2 between the INS data NM1 and NM2.likeData (INS measurement time ITh2h, position data B2h, attitude data θ2h) is interpolated.
[0064]
  On the other hand, the interpolation of the INS data NMi corresponding to the laser data RMi is synchronized with the laser data RMi every 50 msec, for example, as shown in FIG.likeInterpolate INS data.
[0065]
  For example, as shown in FIG. 8, the accurate time At1 of the laser data every 50 msec is read and synchronized with this At1.likeThe measurement time ATs1h (interpolated measurement time ATs1h) of the INS data (NMD2 in FIG. 8) is interpolated.
[0066]
At the same time, as shown in FIG. 9, based on the measurement time ATs1h, the posture data θ1 is interpolated to θθ1h, and the position data B1 is interpolated to BB1h.
[0067]
  That is, the data synchronization processing unit 21 interpolates the captured camera data, INS data, and laser data.INS dataThey are linked based on time data.
[0068]
  Then, as shown in FIG. 10, it is synchronized with each laser data RM1, RM2,.To makeInterpolate the position and orientation data of the interpolated INS data.
[0069]
However, the laser and the INS and the camera and the INS are arranged at different positions on the flat plate 1. For this reason, the sensor calibration processing unit 22 performs processing described below.
[0070]
As shown in FIG. 11, the sensor calibration processing unit 22 measures a plurality of known reference points with a laser as shown in FIG.^bAnd R are defined as shown in Equation 1 (for example, once in February).
[0071]
[Expression 1]

Then, each time the laser data in the memory 32 is input, the unidirectional model generation processing unit 23 uses the calibration data to define a coordinate value (which is obtained from the distance, angle, position, and orientation) in the map coordinates, Generate a laser data model in spatial coordinates. This is done for the three directions of the laser.
[0072]
That is, the laser data RM (RMD) in FIG. 10B is converted into a coordinate group of measurement points as shown in FIG. 12 and stored. Then, the multidirectional model synthesis processing unit synthesizes these three-direction laser data in the geographic coordinate space to obtain the laser model shown in FIG.
[0073]
The surface model generation processing unit 25 divides the laser model to form a surface as shown in FIGS. 14 (a), 14 (b), and 14 (c).
[0074]
In the extraction of a planar patch for generating a surface (FIG. 14B), first, the entire range point (measurement point) is divided according to a certain rule. The divided one is called a range cell. In the division, pay attention to each range cell and determine whether the corresponding range point can return to the space plane within the preset threshold range (the variance value of the residual of regression is smaller than the threshold value) And can be returned to a single spatial plane).
[0075]
  Therefore, it returns to one space plane.ThisAny range cells that do not divide further and repeat this process.
[0076]
The range cell after division is the smallest unit in the process of extracting the planar patch.
[0077]
By such initialization, the range point is divided into several range cells ((b) in FIG. 14), and each range cell corresponds to a plane patch unique to the space by the least square method. Approximated by these planar patches.
[0078]
However, the approximation at this time has a large number of plane patches and the approximation efficiency is poor. In order to increase the approximation efficiency, the range cells corresponding to the same plane are merged to generate a polygon (FIG. 14 (c), FIG. 15). (A), (b)). In this process, the corresponding planes are approximately on the same plane. That is, it starts with the two range cells that are closest to the position of the perpendicular foot drawn from the origin of the same sensor coordinates to the plane.
[0079]
As a result of merging the range cells, the approximation efficiency is increased, but the approximation accuracy may be deteriorated.
[0080]
Therefore, whether the two range cells are merged, that is, the balance between the approximation efficiency and the approximation accuracy is determined by whether the MDL (Minimum Description Length) code length is small.
[0081]
Next, when the surface model is generated, the image mapping processing unit 26 reads each polygon of the surface model, assigns camera data corresponding to the polygon from the position and orientation data and the camera calibration data, and this position and orientation. A three-dimensional graphic image shown in FIG. 13B in which all image data corresponding to the data is assigned to the polygon is generated.
[0082]
In other words, since the three-dimensional graphic image is associated with the three-dimensional position and the data, for example, if this three-dimensional image data is stored in the car navigation defined by the map coordinates, the car navigation image When a predetermined point on the road is selected, a three-dimensional image of the building at that point can be obtained.
[0083]
A typical application example of the apparatus according to the present invention will be described below.
[0084]
(1) Urban disaster prevention
In cities, roads, buildings, and large and small facilities are densely populated, and there are some features that cannot be captured from above. However, the device of the present invention accurately determines the spatial arrangement and three-dimensional shape of urban structures. Therefore, it is possible to provide information that is extremely effective in building a simulation / evacuation simulation and damage prediction system during a disaster.
[0085]
(2) Facility management
In a large city, a vast lifeline network such as electricity, gas, and water and sewage is complicated. Although public companies have already built their own management systems, they are effective in acquiring new reference information and missing information for interoperating information. By using this information, it is possible to determine the location of the construction by looking at the positional relationship between the utility pole and the building on the three-dimensional map image, and to accurately grasp the position and deterioration status of the existing equipment.
[0086]
<Embodiment 2>
Since the three-dimensional graphic image obtained by the present invention has texture information of real images, it can be used for realistic landscape simulation using three-dimensional CG, walk-through moving around in a city, and cyber shopping.
[0087]
For example, taxis always go around the city, but if there are no passengers in the meantime, it will be a wasteful run, but you will know the latest state of the city. Even if you run with a passenger, the fee is only the distance traveled. Therefore, as shown in FIG. 16, for example, an image acquisition device including a multi-line camera, a GPS, and a camera control device used in the first embodiment is mounted on a taxi, and this image data is transmitted to the center.
[0088]
  The center is a point requested from a mobile phone or PCTheAcquire and send the image of this point to the mobile phone or personal computer for charging, and distribute the image acquisition history to the taxi company server, and pay for that later.
[0089]
The above-described image acquisition device 40 (preferably rental) is connected to the mobile phone 41, and when the center side instructs the mobile phone 41 to "acquire xxx buildings per xxx building", the image acquisition device 40 On the other hand, a photographing instruction is input and photographing is performed, and image data of each line sensor of the multi-line camera is sequentially stored in a memory (not shown) in association with the date and the GPS time. This data is generically called camera data.
[0090]
When the taxi driver finishes shooting, the camera data in the memory is transmitted to the center via the network using the mobile phone 41. Each time the camera data is transmitted from the taxi, the center image collecting device 42 stores the telephone number (used as taxi ID) of the mobile phone 41 and the camera data in a database (not shown) in association with each other.
[0091]
Further, when the center distribution / billing server 43 receives a distribution request for an area map of, for example, “Shibuya” from the user's mobile phone 44 or personal computer 45, the telephone number of the mobile phone 44 or the network ID of the personal computer is stored in the database 46. In addition to storing it, a map of Shibuya is searched from the database 47 and transmitted. This map is defined in geographical coordinates. When a desired point on this map is selected, the latitude and longitude of that point can be known.
[0092]
Then, the user selects a desired point on the map of Shibuya displayed on the screen of the mobile phone 44 or the personal computer 45, and inputs a desired time zone for the camera data.
[0093]
As a result, the latitude / longitude and time zone of the selected point are transmitted to the center via the network, and are received by the distribution / billing server 43.
[0094]
The distribution / billing server 43 determines whether or not the camera data having the latitude and longitude and the time zone from the user exists in the database 42, and if so, the camera data is sent to the mobile phone 44 or the personal computer 45 on the user side. Send.
[0095]
As a result, the mobile phone 44 or the personal computer 45 can determine the state of the building in front of the building at a desired point in Shibuya from an image. For example, whether the store you want to go to is crowded or whether there are multiple cars on the road. I understand.
[0096]
Then, the distribution / billing server 43 stores the taxi ID and the acquired image distribution date and time at which the camera data distributed to the user is acquired in the database 48, and distributes this to the taxi company server 49 at a later date. . The taxi company charges the center a predetermined amount from the number of distribution data of this server.
[0097]
In addition, the distribution / billing server 43 stores the ID or telephone number of the requested personal computer or mobile phone 44, the requested area, time, etc. in the database 46, and performs a predetermined billing based on these data. , Withdraw from the bank user account to the center account.
[0098]
Furthermore, when the camera data having the latitude / longitude and time zone from the user does not exist in the database 42, a call is made to the taxi mobile phone 41, and the message “Notify that the xx building in Shibuya is acquired in the xxx time zone”. .
[0099]
Since this taxi is equipped with a GPS, it always obtains the current position from the taxi side, finds the taxi that passes the xx building in Shibuya first in the xxx time zone, and reports it.
[0100]
Note that the user may be a news report or a police. Especially in the case of the police, the parking violation and the vehicle number of the street can be known immediately and it is useful for security.
[0101]
【The invention's effect】
  As described above, according to the present invention, camera data obtained while moving two multiline cameras, a DGPS receiver, three laser scanners, and an inertial navigation device fixed on a moving body, When generating 3D graphic images using laser data, INS data, and GPS time, INS dataIspositionas well asAttitude dataetcAre combined with the camera data and laser data, and then these laser data are combined in geographical coordinates, and image data associated with each distance value and angle is formed into a plane polygon obtained by dividing the three-dimensional laser data according to predetermined conditions. Search based on INS data of camera data. Then, the searched image data is assigned to a polygon to generate a three-dimensional graphic image.
[0102]
For this reason, city space etc. can be reproduced more faithfully. In addition, since the camera data and the laser data are associated with the position / orientation data and the time, it is possible to easily obtain a high-accuracy image from a desired direction at each point.
[0103]
In addition, since each camera data is acquired with the angle of the imaging direction being 45 degrees, even if there is a tree, it is possible to generate an image of a building excluding the tree. ing.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of a mobile mapping spatial information generation apparatus according to a first embodiment of the present invention.
FIG. 2 is a side view of a multi-line camera.
FIG. 3 is a front view of a multi-line camera.
FIG. 4 is a plan view showing attachment positions of a multiline camera and lasers, INS, and DGPS on a gantry.
FIG. 5 is an explanatory diagram for explaining the direction of imaging and distance measurement of the image acquisition device.
FIG. 6 is an explanatory diagram for explaining acquired camera data, laser data, and INS data.
FIG. 7 is an explanatory diagram of INS data set at an accurate time.
FIG. 8 is a timing chart for synchronization and interpolation of camera data, laser data, and INS data.
FIG. 9 is an explanatory diagram for explaining synchronized and interpolated INS data.
FIG. 10 is an explanatory diagram for explaining camera data and laser data that are synchronized and interpolated and associated with INS data;
FIG. 11 is an explanatory diagram for explaining calibration processing;
FIG. 12 is an explanatory diagram of laser data reflecting a calibration result.
FIG. 13 is an explanatory diagram illustrating distance measurement data and a surface image.
FIG. 14 is an explanatory diagram for explaining a synthesis model and division based on laser data.
FIG. 15 is an explanatory diagram illustrating an extracted image according to the present embodiment.
FIG. 16 is a schematic configuration diagram of a mobile mapping acquisition data distribution system according to the second embodiment.
[Explanation of symbols]
1 frame
2 Laser scanner
3 Multi-line camera
20 Data acquisition processing section
21 Data synchronization processor
22 Sensor calibration processor
23 Unidirectional model generation processing unit
24 Multi-directional model synthesis processing unit
25 Surface model generation processing unit
26 Image mapping processor

Claims (5)

Two multi-line cameras, which are a combination of three fisheye lenses and line sensors with a predetermined imaging angle horizontally and a DGPS receiver, emit lasers in different directions and receive them 3 Mobile mapping that generates a three-dimensional graphic image using a camera scanner, laser data, INS data consisting of position and orientation, and GPS time obtained by moving a fixed laser scanner and an inertial navigation device on a moving body Spatial information generation device,
Means for linking the INS data having the GPS time to the camera data and laser data having the GPS time;
Means for reading the laser data of each laser scanner and defining the distance value to the measurement point indicated by the respective laser data and the angle of the laser direction with respect to the measurement point in the respective three-dimensional geographic coordinates based on the INS data When,
Means for synthesizing the unidirectional model of each laser defined in the respective three-dimensional geographic coordinates on the three-dimensional coordinates;
Means for dividing the composite model into planes according to a predetermined condition and polygonizing the divided area;
Means for retrieving image data associated with each distance value and angle of the polygon based on the INS data of the camera data, and assigning the retrieved image data to the polygon to generate the three-dimensional graphic image. A mobile mapping spatial information generating device characterized by the above. The camera data is a shooting time at a first predetermined interval, the INS data is a measurement time at a second predetermined interval (first predetermined interval <second predetermined interval), and the laser data is a third predetermined interval. Each obtained at a measurement time of (second predetermined interval <third predetermined interval),
The linking means is
When the camera data at the shooting time is acquired, the measurement time of the INS data is interpolated so as to be synchronized with the shooting time, and the INS data and the camera data are linked.
2. The mobile mapping according to claim 1, wherein when the laser data is acquired, the measurement time of the INS data is interpolated so as to be synchronized with the measurement time, and the INS data and the laser data are linked. Spatial information generator. The measurement time of the INS data is
When the INS data is acquired at the second predetermined interval, a means for obtaining an accurate measurement time of the acquired INS data from a measurement time and a delay time that are not more than a predetermined accuracy synchronized with the GPS time is activated. The mobile mapping spatial information generating apparatus according to claim 2, wherein the measurement time is obtained. The divided area is
The measurement point is a range point, and each of the range points scattered in the composite model is divided based on the predetermined condition, which is defined as a range cell, and the range point in the range cell is within a preset threshold range. 4. The mobile mapping spatial information generating device according to claim 1, wherein when these range points return to a spatial plane, the plane is extracted and used as the divided area.   5. The mobile mapping spatial information generating apparatus according to claim 4, wherein when the range point does not return to the spatial plane, the process until the extraction of the plane is repeated for a predetermined period to extract the minimum plane.

Images