JP6699323B2 - Three-dimensional measuring device and three-dimensional measuring method for train equipment - Google Patents

Three-dimensional measuring device and three-dimensional measuring method for train equipment Download PDF

Info

Publication number
JP6699323B2
JP6699323B2 JP2016087671A JP2016087671A JP6699323B2 JP 6699323 B2 JP6699323 B2 JP 6699323B2 JP 2016087671 A JP2016087671 A JP 2016087671A JP 2016087671 A JP2016087671 A JP 2016087671A JP 6699323 B2 JP6699323 B2 JP 6699323B2
Authority
JP
Japan
Prior art keywords
train
dimensional
equipment
image
optical flow
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2016087671A
Other languages
Japanese (ja)
Other versions
JP2017196948A (en
Inventor
寛修 深井
寛修 深井
庭川　誠
誠 庭川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Meidensha Corp
Original Assignee
Meidensha Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Meidensha Corp filed Critical Meidensha Corp
Priority to JP2016087671A priority Critical patent/JP6699323B2/en
Publication of JP2017196948A publication Critical patent/JP2017196948A/en
Application granted granted Critical
Publication of JP6699323B2 publication Critical patent/JP6699323B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Measurement Of Optical Distance (AREA)
  • Studio Devices (AREA)
  • Electric Propulsion And Braking For Vehicles (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Description

本発明は、三次元計測装置及び三次元計測方法に関し、とくに一台のカメラで走行中の電車の車両前方又は車両後方を撮像した画像に基づいて電車設備の三次元形状の計測を行う電車設備の三次元計測装置及び三次元計測方法に関する。   The present invention relates to a three-dimensional measuring device and a three-dimensional measuring method, and in particular, a train facility that measures a three-dimensional shape of a train facility based on an image obtained by capturing an image of a front side or a rear side of a train running with one camera 3D measuring device and 3D measuring method.

一般的にカメラで三次元の計測を行うには、計測する対象物の三次元形状モデルが既知であるか、一つの対象物を複数個所から撮影して対象物の三次元座標を求める三角測量を行うことが必要となる。しかしながら電車設備の三次元計測を行うことを想定した場合、電車設備はその数が非常に多く、全ての設備について三次元形状モデルの設定を行うことは現実的ではない。また一つの対象物を複数個所から撮影する場合、カメラを複数台用意して撮影するステレオカメラシステムがよく用いられるが、設置個所によってはカメラを複数台設置できない場合や、設置できても精度を得難い近接した位置にしか設置できない場合があるなどの問題があった。   Generally, in order to perform three-dimensional measurement with a camera, the three-dimensional shape model of the object to be measured is known, or one object is photographed from multiple locations to obtain the three-dimensional coordinates of the object. Will be required. However, when it is assumed that three-dimensional measurement of train equipment is performed, the number of train equipment is very large, and it is not realistic to set a three-dimensional shape model for all equipment. Also, when shooting one object from multiple locations, a stereo camera system that prepares multiple cameras and shoots is often used, but depending on the installation location, it is not possible to install multiple cameras, or even if it can be installed, accuracy is There was a problem that it could only be installed in close proximity, which is difficult to obtain.

このような問題に対し、下記非特許文献１には、一台のカメラによって取得した画像から三次元座標の計測を行う手法として、オプティカルフローや、三次元運動復元等を利用したものが記載されている。   In order to solve such a problem, Non-Patent Document 1 below describes a method that uses optical flow, three-dimensional motion restoration, or the like, as a method for measuring three-dimensional coordinates from an image acquired by one camera. ing.

また、下記特許文献１には、車両に搭載されたカメラによって車両の移動中に撮影された複数の撮影画像から、ＳｆＭ（Structure from Motion）の技術に基づいて、カメラの三次元位置・姿勢や、被写体の三次元位置を求める画像処理装置が開示されている。   Further, in Patent Document 1 below, based on a SfM (Structure from Motion) technique, a three-dimensional position/orientation of a camera is obtained from a plurality of captured images captured by a camera mounted on the vehicle while the vehicle is moving. An image processing apparatus for obtaining a three-dimensional position of a subject is disclosed.

また、下記特許文献２には移動ステレオ方式により物体までの距離を計測する画像処理装置が記載され、下記特許文献３には移動ステレオ方式を用いた三次元環境地図作成を行う三次元形状生成装置が記載されている。   Further, Patent Document 2 below describes an image processing device that measures a distance to an object by a moving stereo system, and Patent Document 3 below describes a three-dimensional shape generation device that creates a three-dimensional environment map using the moving stereo system. Is listed.

国際公開第WO2014/192061号パンフレットInternational Publication No. WO2014/192061 Pamphlet 特許第５４３９８７６号公報Japanese Patent No. 5439876 特許第４５１１１４７号公報Japanese Patent No. 4511147

「知識の森」２群−２編−４章 動画解析、電子情報通信学会、２０１３年、p.1/(18)-18/(18)"Forest of Knowledge" 2 groups-2 volumes-4 chapters Video Analysis, The Institute of Electronics, Information and Communication Engineers, 2013, p.1/(18)-18/(18)

しかしながら、特許文献１〜３に記載されたものは、計測された三次元特徴が何かという判定を行うものではなかった。また、三次元復元の際に回転並進の６自由度のパラメータ全てを求める必要があるという問題もあった。   However, the ones described in Patent Documents 1 to 3 do not judge what the measured three-dimensional characteristics are. In addition, there is also a problem that it is necessary to obtain all 6-degree-of-freedom parameters of rotational translation during three-dimensional reconstruction.

また引用文献２，３に共通した問題としてオプティカルフロー（もしくは特徴量）を求めるために特徴点を算出する必要があるが、実問題を扱う際の最適な特徴点抽出法が明記されていないという問題があった。   In addition, it is necessary to calculate feature points in order to obtain an optical flow (or feature amount) as a problem common to the cited documents 2 and 3, but it is said that the optimum feature point extraction method when dealing with real problems is not specified. There was a problem.

このようなことから本発明は、電車設備に対応する特徴点を好適に抽出し、電車設備の判定を行うことを可能とした電車設備の三次元計測装置及び三次元計測方法を提供することを目的とする。   From the above, the present invention provides a three-dimensional measuring device and a three-dimensional measuring method for a train facility, which is capable of suitably extracting feature points corresponding to the train facility and determining the train facility. To aim.

上記の課題を解決するための第１の発明に係る電車設備の三次元計測装置は、
電車車両の前方又は後方を撮像する撮像装置と、当該撮像装置によって撮像した画像に基づいて前記電車車両の周囲にある設備を検出する画像処理装置とを備える電車設備の三次元計測装置であって、
前記画像処理装置が、
前記撮像装置によって撮像された画像データを入力する画像入力部と、
前記撮像装置によって画像を撮像した位置に対応する前記電車車両の走行速度を取得する車両速度データ取得部と、
前記画像データから特徴点を検出する特徴点検出部と、
前記画像データの時系列的に連続する二つの画像から検出した前記特徴点に基づいてオプティカルフローを算出するオプティカルフロー算出部と、
前記電車車両の走行速度、前記オプティカルフロー、及びカメラパラメータに基づいて前記電車車両の移動量及び回転量を求め、当該移動量及び回転量に基づいて前記特徴点の三次元座標を求める三次元復元部と、
前記特徴点の三次元座標に基づいて前記設備の有無を判定する設備判断部と
を備える
ことを特徴とする。 A three-dimensional measuring device for train equipment according to a first aspect of the present invention for solving the above-mentioned problems,
A three-dimensional measuring device for train equipment, comprising: an imaging device for imaging the front or rear of a train car; and an image processing device for detecting equipment around the train car based on an image taken by the imaging device. ,
The image processing device,
An image input unit for inputting image data captured by the image capturing device,
A vehicle speed data acquisition unit that acquires a traveling speed of the electric train vehicle corresponding to a position where an image is captured by the imaging device;
A feature point detection unit for detecting feature points from the image data,
An optical flow calculation unit that calculates an optical flow based on the characteristic points detected from two images that are consecutive in time series of the image data,
Three-dimensional reconstruction that obtains the movement amount and rotation amount of the train vehicle based on the traveling speed of the train vehicle, the optical flow, and camera parameters, and obtains the three-dimensional coordinates of the feature points based on the movement amount and rotation amount. Department,
An equipment determination unit that determines the presence or absence of the equipment based on the three-dimensional coordinates of the feature points.

また、第２の発明に係る電車設備の三次元計測装置は、
前記特徴点検出部が、特徴点として前記画像データからコーナー点を検出する
ことを特徴とする。 A three-dimensional measuring device for electric train equipment according to the second invention is
The feature point detection unit detects a corner point from the image data as a feature point.

また、第３の発明に係る電車設備の三次元計測装置は、
前記三次元復元部が、鉛直方向及びピッチング角を除く前記電車車両の移動量及び回転量を求める
ことを特徴とする。 In addition, a three-dimensional measuring device for train equipment according to the third invention,
It is characterized in that the three-dimensional reconstruction unit obtains a movement amount and a rotation amount of the train car excluding the vertical direction and the pitching angle.

また、第４の発明に係る電車設備の三次元計測装置は、
前記設備判断部が、前記特徴点が近接して所定値以上存在する領域を前記設備と判断し、前記特徴点が所定値未満の領域をノイズと判断する
ことを特徴とする。 A three-dimensional measuring device for train equipment according to a fourth aspect of the present invention,
The equipment determining unit may determine an area where the feature points are close to each other and have a predetermined value or more as the equipment, and an area where the feature points are less than the predetermined value as noise.

また、第５の発明に係る電車設備の三次元計測方法は、
撮像装置によって電車車両の前方又は後方を撮像し、前記撮像装置によって撮像した画像に基づいて前記電車車両の周囲にある設備を検出する三次元計測方法であって、
前記撮像装置によって撮像された画像データを入力する画像入力工程と、
前記撮像装置によって画像を撮像した位置に対応する前記電車車両の走行速度を取得する車両速度データ取得工程と、
前記画像データから特徴点を検出する特徴点検出工程と、
前記画像データの時系列的に連続する二つの画像から検出した前記特徴点に基づいてオプティカルフローを算出するオプティカルフロー算出工程と、
前記電車車両の走行速度、前記オプティカルフロー、及びカメラパラメータに基づいて前記電車車両の移動量及び回転量を求め、当該移動量及び回転量に基づいて前記特徴点の三次元座標を求める三次元復元工程と、
前記特徴点の三次元座標に基づいて前記設備の有無を判定する設備判定工程と
を有することを特徴とする。 A three-dimensional measuring method for train equipment according to the fifth aspect of the invention is
A three-dimensional measuring method for imaging the front or rear of a train car with an imaging device, and detecting equipment around the train car based on the image captured by the imaging device,
An image input step of inputting image data picked up by the image pickup device;
A vehicle speed data acquisition step of acquiring a traveling speed of the train car corresponding to a position where an image is captured by the imaging device;
A feature point detecting step of detecting feature points from the image data,
An optical flow calculation step of calculating an optical flow based on the feature points detected from two images that are consecutive in time series of the image data,
Three-dimensional reconstruction that obtains the movement amount and rotation amount of the train vehicle based on the traveling speed of the train vehicle, the optical flow, and camera parameters, and obtains the three-dimensional coordinates of the feature points based on the movement amount and rotation amount. Process,
And a facility determining step of determining the presence or absence of the facility based on the three-dimensional coordinates of the feature points.

また、第６の発明に係る電車設備の三次元計測方法は、
前記特徴点検出工程が、前記特徴点として前記画像データからコーナー点を検出する
ことを特徴とする。 A three-dimensional measuring method for train equipment according to a sixth aspect is
The feature point detecting step detects a corner point from the image data as the feature point.

また、第７の発明に係る電車設備の三次元計測方法は、
前記三次元復元工程が、鉛直方向及びピッチング角を除く前記電車車両の移動量及び回転量を求める
ことを特徴とする。 A three-dimensional measuring method for train equipment according to the seventh invention is
It is characterized in that the three-dimensional restoration step obtains a moving amount and a rotating amount of the train car excluding the vertical direction and the pitching angle.

また、第８の発明に係る電車設備の三次元計測方法は、
前記設備判定工程が、前記特徴点が近接して所定値以上存在する領域を前記設備と判断し、前記特徴点が所定値未満の領域をノイズと判断する
ことを特徴とする。 A three-dimensional measuring method for train equipment according to the eighth invention is
In the facility determining step, a region in which the feature points are close to each other and a predetermined value or more is determined to be the facility, and a region in which the feature points are less than the predetermined value is determined to be noise.

本発明に係る電車設備の三次元計測装置及び三次元計測方法によれば、電車設備に対応する特徴点を好適に抽出し、電車設備の判定を行うことが可能になる。   According to the three-dimensional measuring apparatus and the three-dimensional measuring method for train equipment according to the present invention, it becomes possible to appropriately extract the feature points corresponding to the train equipment and determine the train equipment.

本発明の実施例に係る電車設備の三次元計測装置の設置例を示す装置構成図である。It is an apparatus block diagram which shows the example of installation of the three-dimensional measuring apparatus of the train equipment which concerns on the Example of this invention. 図１に示す画像処理装置の構成を示すブロック図である。FIG. 2 is a block diagram showing the configuration of the image processing device shown in FIG. 1. 図２に示す画像入力部により取得した画像データの一例を示す説明図である。FIG. 3 is an explanatory diagram showing an example of image data acquired by the image input unit shown in FIG. 2. 図２に示す特徴点検出部による処理の一例を示す説明図である。FIG. 4 is an explanatory diagram showing an example of processing by a feature point detection unit shown in FIG. 2. 図２に示すオプティカルフロー算出部による処理の一例を示す説明図である。FIG. 3 is an explanatory diagram showing an example of processing by an optical flow calculation unit shown in FIG. 2. 図２に示す設備判断部による処理の一例を示す説明図である。It is explanatory drawing which shows an example of a process by the equipment determination part shown in FIG. 図１に示す電車設備の三次元計測装置による処理の流れを示すフローチャートである。It is a flowchart which shows the flow of a process by the three-dimensional measuring device of the train equipment shown in FIG.

以下、図面を参照しつつ本発明の実施の形態について説明する。   Hereinafter, embodiments of the present invention will be described with reference to the drawings.

図１から図５を用いて本発明に係る電車設備の三次元形状計測装置の実施例について詳細に説明する。   An embodiment of a three-dimensional shape measuring apparatus for electric train equipment according to the present invention will be described in detail with reference to FIGS. 1 to 5.

図１に示すように、本実施例において電車車両１の屋根上には一台の車上カメラ１１（撮像装置）が設置され、この車上カメラ１１が電車車両１の内部に設置された画像収録及び解析コンピュータ（以下、画像処理装置という）１２に接続されている。   As shown in FIG. 1, in this embodiment, an on-vehicle camera 11 (imaging device) is installed on the roof of the train vehicle 1, and the on-board camera 11 is installed inside the train vehicle 1. It is connected to a recording and analysis computer (hereinafter referred to as an image processing device) 12.

車上カメラ１１は、エリアカメラであり、電車車両１の前方又は後方を撮影可能に設置されている。
画像処理装置１２は、車上カメラ１１によって撮像した画像に基づき電車車両１の走行領域の周辺にある電車設備の三次元形状の計測を行うものである。すなわち、電車設備の形状モデルが未知の場合、パターンマッチングなどで画像中から電車設備を検出することは困難である。そこで、本実施例では、一台の車上カメラ１１で電車設備を撮像した動画像の複数の画像（フレーム）から抽出した特徴を用いて、電車設備の計測を行う。 The on-board camera 11 is an area camera, and is installed so as to be able to image the front or the rear of the train car 1.
The image processing device 12 measures the three-dimensional shape of the train equipment around the traveling area of the train vehicle 1 based on the image captured by the on-vehicle camera 11. That is, when the shape model of the train equipment is unknown, it is difficult to detect the train equipment from the image by pattern matching or the like. Therefore, in the present embodiment, the train equipment is measured using the features extracted from a plurality of images (frames) of the moving image obtained by capturing the train equipment with one on-vehicle camera 11.

具体的に説明すると、図２に示すように本実施例において画像処理装置１２は、画像入力部１２ａ、車両速度データ取得部１２ｂ、特徴点検出部１２ｃ、オプティカルフロー算出部１２ｄ、三次元復元部１２ｅ、設備判断部１２ｆ、及び記憶部１２ｇを備えている。   More specifically, as shown in FIG. 2, in the present embodiment, the image processing device 12 includes an image input unit 12a, a vehicle speed data acquisition unit 12b, a feature point detection unit 12c, an optical flow calculation unit 12d, and a three-dimensional restoration unit. 12e, the equipment determination part 12f, and the memory|storage part 12g are provided.

画像入力部１２ａは、車上カメラ１１によって取得した画像データＩｔ１，Ｉｔ２，…（図３（ａ），（ｂ）参照）を入力し、記憶部１２ｇに保管する。
車両速度データ取得部１２ｂは、図示しない自動列車制御装置（ＡＴＣ）からの信号やオドメトリ情報などから車上カメラ１１によって撮像を行った位置に対応した電車車両１の速度データを取得し、記憶部１２ｇに保管する。 The image input unit 12a inputs the image data It1, It2,... (See FIGS. 3A and 3B) acquired by the on-vehicle camera 11 and stores it in the storage unit 12g.
The vehicle speed data acquisition unit 12b acquires speed data of the train car 1 corresponding to the position imaged by the on-board camera 11 from a signal from an automatic train control device (ATC) (not shown) or odometry information, and the storage unit Store at 12g.

特徴点検出部１２ｃは、車上カメラ１１により取得した画像データＩｔ１，Ｉｔ２，…から図４に＋で示すようなコーナー点（特徴点）Ｃをそれぞれ検出し、検出した特徴点Ｃの情報を特徴点データとして記憶部１２ｇに保管する。
ここで、特徴点Ｃの検出には固有値ベースのコーナー検出手法に代表される簡易で高速な手法を用いる。これにより、等間隔に特徴点を設ける手法に比較して電車設備に好適に特徴点を割り振ることができる。また、電車設備は通常コーナー点検出で特徴点Ｃが複数出るという特徴があるのも採用の理由である。 The feature point detection unit 12c detects corner points (feature points) C as indicated by + in FIG. 4 from the image data It1, It2,... Acquired by the on-vehicle camera 11 and outputs information on the detected feature points C. It is stored in the storage unit 12g as feature point data.
Here, a simple and high-speed method typified by an eigenvalue-based corner detection method is used to detect the characteristic point C. As a result, the feature points can be suitably allocated to the train equipment as compared with the method of providing the feature points at equal intervals. Another reason for adoption is that train equipment usually has a feature that a plurality of feature points C appear when corner points are detected.

オプティカルフロー算出部１２ｄは、画像データと特徴点データとに基づいて、車上カメラ１１により取得した画像データＩｔ１，Ｉｔ２，…の時系列的に連続した二枚の画像（例えば、図４に示す時刻ｔ１における画像Ｉｔ１と、時刻ｔ２における画像Ｉｔ２）の特徴点データから、図５に矢印で示すようにオプティカルフローＦを算出し、オプティカルフローデータとして記憶部１２ｇに保管する。   The optical flow calculation unit 12d, based on the image data and the feature point data, two image data It1, It2,... Acquired by the on-vehicle camera 11 that are continuous in time series (for example, shown in FIG. 4). An optical flow F is calculated from the feature point data of the image It1 at the time t1 and the image It2 at the time t2) as indicated by an arrow in FIG. 5, and is stored in the storage unit 12g as the optical flow data.

ここで、電車設備は背景と比較すると電車車両１に近いため、時系列的に連続する画像間での写りの変化が大きくなる。そのため、前後の画像で同一点の移動量を表すオプティカルフローＦを算出し、そのフロー情報を用いて電車設備と背景の分離を行う。   Here, since the train equipment is closer to the train car 1 compared to the background, the change in the reflection between images that are continuous in time series becomes large. Therefore, the optical flow F representing the amount of movement of the same point is calculated in the front and rear images, and the train equipment and the background are separated using the flow information.

なお、オプティカルフローＦの算出手法としては、Lucas-Kanade法に代表される小領域ごとのフローの対応探索を独立に行う手法を採用する。特徴点ベースの手法には、オプティカルフローではなく、リッチな特徴量を持つ特徴抽出方法を用いて、特徴量を用いた特徴点マッチングを行う手法もあるが、オプティカルフローに比べて計算コストが高く、また特徴点の数によって結果が不安定となることからも、オプティカルフローを用いるメリットがある。   As a method of calculating the optical flow F, a method of independently performing flow correspondence search for each small region, which is represented by the Lucas-Kanade method, is adopted. There is also a feature point-based method that performs feature point matching using a feature amount using a feature extraction method that has a rich feature amount instead of optical flow, but the calculation cost is higher than optical flow. Also, there is an advantage in using the optical flow because the result becomes unstable depending on the number of feature points.

三次元復元部１２ｅは、オプティカルフローデータ、車両速度、及び事前に求めたカメラパラメータに基づき、電車車両１の移動量及び回転量を求め、その値をもとにフロー情報を持つ各特徴点Ｃを三次元復元し、復元した三次元座標データと、電車車両１の移動量及び回転量を表す回転並進データを記憶部１２ｇに保管する。   The three-dimensional restoration unit 12e obtains the movement amount and the rotation amount of the train vehicle 1 based on the optical flow data, the vehicle speed, and the camera parameters obtained in advance, and based on the values, each characteristic point C having flow information. Is three-dimensionally restored, and the restored three-dimensional coordinate data and the rotational translation data indicating the movement amount and the rotation amount of the train car 1 are stored in the storage unit 12g.

すなわち、フロー情報、電車の移動速度、及びカメラパラメータを用いて三角測量を行い、各特徴点Ｃのカメラの焦点を原点とする三次元座標を計算する。なお、電車車両１は略水平とみなせるレール３上を移動するため、座標系をレール３上面と水平な電車車両１の前方（または後方）、レール３上面と水平な車両横方向（枕木方向）、電車車両１の鉛直方向にとった場合、車両鉛直方向の変化量は０、ピッチング角（車両鉛直方向周りの回転角）の変化は０とみなせる。そのため、通常であれば三次元復元は回転３自由度、並進３自由度の計６自由度の探索を行う必要があるが、本実施例では４自由度の探索で済むため安定した三次元計測が可能となる。   That is, triangulation is performed using the flow information, the moving speed of the train, and the camera parameters, and the three-dimensional coordinates with the focus of the camera of each feature point C as the origin are calculated. Since the train car 1 moves on the rail 3 that can be regarded as substantially horizontal, the coordinate system is in front of (or behind) the train car 1 that is horizontal with the upper surface of the rail 3 and in the vehicle lateral direction (sleeper direction) that is horizontal with the top surface of the rail 3. When taken in the vertical direction of the train vehicle 1, the change amount in the vehicle vertical direction can be regarded as 0, and the change in pitching angle (rotation angle around the vehicle vertical direction) can be regarded as 0. Therefore, normally, for three-dimensional reconstruction, it is necessary to search for a total of six degrees of freedom including three degrees of freedom of rotation and three degrees of translation. However, in the present embodiment, a search of four degrees of freedom is sufficient, so stable three-dimensional measurement is possible. Is possible.

設備判断部１２ｆは、三次元座標データを、各特徴点Ｃの座標値を基準にグループ分けし、ある程度近接して特徴点Ｃが存在する領域（例えば、図６に網掛けで示す領域Ａ１，Ａ２。以下、点群という）を電車設備として判定し、設備判定データを記憶部１２ｇに保管する。   The equipment determination unit 12f divides the three-dimensional coordinate data into groups based on the coordinate values of the respective characteristic points C, and the areas where the characteristic points C are close to each other to some extent (for example, the area A1 shown by hatching in FIG. 6). A2. Hereinafter, referred to as a point cloud) is determined as a train facility, and facility determination data is stored in the storage unit 12g.

すなわち、電車設備は複数の特徴点Ｃを有すると考えられるため、特徴点Ｃが近接して所定値以上存在した場合は電車設備と判断し、近接して存在する特徴点Ｃが所定値未満である場合はノイズと判断する。三次元復元後に各点群を特徴点Ｃの座標値に基づいてグループ分けすることで、座標値が近接して複数存在する点群を電車設備として判断する。電車設備としては、例えば図１に示す電柱２のほか、支障物、ビームなどがある。
図６に示す例では、電柱２に対応する二つの領域が電車設備と判断されている。 That is, since the train equipment is considered to have a plurality of feature points C, when the feature points C are close to each other and a predetermined value or more exists, it is determined that the train equipment is a train equipment, and the feature points C that are close to each other are less than the predetermined value. If there is, it is judged as noise. After the three-dimensional reconstruction, each point group is divided into groups based on the coordinate value of the characteristic point C, so that the point group having a plurality of coordinate values close to each other is determined as train equipment. As the train equipment, for example, in addition to the electric pole 2 shown in FIG. 1, there are obstacles, beams, and the like.
In the example shown in FIG. 6, two areas corresponding to the telephone pole 2 are determined to be train equipment.

記憶部１２ｇは、カメラパラメータ、画像データ、速度データ、特徴点座標データ、オプティカルフローデータ、三次元座標データ、回転並進データ、及び設備判定データを保管する。   The storage unit 12g stores camera parameters, image data, speed data, feature point coordinate data, optical flow data, three-dimensional coordinate data, rotational translation data, and equipment determination data.

以下、図７を用いて本実施例に係る三次元計測装置による処理の流れについて簡単に説明する。   The flow of processing by the three-dimensional measuring apparatus according to this embodiment will be briefly described below with reference to FIG. 7.

本実施例では、まず、電車車両１の屋根上に設置した車上カメラ１１により、電車車両１の運行中に電車車両１の前方（または後方）の動画像を撮像して画像入力部１２ａにより画像データＩｔ１，Ｉｔ２，…を取得する（ステップＳ１）。続いて車両速度データ取得部１２ｂにより、車上カメラ１１によって動画像を取得したタイミングの車両速度をＡＴＣ信号又はオドメトリ等のデータから取得する（ステップＳ２）。   In the present embodiment, first, the on-board camera 11 installed on the roof of the train vehicle 1 captures a moving image in front of (or behind) the train vehicle 1 while the train vehicle 1 is in operation, and the image input unit 12a. Image data It1, It2,... Are acquired (step S1). Then, the vehicle speed data acquisition unit 12b acquires the vehicle speed at the timing when the moving image is acquired by the on-vehicle camera 11 from the data such as the ATC signal or the odometry (step S2).

その後、特徴点検出部１２ｃにより、取得した各画像Ｉｔ１，Ｉｔ２，…について特徴点（コーナー）Ｃを検出し（ステップＳ３）、オプティカルフロー算出部１２ｄにより、取得した各画像Ｉｔ１，Ｉｔ２，…について時系列的に連続する次の画像との間のオプティカルフローＦを算出する（ステップＳ４）。   Thereafter, the feature point detection unit 12c detects a feature point (corner) C for each of the acquired images It1, It2,... (Step S3), and the optical flow calculation unit 12d detects each of the acquired images It1, It2,. An optical flow F with the next image that is continuous in time series is calculated (step S4).

続いて、三次元復元部１２ｅにより、オプティカルフローの存在する各特徴点Ｃのフロー情報と車両速度、カメラパラメータを用いて車両の移動量、回転量を求め、その値をもとに各点を三次元復元し（ステップＳ５）、最後に、設備判断部１２ｆにより、三次元復元した点群について座標値をもとにグループ分けを行って電車設備かその他の設備かを判定する（ステップＳ６）。   Subsequently, the three-dimensional reconstruction unit 12e obtains the amount of movement and the amount of rotation of the vehicle using the flow information of each feature point C in which the optical flow exists, the vehicle speed, and the camera parameter, and the points are determined based on the values. Three-dimensional reconstruction is performed (step S5), and finally, the equipment determination unit 12f performs grouping on the three-dimensionally restored point group based on the coordinate values to determine whether it is a train facility or another facility (step S6). .

このように構成される本実施例に係る電車設備の三次元計測装置によれば、電車設備の形状モデルなどが無い場合であっても電車設備がコーナー点を多く含むこと及び電車設備は近接してコーナー点（特徴点）Ｃを含むことを利用して電車設備の計測を行うことが可能となる。また、特徴量マッチングではなくオプティカルフローＦを利用することで二枚の画像の対応付けを高速かつ高精度に行うことができる。また、電車車両１の特徴を考慮することにより探索パラメータの少ない三次元復元を行うことができる。   According to the three-dimensional measuring device for a train facility according to the present embodiment configured as described above, the train facility includes many corner points and the train facility is close to each other even when there is no shape model of the train facility. By including the corner point (feature point) C, it is possible to measure the train equipment. Further, by using the optical flow F instead of the feature amount matching, the two images can be associated with each other at high speed and with high accuracy. Further, considering the characteristics of the train car 1, it is possible to perform three-dimensional reconstruction with few search parameters.

本発明は、電車設備の三次元計測装置及び三次元計測方法に適用することができる。   INDUSTRIAL APPLICABILITY The present invention can be applied to a three-dimensional measuring device and a three-dimensional measuring method for train equipment.

１ 電車車両
２ 電柱（電車設備）
３ レール
１１ 車上カメラ
１２ 画像処理装置
１２ａ 画像入力部
１２ｂ 車両速度データ取得部
１２ｃ 特徴点検出部
１２ｄ オプティカルフロー算出部
１２ｅ 三次元復元部
１２ｆ 設備判断部
１２ｇ 記憶部
Ｃ 特徴点
Ｆ オプティカルフロー 1 Train car 2 Utility pole (train equipment)
3 rails 11 on-board camera 12 image processing device 12a image input unit 12b vehicle speed data acquisition unit 12c feature point detection unit 12d optical flow calculation unit 12e three-dimensional restoration unit 12f equipment determination unit 12g storage unit C feature point F optical flow

Claims (10)

電車車両の前方又は後方を撮像する撮像装置と、当該撮像装置によって撮像した画像に基づいて前記電車車両の周囲にある設備を検出する画像処理装置とを備える電車設備の三次元計測装置であって、
前記画像処理装置が、
前記撮像装置によって撮像された画像データを入力する画像入力部と、
前記撮像装置によって画像を撮像した位置に対応する前記電車車両の走行速度を取得する車両速度データ取得部と、
前記画像データから特徴点を検出する特徴点検出部と、
前記画像データの時系列的に連続する二つの画像から検出した前記特徴点に基づいてオプティカルフローを算出するオプティカルフロー算出部と、
前記電車車両の走行速度、前記オプティカルフロー、及びカメラパラメータに基づいて前記電車車両の移動量及び回転量を求め、当該移動量及び回転量に基づいて前記特徴点の三次元座標を求める三次元復元部と、
前記特徴点の三次元座標に基づいて前記設備の有無を判定する設備判断部と
を備える
ことを特徴とする電車設備の三次元計測装置。 A three-dimensional measuring device for train equipment, comprising: an imaging device for imaging the front or rear of a train car; and an image processing device for detecting equipment around the train car based on an image taken by the imaging device. ,
The image processing device,
An image input unit for inputting image data captured by the image capturing device,
A vehicle speed data acquisition unit that acquires a traveling speed of the electric train vehicle corresponding to a position where an image is captured by the imaging device;
A feature point detection unit for detecting feature points from the image data,
An optical flow calculation unit that calculates an optical flow based on the characteristic points detected from two images that are consecutive in time series of the image data,
Three-dimensional reconstruction that obtains the movement amount and rotation amount of the train vehicle based on the traveling speed of the train vehicle, the optical flow, and camera parameters, and obtains the three-dimensional coordinates of the feature points based on the movement amount and rotation amount. Department,
A three-dimensional measuring device for train equipment, comprising: an equipment determination unit that determines the presence or absence of the equipment based on the three-dimensional coordinates of the feature points. 前記三次元復元部が、前記電車車両の走行速度、前記オプティカルフロー、及びカメラパラメータを用いて三角測量を行い、前記撮像装置の焦点を原点とする前記特徴点の三次元座標を計算する請求項１記載の電車設備の三次元計測装置。The three-dimensional reconstruction unit performs triangulation using the traveling speed of the train car, the optical flow, and camera parameters, and calculates three-dimensional coordinates of the feature point with the focus of the imaging device as the origin. A three-dimensional measuring device for train equipment according to 1. 前記特徴点検出部が、前記特徴点として前記画像データからコーナー点を検出する
ことを特徴とする請求項１又は請求項２記載の電車設備の三次元計測装置。 The feature point detection unit, the three-dimensional measuring apparatus of the train equipment according to claim 1 or claim 2, wherein the detecting the corner points from the image data as a characteristic point. 前記三次元復元部が、鉛直方向及びピッチング角を除く前記電車車両の移動量及び回転量を求める
ことを特徴とする請求項１から請求項３のいずれか１項に記載の電車設備の三次元計測装置。 The three-dimensional restoration unit according to any one of claims 1 to 3, wherein the three-dimensional restoration unit obtains a movement amount and a rotation amount of the train vehicle excluding a vertical direction and a pitching angle. Measuring device. 前記設備判断部が、前記オプティカルフローを用いて電車設備と背景の分離を行い、前記特徴点が近接して所定値以上存在する領域を前記設備と判断し、前記特徴点が所定値未満の領域をノイズと判断する
ことを特徴とする請求項１から請求項４のいずれか１項に記載の電車設備の三次元計測装置。 The equipment determining unit separates the train equipment from the background using the optical flow , determines that the area where the feature points are close to each other and has a predetermined value or more is the equipment, and the area where the feature points are less than the predetermined value. Is determined as noise, and the three-dimensional measuring device for train equipment according to any one of claims 1 to 4 . 撮像装置によって電車車両の前方又は後方を撮像し、前記撮像装置によって撮像した画像に基づいて前記電車車両の周囲にある設備を検出する電車設備の三次元計測方法であって、
前記撮像装置によって撮像された画像データを入力する画像入力工程と、
前記撮像装置によって画像を撮像した位置に対応する前記電車車両の走行速度を取得する車両速度データ取得工程と、
前記画像データから特徴点を検出する特徴点検出工程と、
前記画像データの時系列的に連続する二つの画像から検出した前記特徴点に基づいてオプティカルフローを算出するオプティカルフロー算出工程と、
前記電車車両の走行速度、前記オプティカルフロー、及びカメラパラメータに基づいて前記電車車両の移動量及び回転量を求め、当該移動量及び回転量に基づいて前記特徴点の三次元座標を求める三次元復元工程と、
前記特徴点の三次元座標に基づいて前記設備の有無を判定する設備判定工程と
を有することを特徴とする電車設備の三次元計測方法。 A three-dimensional measuring method for a train facility, which captures an image of the front or rear of a train vehicle by an image capturing device, and detects the facility around the train vehicle based on the image captured by the image capturing device,
An image input step of inputting image data picked up by the image pickup device;
A vehicle speed data acquisition step of acquiring a traveling speed of the electric train vehicle corresponding to a position where an image is captured by the imaging device;
A feature point detecting step of detecting feature points from the image data,
An optical flow calculation step of calculating an optical flow based on the feature points detected from two images that are consecutive in time series of the image data,
Three-dimensional reconstruction that obtains the movement amount and rotation amount of the train vehicle based on the traveling speed of the train vehicle, the optical flow, and camera parameters, and obtains the three-dimensional coordinates of the feature points based on the movement amount and rotation amount. Process,
A three-dimensional measuring method for a train facility, comprising a facility determining step of determining the presence or absence of the facility based on the three-dimensional coordinates of the feature points. 前記三次元復元工程が、前記電車車両の走行速度、前記オプティカルフロー、及びカメラパラメータを用いて三角測量を行い、前記撮像装置の焦点を原点とする前記特徴点の三次元座標を計算するThe three-dimensional reconstruction step performs triangulation using the traveling speed of the train car, the optical flow, and camera parameters, and calculates the three-dimensional coordinates of the feature point with the focus of the imaging device as the origin.
ことを特徴とする請求項６記載の電車設備の三次元計測方法。The three-dimensional measuring method for train equipment according to claim 6, wherein. 前記特徴点検出工程が、前記特徴点として前記画像データからコーナー点を検出する
ことを特徴とする請求項６又は請求項７記載の電車設備の三次元計測方法。 8. The three-dimensional measuring method for train equipment according to claim 6 or 7, wherein the feature point detecting step detects a corner point from the image data as the feature point. 前記三次元復元工程が、鉛直方向及びピッチング角を除く前記電車車両の移動量及び回
転量を求める
ことを特徴とする請求項６から請求項８のいずれか１項に記載の電車設備の三次元計測方法。 The three-dimensional restoration of the train equipment according to any one of claims 6 to 8, wherein the three-dimensional restoration step obtains a movement amount and a rotation amount of the train vehicle excluding a vertical direction and a pitching angle. Measuring method. 前記設備判定工程が、前記オプティカルフローを用いて電車設備と背景の分離を行い、前記特徴点が近接して所定値以上存在する領域を前記設備と判断し、前記特徴点が所定値未満の領域をノイズと判断する
ことを特徴とする請求項６から請求項９のいずれか１項に記載の電車設備の三次元計測方法。 The equipment determining step performs separation of the train equipment and the background by using the optical flow, determines that the area where the feature points are close to each other and has a predetermined value or more is the equipment, and the area where the feature points are less than the predetermined value. 10. The three-dimensional measurement method for train equipment according to claim 6 , wherein the noise is determined as noise.
JP2016087671A 2016-04-26 2016-04-26 Three-dimensional measuring device and three-dimensional measuring method for train equipment Active JP6699323B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2016087671A JP6699323B2 (en) 2016-04-26 2016-04-26 Three-dimensional measuring device and three-dimensional measuring method for train equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2016087671A JP6699323B2 (en) 2016-04-26 2016-04-26 Three-dimensional measuring device and three-dimensional measuring method for train equipment

Publications (2)

Publication Number Publication Date
JP2017196948A JP2017196948A (en) 2017-11-02
JP6699323B2 true JP6699323B2 (en) 2020-05-27

Family

ID=60238715

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2016087671A Active JP6699323B2 (en) 2016-04-26 2016-04-26 Three-dimensional measuring device and three-dimensional measuring method for train equipment

Country Status (1)

Country Link
JP (1) JP6699323B2 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109109859A (en) * 2018-08-07 2019-01-01 庄远哲 A kind of electric car and its distance detection method
WO2021044564A1 (en) * 2019-09-05 2021-03-11 三菱電機株式会社 Forward monitoring device and method for calibrating forward monitoring device
JP7461275B2 (en) 2020-10-28 2024-04-03 株式会社日立製作所 Track transportation system, control method for track transportation system, and trackside equipment shape measurement system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4305073B2 (en) * 2003-07-08 2009-07-29 株式会社明電舎 Pantograph obstacle detection method and apparatus
JP5439876B2 (en) * 2009-03-11 2014-03-12 オムロン株式会社 Image processing apparatus and method, and program
JP5293429B2 (en) * 2009-06-10 2013-09-18 日産自動車株式会社 Moving object detection apparatus and moving object detection method
JP5516561B2 (en) * 2011-12-08 2014-06-11 株式会社デンソーアイティーラボラトリ Vehicle driving support device
JP6277468B2 (en) * 2013-12-18 2018-02-14 株式会社明電舎 Vehicle power pole position inspection device
US20160026898A1 (en) * 2014-07-24 2016-01-28 Agt International Gmbh Method and system for object detection with multi-scale single pass sliding window hog linear svm classifiers
JP6209141B2 (en) * 2014-09-04 2017-10-04 公益財団法人鉄道総合技術研究所 Obstacle detection apparatus and method

Also Published As

Publication number Publication date
JP2017196948A (en) 2017-11-02

Similar Documents

Publication Publication Date Title
US10061993B2 (en) Warning method of obstacles and device of obstacles
KR101997436B1 (en) Object position measurement by car camera using vehicle motion data
JP5926228B2 (en) Depth detection method and system for autonomous vehicles
JP6524529B2 (en) Building limit judging device
JP4631096B2 (en) Vehicle periphery monitoring device
JP5075672B2 (en) Object detection apparatus and method
JP6499047B2 (en) Measuring device, method and program
JP2018124787A (en) Information processing device, data managing device, data managing system, method, and program
WO2014002692A1 (en) Stereo camera
JP6021689B2 (en) Vehicle specification measurement processing apparatus, vehicle specification measurement method, and program
US9715632B2 (en) Intersection recognizing apparatus and computer-readable storage medium
US20190078876A1 (en) Road surface displacement detection device and suspension control method
KR20170064652A (en) Egomotion estimation system and method thereof
JP6699323B2 (en) Three-dimensional measuring device and three-dimensional measuring method for train equipment
JP6547841B2 (en) Image processing apparatus, object recognition apparatus, device control system, image processing method and program
JP6543935B2 (en) PARALLEL VALUE DERIVING DEVICE, DEVICE CONTROL SYSTEM, MOBILE OBJECT, ROBOT, PARALLEL VALUE DERIVING METHOD, AND PROGRAM
JP6905390B2 (en) Own vehicle position estimation environment map generator, own vehicle position estimation device, own vehicle position estimation environment map generation program, and own vehicle position estimation program
KR20180122691A (en) Mainly a detection method and mainly a detection device
JP6096601B2 (en) Station platform fall detection device
JP3629935B2 (en) Speed measurement method for moving body and speed measurement device using the method
JP5981284B2 (en) Object detection device and object detection method
JP6466679B2 (en) Object detection device
KR102065337B1 (en) Apparatus and method for measuring movement information of an object using a cross-ratio
JP2015041187A (en) Traffic measurement device, and traffic measurement method
JP6416654B2 (en) White line detector

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20190219

RD02 Notification of acceptance of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7422

Effective date: 20190524

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20190605

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20190529

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20191211

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20191217

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20200213

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20200331

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20200413

R150 Certificate of patent or registration of utility model

Ref document number: 6699323

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150