JP6916091B2 - Position / orientation estimation system and position / orientation estimation device - Google Patents

Description

The present invention relates to a position / orientation estimation system and a position / attitude estimation device that estimate the position and orientation of the self-propelled body based on optical sensing data obtained by observation from the self-propelled body.

Conventionally, a Global Positioning System (hereinafter referred to as "GPS") has been used to measure the position and posture of a self-propelled body (hereinafter, a vehicle will be described as an example). The vehicle is equipped with a GPS receiver, and by receiving signals from a plurality of GPS satellites with the GPS receiver, the position of the own vehicle can be measured by a code positioning method or a carrier wave positioning method.

However, the position of the own vehicle cannot be measured by GPS in a place such as a tunnel where the GPS receiver cannot receive the signal from the GPS satellite. As one of the methods other than GPS for measuring the position of the own vehicle, there are wheel odometry and visual odometry. The wheel odometry measures the movement locus of the own vehicle by integrating the direction and the number of rotations of the wheels of the own vehicle, and estimates the position and posture of the own vehicle. The visual odometry estimates the position and posture of the own vehicle by estimating the movement locus of the own vehicle based on a plurality of continuous images taken by a camera fixed to the own vehicle.

For this visual odometry, a model-based method has been studied for a long time, but in recent years, a learning-based method using a deep neural network (hereinafter referred to as "DNN") has attracted attention (for example, a patent). Document 1).

e.g., R. Clark, S. Wang, H. Wen, A. Markham, and N. Trigoni. Vinet: Visual-inertial odometry as a sequence-to-sequence learning problem. In AAAI, pages 3995-4001, 2017

In visual odometry, it is necessary to model the long-term correlation using an image sensor with high temporal resolution in order to accurately estimate the position and orientation of the own vehicle. Event cameras are attracting attention as such image sensors with high temporal resolution.

However, although the conventional DNN can grasp the correlation for a short time, it is not realistic because the processing load (processing time, memory capacity used) of the system becomes excessive to grasp the correlation for a long time. rice field.

Therefore, the present invention presents a position / orientation estimation system, a position / orientation estimation method, and a position / orientation estimation that perform visual odometry using a CNN (Long Short-Term CNN, hereinafter referred to as “LSTCNN”) that can model a long-term correlation. The purpose is to provide a program.

The position / orientation estimation device of the present invention decomposes the space-time three-dimensional CNN performed on the image sensor data into the space two-dimensional CNN and the time one-dimensional CNN for visual odometry. As a result, the time range in which the convolution process can be performed can be extended without increasing the processing load.

The position / orientation estimation system according to one aspect of the present invention includes an optical sensing device that generates three-dimensional optical sensing data including two-dimensional position information and one-dimensional time information, and the optical sensing data input in a time series. Based on this, a position / orientation estimation device for estimating the position / orientation of the optical sensing device by visual odometry is provided. The position / orientation estimation device includes a two-dimensional convolution unit composed of a plurality of two-dimensional CNN modules that input the position information of each of a plurality of continuous frames composed of the optical sensing data and output a feature amount. A one-dimensional CNN module composed of a one-dimensional CNN module that inputs a plurality of the feature amounts output from each of the plurality of two-dimensional CNN modules and outputs the amount of change in position and orientation between adjacent frames as a local change amount. The cumulative value of the local change amount is obtained by accumulating the local change amounts of the folding portion and the plurality of frames, and by adding the cumulative value to the initial value of the position and orientation, after the plurality of frames. It includes a cumulative unit for determining the position and orientation of the optical sensing device.

With this configuration, the 3D CNN to be executed for optical sensing data consisting of 2D position information and 1D time information input in time series is divided into 2D CNN for position information and 1D CNN for time information. Therefore, it is possible to lengthen the time range in which convolution processing is possible (increase the number of optical sensing data in the time direction) without increasing the processing load.

In the position / orientation estimation system, the optical sensing device may be an event camera. The time resolution of the event camera is high, and the number of frames per unit time is large. According to this configuration, the position and orientation can be estimated by effectively performing the convolution even for such a large number of frames (long time).

In the position / orientation estimation system, the position / orientation estimation device reduces the time resolution of the optical sensing data input from the optical sensing device to generate the plurality of frames having the reduced time resolution. May further be included. With this configuration, it is possible to prevent the time resolution of the optical sensing data from the optical sensing device from being too high and the input data in the convolution process becoming too sparse in the time direction.

In the position / orientation estimation system, the optical sensing device may be fixed to the vehicle so as to sense the outside of the vehicle, and the cumulative unit accumulates the local change amount on a model basis and the local change. The quantity may be expressed by the parameters of the straight-ahead change amount and the angle change amount. With this configuration, it is possible to accumulate local change cormorants with a model with a small number of parameters by taking advantage of the restrictions on the movement of the vehicle.

In the above position / orientation estimation system, each of the two-dimensional CNN modules may be an LSTM module.

The position / orientation estimation device of one aspect of the present invention is used together with an optical sensing device that generates three-dimensional optical sensing data including two-dimensional position information and one-dimensional time information, and the optical sensing is input in a time series. It is a position / orientation estimation device that estimates the position / orientation of the optical sensing device by visual odometry based on the data, and is characterized by inputting the position information of each of a plurality of consecutive frames composed of the optical sensing data. A position between the two-dimensional convolution unit composed of a plurality of two-dimensional CNN modules that output quantities and the plurality of feature quantities output from each of the plurality of two-dimensional CNN modules and adjacent frames. The cumulative value of the local change amount is obtained by accumulating the one-dimensional convoluted portion composed of the one-dimensional CNN module that outputs the change amount of the posture as the local change amount and the local change amount of the plurality of frames, and the position and orientation. By adding the cumulative value to the initial value of the above, the cumulative portion for obtaining the position and orientation of the optical sensing device after the plurality of frames is provided.

Even with this configuration, the 3D CNN to be executed for the optical sensing data consisting of the 2D position information and the 1D time information input in time series is divided into the 2D CNN for the position information and the 1D CNN for the time information. Since the data are executed separately, the time range in which the convolution process can be performed can be lengthened (the number of optical sensing data in the time direction is large) without increasing the processing load.

According to the present invention, the three-dimensional CNN to be executed for the optical sensing data consisting of the two-dimensional position information and the one-dimensional time information input in time series are the two-dimensional CNN for the position information and the one-dimensional CNN for the time information. Since the data is executed separately, the time range in which the convolution process can be performed can be lengthened (the number of optical sensing data in the time direction is large) without increasing the processing load.

A block diagram showing a configuration of a position / orientation estimation system according to an embodiment of the present invention. The figure which compares the time resolution of the event data by the event camera of embodiment of this invention, and the image by a normal camera. The figure which shows the network structure and data flow in the visual odometry of embodiment of this invention. The figure which shows the network structure of the one-dimensional CNN module 23 and the cumulative part 24 of embodiment of this invention. The figure which shows the model of the parallel two-wheeled vehicle of embodiment of this invention

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described below show an example of the case where the present invention is carried out, and the present invention is not limited to the specific configuration described below. In carrying out the present invention, a specific configuration according to the embodiment may be appropriately adopted.

FIG. 1 is a block diagram showing a configuration of a position / orientation estimation system according to an embodiment of the present invention. The position / orientation estimation system 100 includes an optical sensing device 10 and a position / orientation estimation device 20. The position / posture device 20 includes a pretreatment unit 21, a two-dimensional folding unit 22, a one-dimensional folding unit 23, and a cumulative unit 24. The optical sensing device 10 is fixed to a vehicle that is a self-propelled body, generates an event frame as optical sensing data by optically sensing the outside of the vehicle, and generates an event frame in chronological order. Output to 20.

In the present embodiment, as the optical sensing device 10, an event camera as a biologically inspired camera is adopted. A normal camera measures the number of photons accumulated in a predetermined exposure time for each pixel, and simultaneously outputs the measurement results of all the pixels as one frame. On the other hand, in the event camera, each pixel operates asynchronously. Further, the event camera outputs an event frame as one frame of optical sensing data when it detects a difference between the previous detection intensity (brightness) and the current detection intensity (brightness). In the event data constituting the event frame (hereinafter, also simply referred to as "frame"), the position information of the pixel in which the detection intensity is different, the time stamp as time information, and the detection intensity are increased or decreased. It contains polarity information that indicates whether or not it is done.

FIG. 2 is a diagram comparing the time resolutions of the event data obtained by the event camera and the image obtained by the normal camera. The event camera has a time resolution on the order of microseconds, and has an extremely high time resolution as compared with a normal camera (for example, 30 frames / second). Further, the event camera does not detect the absolute value of the intensity, but detects only the polarity of the intensity change, so that the dynamic range is wide. For example, the dynamic range of a normal camera is about 50 dB, while the dynamic range of an event camera is about 120 dB. Further, while a normal camera can detect only a portion having a relatively high intensity, an event camera can detect a dark portion and a bright portion at the same time.

Due to the above characteristics of the event camera, the event camera is effectively used in the scene of automatic driving of a vehicle. However, since the time resolution of the event camera is high as described above, the position / orientation estimation device 20 needs to be able to handle the long-term (multi-frame) correlation.

The position / orientation estimation device 20 acquires a plurality of event frames arranged in time series from the optical sensing device 10 and reduces the number of these frames to extract frames e1 to eK. Here, K is the number of frames that can be processed by the position / orientation estimation device 20 at one time (hereinafter, also referred to as “allowable frame number”). The position / orientation estimation device 20 obtains the position / orientation pK of the own vehicle after one frame to K frames by visual odometry from the plurality of frames e1 to eK.

ここで、

は、タイムステップｋのフレーム（以下、「第ｋフレーム」等と表現する。）であり、Ｍ×Ｎは、イベントカメラの画素数である。また、ｍは、ホイールオドメトリ、慣性測定装置等のビジュアルオドメトリ以外の方法で得られた付加的なセンシングデータである。入力データにｍを含めるか否かは任意である。また、ｐｋは、第ｋフレームにおける光学センシング装置（が固定された自車両）の位置姿勢である。 That is, the position / orientation estimation device 20 executes the visual odometry according to the following equation (1).

here,

Is a frame of the time step k (hereinafter, referred to as “kth frame” or the like), and M × N is the number of pixels of the event camera. Further, m is additional sensing data obtained by a method other than visual odometry such as wheel odometry and inertial measurement device. Whether or not to include m in the input data is optional. Further, pk is the position and orientation of the optical sensing device (own vehicle to which the optical sensing device is fixed) in the kth frame.

Specifically, the position / orientation estimation device 20 obtains Δp1 to ΔpK of changes in the position and attitude of the own vehicle (hereinafter, also referred to as “local change amount”) between adjacent frames, and accumulates (connects) them in order. Then, the position / orientation pK of the own vehicle in the K frame is calculated by adding it to the initial values (hereinafter, referred to as “initial position / attitude”) p0 of the position and attitude of the own vehicle. When this is expressed by an equation, it becomes the following equation (2).

As described above, since the event camera has a high time resolution, the LSTCNN in the position / orientation estimation device 20 needs to be able to handle long-term (multi-frame, for example, several thousand frames) correlation. Therefore, the position / orientation estimation device 20 decomposes the three-dimensional CNN in space and time to be executed for a plurality of frame data input from the optical sensing device 10 into a two-dimensional CNN in space and a one-dimensional CNN in time. Execute. For this purpose, the position / orientation estimation device 20 includes a preprocessing unit 21 that reduces the number of a plurality of event frames input from the optical sensing device 10. Further, the position / orientation estimation device 20 has a two-dimensional folding unit 22 that executes a two-dimensional CNN in space and a one-dimensional folding unit 23 that executes a one-dimensional CNN in time, and performs the position / orientation estimation device 20 for a plurality of frames. The dimensional CNN is divided into a two-dimensional CNN and a one-dimensional CNN.

The preprocessing unit 21 will be described. The plurality of event frames output from the optical sensing device 10 are asynchronous at each pixel, and each event frame is composed of {u, v, t, p} four-dimensional event data. Here, u and v are the positions where the event was detected, t is the time (time stamp) when the event was detected, and p is the polarity of the detected event. These event data are converted into spatiotemporal event frames by the preprocessing unit 21.

The preprocessing unit 21 projects each data u, v, t, p of the event frame onto the corresponding spatiotemporal position of the three-dimensional tensor in order to form the event frame. Since the time resolution of the event camera is very small, about 1 microsecond, if a 3D tensor is prepared with that particle size, multiple frames input to the 2D convolution unit 22 become too sparse (sparse) and CNN. Becomes inefficient to process with. Therefore, the preprocessing unit 21 is sufficiently coarse, but the time resolution of the event frame is reduced to a time resolution τ (for example, about 1,000 microseconds) smaller than the frame rate of a normal camera (for example, 30 frames / second). To reduce.

の３つの重み係数を下式（４）〜（６）によって計算する。

ここで、ｔはイベントのタイムスタンプであり、

は、ｔに最も近い離散化タイムスタンプであり、

である。 The preprocessing unit 21 responds to each event in order to maintain detailed time information obtained from the optical sensing device 10.

The three weighting coefficients of are calculated by the following equations (4) to (6).

Here, t is the time stamp of the event,

Is the discretized time stamp closest to t,

Is.

As described above, the preprocessing unit 21 generates a plurality of frames whose time resolution is smaller than the plurality of event frames input from the optical sensing device 10 and inputs them to the two-dimensional convolution unit 22.

The two-dimensional folding unit 22 is composed of a plurality of two-dimensional CNN modules 22-1 to 22-K that execute two-dimensional CNN for each frame input from the preprocessing unit 21, and the one-dimensional folding unit 22. Reference numeral 23 denotes a one-dimensional CNN module. Each 2D CNN module 22-1 to 22-K processes a time-divided M × N × L short-time event frame, and the 1-dimensional CNN module has a long-time F × 1 × T size. Process features. Here, F is the length of the feature amount output from each of the two-dimensional CNN modules 22-1 to 22-K, and T satisfies T = K / L.

FIG. 3 is a diagram showing a network structure and a data flow in the visual odometry according to the embodiment of the present invention. L is arbitrarily set between 1 and K according to the characteristics of the input frame. For example, when K = 3000, L = 100 may be set.

Each of the two-dimensional CNN modules 22-1 to 22-K convolves in space (two-dimensional), and the one-dimensional CNN module 23 convolves in time (one dimension). The structure of each 2D CNN module 22-1 to 22-K is, for example, VGG-16 (Simonyan, K., and isserman, A. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs / 1409.1556.) It may be similar to the convolutional part of the network. The one-dimensional CNN module 23 includes WaveNet (van den Oord, A .; Dieleman, S .; Zen, H .; Simonyan, K .; Vinyals, O .; Graves, A .; Kalchbrenner, N .; Senior, AW; and Kavukcuoglu, K. 2016. Wavenet: A generative model for raw audio. CoRR abs / 1609.03499.) It is composed by stacking ordinary convolutional modules.

FIG. 4 is a diagram showing a network structure of the one-dimensional CNN module 23 and the cumulative unit 24. The one-dimensional CNN module 23 models a complex time dependence using a gated activation unit represented by the following equation (3), as used in WaveNet.

Here, f is a network of two-dimensional CNN modules 22-1 to 22-K, and g is a network of one-dimensional CNN modules 23. The one-dimensional CNN network g has a layer structure of an O layer (O is a natural number), and the output of the (O-1) layer is a change (local change amount) between adjacent frames estimated from consecutive frames. The Oth layer is a parameterless model-based Pose Concatenation (hereinafter referred to as "MPC"). The MPC is implemented as a cumulative unit 24. The MPC module, that is, the cumulative unit 24 estimates the position / orientation pk + 1 of the time step k + 1 by updating the position / orientation pk of the time step k using the local change amount Δpk of the time step k.

In the final layer of the one-dimensional folding unit 23, the estimated position / orientation error is calculated. This error is used to update the parameters of the two-dimensional folding unit 22 and the one-dimensional folding unit 23. The cumulative unit 24 as an MPC module can effectively calculate a stable derivative (that is, the derivative with respect to the parameter of each neuron) from the final error. This MPC module is obtained from the following findings.

FIG. 5 is a diagram showing a model of a parallel two-wheeled vehicle. As shown in FIG. 5, the movement of the vehicle is restricted and can be expressed by a parameter smaller than the parameter set of the Lie algebra se (2) or se (3). That is, in the parallel two-wheeled vehicle model, the local movement can be expressed by the parameters of the straight-ahead speed v and the angular velocity ω, but the cumulative portion 24 of the present embodiment uses the straight-ahead change ΔL and the angular change Δθ, and the position and orientation. The error is defined by ΣΔL and ΣΔθ. By such parameterization and change of the definition of the position / orientation error, it is possible to easily and stably calculate the derivative of the error function for each local change amount.

で表示できる。 In the following, for the sake of simplicity, a two-dimensional case in which the vehicle travels on a two-dimensional plane (movement in the height direction of the vehicle is not considered) will be described first. In the case of two dimensions, the position and orientation of the vehicle is the position of the vehicle and the straight-ahead angle.

Can be displayed with.

を用いて下式（７）で更新される。

ここで、Δθｋは、

によって与えられ、ΔＬは、ｖ及びωを用いて下式（８）により計算される。

Normally, the position and orientation are the amount of local change during the local time Δt in the kth frame.

Is updated by the following equation (7).

Here, Δθk is

Given by, ΔL is calculated by equation (8) below using v and ω.

Since the error of the time step k-1 depends on the error of the time step k, the derivative for each local change amount zk of the position-posture pK is non-linearly related to the equations (7) and (8). Therefore, the calculation load is high, the implementation is difficult, and the derivative itself becomes unstable when the cumulative position and orientation are far from the true position and orientation.

In the case of the conventional definition of error, the error of the position and orientation of the time step k depends on the error of the position and orientation of the time steps 1 to k-1, so that the differentiation is via the past equation (8). Affected by past differentiation. Therefore, the differentiation becomes a function of the error of the entire interval for integrating the position and orientation, and as a result, the calculation load becomes large. On the other hand, in the present embodiment, the derivative of each time step (time) of the finally integrated error with respect to the local change amount becomes independent for each time step, so that the calculation is simple and lightweight.

と表現する代わりに、

と表現し、累積された経路と角度のエラーを

ではなく、

と表記する。 Specifically, in the cumulative unit 24 of the present embodiment, the amount of local change is determined.

Instead of expressing

And the accumulated path and angle errors

not,

Notated as.

を出力し、第Ｏ層はＭＰＣモジュールとして現在の車両の動きを式（７）を用いて更新する。ここで、ｑｋは、下式（９）のように定義できる。

The first (O-1) layer, that is, the output layer of the one-dimensional CNN unit 23 is

Is output, and the Oth layer updates the current movement of the vehicle as an MPC module using the equation (7). Here, qk can be defined as in the following equation (9).

ここで、

は、累積された経路及び角度の真値である。 The cumulative position / orientation error Laccom calculated by the MPC module is defined by the following equation (10).

here,

Is the true value of the accumulated path and angle.

の局所変化量

に関するヤコビ行列は、下式（１１）で計算される。

Local change amount of

The Jacobian matrix with respect to is calculated by the following equation (11).

In addition to the above-mentioned cumulative position / orientation error Laccum, the MPC module also calculates the error Llocal of the amount of local change by the following equation (12).

は、ネットワークを学習するのに用いられる。調整パラメータλ１、λ２は、学習の初期にはＬｌｏｃａｌを強調し、学習の後期にはＬａｃｃｕｍを強調するように調整する。これにより、連結された位置姿勢の推定の精度を向上できる。 The sum of these cumulative position / orientation error Laccoum and local movement error Llocal

Is used to learn the network. The adjustment parameters λ1 and λ2 are adjusted so as to emphasize Llocal in the early stage of learning and emphasize Laccoum in the later stage of learning. As a result, the accuracy of estimating the connected position and orientation can be improved.

Next, learning of the position / orientation estimation device 20 will be described. When learning the position / orientation estimation device 20, a continuous sequence of 2K frames having a length twice the allowable number of frames K is randomly extracted from the data set as input data. The position / orientation estimation device 20 acquires a tensor having a size of F × 1 × 2T by inputting time-divisioned (M × N × L) input data into the two-dimensional convolution unit 22 over 2T times. ..

These tensors are input to the one-dimensional folding unit 23, and the position-posture error and the position-posture error are calculated in the one-dimensional folding unit 23. The one-dimensional convolution unit 23 is updated using this error. The error immediately before the two-dimensional folding unit 22 is divided into T this short-time error, and is used for updating the two-dimensional folding unit 22 over T times. For the 2K frame, it is necessary to acquire a valid T from the one-dimensional folding unit 23.

In this embodiment, since the normal convolutional layer is not padded, the output can be reduced to half the kernel size. If additional sensing data m is available, they are linked to the output of the two-dimensional convolution unit 22, and the time information is modeled by the one-dimensional convolution unit 23 together with the output of the two-dimensional convolution unit 22. Be transformed.

Adam (Kingma, D., and Ba, J. 2014. Adam: A method for stochastic optimization. ArXiv preprint arXiv: 1412.6980.) With hyperparameters (learning rate is l = 0.001, β1 = 0.9, β2 = 0.999, ε = 0.00000001) It can be used.

As described above, according to the position / orientation estimation system 100 of the present embodiment, the three-dimensional CNN to be executed for the optical sensing data consisting of the two-dimensional position information and the one-dimensional time information input in time series. Is executed separately for the two-dimensional CNN for the position information and the one-dimensional CNN for the time information, so that the time range in which the convolution process can be performed is extended without increasing the processing load (the number of optical sensing data in the time direction is increased). Many) can.

In the present invention, the three-dimensional CNN to be executed for the optical sensing data consisting of the two-dimensional position information and the one-dimensional time information input in time series are the two-dimensional CNN for the position information and the one-dimensional CNN for the time information. Since it is executed separately, the time range in which convolution processing can be performed can be lengthened (the number of optical sensing data in the time direction is large) without increasing the processing load, and the image taken from the self-propelled body. It is useful as a position / orientation estimation system or the like that estimates the position and orientation of the self-propelled body based on the above.

10 Optical sensing device 20 Position / orientation estimation device 21 Preprocessing unit 22 Two-dimensional folding unit 23 One-dimensional folding unit 24 Cumulative unit 100 Position / orientation estimation system

Claims (5)

An optical sensing device that generates three-dimensional optical sensing data including two-dimensional position information and one-dimensional time information, and
A position / orientation estimation device that estimates the position / orientation of the optical sensing device by visual odometry based on the optical sensing data input in time series.
With
The position / orientation estimation device is
A two-dimensional convolutional unit composed of a plurality of two-dimensional CNN modules for inputting the position information of each of a plurality of consecutive frames composed of the optical sensing data and outputting the feature amount.
A one-dimensional CNN module composed of a one-dimensional CNN module that inputs a plurality of the feature amounts output from each of the plurality of two-dimensional CNN modules and outputs the amount of change in position and orientation between adjacent frames as a local change amount. Folding part and
By accumulating the local change amounts of the plurality of frames, the cumulative value of the local change amount is obtained, and by adding the cumulative value to the initial value of the position and orientation, the optical sensing device after the plurality of frames Cumulative part for finding position and posture,
Equipped with a,
The optical sensing device is fixed to the vehicle so as to sense the outside of the vehicle.
The cumulative unit is a position / posture estimation system that accumulates the local change amount on a model basis and expresses the local change amount with parameters of a straight-ahead change amount and an angle change amount. The position / orientation estimation system according to claim 1, wherein the optical sensing device is an event camera. The position / orientation estimation device further includes a preprocessing unit that lowers the time resolution of the optical sensing data input from the optical sensing device to generate the plurality of frames whose time resolution is lowered. 2. The position / orientation estimation system according to 2. The cumulative unit uses the first error for the parameter and the second error for the local change amount for learning by weighting them with a first weight and a second weight, respectively. The position / orientation estimation system according to any one of claims 1 to 3, wherein the second weight is made heavier at the initial stage of the learning, and the first weight is made heavier at the latter stage of learning to perform learning. .. The optical sensing device is used together with an optical sensing device that generates three-dimensional optical sensing data including two-dimensional position information and one-dimensional time information, and is based on the optical sensing data input in a time series by visual odometry. It is a position / orientation estimation device that estimates the position / orientation of
A two-dimensional convolutional unit composed of a plurality of two-dimensional CNN modules for inputting the position information of each of a plurality of consecutive frames composed of the optical sensing data and outputting the feature amount.
A one-dimensional configuration consisting of a one-dimensional CNN module that inputs a plurality of the feature amounts output from each of the plurality of two-dimensional CNN modules and outputs the amount of change in position and orientation between adjacent frames as a local change amount. Folding part and
By accumulating the local change amounts of the plurality of frames, the cumulative value of the local change amount is obtained, and by adding the cumulative value to the initial value of the position and orientation, the optical sensing device after the plurality of frames Cumulative part for finding position and posture,
Equipped with a,
The optical sensing device is fixed to the vehicle so as to sense the outside of the vehicle.
The cumulative unit is a position / posture estimation device that accumulates the local change amount on a model basis and expresses the local change amount with parameters of a straight-ahead change amount and an angle change amount.

Images