KR101996169B1 - Method and apparatus for estimating vehicle position based on visible light communication that considering camera displacement - Google Patents

Abstract

Disclosed are a method and device for estimating a vehicle location based on visible light communication considering camera displacement. A vehicle location estimating method using visible light communication, according to an embodiment, comprises the steps of: photographing street lamps while a vehicle is moving to obtain a photographed image including LEDs within the street lamps; acquiring image coordinates corresponding to the LEDs and visible light signals of the street lamps based on the photographed image; determining identification codes of the street lamps based on the visible light signals; extracting world coordinates of the street lamps corresponding to the determined identification codes from a database in which the identification codes of the street lamps and the world coordinates of the street lamps are matched and stored; constructing equations based on the geometric relationship between the obtained image coordinates and the extracted world coordinates; and estimating the location of the vehicle based on the constructed equations. The step of constructing the equations includes a step of reflecting a displacement value of the camera according to the movement of the vehicle in at least a part of the extracted world coordinates.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and apparatus for estimating a position of a vehicle based on visible light communication,

The following embodiments relate to a method and apparatus for vehicle position estimation based on visible light communication in consideration of camera displacement.

Two types of sensors, CCD (charge coupled device) and CMOS (complementary metal-oxide semiconductor), can be used in the imaging field. CCDs have been considered to provide excellent image quality, and in the early years of the digital age most cameras were equipped with CCD sensors. However, in recent years, with the rapid development of CMOS sensors, CMOS sensors have performed better than CCD sensors in various aspects including image quality, frame rate and production cost. As a result, CMOS sensors are used as building blocks in most imaging devices and are becoming widespread and are suitable for use as receiving devices in the field of visible light communication (VLC) and VLC based positioning.

There are three methods for estimating the current position of the vehicle. First, it uses a global positioning system (GPS). This method is inexpensive, but GPS can not be used in urban tunnels or streets where high buildings can interfere with GPS signals. Secondly, by estimating the position of a vehicle based on data communication such as 4G and long term evolution (LTE), this method can overcome the limitation related to the availability of GPS. However, this method only provides position accuracy of tens of meters and is not sufficient for applications requiring high levels of accuracy such as autonomous vehicles. Third, by using LIDAR (light detection and ranging), this method provides very high accuracy. However, implementing LIDAR generally requires additional equipment at very high cost. For this reason, a new technique for estimating the position of the vehicle is required.

In recent years, a new kind of technology based on visible light communication has been proposed to estimate vehicle location. For example, a vehicle's taillamp or headlight can be used to transmit position signals to two photodiodes installed in other vehicles, and a time difference of arrival (TDOA) algorithm is used to determine the relative position between the vehicles Can be used. However, when using TDOA, it is difficult to achieve high accuracy in practice because a large error occurs due to a small error occurring in time measurement. As another example, the position of the vehicle can be determined using a light emitting diode (LED) signal light and a photodiode. However, since a photodiode can not distinguish between different light sources, this method may be very vulnerable to ambient light, especially sunlight. As another example, if the first vehicle sends a VLC positioning signal via the rear LED, the second vehicle can use the two cameras to receive the positioning signal and use the neural network to estimate the position of the target vehicle. In this case, the accuracy in camera-based position estimation can be highly dependent on the distance between reference points, which is the LED. If the rear LED of the first vehicle is relatively close to the distance between the two vehicles, the position accuracy may not be high.

A method of estimating the position of a vehicle using visible light communication in accordance with an embodiment includes the steps of photographing streetlights while the vehicle is moving to obtain a shot image including LEDs in the streetlights; Obtaining visible light signals of the streetlights based on the sensed image and image coordinates corresponding to the LEDs; Determining identification codes of the streetlights based on the visible light signals; Extracting world coordinates of the streetlights corresponding to the determined identification codes in a database in which the identification codes of the streetlights and the world coordinates of the streetlights are matched with each other; Constructing equations based on the geometric relationships between the obtained image coordinates and the extracted world coordinates; And estimating the position of the vehicle based on the equations constructed, wherein the step of constructing the equations includes reflecting the displacement value of the camera as the vehicle moves into at least a portion of the extracted world coordinates .

An apparatus for estimating a position of a vehicle using visible light communication according to an exemplary embodiment of the present invention includes a photographing unit for photographing streetlights while the vehicle is moving to generate a photographing image including LEDs in the streetlights; And obtaining image coordinates corresponding to the visible light signals of the street lamps and the LEDs based on the shot image, determining identification codes of the street lamps based on the visible light signals, Extracting world coordinates of the streetlamps corresponding to the determined identification codes in a database in which world coordinates of the streetlamps are matched with each other and extracting world coordinates of the streetlamps corresponding to the determined identification codes based on the geometric relationship between the obtained image coordinates and the extracted world coordinates, And the position estimating unit reflects the displacement value of the camera according to the movement of the vehicle on at least a part of the extracted world coordinates when constructing the equations.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates a vehicle position estimation system in accordance with one embodiment.
2 is a flowchart illustrating a vehicle position estimation process according to an exemplary embodiment of the present invention.
3 illustrates a pinhole camera model according to one embodiment.
4 illustrates an image acquisition process of a CMOS sensor according to an exemplary embodiment;
5 is a view showing a photographed image according to a movement of a camera according to an embodiment.
6 illustrates a relationship between camera displacement and LED displacement in accordance with one embodiment;
FIG. 7 is an operational flowchart illustrating a process of constructing an equation by merging photographed images according to an embodiment. FIG.
8 is an operational flow diagram illustrating a process for constructing an equation by compensating for rolling shutter artifacts according to an embodiment;
9 is a view illustrating a position estimation process according to a camera displacement according to an embodiment.
10 is a diagram illustrating a position estimation process according to an LED displacement according to an exemplary embodiment.

It is to be understood that the specific structural or functional descriptions of embodiments of the present invention disclosed herein are presented for the purpose of describing embodiments only in accordance with the concepts of the present invention, May be embodied in various forms and are not limited to the embodiments described herein.

Embodiments in accordance with the concepts of the present invention are capable of various modifications and may take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. However, it is not intended to limit the embodiments according to the concepts of the present invention to the specific disclosure forms, but includes changes, equivalents, or alternatives falling within the spirit and scope of the present invention.

The terms first, second, or the like may be used to describe various elements, but the elements should not be limited by the terms. The terms may be named for the purpose of distinguishing one element from another, for example without departing from the scope of the right according to the concept of the present invention, the first element being referred to as the second element, Similarly, the second component may also be referred to as the first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Expressions that describe the relationship between components, for example, "between" and "immediately" or "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms " comprises ", or " having ", and the like, are used to specify one or more of the features, numbers, steps, operations, elements, But do not preclude the presence or addition of steps, operations, elements, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

In the following embodiments, a VLC-based vehicle position estimation system using a streetlight and a CMOS image sensor is proposed. Street lamps are becoming increasingly popular around the world with many benefits, including environmental friendliness, long life span and high energy efficiency. These high power and high density lighting systems can be utilized for communication and positioning purposes. In addition, location estimation using a streetlight is highly likely to provide high accuracy due to the tight arrangement of street lights in principle.

The problem of using cameras for positioning has long been studied in the field of computer vision. However, there are two limitations of the conventional positioning algorithm in estimating the vehicle position. First, the conventional positioning algorithm can not obtain sufficient spin-off when the LED streetlights are present in the same line, thus failing to estimate the position of the vehicle. Second, regardless of the arrangement of the LEDs, the accuracy of the position estimation may be degraded due to the rolling shutter artifacts of the CMOS sensor. Using a rolling shutter mechanism, each pixel row of the CMOS sensor is exposed at different times. Therefore, if the position of the object changes during the exposure period, artifacts such as object distortion of the photographed image may occur. If the object moves at high speed, such distortion may give erroneous information to the position estimation algorithm, which may cause errors in the position estimation result. Conventional positioning algorithms originally designed for use with CCD sensors that do not use a rolling shutter mechanism or recent algorithms designed for use with smartphone CMOS sensors do not address these artifacts.

According to the embodiments described below, the above problems can be solved. According to the embodiments, in order to solve the LED arrangement problem in the same line, two LEDs disposed at different positions of the vehicle are used to photograph an image of the LED street lamp, and the position of the camera is estimated by integrating the data of the two LEDs . Rolling shutter artifacts may also be corrected according to embodiments. For example, the artifacts of the LED image can be corrected based on the speed of the vehicle and the read time of the sensor. Therefore, high positioning accuracy can be achieved even when the vehicle moves at high speed.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

1 is a diagram illustrating a vehicle position estimation system in accordance with one embodiment.

Referring to FIG. 1, the streetlight 120 may transmit a visible light signal. The visible light signal may include the identification code of the streetlight 120 and the world coordinates of the streetlight 120 corresponding to the location of the streetlight 120 may be obtained through the identification code.

The apparatus 110 may include a photographing unit 111 and a position estimating unit 112 and may estimate the position of the vehicle based on visible light communication. The photographing unit 111 may include at least one camera based on a CMOS image sensor, and may photograph the street lamp 120 to generate a photographed image. The position estimating unit 112 may acquire a photographed image from the photographing unit 111 and obtain a visible light signal of the streetlight 120 based on the photographed image.

The position estimating unit 112 extracts an identification code from the visible light signal and refers to the database in which the identification code of the streetlight 120 and the world coordinates of the streetlight 120 are matched and stored to store the world coordinate corresponding to the extracted identification code Can be obtained.

The streetlight 120 may include an LED 121. The position estimating unit 112 can obtain the image coordinates of the LED 121 in the photographed image. The position estimating unit 112 can estimate the position of the vehicle based on the geometric relationship between the world coordinates and the image coordinates, and the intrinsic parameters of the camera. The intrinsic parameters of the camera may include the size of the sensor, the sensor resolution, and the focal length of the lens. Further, the speed of the vehicle can be measured through the speedometer of the vehicle, and the speed of the vehicle can be used for the rolling shutter correction.

2 is a flowchart illustrating a vehicle position estimation process according to an exemplary embodiment of the present invention.

Referring to FIG. 2, in step 210, the photographing section photographs the streetlight, and the position estimating section obtains the photographing image from the photographing section. The photographed image may include LEDs in street lamps.

In step 220, the position estimating unit obtains the image coordinates corresponding to the visible light signals and the LEDs of the streetlamps based on the shot image. Each of the streetlights can transmit data through the visible light output from the LED through the VLC technique, and the position estimator can acquire data from the visible light signal transmitted by the streetlight.

In step 230, the position estimator extracts the world coordinates of the street lamp in the database based on the visible light signal. The position estimating unit may determine the identification codes of the streetlights based on the visible light signals. For example, the position estimator may demodulate the visible light signals to obtain the identification codes of the respective streetlights from the visible light signals. The identification code can be uniquely assigned to each streetlight to identify the streetlight. In the database, the identification codes of the streetlights and the world coordinates of the streetlights may be matched and stored. The position estimating unit may extract, from the database, world coordinates of streetlights corresponding to the determined identification codes.

In step 240, the position estimator constructs equations related to the geometric relationship between the obtained image coordinates and the extracted world coordinates. According to the pinhole camera model, a certain geometric relationship can be formed between the image coordinates and the world coordinates, and the equations can be constructed according to this geometrical relationship. The concrete procedure for constructing the equations will be described later.

In step 250, the position estimating unit estimates the position of the vehicle based on the equations constructed above. The position estimating unit estimates the position of the camera at the moment the streetlight is photographed by obtaining the solution through the equations, and estimates the position of the vehicle based on the position of the camera. A concrete procedure for obtaining a solution of the equations will be described later.

3 is a view illustrating a model of a pinhole camera according to an exemplary embodiment of the present invention. The camera position estimation algorithm can be based on a pinhole camera model.

로 표시된다. 주점(principal point)은 이상적으로는 센서의 중심에 위치하며,

와 같은 2D 이미지 좌표로 나타낼 수 있다.Referring to FIG. 3, there are shown three types of coordinate systems: a three-dimensional world coordinate system, a three-dimensional camera coordinate system, and a two-dimensional image coordinate system. The three-dimensional world coordinates of the camera are

. The principal point is ideally located at the center of the sensor,

As shown in FIG.

,

가 i 번째 LED의 월드 좌표 및 이미지 좌표를 나타낸다고 가정하면, 각 월드 좌표와 각 이미지 좌표 간의 기하학적 관계는 수학식 1로 나타낼 수 있다.In general, multiple LEDs can be photographed in an image.

,

Is the world coordinate and the image coordinate of the i-th LED, the geometric relationship between each world coordinate and each image coordinate can be expressed by Equation (1).

는 스칼라이고, K는 렌즈 초점 거리 f와 주점

을 포함하는 카메라의 고유 파라미터(intrinsic parameter)에 의해 정의 된 3 x 4 카메라 캘리브레이션 행렬(calibration matrix)이며, R은 4 x 3 회전 행렬(rotation matrix)이고

는 4 x 1 변환 행렬(translation matrix)이다.here

Is a scalar, K is a lens focal length f,

R is a 4 x 3 rotation matrix and is a 3 x 4 camera calibration matrix defined by intrinsic parameters of the camera,

Is a 4 x 1 translation matrix.

 The camera-based positioning algorithm relies on solving several sets of equations in the form of Equation (1). There are two methods for deriving the camera position in equation (1), depending on whether an inertial sensor is used to obtain information about the camera after the camera's unique parameters have been computed. A method using an inertial sensor and a method using a positioning algorithm. Each of these methods may have advantages and disadvantages.

A simple algorithm to find the camera position is to use an inertial sensor to obtain information about the camera's pose (ie, direction). From this information, a rotation matrix R can be obtained. Then, equation (1) can be determined as a collinearity condition in which the camera position can be found. The advantage of this method is that it requires only two scene points to determine the camera position. In this case, however, the accuracy is highly dependent on the accuracy of the inertial sensor used to obtain the camera pose. The accuracy of an inertial sensor in an indoor environment is generally sufficient. However, in the case of the system according to the embodiment, since the distance from the camera to the LED is much longer than the distance of the indoor system, the positioning error may be increased due to the same error in the angle measurement.

The camera rotation matrix R is unknown because the position of the camera in the positioning algorithm is unknown. The problem of finding the camera position then results in finding the matrix [R t] using equation (1). To solve this problem, Equation 1 needs to be transformed into a kind of linear equation, which can be performed by a direct linear transformation (DLT) technique.

가 행렬 [R t]를 나타낸다고 가정하자. 행렬 [R t]의 각 행

, m = 1, 2, 3은 1 x 4 벡터

으로 나타낼 수 있다.

이고, 두 벡터

와

는 평행하므로, 이들의 외적(cross-product)은 0이다. 외적을 계산함으로써 수학식 2를 얻을 수 있다.

Let Rt denote the matrix [R t]. Each row of the matrix [R t]

, m = 1, 2, and 3 are 1 x 4 vectors

.

, And the two vectors

Wow

Are parallel, so their cross-product is zero. Equation 2 can be obtained by calculating the outer product.

각각은 수학식 2의 형태로 두 선형 방정식의 집합을 공식화 할 수 있다. 모든 장면 점에 대응하는 방정식들을 결합하여 수학식 3을 얻을 수 있다.In Equation (2), 0 = [0 0 0 0]. pair

Each can formulate a set of two linear equations in the form of equation (2). Equation (3) can be obtained by combining equations corresponding to all scene points.

Equation (3) can be expressed by Equation (4).

In Equation (4), M is a 2N x 12 matrix and v is a 12 x 1 vector. Where N is the number of scene points.

Since there are 12 unknown elements in Equation 4, six scene points are needed to solve them. In the ideal case with no noise, the exact solution of v can be found so that Mv = 0 is satisfied. Mv = 0 may not have the correct solution due to noise in actual implementation. Therefore, this problem becomes a least squares problem of finding the solution v as in equation (5).

가 수학식 5에 대해 선형이 되지 않아야 한다. 다시 말해, 카메라 위치를 찾기 위해서는 LED가 동일 선상에 위치하지 않아야 한다. 이러한 솔루션을 통해 카메라의 위치가 추정될 수 있다.The least squares problem above can have a solution created with a technique called singular value decomposition (SVD). However,

Should not be linear with respect to Equation (5). In other words, the LEDs must not be on the same line to find the camera position. With this solution, the position of the camera can be estimated.

4 is a diagram illustrating an image acquisition process of a CMOS sensor according to an exemplary embodiment of the present invention.

로 표시되며, 이는 한 행의 판독 시간과 동일하다. 센서의 다른 행이 서로 다른 시간에 노출되기 때문에, 센서의 다른 행에서 촬영된 LED 이미지는 다른 위치 상의 LED 이미지에 해당하므로, LED가 카메라에 더 가까운 것과 같은 이미지 아티팩트가 발생한다. 도 5는 일 실시예에 따른 카메라의 이동에 따른 촬영 이미지를 나타낸 도면이다. 카메라(510)가 이동하면서 LED들(520)을 촬영할 때, 아티팩트가 발생하지 않는다면 촬영 이미지(530)가 획득되지만, 아티팩트가 발생한다면 촬영 이미지(540)가 획득되게 된다. 즉, LED들(520)이 카메라(510)에 더 가까운 것과 같은 이미지 아티팩트가 발생할 수 있다.Referring to FIG. 4, each row of sensors is sequentially exposed at different times. The spacing between the starting points of the exposures in two consecutive rows is

, Which is equal to the read time of one row. Since different rows of the sensor are exposed at different times, the LED images taken in different rows of the sensor correspond to the LED images on different locations, resulting in image artifacts such that the LED is closer to the camera. 5 is a view showing a captured image according to the movement of a camera according to an embodiment. When the camera 510 captures the LEDs 520 as it moves, the captured image 530 is acquired if an artifact does not occur, but the captured image 540 is acquired if an artifact occurs. That is, image artifacts such as the LEDs 520 being closer to the camera 510 may occur.

및

가 i번째 LED의 월드 좌표 및 이미지 좌표인 경우, 이들 두 좌표 간의 관계는 수학식 6과 같이 나타낼 수 있다.

And

Is the world coordinate and the image coordinate of the i-th LED, the relationship between these two coordinates can be expressed by Equation (6).

는 i번째 LED가 캡쳐된 때 카메라의 위치

에 대응하는 변환 벡터를 나타낸다.In Equation (6)

Is the position of the camera when the i-th LED is captured

≪ / RTI >

사이의 차이는 더 커지고, 그에 따른 포지셔닝 오류를 발생시킨다.It is assumed that the pose of the camera in Equation (6) does not change during the shooting of a single image. Therefore, the camera rotation matrix remains the same as the other LEDs. Since all of the LEDs in the image correspond to different locations in the camera, finding a single location of the camera based on these LEDs does not lead to accurate results. When the vehicle moves faster, the conversion vector of equation (6)

The greater the difference between the two, resulting in a positioning error.

Figure 6 is a diagram illustrating the relationship between camera displacement and LED displacement in accordance with one embodiment.

Among the two types of position estimation algorithms described above, the inertial sensor method may be more suitable for use in an indoor environment where the distance from the camera to the LED is relatively short. Most inertial sensors in the system considered in the embodiments may not be accurate enough to produce the desired position accuracy. Also, there is a problem that the positioning error due to the error of the inertial sensor is very difficult to manage.

Therefore, in the embodiments, the position of the camera is estimated using the positioning algorithm. For this, collinear arrangement of LEDs and rolling shutter artifacts may be a problem. In the embodiments, the first problem can be solved by using two cameras and the fusion algorithm, and the second problem can be solved by using the compensation method. The solution to these two problems is based on the conversion between the camera displacement and the LED displacement described below.

x and X denote the image coordinates and world coordinates of the scene point, respectively. C represents the camera position, and [Delta] C represents the change in the camera position, i.e., the displacement. The result for the image coordinate x of the scene point X due to the change of the camera position is equivalent to the change of the scene position as shown in Equation (7).

,

로 나타내면 수학식 8을 얻을 수 있다.Referring to FIG. 6, initially the camera remains at position C, which is considered to be the camera position, and then the camera moves to position C + DELTA C to take an image of the LED located at X. According to Equation (7), the image taken at C +? C is the same as the image of the LED placed at position X. The coordinates of the two LEDs in the two images

,

The equation (8) can be obtained.

및

는 두 임의의 스케일을 나타낸다. 수학식 7에 따르면

이다. 따라서 두 개의 촬영된 이미지에서 두 LED의 이미지 좌표는 동일하다.In Equation (8)

And

Represents two arbitrary scales. According to equation (7)

to be. Therefore, the image coordinates of the two LEDs in the two captured images are the same.

As described below, solutions to two problems of collinear LED arrays and rolling shutter artifacts all relate to multiple LED images taken with cameras placed at different locations. The equations formulated with these LED images are constructed corresponding to various positions of the camera, and the problem is that they can not solve this set of equations in terms of camera displacement.

Equation 7 can be used to transform the camera displacement into an LED displacement. Different images for the same LED taken at cameras at different positions can then be considered as images of separate LEDs placed at different positions photographed at the camera at the fixed location. In this case, when the difference of the camera position is known, the position of the virtual LED can be known. Thus, a set of solvable equations can be determined.

FIG. 7 is an operational flowchart illustrating a process of constructing an equation by merging photographed images according to an embodiment.

Referring to FIG. 7, in step 710, the position estimator constructs an equation by merging images photographed by a plurality of cameras. In the following, an embodiment in which a plurality of cameras includes a first camera and a second camera will be described.

As mentioned above, in the embodiment, a position estimation algorithm that does not use the inertial sensor is used. Therefore, at least six non-collinear LED images should be taken to estimate the position of the camera. However, in many cases streetlights are arranged in a straight line along both sides of the street. Thus, three or more LEDs selected from both sides of the distance are usually collinear. In other words, there are up to four non-collinear LEDs on both sides of the distance, so that the camera position can not be estimated through the conventional algorithm. A sufficient number of equations can not be obtained.

To solve the collinear LED array problem, it is necessary to obtain more than 6 non-collinear LED images to formulate the geometric equations of the camera. This can be done via two cameras arranged at different positions to capture the image of the LED and by Equation (7). Since the camera displacement can be converted to an LED displacement, the same LED image taken by both cameras can be regarded as two different LED images taken with a single camera.

It is assumed that each camera shoots six LED images arranged in two lines. At this time, the photographed image photographed by the second camera may be regarded as an image photographed in a state where the first camera moves to the position of the second camera. That is, the first camera photographs a first image including six LEDs at the original position, and the second image captures a second image including six LEDs at a position of the second camera. When a single image is fused, it can be considered that a single image including 12 LEDs is taken by the first camera. In the fused single image, the six LEDs are in a non-collinear fashion.

Thus, two sets of equations formulated from two images can be fused, and each set of equations includes six equations, such as Equation (1). Originally, these two sets of equations have different camera positions, but in the fused equation set there are 12 LEDs for a single camera position, six of which are on the same line. The camera position can be calculated by selecting an equation corresponding to six LEDs positioned on non-collinear lines among 12 LEDs.

를 i번째 LED의 월드 좌표로 두고,

및

를 각각 제1 카메라 및 제2 카메라에 의해 촬영된 두 이미지 상 i번째 LED의 이미지 좌표로 둔다. ΔC는 두 카메라 사이의 카메라 간격으로 이해될 수 있다. 이에 따라 수학식 9를 도출할 수 있다.Let C be the world coordinate of the first camera, C + C be the world coordinate of the second camera,

Is set as the world coordinate of the i-th LED,

And

Are placed at the image coordinates of the i-th LED on the two images taken by the first camera and the second camera, respectively. ΔC can be understood as a camera interval between two cameras. Accordingly, equation (9) can be derived.

Equation (10) can be obtained by applying Equation (7) to Equation (9).

라 하면, N개의 LED에 대해, 수학식 10과 유사한 2N개의 방정식이 도출될 수 있다. DLT를 적용하여 2N개의 방정식이 수학식 11과 같이 도출될 수 있다.

2N equations similar to Equation 10 can be derived for N LEDs. 2N equations can be derived as Equation (11) by applying DLT.

는 j번째 카메라의 i번째 LED의 이미지 좌표이다. SVD를 통해 카메라의 세계 좌표, 즉 카메라의 위치를 추정할 수 있다.here

Is the image coordinate of the i-th LED of the j-th camera. SVD can be used to estimate the world coordinates of the camera, that is, the position of the camera.

8 is an operational flow diagram illustrating a process for constructing an equation by compensating for rolling shutter artifacts according to an embodiment.

Referring to FIG. 8, in step 810, the position estimator compares the rolling shutter artifacts to form an equation. To calculate the rolling shutter artifact, we can derive equation (12) based on equation (6).

는 i번째 LED가 촬영된 시점의 카메라의 세계 좌표이다.here

Is the world coordinate of the camera when the i-th LED is photographed.

Equation (12) states that the solution can not be found because there are more unknowns than the number of equations. However, if the difference of the camera world coordinate C, i.e.,? C, is known, the camera displacement can be converted to the LED displacement using equation (7), and thus the solution of equation (12) can be found.

, i = 1, 2, 3 ...이 i번째 LED를 캡처하는 동안 C와 카메라 위치 사이의 차이라고 둘 수 있다 수학식 12에

를 대입하여 수학식 13을 얻을 수 있다.More specifically, C is the camera position at the beginning of exposure of the current frame,

, i = 1, 2, 3 ... can be said to be the difference between C and the camera position while capturing the i-th LED.

(13) can be obtained.

가 먼저 결정될 필요가 있다.

는 차량의 속도 및 센서의 판독 시간에 기초하여 결정될 수 있다.

을 센서의 행 판독 시간으로 두고,

를 i번째 LED가 센서에 나타나는 행으로 두면, i번째 LED에 대응하는 경과 시간

은 수학식 14와 같이 나타낼 수 있다.To compensate for the rolling shutter artifact, the difference between the camera positions

Needs to be determined first.

May be determined based on the speed of the vehicle and the read time of the sensor.

As the row read time of the sensor,

Is set to the row in which the i-th LED appears in the sensor, the elapsed time corresponding to the i-th LED

Can be expressed by Equation (14).

라고 하면, 차이

는 수학식 15와 같이 나타낼 수 있다.The speed of the vehicle

In other words,

Can be expressed by Equation (15).

Equation (16) can be obtained by applying Equation (15) to Equation (13).

이다.In equation (16)

to be.

Equation (16) can be a basic equation for rolling shutter correction. To calculate the real solution, equation (16) can be modified as shown in equation (17).

The position of the camera can be estimated by applying SVD to Equation (17).

9 is a diagram illustrating a position estimation process according to a camera displacement according to an embodiment.

Referring to FIG. 9, the position of the camera changes from C = t0 to C + tC1, from t = t1 to C +? C1, and from t = t2 to C +? C2. The displacement value of the camera as the vehicle moves is ΔC1 at t = t1 and ΔC2 at t = t2. The position estimating unit may record the displacement value of the camera according to the movement of the vehicle when the streetlight is photographed. Let X be the image coordinate x of the LED, X be the world coordinate of the street lamp, K be the intrinsic parameter of the camera, R be the rotation matrix, C be the position of the camera, and? C be the displacement of the camera as the vehicle moves. . ≪ / RTI >

However, since? C is reflected in C in Equation (18), it is impossible to construct a set of equations that can be solved with respect to C when Equation (18) is used. According to the embodiment, a set of equations is constructed to obtain a solution for C by reflecting ΔC to X.

10 is a diagram illustrating a position estimation process according to an LED displacement according to an embodiment.

Referring to FIG. 10, the displacement of the camera as the vehicle moves is substituted by the displacement of the LED. For example, the position of the camera in FIG. 9 changes from C = t0 to C, t = t1 to C +? C1, and t = t2 to C +? C2. This is replaced with a change in the position of the LED in FIG. 10 from X -? C1 at t = t0 to X -? C2 at t = t1. The displacement value of the camera as the vehicle moves is ΔC1 at t = t1 and ΔC2 at t = t2, as in FIG.

Thus, the position estimating apparatus can reflect the displacement value of the camera according to the movement of the vehicle in at least a part of the world coordinates of the LED. Let X be the image coordinate x of the LED, X be the world coordinate of the street lamp, K be the intrinsic parameter of the camera, R be the rotation matrix, C be the position of the camera and? C be the displacement of the camera as the vehicle moves. . ≪ / RTI >

Since? C is reflected in X in equation (19), equation (19) can be used to construct a set of equations that can be solved with respect to C.

For example, when the photographed image includes the first to nth streetlights, the position estimation device determines a first displacement value of the camera according to the movement of the vehicle at the moment the first LED in the first streetlight is photographed, The first displacement value may be reflected in the first world coordinate corresponding to the first streetlight. The position estimating device may determine a first equation relating to the first streetlight based on image coordinates corresponding to the first LED, first world coordinates reflecting the first displacement value, and intrinsic parameters of the camera. The position estimating device can determine the second to nth equations in this manner also for the remaining second street lamps to the nth street lamps, thereby obtaining a solution of the equation set and estimating the position of the vehicle.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented as a computer-readable recording medium, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (10)

A method for estimating a position of a vehicle using visible light communication,
Capturing streetlights while the vehicle is moving to obtain a shot image including LEDs in the streetlights;
Obtaining visible light signals of the streetlights based on the sensed image and image coordinates corresponding to the LEDs;
Determining identification codes of the streetlights based on the visible light signals;
Extracting world coordinates of the streetlights corresponding to the determined identification codes in a database in which the identification codes of the streetlights and the world coordinates of the streetlights are matched with each other;
Constructing equations based on the geometric relationships between the obtained image coordinates and the extracted world coordinates; And
Estimating a position of the vehicle based on the equations;
Lt; / RTI >
The steps of constructing the equations
Determining a first displacement value of the camera according to the movement of the vehicle at a moment when the first LED in the first streetlight among the streetlights displayed in the captured image is photographed;
Adjusting a first world coordinate corresponding to the first streetlight by a first displacement value; And
Determining a first one of the equations based on the image coordinates corresponding to the first LED, the adjusted first world coordinate, and the intrinsic parameters of the camera
/ RTI > delete The method according to claim 1,
Recording the displacement value of the camera according to the movement of the vehicle when photographing the streetlights
≪ / RTI > The method according to claim 1,
The above equations are based on the following equations:

X represents the world coordinate of the street lamp; K represents the camera's intrinsic parameter; R represents the rotation matrix; C represents the camera position; And ΔC represents the displacement of the camera as the vehicle moves.
Way. The method according to claim 1,
Wherein when the captured image is captured by a first camera and a second captured image is captured by a second camera,
Reflecting a camera interval between the first camera and the second camera on the world coordinates of the streetlights displayed in the second shot image; And
Constructing the equations based on world coordinates of streetlights in the shot image and world coordinates of second streetlights in the second shot image reflecting the camera interval
/ RTI > The method according to claim 1,
The steps of constructing the equations
And constructing the equations based on direct linear transformation (DLT). The method according to claim 1,
The step of estimating the position of the vehicle
And applying a singular value decomposition (SVD) to the equations to estimate the position of the vehicle. An apparatus for estimating a position of a vehicle using visible light communication,
A photographing unit for photographing streetlights while the vehicle is moving to generate a photographing image including LEDs in the streetlights; And
Acquiring visible light signals of the street lamps and image coordinates corresponding to the LEDs based on the sensed image, determining identification codes of the streetlights based on the visible light signals, identifying the identification codes of the streetlights, Extracting world coordinates of the streetlamps corresponding to the determined identification codes in a database in which the world coordinates of the street coordinates of the streetlamps are matched with each other and calculating equations based on the obtained image coordinates and the geometric relationship between the extracted world coordinates The position estimating unit
Lt; / RTI >
When constructing the above equations, the position estimator
Determining a first displacement value of the camera according to the movement of the vehicle at the instant when the first LED in the first streetlight among the streetlights displayed in the captured image is photographed, 1 displacement value, and determines a first one of the equations based on the image coordinates corresponding to the first LED, the adjusted first world coordinate, and the intrinsic parameters of the camera. delete 9. The method of claim 8,
When the shot image is shot by the first camera and the second shot image is shot by the second camera,
Wherein the distance between the first camera and the second camera is reflected in the world coordinates of the streetlights displayed in the second shot image, and the world coordinates of the streetlights in the shot image and the second shot image And constructs the equations based on world coordinates of the second streetlights.

Images