KR101798043B1 - Method and apparatus for calculating 3 dimensional coordinate of vlc light source - Google Patents

Abstract

The present invention relates to a method and apparatus for calculating the three-dimensional coordinate of a VLC light source. The method includes a step of allowing a target image block selection part to select a target image block including the VLC light source of a preceding vehicle among a plurality of image blocks obtained by dividing a plurality of forward images collected to correspond to each of a plurality of cameras with regard to each of the plurality of forward images, a step of allowing a two-dimensional image coordinate calculation part to calculate the two-dimensional image coordinate of the VLC light source of the preceding vehicle for each of the plurality of forward images based on the target image block included in each of the plurality of forward images, and a step of allowing a three-dimensional image coordinate calculation part to calculate the three dimensional coordinate of the VLC light source of the preceding vehicle based on the two-dimensional image coordinate of the VLC light source of the preceding vehicle and a positional relationship between the plurality of cameras.

Description

TECHNICAL FIELD The present invention relates to a three-dimensional coordinate calculation method and apparatus for a VLC light source,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional coordinate calculation method of a VLC light source for calculating three-dimensional coordinates of a VLC (Visible Light Communication) light source disposed in a front vehicle.

Smart Car, which collectively refers to automobile technology that has been attracting attention in the fusion between different industries, refers to a vehicle having a physical system in which a network can be connected and internal and external networks of a vehicle are interconnected through wireless communication.

Typical technologies applied to smart cars include ADAS (Advanced Driver Assistance Systems) and autonomous driving systems. In order to implement such a smart car, communication between the vehicle ahead and the vehicle behind the vehicle is important, A VLC (Visible Light Communication) communication method is used for communication between the front vehicle and the rear vehicle.

VLC is a communication technology that transmits information by using visible light in a wavelength range of 400 nm to 700 nm. It rapidly blinks visible light emitted from a light source such as a light emitting diode (LED) Technology.

When the VLC is utilized for communication between the front vehicle and the rear vehicle, the VLC light source disposed at the rear of the front vehicle blinks at a high speed, and the camera disposed in front of the rear vehicle recognizes the VLC light source of the front vehicle, .

In a conventional VLC-based vehicle aiding system, a signal of a VLC light source transmitted from a forward vehicle is received by a single camera, and information such as a brake deceleration included in the signal of the VLC light source is utilized to prevent collision between the vehicles There is a problem of level.

Korean Patent Publication No. 10-2009-0059343 (2009.06.11.)

SUMMARY OF THE INVENTION The object of the present invention is to solve the problem described above, and it is an object of the present invention to extract a target image block which is a video including a VLC light source of a forward vehicle among a plurality of forward images collected from a plurality of cameras, Dimensional coordinate of the VLC light source of the front vehicle and calculates the three-dimensional coordinates of the VLC light source of the front vehicle based on the calculated two-dimensional coordinate.

Furthermore, it is an object of the present invention to calculate the movement locus of the front vehicle based on the calculated three-dimensional coordinate of the VLC light source of the front vehicle, acceleration information of the front vehicle received from the VLC light source of the front vehicle, and angular velocity information of the front vehicle .

The problems to be solved by the present invention are not limited to the above-mentioned problem (s), and another problem (s) not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method for calculating three-dimensional coordinates of a VLC light source, the method comprising: a target image block selection unit for selecting a plurality of target images, Selecting a target image block including a VLC light source of a forward vehicle among a plurality of image blocks, for each of a plurality of forward images, calculating a two-dimensional image coordinate of the forward vehicle based on a target image block included in each of the plurality of forward images, Calculating a two-dimensional image coordinate of the VLC light source for each of the plurality of forward images, and calculating a three-dimensional coordinate of the front vehicle based on the two-dimensional coordinate of the VLC light source of the front vehicle and the positional relationship between the plurality of cameras. And calculating the three-dimensional coordinates of the VLC light source.

For example, the step of selecting the target image block for each of the plurality of forward images may include the steps of: dividing each of the plurality of forward images to determine a plurality of image blocks; And selecting a target image block for each of the plurality of forward images based on the sum of the intensity values by the image frames.

For example, after the step of selecting the target image block, the step of comparing the size of the target image block with the size of the predetermined lowest image block is further included.

According to an embodiment, when the size of the target image block is larger than the size of the preset lowest level image block, setting the target image block as an upper image block includes determining a plurality of image blocks, Selecting a block and performing a comparison are performed for an upper image block, and a target image block selection cycle is performed until a size of a target image block becomes equal to a size of a lowest-ranked image block set in advance Repeatedly.

For example, the step of selecting a target image block may include calculating a sum of intensity values of each of a plurality of pixels included in each of the plurality of image blocks for each of the image frames for a predetermined period of time, Generating a plurality of frequency domain waveforms corresponding to each of the plurality of image blocks by applying a predetermined Fourier transform to each of the plurality of frequency domain waveforms; As a target image block.

), 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표(

), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix) 중 적어도 하나에 기초하여 산출된다.For example, when the plurality of cameras are two cameras, the three-dimensional coordinates of the VLC light source of the front vehicle are obtained by dividing the two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the first camera

), The two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera

A translation vector determined based on a positional relationship between the first camera and the second camera, a rotation matrix determined based on a positional relationship between the first camera and the second camera, .

According to one embodiment, when the plurality of cameras is two cameras, the three-dimensional coordinates of the VLC light source of the front vehicle are calculated based on the following equation (1).

[Equation 1]

는 전방 차량의 VLC 광원의 3차원 좌표,

는 제1 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표,

는 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표, x는

, t는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), r_k는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix)의 k번째 행을 의미하고,

은 내적을 의미한다.At this time,

Dimensional coordinate of the VLC light source of the front vehicle,

Dimensional image coordinate of the VLC light source of the front vehicle in the forward image collected through the first camera,

Dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera, x

, t is a translation vector determined based on a positional relationship between the first camera and the second camera, r _k is a rotation matrix determined based on a positional relationship between the first camera and the second camera, ≪ / RTI >< RTI ID =

Is the inner product.

According to one embodiment, the movement locus calculating section calculates the movement locus of the preceding vehicle based on the three-dimensional coordinate of the VLC light source of the preceding vehicle, the acceleration information of the preceding vehicle received from the VLC light source of the preceding vehicle, and the angular velocity information of the preceding vehicle The method comprising the steps of:

According to an aspect of the present invention, there is provided an apparatus for calculating three-dimensional coordinates of a VLC light source, the apparatus comprising: a plurality of image blocks, Dimensional image coordinates of the VLC light source of the preceding vehicle based on the target image block included in each of the plurality of forward images, the target image block selecting unit selecting the target image block including the light source for each of the plurality of forward images, Dimensional coordinates of the VLC light source of the preceding vehicle based on the two-dimensional image coordinate of the VLC light source of the preceding vehicle and the positional relationship between the plurality of cameras, And a coordinate calculation unit.

According to an embodiment, the target image block selection unit determines the plurality of image blocks by dividing each of the plurality of forward images, and calculates a sum of the intensity values of the plurality of pixels included in each of the plurality of image blocks, The target image block for each of the plurality of forward images is selected.

For example, after selecting the target image block, the target image block selection unit compares the size of the target image block with the size of the lowest-ranked image block set in advance.

For example, when the size of the target image block is larger than the size of the preset lowest image block, the target image block selecting unit sets the target image block as an upper image block, and the target image block selecting unit, A target image block selection cycle consisting of an operation of selecting a target image block and an operation of comparing a size of a target image block with a size of a preset lowest image block is performed for an upper image block, The cycle is repeated until the size of the target image block becomes equal to the size of the lowest-order image block set in advance.

For example, the target image block selection unit may calculate the sum of the intensity values of the plurality of pixels included in each of the plurality of image blocks for each image frame over a predetermined period of time, And generating a plurality of frequency domain waveforms corresponding to each of the plurality of image blocks by applying a transform (Fourier Transform), generating a plurality of frequency domain waveforms corresponding to frequency domain waveforms including frequency components of a plurality of frequency domain waveforms, As a target image block.

), 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표(

), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix) 중 적어도 하나에 기초하여 산출된다.For example, when the plurality of cameras are two cameras, the three-dimensional coordinates of the VLC light source of the front vehicle are the two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the first camera

), The two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera

A translation vector determined based on a positional relationship between the first camera and the second camera, a rotation matrix determined based on a positional relationship between the first camera and the second camera, .

According to one embodiment, when the plurality of cameras is two cameras, the three-dimensional coordinates of the VLC light source of the front vehicle are calculated based on the following equation (1).

[Equation 1]

는 전방 차량의 VLC 광원의 3차원 좌표,

는 제1 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표,

는 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표, x는

, t는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), r_k는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix)의 k번째 행을 의미하고,

은 내적을 의미한다.At this time,

Dimensional coordinate of the VLC light source of the front vehicle,

Dimensional image coordinate of the VLC light source of the front vehicle in the forward image collected through the first camera,

Dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera, x

, t is a translation vector determined based on a positional relationship between the first camera and the second camera, r _k is a rotation matrix determined based on a positional relationship between the first camera and the second camera, ≪ / RTI >< RTI ID =

Is the inner product.

For example, a movement locus calculating section that generates a locus of movement of the front vehicle based on the three-dimensional coordinate of the VLC light source of the front vehicle, the acceleration information of the front vehicle received from the VLC light source of the front vehicle, and the angular velocity information of the front vehicle .

According to an embodiment of the present invention, a target image block, which is a video including a VLC light source of a preceding vehicle, is extracted from a plurality of forward images collected from a plurality of cameras, and then a VLC light source Dimensional coordinate of the forward vehicle and calculates the three-dimensional coordinates of the VLC light source of the preceding vehicle based on the calculated two-dimensional coordinates. Thus, even if only a part of the image block is utilized without utilizing the entire image of the forward image, It is possible to increase the frame rate of each of the plurality of cameras as compared with a method of determining the position of the VLC light source from the entire forward image.

Further, according to the present invention, the movement locus of the preceding vehicle is calculated based on the three-dimensional coordinate of the calculated VLC light source of the front vehicle, the acceleration information of the front vehicle received from the VLC light source of the preceding vehicle, and the angular velocity information of the preceding vehicle Thus, collision can be prevented more precisely.

1 is a view for explaining a vehicle system to which a three-dimensional coordinate calculation apparatus for a VLC light source is applied, according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a three-dimensional coordinate calculation apparatus for a VLC light source according to an embodiment of the present invention.
3 is a flowchart for explaining a three-dimensional coordinate calculation method of a VLC light source according to an embodiment of the present invention.
FIG. 4 is a flow chart for explaining a step of selecting a target image block in a three-dimensional coordinate calculation method of a VLC light source according to an embodiment of the present invention.
5 is a flowchart for explaining a concrete example of selecting a target image block among a plurality of image blocks in the method of calculating three-dimensional coordinates of a VLC light source according to an embodiment of the present invention.
6 is a diagram for explaining a step of selecting a target image block in the three-dimensional coordinate calculation method of the VLC light source according to the embodiment of the present invention.
7 is a diagram for explaining a structure of a signal transmitted by a VLC light source of a forward vehicle in a method and apparatus for three-dimensional coordinate calculation of a VLC light source according to an embodiment of the present invention.
8 is a view for explaining a method for calculating three-dimensional coordinates of a VLC light source of a front vehicle in a three-dimensional coordinate calculation method and apparatus of a VLC light source according to an embodiment of the present invention.
FIG. 9 is a diagram for explaining a method for calculating a movement locus of a front vehicle in a method and apparatus for three-dimensional coordinate calculation of a VLC light source according to an embodiment of the present invention.
10 is a view for explaining a method of calculating a movement locus of a front vehicle in a method and apparatus for three-dimensional coordinate calculation of a VLC light source according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to facilitate a person skilled in the art to easily carry out the technical idea of the present invention. . In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, a method and an apparatus for calculating three-dimensional coordinates of a VLC light source according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1, a method and apparatus for calculating three-dimensional coordinates of a VLC light source according to an embodiment of the present invention will be described with reference to FIG. .

1 is a view for explaining a vehicle system to which a three-dimensional coordinate calculation apparatus for a VLC light source is applied, according to an embodiment of the present invention.

1, a vehicle system 10 to which a three-dimensional coordinate calculation device for a VLC light source according to an embodiment of the present invention is applied includes a plurality of cameras 11 and 12, a three-dimensional coordinate calculation device 200 and a VLC light source 13.

The plurality of cameras 11 and 12 can be disposed in front of the vehicle system 10 and can recognize a VLC light source (not shown) positioned behind the front vehicle by photographing a forward image of the vehicle.

According to one embodiment, the plurality of cameras 11, 12 may be attached to the front of the vehicle system 10 and may refer to a high-speed camera having a frame rate of at least 1000 fps or higher, but the invention is not so limited Do not.

For example, when the plurality of cameras 11 and 12 are high-speed cameras, the VLC signal can be received even when the VLC light source (not shown) of the preceding vehicle blinks at a speed that can not be recognized at the user's time.

For example, the plurality of cameras 11 and 12 have a very high frame rate (frame rate per second) as a high-speed camera and can receive a part of a forward image by using a windowing function or the like, Function can be mounted.

For example, the reason why the plurality of cameras 11 and 12 are high-speed cameras is that the frame rate of a general camera is about 30 fps, and the flickering speed of the VLC light source, which can be decoded using the frame rate, is about 15 Hz. The blinking phenomenon of the light source may be recognized by the user, which may cause inconvenience to the user.

Also, when the flickering rate of the VLC light source of the front vehicle is decoded at 15 Hz, data is transmitted at a very low rate, and not only the amount of data delivered per hour, but also rapid delivery can cause excessive delays in delivering important safety- It is because.

Accordingly, the high-speed camera, which is one embodiment of the plurality of cameras 11 and 12, may have a frame rate of 500 fps to 1000 fps or higher, but the present invention is not limited thereto.

According to one embodiment, the plurality of cameras 11 and 12 can include a windowing function for detecting only a part of the forward image, and as a result, the frame rates of the plurality of cameras 11 and 12 are dramatically reduced The frame rate of 100000 fps or more can be achieved, but the present invention is not limited thereto.

That is, the plurality of cameras 11 and 12 can receive the VLC information without delay by additionally increasing the frame rate by receiving only the partial block corresponding to the target image block in the forward image through the windowing function described above.

In this case, when the plurality of cameras 11 and 12 utilize the windowing function, only the target image block, which is a part of the forward image, is received and the position of the VLC light source of the forward vehicle is detected. A method of detecting the position corresponding to the light source may be required. A more detailed description of this method will be described later with reference to Fig. 6 below.

The VLC light source 13 may be attached to the rear surface of the vehicle and transmit information through visible light. The VLC light source 13 may be an LED array including LEDs, various devices capable of generating visible light, .

The VLC light source 13 can transmit information using various modulation techniques such as on-off keying (OOK) using flickering of LEDs, OFDM, pulse position modulation (PPM) It is possible to periodically transmit a VLC frame including an identifier of the vehicle, angular velocity information of the vehicle, and acceleration information of the vehicle.

According to one embodiment, the VLC light source 13 may refer to a tail lamp attached to the rear of the vehicle, a LED array buried separately for VLC communication, and the like. However, the VLC light source 13, The present invention is not limited thereto.

Referring now to FIG. 2, an apparatus 200 for calculating three-dimensional coordinates of a VLC light source according to an embodiment of the present invention will be described.

FIG. 2 is a block diagram illustrating a three-dimensional coordinate calculation apparatus for a VLC light source according to an embodiment of the present invention.

2, the VLC light source three-dimensional coordinate calculation device 200 includes a target image block selection unit 210, a two-dimensional image coordinate calculation unit 220, and a three-dimensional coordinate calculation unit 220. [ And a calculation unit 230.

2, the apparatus 200 for calculating three-dimensional coordinates of a VLC light source may further include a movement locus calculating unit 240, but the present invention is not limited thereto.

At this time, the target image block selection unit 210 receives the forward image from each of the plurality of cameras 11 and 12 attached to the vehicle.

Each of the plurality of cameras 11 and 12 receives the forward image including the VLC light source 23 of the forward vehicle and transmits the forward image To the image block selection unit 210.

The target image block selection unit 210 selects a target image block including the VLC light source of the forward vehicle among a plurality of image blocks obtained by dividing the plurality of forward images collected so as to correspond to each of the plurality of cameras, Select.

At this time, each of the plurality of forward images may refer to an image in which a plurality of cameras 11 and 12 are photographed forward.

For example, the plurality of image blocks may be a small unit of a forward image obtained by dividing each of the plurality of forward images into predetermined sizes, and each of the plurality of forward images may be divided into a plurality of image blocks.

For example, the target image block may be an image block including an image of the VLC light source 23 of the forward vehicle among a plurality of image blocks obtained by dividing the forward image, and the target image block may be selected for each of the plurality of forward images .

More specifically, the target image block selection unit 210 determines a plurality of image blocks by dividing each of the plurality of forward images, and calculates a sum of the intensity values of the plurality of pixels included in each of the plurality of image blocks, The target image block for each of the plurality of forward images is selected.

For example, after selecting the target image block, the target image block selection unit 210 may compare the size of the target image block with the size of the lowest image block set in advance.

For example, the lowest image block is a minimum image block in which the apparatus 200 for calculating the three-dimensional coordinates of the VLC light source according to the embodiment of the present invention calculates a minimum image unit Of the image block.

If it is determined that the size of the target image block is larger than the predetermined size of the lowest image block, the target image block selection unit 210 sets the target image block as an upper image block, ) Selects a plurality of image blocks, selects a target image block, and compares the size of the target image block with the size of the lowest-ranked image block set in advance, And the target image block selection cycle can be repeated until the size of the target image block becomes equal to the size of the lowest-ranked image block set in advance.

For example, when the size of the forward image is 16 × 16, the size of the preset lowest image block is 4 × 4, and the object image block selection unit 210 divides the upper image block into two image blocks to select the object image block The operation of the target image block selection unit 210 will be described.

In this case, the target image block selection unit 210 divides the forward image into two image blocks each having a size of 8 × 16, and then outputs the image block including the VLC light source 23 of the forward vehicle among the two image blocks, Block, and compares the size of the target image block, 8X16, with the size of the lowest-ranked image block, 4X4.

At this time, since the size of the target image block 8X16 is larger than the size of the lowest image block 4X4, the target image block selection unit 210 sets the target image block as an upper image block.

Thereafter, the target image block selection unit 210 divides the set upper image block into two image blocks each having a size of 8 × 8, and then outputs the image block including the VLC light source 23 of the forward vehicle among the two image blocks The target image block is selected and the size 8X8 of the target image block is compared with the size 4X4 of the lowest image block set in advance.

Likewise, since the size of the object image block 8X8 is larger than the size of the lowest image block 4X4, the object image block selection unit 210 sets the object image block as an upper image block again.

The above operation can be repeated until the size of the determined target image block becomes 4X4, and if the size of the finally determined target image block is 4X4, the above operation can be stopped.

That is, the target image block selection unit 210 repeatedly divides the forward image collected by at least one camera among the plurality of cameras 11 and 12 to generate a target image block having the same size as the lowest- The above-described operation can be repeated until it is determined.

As a result, the finally determined target image block has the same size as the lowest image block, and can be determined as an image block including the VLC light source 23 of the forward vehicle.

For example, the lowest image block may be an image block having the smallest size among a plurality of image blocks, which are small units for dividing a forward image, and the size of the lowest image block may be preset, Lt; / RTI >

For example, when the size of the target image block determined through one target image block selection cycle is larger than the size of the lowest image block, the upper image block is set by the target image block selection unit 210, It may mean an image block to be segmented in the cycle.

At this time, the target image block selection cycle includes a first step of dividing the forward image or the upper image block into a plurality of image blocks, the image block including the VLC light source 23 of the forward vehicle by applying predetermined criteria to the plurality of image blocks, As a target image block, and a third step of comparing the size of the target image block with the size of the lowest-ranked image block set in advance.

At this time, the object block selection cycle is performed for the forward image and the object image block selection cycle for the upper image block is performed from the second round of execution.

At this time, the following method may be applied to the target image block selection unit 210 to determine a target image block, which is an image block including the VLC light source 23 of the preceding vehicle among the plurality of image blocks.

The target image block selection unit 210 calculates the sum of the intensity values of the plurality of pixels included in each of the plurality of image blocks for each image frame over a predetermined period of time, A plurality of frequency domain waveforms corresponding to each of the plurality of image blocks are generated by applying a set Fourier transform to each of the plurality of frequency domain waveforms, Select the image block as the target image block.

The two-dimensional image coordinate calculation unit 220 calculates the two-dimensional image coordinates of the VLC light source of the preceding vehicle for each of the plurality of forward images based on the target image block included in each of the plurality of forward images.

The three-dimensional coordinate calculation unit 230 calculates the three-dimensional coordinates of the VLC light source of the preceding vehicle based on the two-dimensional image coordinate of the VLC light source of the preceding vehicle and the positional relationship between the plurality of cameras.

), 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표(

), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix) 중 적어도 하나에 기초하여 산출된다.For example, when the plurality of cameras are two cameras, the three-dimensional coordinates of the VLC light source of the front vehicle are the two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the first camera

), The two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera

A translation vector determined based on a positional relationship between the first camera and the second camera, a rotation matrix determined based on a positional relationship between the first camera and the second camera, .

Here, the motion vector and the rotation transformation matrix are known as parameters for indicating the positional relationship between cameras in various conventional stereo cameras, and a detailed description thereof will be omitted.

For example, when the plurality of cameras is two cameras, the three-dimensional coordinates of the VLC light source of the front vehicle are calculated based on the following equation (1).

[Equation 1]

는 전방 차량의 VLC 광원의 3차원 좌표,

는 제1 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표,

는 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표, x는

, t는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), r_k는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix)의 k번째 행을 의미하고,

은 내적을 의미한다.At this time,

Dimensional coordinate of the VLC light source of the front vehicle,

Dimensional image coordinate of the VLC light source of the front vehicle in the forward image collected through the first camera,

Dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera, x

, t is a translation vector determined based on a positional relationship between the first camera and the second camera, r _k is a rotation matrix determined based on a positional relationship between the first camera and the second camera, ≪ / RTI >< RTI ID =

Is the inner product.

For example, based on the three-dimensional coordinate of the VLC light source of the preceding vehicle, the acceleration information of the preceding vehicle received from the VLC light source of the preceding vehicle, and the angular velocity information of the preceding vehicle, the movement locus calculating section 240 calculates the moving locus .

Now, a more detailed description of the apparatus 200 for calculating a three-dimensional coordinate of a VLC light source according to an embodiment of the present invention will be described later with reference to FIGS. 3 to 10, and redundant description will be omitted.

Now, referring to FIG. 3, a three-dimensional coordinate calculation method of a VLC light source according to an embodiment of the present invention will be described.

3 is a flowchart for explaining a three-dimensional coordinate calculation method of a VLC light source according to an embodiment of the present invention.

3, a method for calculating three-dimensional coordinates of a VLC light source according to an exemplary embodiment of the present invention includes selecting a target image block S310, calculating a two-dimensional image coordinate of a VLC light source of a forward vehicle (S320) and calculating the three-dimensional coordinates of the VLC light source of the front vehicle (S330).

In step S310, the target image block selection unit 210 selects the VLC light source 23 of the forward vehicle among the plurality of image blocks obtained by dividing the plurality of forward images collected so as to correspond to the plurality of cameras 11 and 12, And selecting a target image block to be included for each of the plurality of forward images.

In this case, the target image block selected in step S310 is an image block including the VLC light source 23 of the preceding vehicle among a plurality of image blocks obtained by dividing the plurality of forward images collected by the plurality of cameras 11 and 12 can do.

Referring now to Figures 4-6, an embodiment of step S310 will be described.

FIG. 4 is a flow chart for explaining a step of selecting a target image block in a three-dimensional coordinate calculation method of a VLC light source according to an embodiment of the present invention. 5 is a flowchart for explaining a concrete example of selecting a target image block among a plurality of image blocks in the method of calculating three-dimensional coordinates of a VLC light source according to an embodiment of the present invention. 6 is a diagram for explaining a step of selecting a target image block in the three-dimensional coordinate calculation method of the VLC light source according to the embodiment of the present invention.

As shown in FIG. 4, step S310 includes steps S311, S312, S313, and S314.

Step S311 may refer to a step of determining a plurality of image blocks by dividing each of the plurality of forward images.

For example, if an upper image block is set in step S314, step S311 may be a step of dividing an upper image block to determine a plurality of image blocks.

That is, in the above-described first to third steps of the target image block selection cycle, step S311 may be a step of dividing each of the plurality of forward images to determine a plurality of image blocks. And determining a plurality of image blocks.

At this time, the number of the plurality of image blocks generated by dividing the forward image or the upper image block may be preset and stored in the object image block selection unit 210. [

For example, in step S311, when the target image block selection unit 210 generates two image blocks by dividing a forward image or an upper image block, the target image block selection unit 210 selects a forward image or an upper image block as two It can be divided into image blocks.

For example, if the forward image or the upper image block has a size of 16X16, each of the two image blocks divided through S311 may have a size of 8X16.

Step S312 may include determining a target image block, which is an image block including the VLC light source 23 of the preceding vehicle among the plurality of image blocks.

More specifically, in step S312, the target image block selection unit 210 selects a target image block for each of the plurality of forward images based on the sum of the intensity values of the plurality of pixels included in each of the plurality of image blocks, Select.

Hereinafter, a more detailed description of step S312 will be described later with reference to FIG. 5, and redundant description will be omitted.

The step S313 performed after the step S312 may be a step of comparing the size of the target image block with the size of the lowest image block set in advance.

If the size of the target image block is equal to the size of the lowest-ranked image block as a result of the comparison in step S313, step S320 shown in FIG. 3 may be performed.

As described above, since each of the plurality of cameras 11 and 12 can be a high-speed camera supporting a windowing function, it is possible to receive image data from a divided image block from the whole of the forward image, The smaller the block size, the higher the frame rate can be achieved.

Accordingly, if it is determined in step S313 that the size of the target image block including the VLC light source 23 of the preceding vehicle is equal to the size of the lowest-ranked image block, the size of the image block received by each of the plurality of cameras 11 and 12 The size becomes very small, so that a plurality of cameras 11 and 12 can achieve a very high frame rate.

On the other hand, if it is determined in step S313 that the size of the target image block is larger than the preset size of the lowest image block, step S314 is performed.

In this case, step S314 may mean setting the target image block as an upper image block.

In this case, since the target image block is larger than the lowest image block yet, it corresponds to an image block that can be further divided.

As a result, in step S314, the target image block selection unit 210 sets the target image block as an upper image block, and then repeats steps S311, S312, and S313 for the upper image block recursively, The target image block selection cycle can be repeated until the size of the selected target image block becomes equal to the size of the least significant image block.

As described above, the target image block selection cycle may be a cycle for repeatedly performing the steps S311, S312, and S313 described above. In the first round of the target image block selection cycle, Steps S311, S312, and S313 may be performed, and steps S311, S312, and S313 may be performed on the upper image block in the second and subsequent times.

In other words, a target image block selection cycle including a step S311 of determining a plurality of image blocks, a step S312 of selecting a target image block, and a step S313 of comparing are performed for an upper image block, The image block selection cycle can be repeated until the size of the target image block becomes equal to the size of the lowest-order image block set in advance.

Referring now to FIG. 6, an embodiment of a target image block selection cycle consisting of S311, S312, and S313 and S314 will be described.

The image shown in FIG. 6 shows a forward image collected by one of the cameras 11 and 12, and the size of the lowest image block is the image block a22122111 and the image block a22122112, It is assumed that it is preset to the same size.

Hereinafter, a method of determining a target image block having the same size as the lowest image block among the image blocks including the VLC light source 23 of the forward vehicle in the forward image by repeatedly applying the target image block selection cycle will be described.

In step S311 of the first iteration of the target image block selection cycle, the forward image is divided into two image blocks a1 and a2. In step S312, the VLC light source of the forward vehicle among the two image blocks a1 and a2 23 are determined as the target image block a2 and the size of the target image block a2 is compared with the size of the lowest image block a2122111 in step S313.

At this time, since the size of the target image block a2 is larger than the size of the lowest image block a2122111 as a result of the comparison in step S313, the target image block a2 is set as the upper image block a2 through step S314 .

In this case, in step S311, the upper image block a2 is divided into two image blocks a21 and a22. In step S312, the upper image block a2 is divided into two image blocks a21 and a22. The image block a22 including the VLC light source 23 of the front vehicle among the two image blocks a21 and a22 is determined as the target image block a22 again. In step S313, the size of the target image block a22 Is compared with the size of the lowest image block (the size of a22122111).

At this time, since the size of the target image block a22 is larger than the size of the lowest image block a22122111 as a result of the comparison in step S313, the target image block a22 is reset to the anterior image block a22 through step S314 .

In this case, in step S311, the upper image block a22 is divided into two image blocks a221 and a222, and in step S312, the upper image block a22 is divided into two image blocks a221 and a222, The image block a221 including the VLC light source 23 of the front vehicle among the two image blocks a221 and a222 is determined again as the target image block a221. In step S313, the size of the target image block a221 Is compared with the size of the lowest image block (the size of a22122111).

At this time, since the size of the target image block a221 is larger than the size of the lowest image block a2122111 as a result of the comparison in step S313, the target image block a221 is reset to the anterior image block a221 through step S314 .

In this case, in step S311, the upper image block a221 is divided into two image blocks a2211 and a2212. In step S312, the upper image block a221 is divided into two image blocks a2211 and a2212, The image block a2212 including the VLC light source 23 of the front vehicle among the two image blocks a2211 and a2212 is determined again as the target image block a2212. In step S313, the size of the target image block a2212 Is compared with the size of the lowest image block (the size of a22122111).

At this time, since the size of the target image block a2212 is larger than the size of the lowest image block a22122111 as a result of the comparison in step S313, the target image block a2212 is reset to the anterior image block a2212 through step S314 .

When the upper image block a2212 is set, the fifth iteration of the target image block selection cycle is started. In this case, the upper image block a2212 is divided into two image blocks a22121 and a22122 in step S311, The image block a22122 including the VLC light source 23 of the front vehicle among the two image blocks a22121 and a22122 is determined again as the target image block a22122. In step S313, the size of the target image block a22122 Is compared with the size of the lowest image block (the size of a22122111).

At this time, since the size of the target image block a22122 is larger than the size of the lowest image block a22122111 as a result of the comparison in step S313, the target image block a22122 is reset to the anterior image block a22122 through step S314 .

In this case, the upper image block a22122 is divided into two image blocks a221221 and a221222 in step S311, and in step S312, The image block a221221 including the VLC light source 23 of the front vehicle among the two image blocks a221221 and a221222 is determined again as the target image block a221221. In step S313, the size of the target image block a221221 Is compared with the size of the lowest image block (the size of a22122111).

At this time, since the size of the target image block a221221 is larger than the size of the lowest image block (the size of a22122111) as a result of the comparison in step S313, the target image block a221221 is reset to the anterior image block a221221 through step S314 .

If the upper image block a221221 is set, the seventh iteration of the target image block selection cycle starts. In this case, the upper image block a221221 is divided into two image blocks a2212211 and a2212212 in step S311, The image block a2212211 including the VLC light source 23 of the front vehicle among the two image blocks a2212211 and a2212212 is determined again as the target image block a2212211 and in step S313 the size of the target image block a2212211 Is compared with the size of the lowest image block (the size of a22122111).

At this time, since the size of the target image block a2212211 is larger than the size of the lowest image block (the size of a22122111) as a result of the comparison in step S313, the target image block a2212211 is reset to the anterior image block a2212211 through step S314 .

In this case, the upper image block a2212211 is divided into two image blocks a22122111 and a22122112 in step S311, and in step S312, The image block a22122111 including the VLC light source 23 of the front vehicle among the two image blocks a22122111 and a22122112 is determined again as the target image block a22122111 and in step S313 the size of the target image block a22122111 Is compared with the size of the lowest image block (the size of a22122111).

At this time, since the size of the target image block a22122111 is equal to the size of the lowest image block (the size of a22122111) as a result of the comparison in step S313, the target image block selection cycle described above is terminated, May be the output value of the target image block selection cycle.

Meanwhile, a plurality of VLC light sources may be included in the plurality of forward images received through the plurality of cameras 11 and 12 in step S310.

This is because the forward image may include an image of a plurality of forward vehicles such as an image of another forward vehicle of the adjacent lane, and calculating the three-dimensional coordinates of the VLC light source 23 of the forward vehicle through steps S320 and S330 The two-dimensional image coordinates of the VLC light source 23 of the preceding vehicle extracted from the plurality of forward images received by each of the plurality of cameras 11 and 12 should be the image coordinates for the same vehicle.

To this end, the VLC light source 23 of the front vehicle can transmit its own identifier through the visible light communication method, and the object image block selection unit 210 according to the embodiment of the present invention includes the object image block selection unit 210, A VLC light source having the same identifier among a plurality of VLC light sources can be traced to select target image blocks.

A method of extracting an identifier through a VLC transmission frame transmitted by the VLC light sources 13 and 23 will now be described with reference to FIG.

7 is a diagram for explaining a structure of a signal transmitted by a VLC light source of a forward vehicle in a method and apparatus for three-dimensional coordinate calculation of a VLC light source according to an embodiment of the present invention.

7, the VLC transmission frame transmitted by the VLC light sources 13 and 23 may include a preamble, an identifier, and vehicle dynamics information. The VLC light source 13 and the VLC light source 13, May comprise a plurality of VLC transmission frames.

At this time, the vehicle dynamics information may include at least acceleration information and angular velocity information of the preceding vehicle, and the vehicle dynamics information may be utilized by the movement locus calculating unit 240 to generate a locus of movement of the forward vehicle. But is not limited thereto.

That is, when the target image block selecting unit 210 detects the preamble among the VLC transmission frames, the target image block selecting unit 210 can determine the subsequent signal as an identifier.

At this time, the target image block selection unit 210 may include a correlator (not shown) for detecting the preamble.

If the preamble signal in the sample t is a (t), then the correlator (not shown) is identical to the lowest video block described above It is possible to calculate the determination signal x (t) with respect to the signal value s _m (t) received in the target image block m of the size.

The above-described operation of the correlator can be expressed by the following equation (2).

&Quot; (2) "

(T) is a decision signal, t is a sample, T is a length of a preamble among VLC transmission frames, a (t) is a preamble signal, and s _m Lt; / RTI >

At this time, a correlator (not shown) detects a preamble by detecting a peak in the decision signal x (t), and through this, an identifier can be determined. The identifier includes OOK, OFDM, PPM used in VLC communication By receiving it, a correlator (not shown) can detect the unique identifier of the VLC light source 23 of the preceding vehicle.

At this time, a more specific operation in which the correlator (not shown) detects the peak and detects the preamble and detects the identifier is already known, and a detailed description thereof will be omitted.

In order to determine the target image block, which is the image block including the VLC light source 23 of the preceding vehicle, among the plurality of image blocks in the above-described S312, the position of the VLC light source 23 is tracked It is important to do.

For example, in order to determine the image block a2 including the VLC light source 23 among the two image blocks a1 and a2 shown in Fig. 6 as a target image block a2, two image blocks a1 , a2) of the VLC light source (23).

Hereinafter, with reference to FIG. 5, a method of determining a target image block, which is an image block including the VLC light source 23 of the preceding vehicle, among a plurality of image blocks will be described.

5, in operation S312, a sum of intensity values of a plurality of pixels included in each of the plurality of image blocks is calculated for each image frame (S510), and a Fourier transform is applied to the sum of image frames A step S520 of generating a waveform and a step S530 of selecting an image block corresponding to a frequency domain waveform including a frequency component equal to or higher than a preset threshold value as a target image block in the frequency domain waveform.

In operation S510, the sum of the intensity values of the plurality of pixels included in each of the plurality of image blocks may be calculated for a predetermined time.

At this time, in step S510, the target image block selection unit 210 determines a target image block, which is an image block including the VLC light source 23 of the preceding vehicle, based on the intensity of each of the plurality of pixels included in each of the plurality of image blocks The sum of the values of the image frames can be calculated.

Here, the meaning of the intensity value of the pixel may be various values indicating the properties of the pixel including the weight sum of R, G and B colors according to the color of the VLC light source 23 of the front vehicle, grayscale, And the present invention is not limited to a particular pixel intensity value.

For example, in a particular image frame t, the first image block includes a total of five pixels having intensity values of 3, 9, 10, 5, and 7, and the second image block includes 2, 1, 6, , And a total of five pixels having intensity values of " 3 " and " 3 ".

In the above example, the sum of the pixel intensity values of the first image block in a particular image frame t with respect to the image frame t may be expressed as 3 + 9 + 10 + 5 + 7 = 34, The sum of the pixel intensity values for the image frame t may be expressed as 2 + 1 + 6 + 9 + 3 = 21.

In this case, the above-described step S510 may be expressed by the following equation (3).

&Quot; (3) "

At this time, m is the image blocks, t is the image frame, s _m (t) the sum of the pixel intensity values contained in the image block m in the image frame t, (x, y) are the pixel coordinates contained in the video block m, B _m Is the set of all pixels included in the image block m and P _{x, y} (t) is the intensity value of the (x, y) pixel in the image frame t.

When the above-described expression (3) is repeatedly calculated for the predetermined time T, the above-described step S510 is terminated.

In operation S520, a plurality of frequency domain waveforms corresponding to each of the plurality of image blocks may be generated by applying a predetermined Fourier transform to the total of the image frames calculated over a predetermined period of time.

In this case, the predetermined Fourier transform used in step S520 may mean Fast Fourier Transform, but the Fourier transform used in the present invention is not limited to the fast Fourier transform.

For example, if the sum s _m (t) of the pixel intensity values included in the image block m in the image frame t for the image block m calculated in step S510 is calculated for a preset time T, S520 The frequency domain waveform generated as a result of the fast Fourier transform is defined as FFT _m (f, T).

At this time, a specific method of generating the frequency domain waveform by applying the fast Fourier transform to the waveform in the time domain is well known in the art, and a detailed description thereof will be omitted.

In operation S530, the image block corresponding to the frequency domain waveform including a frequency component equal to or higher than a predetermined threshold value among the plurality of frequency domain waveforms may be selected as the target image block.

) 이상의 주파수 성분을 가지는 주파수 영역 파형에 대응되는 영상 블록을 상술한 대상 영상 블록으로 결정할 수 있다.For example, in step S530, the target image block selection unit 210 selects a target image block among a plurality of frequency domain waveforms calculated for each of the plurality of image blocks,

) Or more can be determined as the above-mentioned object image block.

The above process can be expressed by the following equation (4).

&Quot; (4) "

는 미리 설정된 임계값을 의미한다.At this time, FFT _m (f, T) is a frequency domain waveform for the image block m,

Quot; means a preset threshold value.

That is, in step S530, the target image block selection unit 210 may determine an image block satisfying the condition of Equation (4) as a target image block.

)는 VLC 광원(23)이 포함된 영상의 주파수 영역 특징을 결정하기 위한 임계값으로 실험적으로 결정될 수 있으며, 본 발명은 특정 임계값으로 한정되지 않는다.At this time, a preset threshold value for determining a target image block (

May be experimentally determined as a threshold value for determining the frequency domain characteristic of the image including the VLC light source 23, and the present invention is not limited to a specific threshold value.

Meanwhile, in step S530, the target image block selection unit 210 may increase the accuracy of the target image block determination by comparing each of a plurality of frequency components included in the frequency domain waveform with preset threshold values, but the present invention is not limited thereto .

Referring back to FIG. 2, step S320 will be described.

In step S320, the two-dimensional image coordinate calculation unit 220 calculates the two-dimensional image coordinates of the VLC light source 23 of the preceding vehicle on each of the plurality of forward images based on the target image block included in each of the plurality of forward images . ≪ / RTI >

At this time, in step S320, the two-dimensional image coordinates calculation unit 220 calculates the two-dimensional image coordinates of the VLC light source 23 of the preceding vehicle on each of the plurality of forward images based on the target image block included in the plurality of forward images The two-dimensional image coordinates of the VLC light source 23 of the front vehicle in each of the plurality of forward images can be calculated more quickly.

For example, if the forward image includes a total of 10000 pixels from (0, 0) to (100, 100), and the target image block includes (0,0), (10,0) If the VLC light source 23 of the front vehicle is located at (5, 5) pixels, the two-dimensional image coordinate calculation unit 220 at step S320 includes a total of 10000 pixels Instead of calculating the position of the VLC light source 23 of the forward vehicle in the pixel, the position of the VLC light source 23 is calculated only in a total of 100 pixels included in the target image block, and based on the relationship between the target image block and the forward image The two-dimensional image coordinates of the VLC light source 23 in the forward image can be calculated, so that the two-dimensional image coordinates can be calculated more quickly.

According to one embodiment, in step S320, the two-dimensional image coordinate calculator 220 calculates the two-dimensional image coordinate of the VLC light source 23 on the object image block by using a known method of finding the coordinates of the light source in the two- Various algorithms may be applied, but the present invention is not limited thereto.

In this case, in step S320, the two-dimensional image coordinate calculator 220 calculates the two-dimensional image coordinates of the VLC light source 23 of the front vehicle 23) in the target image block and reset the image block adjacent to the area outside the target image block of the VLC light source 23 of the preceding vehicle to the target image block.

On the other hand, when the movement of the VLC light source 23 of the forward vehicle is missed in the case where the VLC light source 23 of the forward vehicle moves out of the target image block determined by the movement of the VLC light source 23, S510, S520, and S530 may be applied to re-search the target image block in which the VLC light source 23 of the preceding vehicle exists.

Meanwhile, the method and apparatus for calculating three-dimensional coordinates of a VLC light source according to an embodiment of the present invention may periodically search for a forward image to find a VLC light source of another vehicle while receiving the VLC signal from the VLC light source 23 of the preceding vehicle In this case, multiple VLC signals may be simultaneously decoded using multiple windowing.

According to one embodiment, in step S320, the two-dimensional image coordinate calculator 220 calculates the two-dimensional image coordinates of the VLC light source 23 of the preceding vehicle in the forward image collected from the first camera 11, 2 positional relationship between the first camera 11 and the second camera 12 in order to calculate the two-dimensional image coordinates of the VLC light source 23 of the front vehicle in the forward image collected from the camera 12 .

For example, the two-dimensional image coordinate calculation unit 220 can previously store the layout configuration in which the two cameras 11 and 12 are aligned and the positional relationship between the two cameras 11 and 12, A translation vector t representing a displacement between the cameras 11 and 12 and a rotation matrix R representing a posture difference between the two cameras 11 and 12. [

At this time, a more detailed description of the motion vector t and the rotation transformation matrix R is omitted since it is known in a conventional stereo camera system.

An essential matrix E that can be calculated through the camera calibration between the two cameras 11 and 12 is calculated as E = R [t] x using the motion vector t and the rotation transformation matrix R Where [t] x denotes a matrix representing an outward product to a vector t.

If the two-dimensional image coordinates on the forward image of the VLC light source 23 of the forward vehicle detected by each of the two cameras 11 and 12 are x 'and x, respectively, the relationship of x'Ex = 0 for the essential matrix .

At this time, if the first camera 11 detects the VLC light source 23 of the preceding vehicle and knows the two-dimensional image coordinate x, x 'can be defined as a line by the above-described relational expression.

In step S320, the two-dimensional image coordinate calculator 220 calculates the two-dimensional coordinate of the VLC light source 23 of the front vehicle detected by the first camera 11, The second camera 12 detects a linear region in which the VLC light source 23 of the vehicle can appear and uses the linear region to calculate the two-dimensional image coordinates of the VLC light source 23 of the front vehicle detected by the second camera 12. [ Dimensional coordinate of the VLC light source 23 of the preceding vehicle on the forward image collected by the VLC light source 23, but the present invention is not limited thereto.

With continued reference now to Figures 3 and 8, step S330 will be described.

In step S330, the three-dimensional coordinate calculation unit 230 calculates the three-dimensional coordinates of the VLC light source of the preceding vehicle based on the two-dimensional coordinate of the VLC light source 23 of the preceding vehicle and the positional relationship between the plurality of cameras It can mean a step.

As shown in FIG. 8, by using the two-dimensional image coordinates of the VLC light source 23 of the front vehicle with respect to each of the two cameras, depth (z) information of the VLC light source 23 of the preceding vehicle is generated .

은 좌측 카메라의 렌즈의 중심축으로부터 전방 차량의 VLC 광원(23)의 상이 맺히는 거리,

은 우측 카메라의 렌즈의 중심축으로부터 전방 차량의 VLC 광원(23)의 상이 맺히는 거리를 의미한다.For example, as shown in FIG. 8, the VLC light source coordinates are (x, y, z), b is the distance between the lens axes of the two cameras 11 and 12, C _l is the center of the left camera lens, C _r is the center of the right camera lens, f is the focal length,

The distance of the image of the VLC light source 23 of the front vehicle from the center axis of the lens of the left camera,

Means a distance at which the image of the VLC light source 23 of the front vehicle is formed from the center axis of the lens of the right camera.

In this case, z means the depth of the VLC light source 23 of the front vehicle.

At this time, if the similarity ratio of the triangle is used, the following equations (5) and (6) can be determined.

&Quot; (5) "

&Quot; (6) "

Here, z, which is the depth information, is calculated using Equations (5) and (6).

&Quot; (7) "

As described above, the two-dimensional image coordinates of the VLC light source 23 of the front vehicle on the front image collected from the first camera 11 and the VLC light source 23 of the front vehicle on the front image collected from the second camera 12 The three-dimensional coordinates of the VLC light source 23 of the front vehicle can be calculated.

), 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표(

), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix) 중 적어도 하나에 기초하여 산출될 수 있다.For example, when the plurality of cameras are two cameras 11 and 12 in step S330, the three-dimensional coordinates of the VLC light source 23 of the front vehicle are obtained by using the VLC Two-dimensional image coordinates of the light source (

), The two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera

A translation vector determined based on a positional relationship between the first camera and the second camera, a rotation matrix determined based on a positional relationship between the first camera and the second camera, Can be calculated.

More specifically, the three-dimensional coordinates of the VLC light source of the front vehicle can be calculated based on the following equation (1).

[Equation 1]

는 전방 차량의 VLC 광원의 3차원 좌표,

는 제1 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표,

는 제2 카메라를 통해 수집된 전방 영상에서 전방 차량의 VLC 광원의 2차원 영상 좌표, x는

, t는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 이동 벡터(translation vector), r_k는 제1 카메라 및 제2 카메라 상호간의 위치관계에 기초하여 결정된 회전변환행렬(rotation matrix)의 k번째 행을 의미하고,

은 내적을 의미한다.At this time,

Dimensional coordinate of the VLC light source of the front vehicle,

Dimensional image coordinate of the VLC light source of the front vehicle in the forward image collected through the first camera,

Dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera, x

, t is a translation vector determined based on a positional relationship between the first camera and the second camera, r _k is a rotation matrix determined based on a positional relationship between the first camera and the second camera, ≪ / RTI >< RTI ID =

Is the inner product.

Although not shown in the drawing, the method for calculating three-dimensional coordinates of a VLC light source according to the embodiment of the present invention is characterized in that, after step S330, the movement locus calculating unit 240 calculates a three-dimensional coordinate of the VLC light source 23 of the front vehicle (Not shown) of generating a movement trajectory of the preceding vehicle based on the coordinate, the acceleration information of the preceding vehicle received from the VLC light source 23 of the preceding vehicle, and the angular velocity information of the preceding vehicle.

According to one embodiment, the step (not shown) of generating the movement trajectory of the front vehicle further includes, based on the velocity information and the rotation angle information of the front vehicle received from the VLC light source 23 of the front vehicle, But the present invention is not limited thereto.

For example, in a step (not shown) of generating a movement trajectory of the front vehicle, the movement locus calculating unit 240 calculates the movement locus of the front vehicle based on the three-dimensional coordinates of the VLC light source 23 of the front vehicle and the kinematics information of the front vehicle The movement trajectory of the forward vehicle can be calculated through the equation of motion, but the present invention is not limited thereto.

For example, in a step (not shown) of generating a movement trajectory of the front vehicle, the movement locus calculating unit 240 calculates three-dimensional coordinates of the VLC light source of the preceding vehicle, acceleration information of the preceding vehicle, and angular velocity information of the preceding vehicle Therefore, it is possible to calculate the trajectory of the movement of the preceding vehicle, thereby preventing the risk of collision with the preceding vehicle in advance.

In other words, when calculating the three-dimensional coordinate, which is a relative position with respect to the preceding vehicle, the moving-locus calculating unit 240 combines the three-dimensional coordinates with the dynamic information such as the acceleration and angular velocity received through VLC communication to track the trajectory of the preceding vehicle .

Here, the acceleration information of the front vehicle may refer to information extracted from the accelerometer included in the front vehicle, the Excel / brake of the front vehicle, and transmitted through the VLC light source 23, and the angular velocity information of the front vehicle may be included in the front vehicle A gyroscope that is extracted from the steering wheel of the front vehicle, and transmitted through the VLC light source 23.

At this time, the movement locus calculating unit 240 performs sensor fusion such as an Extended Kalman filter or an Unscented Kalman filter to combine the above-described various kinds of dynamic information with three-dimensional coordinates. Technology, but the present invention is not limited thereto.

In other words, when the three-dimensional coordinates of the VLC light source 23 of the front vehicle are calculated through the step of generating the movement locus of the front vehicle (not shown) (S330), the movement locus calculating unit 240 calculates the three- It is possible to prevent the collision with the forward vehicle by calculating the position and the movement locus of the front vehicle on which the VLC light source 23 is mounted.

At this time, in order to prevent collision with the front vehicle, it is necessary to know the movement information of the front vehicle including the velocity, the attitude, the acceleration, and the angular velocity as well as the three-dimensional coordinates of the VLC light source 23 of the front vehicle indicating the relative position of the front vehicle.

Furthermore, since information such as speed, attitude, acceleration, and angular velocity of the receiving vehicle system 10 including the apparatus 200 for calculating the three-dimensional coordinate of the VLC light source according to the embodiment of the present invention is also necessary for collision detection, The system 10 may also be equipped with a sensor for sensing such information.

At this time, the front vehicle performs at least one of the identification information, the vehicle acceleration (the value obtained from the accelerometer and the brake, the excel, etc.), the angular velocity (the value obtained from the gyroscope and the steering wheel) So that the surrounding receiving vehicle can obtain its own dynamic information.

On the other hand, the movement locus calculating section 240 included in the receiving vehicle system 10 calculates three-dimensional coordinates of the VLC light source 23 of the front vehicle calculated by the three-dimensional coordinate calculating section 230, The mechanical information can be combined to determine the precise relative position, speed, and acceleration of the vehicle ahead.

Now, one embodiment of the operation of the movement locus calculating section 240 in the step (not shown) for generating the locus of movement of the front vehicle will be described with reference to FIG.

FIG. 9 is a diagram for explaining a method for calculating a movement locus of a front vehicle in a method and apparatus for three-dimensional coordinate calculation of a VLC light source according to an embodiment of the present invention.

9, if the coordinate system is simplified to two dimensions, the three-dimensional coordinate calculation device 200 of the VLC light source according to the embodiment of the present invention, with the north (X axis) and the east (Y axis) The vehicle 10 equipped with the apparatus 10 for calculating the three-dimensional coordinates of the VLC light source according to the embodiment of the present invention can be considered to have a coordinate system in parallel with the inertial coordinate system, Dimensional coordinate calculation device 200 of the VLC light source according to an embodiment of the present invention for convenience of explanation, the condition at the measuring point i can be measured as follows. The vehicle 10 equipped with the vehicle 10 will be described as a receiving vehicle and the parameters shown in Fig. 9 will be described.

는 수신 차량의 회전각,

는 수신 차량의 속도,

는 수신 차량의 가속도,

는 수신 차량의 각속도,

는 전방 차량의 회전각,

는 전방 차량의 속도,

는 전방 차량의 가속도,

는 전방 차량의 각속도,

는 S330 단계를 통해 산출된 전방 차량의 VLC 광원(23)의 3차원 좌표를 의미한다.9

The rotation angle of the receiving vehicle,

The speed of the receiving vehicle,

The acceleration of the receiving vehicle,

The angular velocity of the receiving vehicle,

The rotational angle of the front vehicle,

The speed of the front vehicle,

The acceleration of the front vehicle,

The angular velocity of the front vehicle,

Dimensional coordinate of the VLC light source 23 of the front vehicle calculated through step S330.

Hereinafter, variables for explaining Equations (8) to (23) are defined as follows.

는 측정시점 i에서의 수신 차량의 가속도 측정값,

는 측정시점 i에서의 수신 차량의 각속도 측정값,

는 측정시점 i에서의 전방 차량의 가속도 측정값,

는 측정시점 i에서의 전방 차량의 각속도 측정값,

는 측정시점 i에서의 수신 차량의 카메라 좌표 기준 전방 차량의 상대위치 측정값을 의미한다.

The acceleration measurement value of the receiving vehicle at the measurement time i,

The angular velocity measurement value of the receiving vehicle at the measurement point i,

Is an acceleration measurement value of the preceding vehicle at the measurement point i,

Is an angular velocity measurement value of the front vehicle at the measurement point i,

Means the relative position measurement value of the preceding vehicle based on the camera coordinates of the receiving vehicle at the measurement time i.

For example, the change in the rotational angle of the receiving vehicle can be calculated by Equation (8) below.

&Quot; (8) "

는 시점 i+1에서 수신 차량의 회전각, T는 측정 주기를 의미한다.At this time,

Denotes the rotation angle of the receiving vehicle at time i + 1, and T denotes the measurement period.

For example, the rotational angle change of the front vehicle can be calculated by the following equation (9).

&Quot; (9) "

는 시점 i+1에서 전방 차량의 회전각, T는 측정 주기를 의미한다.At this time,

Denotes the rotation angle of the front vehicle at time i + 1, and T denotes the measurement period.

According to one embodiment, the change in the speed of the receiving vehicle can be calculated as: < EMI ID = 10.0 >

&Quot; (10) "

는 시점 i+1에서 수신 차량의 속도, T는 측정 주기를 의미한다.At this time,

Is the speed of the receiving vehicle at time i + 1, and T is the measurement period.

For example, the change of the speed of the front vehicle can be calculated as shown in Equation (11) below.

&Quot; (11) "

는 시점 i+1에서 전방 차량의 속도, T는 측정 주기를 의미한다.At this time,

Is the speed of the front vehicle at time i + 1, and T is the measurement period.

For example, the change in the acceleration of the receiving vehicle can be calculated by Equation (12) below.

&Quot; (12) "

는 시점 i+1에서 수신 차량의 가속도,

는 미리 설정된 가속도 가중치,

는 가속도의 상태 변화를 나타내는 확률 변수를 의미한다.At this time,

Is the acceleration of the receiving vehicle at time i + 1,

A predetermined acceleration weight,

Is a random variable representing the state change of acceleration.

On the other hand, the change of the acceleration of the front vehicle can be calculated by Equation (13) below.

&Quot; (13) "

는 시점 i+1에서 전방 차량의 가속도,

는 미리 설정된 가속도 가중치,

는 가속도의 상태 변화를 나타내는 확률 변수를 의미한다.At this time,

Is the acceleration of the forward vehicle at time i + 1,

A predetermined acceleration weight,

Is a random variable representing the state change of acceleration.

For example, the change of the angular velocity of the receiving vehicle can be calculated by Equation (14) below.

&Quot; (14) "

는 시점 i+1에서 수신 차량의 각속도,

는 미리 설정된 각속도 가중치,

는 각속도의 상태 변화를 나타내는 확률 변수를 의미한다.At this time,

Is the angular velocity of the receiving vehicle at time i + 1,

A predetermined angular velocity weight,

Means a random variable representing the state change of the angular velocity.

For example, the angular velocity change of the front vehicle can be calculated by the following equation (15).

&Quot; (15) "

는 시점 i+1에서 전방 차량의 각속도,

는 미리 설정된 각속도 가중치,

는 각속도의 상태 변화를 나타내는 확률 변수를 의미한다.At this time,

Is the angular velocity of the forward vehicle at time i + 1,

A predetermined angular velocity weight,

Means a random variable representing the state change of the angular velocity.

For example, the X coordinate change of the front vehicle indicating the movement trajectory of the front vehicle can be calculated as shown in Equation (16) below.

&Quot; (16) "

는 시점 i+1에서 전방 차량의 VLC 광원(23)의 X좌표,

는 시점 i에 대하여 산출된 전방 차량의 VLC 광원(23)의 X좌표,

는 시점 i에서의 전방 차량의 속도,

는 시점 i에서의 전방 차량의 회전각,

는 시점 i에서의 수신 차량의 속도,

는 시점 i에서의 수신 차량의 회전각을 의미한다.At this time,

Is the X coordinate of the VLC light source 23 of the front vehicle at the time i + 1,

The X coordinate of the VLC light source 23 of the front vehicle calculated for the time i,

The speed of the front vehicle at time i,

The rotational angle of the front vehicle at the time point i,

The speed of the receiving vehicle at time i,

Means the rotation angle of the receiving vehicle at time i.

For example, the Y coordinate change of the front vehicle indicating the movement trajectory of the front vehicle can be calculated as shown in Equation (17) below.

&Quot; (17) "

는 시점 i+1에서 전방 차량의 VLC 광원(23)의 Y좌표,

는 시점 i에 대하여 산출된 전방 차량의 VLC 광원(23)의 Y좌표,

는 시점 i에서의 전방 차량의 속도,

는 시점 i에서의 전방 차량의 회전각,

는 시점 i에서의 수신 차량의 속도,

는 시점 i에서의 수신 차량의 회전각을 의미한다.At this time,

Is the Y coordinate of the VLC light source 23 of the front vehicle at the time point i + 1,

The Y coordinate of the VLC light source 23 of the front vehicle calculated for the time i,

The speed of the front vehicle at time i,

The rotational angle of the front vehicle at the time point i,

The speed of the receiving vehicle at time i,

Means the rotation angle of the receiving vehicle at time i.

On the other hand, the measurement value of the angular velocity of the receiving vehicle can be calculated by the following equation (18).

&Quot; (18) "

는 수신 차량의 각속도의 측정 오류를 나타내는 확률 변수를 의미한다.At this time,

Is a random variable representing a measurement error of the angular velocity of the receiving vehicle.

On the other hand, the angular velocity measurement value of the front vehicle can be calculated as shown in Equation 19 below.

&Quot; (19) "

는 전방 차량의 각속도의 측정 오류를 나타내는 확률 변수를 의미한다.At this time,

Means a random variable representing a measurement error of the angular velocity of the front vehicle.

For example, the acceleration measurement value of the receiving vehicle can be calculated by Equation (20) below.

&Quot; (20) "

는 수신 차량의 가속도 측정 오류를 나타내는 확률 변수를 의미한다.At this time,

Denotes a random variable representing an acceleration measurement error of the receiving vehicle.

For example, the acceleration measurement value of the front vehicle can be calculated as shown in the following equation (21).

&Quot; (21) "

는 전방 차량의 가속도 측정 오류를 나타내는 확률 변수를 의미한다.At this time,

Means a random variable representing an acceleration measurement error of the preceding vehicle.

On the other hand, the measured value of the distance of the forward vehicle with respect to the camera coordinate of the receiving vehicle at the measuring point i can be calculated by Equation (22) below.

&Quot; (22) "

는 측정시점 i에서의 수신 차량의 카메라 좌표 기준 전방 차량의 거리의 측정값,

는 전방 차량의 거리의 측정값의 측정 오류를 나타내는 확률 변수를 의미한다.At this time,

Is a measured value of the distance of the preceding vehicle with respect to the camera coordinate of the receiving vehicle at the measuring point i,

Means a random variable representing a measurement error of the measured value of the distance of the front vehicle.

On the other hand, the measured value of the displacement of the front vehicle based on the camera coordinates of the receiving vehicle at the measuring point i can be calculated by the following equation (23).

&Quot; (23) "

는 측정시점 i에서의 수신 차량의 카메라 좌표 기준 전방 차량의 변위의 측정값,

는 전방 차량의 변위의 측정값의 측정 오류를 나타내는 확률 변수를 의미한다.At this time,

Is a measurement value of the displacement of the front vehicle based on the camera coordinates of the receiving vehicle at the measuring time i,

Means a random variable representing the measurement error of the measured value of the displacement of the front vehicle.

According to one embodiment, in the step of generating the movement trajectory of the front vehicle (not shown), the movement trajectory calculating unit 240 calculates the trajectory of the front vehicle based on the Extended Kalman filter, The movement locus of the front vehicle can be calculated by applying various known techniques including the Unscented Kalman filter and the like, and the result can be as shown in FIG. 10. However, the present invention is not limited thereto Do not.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but many variations and modifications may be made without departing from the scope of the present invention. It will be understood that the invention may be practiced.

10: vehicle system 11: first camera
12: second camera 13: VLC light source
23: VLC light source
200: Three-dimensional coordinate calculation device of VLC light source
210: target image block selection unit
220: two-dimensional image coordinate calculation unit
230: Three-dimensional coordinate calculation unit
240: movement locus calculating section

Claims (16)

The target image block selection unit selects a target image block including the VLC light source of the forward vehicle among the plurality of image blocks obtained by dividing the plurality of forward images collected so as to correspond to each of the plurality of cameras to each of the plurality of forward images step;
Calculating a two-dimensional image coordinate of the VLC light source of the preceding vehicle based on the target image block included in each of the plurality of forward images, for each of the plurality of forward images; And
Dimensional coordinate of the VLC light source of the preceding vehicle based on the two-dimensional coordinate of the VLC light source of the preceding vehicle and the positional relationship between the plurality of cameras,
Wherein the step of selecting the target image block for each of the plurality of forward images comprises:
Dividing each of the plurality of forward images to determine the plurality of image blocks;
Calculating a sum of intensity values of each of a plurality of pixels included in each of the plurality of image blocks for each image frame for a predetermined time;
Generating a plurality of frequency domain waveforms corresponding to each of the plurality of image blocks by applying a predetermined Fourier transform to a total of the image frames calculated during the predetermined period of time; And
And selecting an image block corresponding to a frequency domain waveform including a frequency component equal to or higher than a predetermined threshold value among the plurality of frequency domain waveforms as the object image block. delete The method according to claim 1,
After selecting the target image block,
And comparing the size of the target image block with a size of a lowest-order image block set in advance. The method of claim 3,
If the size of the target image block is larger than the size of the preset lowest image block,
Further comprising setting the target image block as an upper image block,
The target image block selection cycle including the steps of determining the plurality of image blocks, selecting the target image block, and comparing the target image block is performed on the upper image block,
Wherein the target image block selection cycle is repeatedly performed until the size of the target image block becomes equal to the size of the preset lowest image block. delete The method according to claim 1,
When the plurality of cameras are two cameras,
Dimensional coordinate of the VLC light source of the front vehicle,
Dimensional image coordinates of the VLC light source of the front vehicle in the forward image collected through the first camera

), The two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera

A translation vector determined based on a positional relationship between the first camera and the second camera, a rotation matrix determined based on a positional relationship between the first camera and the second camera, Dimensional coordinate of a VLC light source calculated on the basis of the three-dimensional coordinate of the VLC light source. The method according to claim 1,
When the plurality of cameras are two cameras,
Wherein the three-dimensional coordinate of the VLC light source of the front vehicle is calculated based on the following equation (1).
[Equation 1]

At this time,

Dimensional coordinate of the VLC light source of the front vehicle,

Dimensional image coordinate of the VLC light source of the front vehicle in the forward image collected through the first camera,

Dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera, x

, t is a translation vector determined based on a positional relationship between the first camera and the second camera, r _k is a rotation matrix determined based on a positional relationship between the first camera and the second camera, ≪ / RTI >< RTI ID =

Means inner product. The method according to claim 1,
The movement locus calculating section generates a movement locus of the preceding vehicle based on the three-dimensional coordinates of the VLC light source of the preceding vehicle, the acceleration information of the preceding vehicle received from the VLC light source of the preceding vehicle, and the angular velocity information of the preceding vehicle Dimensional coordinate of the VLC light source. A target image block selector for selecting a target image block including a VLC light source of a forward vehicle among a plurality of image blocks obtained by dividing a plurality of forward images collected to correspond to each of the plurality of cameras, for each of the plurality of forward images;
A two-dimensional image coordinate calculator for calculating a two-dimensional image coordinate of the VLC light source of the preceding vehicle based on the target image block included in each of the plurality of forward images, for each of the plurality of forward images; And
And a three-dimensional coordinate calculation unit for calculating three-dimensional coordinates of the VLC light source of the preceding vehicle based on the two-dimensional image coordinates of the VLC light source of the preceding vehicle and the positional relationship between the plurality of cameras,
Wherein the target image block selection unit comprises:
Determining a plurality of image blocks by dividing each of the plurality of forward images, calculating a sum of intensity values of each of a plurality of pixels included in each of the plurality of image blocks,
Generating a plurality of frequency domain waveforms corresponding to each of the plurality of image blocks by applying a predetermined Fourier transform to the sum of the image frames calculated during the predetermined time,
And selects an image block corresponding to a frequency domain waveform including a frequency component equal to or higher than a predetermined threshold value among the plurality of frequency domain waveforms as the object image block. delete 10. The method of claim 9,
After selecting the target image block,
Wherein the target image block selection unit comprises:
And compares a size of the target image block with a size of a lowest-order image block set in advance. 12. The method of claim 11,
If the size of the target image block is larger than the size of the preset lowest image block,
Wherein the target image block selection unit sets the target image block as an upper image block,
Wherein the target image block selection unit comprises: a target image block selection unit operable to select the plurality of image blocks, to select the target image block, and to compare the size of the target image block with a size of a lowest- A selection cycle is performed for the upper image block,
Wherein the target image block selection cycle is repeatedly performed until the size of the target image block becomes equal to the size of the preset lowest image block. delete 10. The method of claim 9,
When the plurality of cameras are two cameras,
Dimensional coordinate of the VLC light source of the front vehicle,
Dimensional image coordinates of the VLC light source of the front vehicle in the forward image collected through the first camera

), The two-dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera

A translation vector determined based on a positional relationship between the first camera and the second camera, a rotation matrix determined based on a positional relationship between the first camera and the second camera, Dimensional coordinates of the VLC light source. 10. The method of claim 9,
When the plurality of cameras are two cameras,
Wherein the three-dimensional coordinate of the VLC light source of the front vehicle is calculated based on the following equation (1).
[Equation 1]

At this time,

Dimensional coordinate of the VLC light source of the front vehicle,

Dimensional image coordinate of the VLC light source of the front vehicle in the forward image collected through the first camera,

Dimensional image coordinates of the VLC light source of the front vehicle in the front image collected through the second camera, x

, t is a translation vector determined based on a positional relationship between the first camera and the second camera, r _k is a rotation matrix determined based on a positional relationship between the first camera and the second camera, ≪ / RTI >< RTI ID =

Means inner product. 10. The method of claim 9,
A movement locus calculating unit that generates a locus of movement of the preceding vehicle based on three-dimensional coordinates of the VLC light source of the preceding vehicle, acceleration information of the preceding vehicle received from the VLC light source of the preceding vehicle, and angular velocity information of the preceding vehicle Further comprising: a three-dimensional coordinate calculation unit of the VLC light source.

Images