KR102347767B1 - Method and apparatus for determining road shape based on optical camera communication - Google Patents

Abstract

Disclosed are a method and a device for determining a road shape based on an optical camera communication (OCC). According to an embodiment of the present disclosure, the method for determining a road shape based on an OCC, comprises the steps of: receiving interval information between a plurality of taillights of a front vehicle and size information on the taillights using a single camera provided in a vehicle; obtaining a plurality of taillight images of the front vehicle through the single camera, and setting a plurality of taillight coordinates of the front vehicle in the obtained taillight images; and calculating a road bending angle using a preset mathematical algorithm on the basis of the received interval information between the taillights of the front vehicle and size information on the taillights, and the taillight coordinates of the front vehicle. Therefore, reliability and safety of a product can be improved.

Description

Optical camera communication (OCC) based road shape determination method and apparatus

The present disclosure uses a single camera that shoots the vehicle in front as an optical camera communication (OPTICAL CAMERA COMMUNICATION, OCC)-based receiver, based on data received from a plurality of tail lights of the vehicle in front to calculate the bending angle of the road. It relates to a method and apparatus for determining a road type that enables

In general, wireless communication is advantageous in terms of cost, practicality and operability, but a bottleneck occurs. In addition, although the RF (Radio Frequency) band below the 10 GHz frequency part has been widely used for wireless communication, it does not meet the capacity and speed required by the bandwidth, and several technologies use the same frequency band (Wi-Fi, Bluetooth, mobile communication network). , and wireless phones), research to solve the bottleneck problem that occurs in wireless communication has been actively conducted.

Accordingly, in recent years, many wireless applications based on RF technology are being replaced by optical wireless technology. Among them, Visible Light Communication (VLC) is an already established technology, which uses the visible light spectrum (400-790THz), provides high data rates, and uses modulated optical signals of LEDs and LDs. .

On the other hand, Optical Camera Communication (OCC) technology is a technology that is being developed and standardized. It is one of the new promising technologies that is part of the Optical Wireless Communication (OWC) family. Here, the OWC technology uses an unlicensed spectrum as a communication medium to solve the bottleneck, so there is no need for a frequency use license, and it has the characteristics of easy security maintenance due to the straightness of radio waves and low object permeability.

In optical camera communication (OCC), light coming from different angles can be projected at various locations on the image sensor plane. In other words, optical camera communication (OCC) is an ideal technology for spatial division multiplexing and MIMO (Multiple Input and Multiple-Output) system imaging because it can separate light from different sources and directions through these features.

Such optical camera communication (OCC) in particular is one of the most promising candidates for vehicle communication. For example, for optical camera communication (OCC), a lighting lamp such as a tail lamp of a vehicle may be used as a transmitter, and a camera provided in the vehicle may be used as a receiver. That is, the vehicle's front and rear LEDs can be used as an optical camera communication (OCC) transmitter that can transmit vehicle status or other information to other vehicles, and the camera receiver receives data from images captured by the vehicle's front and rear LEDs. It can be decoded to receive data.

The optical camera communication (OCC)-based vehicle communication as described above is also disclosed in Prior Art 1. However, in the conventional optical camera communication (OCC)-based vehicle communication technology such as Prior Art 1, when data is received from a vehicle in front, two cameras provided in the vehicle are used, or a camera and a radar provided in the vehicle are used. It predicts the position of the vehicle ahead.

Also, prior art 1 does not disclose determining a road shape such as a road bending angle during driving, which is another important problem for safe driving. If the view of the host driver is obscured by the driving vehicle or the road curved forward cannot be seen due to driving at night, a major problem may occur while driving. Accordingly, there is a need for a method capable of ensuring safety by allowing the user to grasp the shape of the road ahead through a more efficient and accurate method.

The above-mentioned background art is technical information that the inventor possessed for the derivation of the present invention or acquired in the process of derivation of the present invention, and cannot necessarily be said to be a known technique disclosed to the general public prior to the filing of the present invention.

Prior Art 1: Korean Patent Publication No. 10-1689252 (Registered on Dec. 19, 2016)

One problem of the embodiment of the present disclosure is to calculate the road bending angle for the following vehicle using an OCC system that uses a single camera as a receiver and a plurality of tail lights of a vehicle as a transmitter, thereby improving the accuracy and efficiency of determining the shape of the front road is to improve

One task of the embodiment of the present disclosure is to determine the shape of the road ahead by integrating the results using the mathematical model and the artificial neural network model, to improve the accuracy of determining the road shape, and to identify the road shape in advance, thereby improving driving safety and improving driving safety. is going to sleep

One object of the embodiment of the present disclosure is to determine the shape of the road ahead using only a single camera, thereby simplifying computational complexity and vehicle equipment structure, thereby making the vehicle lighter.

An object of an embodiment of the present disclosure is to provide an optical camera communication (OCC) device-based system applicable to vehicle communication, thereby providing very stable communication for real-time connection and safety.

The object of the embodiment of the present disclosure is not limited to the above-mentioned tasks, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the embodiment of the present invention will be. It will also be appreciated that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations thereof indicated in the claims.

Optical camera communication (OCC)-based road condition determination method according to an embodiment of the present disclosure uses a single camera for photographing a vehicle in front as an optical camera communication (OCC)-based receiver, data received from a plurality of tail lights of the vehicle in front It may include the step of determining the shape of the road ahead based on the.

Specifically, the optical camera communication (OCC)-based road shape determination method according to an embodiment of the present disclosure uses a single camera provided in the vehicle to obtain distance information between a plurality of tail lights of a front vehicle and size information of a plurality of tail lights receiving, acquiring a plurality of tail light images of the front vehicle through a single camera, and setting coordinates of a plurality of tail lights of the front vehicle in the obtained plurality of tail light images; The method may include calculating the road bending angle using a predetermined mathematical algorithm based on the interval information and the size information of the plurality of tail lights and the coordinates of the plurality of tail lights of the vehicle in front.

Through the optical camera communication (OCC)-based road condition determination method according to an embodiment of the present disclosure, by calculating the degree of bending of the driving road of the following vehicle using the OCC system using a single camera, more accurate and efficient front By judging the road type, accidents can be prevented.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium storing a computer program for executing the method may be further provided.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to an embodiment of the present disclosure, an accident can be prevented by using an OCC system that uses a single camera as a receiver and a plurality of tail lights of a vehicle as a transmitter to recognize the shape of the road ahead in a situation in which visibility is limited. .

In addition, by determining the shape of the road ahead using only a single camera instead of using a plurality of cameras or a single camera and an additional sensor to determine the shape of the road ahead, it is possible to reduce computational complexity and simplify the structure of vehicle equipment. In addition, it is possible to reduce the weight of the vehicle by reducing the computational complexity and simplifying the structure of the vehicle equipment, thereby improving user satisfaction.

In addition, by judging the shape of the road ahead by using a mathematical model and an artificial neural network in combination, the accuracy of judging the road shape can be improved, thereby improving product reliability and safety.

In addition, by calculating the road bending angle from the shape of the tail light of the vehicle in front, it can play an important role in reducing road accidents by warning the driver in advance according to the road bending angle value even in bad weather such as rain, snow, and fog.

Effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a schematic illustration of a system for determining a road type based on optical camera communication (OCC) according to an embodiment of the present disclosure.
2 is a view for explaining a basic operating principle of optical camera communication (OCC) according to an embodiment of the present disclosure.
3 is a diagram schematically illustrating a change in the tail light of the front vehicle according to the rotation of the front vehicle in the device for determining a road type according to an embodiment of the present disclosure.
4 is a block diagram schematically illustrating an apparatus for determining a road type according to an embodiment of the present disclosure.
5 is an exemplary view for explaining a process of calculating a road bending angle based on an equation method of an apparatus for determining a road shape according to an embodiment of the present disclosure.
6 is an exemplary diagram for explaining a process of calculating a road bending angle based on an artificial neural network of an apparatus for determining a road type according to an embodiment of the present disclosure.
7 is an exemplary diagram for explaining an artificial neural network training phase according to an embodiment of the present disclosure.
8 is an exemplary view illustrating a shape of a tail light of a vehicle in front of the device for determining a road shape according to an embodiment of the present disclosure.
9 is a flowchart illustrating a method for determining a road type according to an embodiment of the present disclosure.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the detailed description in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments presented below, it can be implemented in a variety of different forms, and should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. . The embodiments presented below are provided so that the disclosure of the present invention is complete, and to completely inform those of ordinary skill in the art to which the present invention pertains to the scope of the invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings, and in the description with reference to the accompanying drawings, the same or corresponding components are given the same reference numerals, and overlapping descriptions thereof are omitted. decide to do

1 is a schematic illustration of an optical camera communication (OCC)-based road type determination system according to an embodiment of the present disclosure, and FIG. 2 is an optical camera communication (OCC) basic operating principle according to an embodiment of the present disclosure. It is a drawing for explanation.

Referring to FIG. 1 , the optical camera communication (OCC)-based road type determination system of this embodiment is based on an optical camera communication (OCC) technology, and optical camera communication (OCC) is applied to vehicle communication. will do Optical camera communication (OCC) uses a light source such as an LED as a transmitter and a camera as a receiver. That is, the lights located at the front and rear of the vehicle can be used as an optical camera communication (OCC) transmitter that can transmit the vehicle status or other information to other vehicles, and the camera receiver decodes data from the captured image of the light to You can check the transmitted information.

In this embodiment, if the distance between the two tail lights of the vehicle in front or the size of the tail lights can be transmitted using optical camera communication (OCC), the road bending angle can be calculated from the shape of the tail lights of the vehicle in front using one camera. characterized by having In addition, in this embodiment, it is characterized in that the road bending direction can be detected from the image captured by the camera receiver of the following vehicle by comparing the shapes of a plurality of tail lights of the vehicle in front. Accordingly, according to the present embodiment, the curvature of the road ahead can be grasped when the front view is unclear, such as being obscured by a vehicle in front, thereby preventing an accident and improving driving stability.

When the technology of this embodiment is applied, data can be received without radio frequency (RF) technology at a data rate of 1 kbps or more with a communication distance of up to 20 m, the error rate is low, and a real-time vehicle information collection service can be supported.

Here, optical camera communication (OCC) is a technology that uses a camera image sensor to receive data bits transmitted from a light source. Many promising technologies are being studied. In addition, optical camera communication (OCC) can provide high performance characteristics including good signal-to-interference + noise ratio (SINR), high security, low interference and high stability over various communication distances.

Optical Camera Communication (OCC) differs from Visible Light Communication (VLC) and Light Fidelity (LiFi) because different types of receivers can be used, and photodiodes in Visible Light Communication (VLC) and LiFi (photodiode, PD) is used only for data reception. In an optical camera communication (OCC) system, without the need to modify hardware to support the communication purpose, a typical camera consisting of a lens and a two-dimensional image sensor is advantageous for communication because light is spatially separated from the conventional lens compared to a photodiode, and high image quality Along with the resolution, the camera can simultaneously demodulate multiple spatially separated light sources.

The shutter mechanism of the camera receiver can determine the pixel exposure of the image, and depending on the shutter mechanism, the camera can be classified as a global or rolling shutter class. A rolling shutter camera that sequentially exposes rows of pixels to light must sample fast enough to detect changes in the intensity of the modulated light in the rolling image. A global shutter camera that exposes all pixels simultaneously requires a frame rate fast enough to detect changes in the luminance of each LED in successive images.

On the other hand, since the main purpose of the light source is lighting and communication is secondary, the light source and its modulation must be appropriately selected. Modulation methods include pulse-based transmission, which may include a modulation technique in which data is encoded as a pulse wave rather than a sine wave. And pulse-based transfer modulation can be implemented with a single high-power, high-efficiency, slow-response DC converter and an additional fast-acting power switch to deliver current to the LED at a determined moment, and when the average value varies with the pulse width of the data signal, The same switch that operates the transmission can provide dimming control, greatly simplifying the DC converter. Dimmable modulation may include modulation techniques such as On-Off Keying (OOK), Variable Pulse Position Modulation (VPPM), and Color Shift Keying (CSK).

Referring to the basic operating principle of optical camera communication (OCC) with reference to FIG. 2 , the optical camera communication (OCC) system may include an OCC transmitter and an OCC receiver.

The OCC transmitter may be implemented by lighting such as a vehicle tail light in this embodiment, and an encoder, an optical signal modulator, a light source driving circuitry, and a light source (LED) shown in FIG. 2 . may include

The OCC transmitter may code an input data sequence to be transmitted in the OCC system. Such coding may be implemented in various ways. For example, when data to be transmitted is 1, the encoder may correspond to on of the light source, and when data is 0, it may correspond to off of the light source. This example may be set differently according to the pulse frequency of the light source. For example, when data is 1, the light source may correspond to on-on, and when data is 0, the light source may correspond to off-off.

As such, in this embodiment, the OCC transmitter may match the on/off states of the light sources corresponding to the data to transmit data through the on/off of the light sources in the future. In this embodiment, the OCC transmitter may code data using, for example, a Manchester coding technique, a 4B6B coding technique, or the like.

Also, in the present embodiment, the optical signal modulator may configure coded data into data symbols and generate a data packet including the data symbols. Such a data packet may be formed by arranging data composed of digital bits 1 and 0 consecutively.

In addition, the light source driving circuit may drive the light source according to the coded data. For example, the light source may be turned on and off according to bits 1 and 0 of the data. Such a light source driving circuit may turn on/off the light source according to a preset pulse frequency. In this way, the light source driving circuit may output data to be transmitted through on/off control of the light source.

That is, the light source serves as a transmitter in an optical camera communication (OCC) system. The light source may be a light emitting diode (LED), and at least one light source may be provided. Such a light source may be turned on or off with a preset pulse frequency by the light source driving circuit according to the coded data as described above. When a plurality of light sources are provided according to the present embodiment, they may be arranged in 1N, may be arranged in M1, or may be arranged in MN. Of course, it may be arranged in various shapes, such as a circular shape, a radial shape, an oval shape, and the like. If the pulse frequency at which the light source is turned on/off is more than 110 times per second, the human eye cannot distinguish the on/off and recognizes that it is in the on-state continuously. These pulse frequencies are, of course, adjustable.

The OCC receiver may be implemented by a single camera provided in the vehicle in this embodiment, and may include an image sensor, a pixel scanner, a demodulator, and a decoder as shown in FIG. 2 . can

That is, a camera including an image sensor may serve as a receiver in an optical camera communication (OCC) system, and the camera may be a camera that captures an image in a rolling shutter method. Specifically, the camera includes a rolling shutter type image sensor combined in a plurality of rows, and can continuously capture the blinking state of the light source for each row according to a preset frame rate. have. To this end, a rolling shutter type image sensor may be provided therein. Each row of the image sensor is sequentially exposed at regular time intervals for a preset exposure time (integration time). The frame time is the last exposure time in the first column and the last exposure time in the last column, and the sum of the exposure time and the frame time is the capture time. An image captured during such a capture time appears as a white band when the light source is on, and appears as a black band when the light source is turned off. Changes in the on/off state of the light source may be sequentially recorded during the capture time. In this case, the white band and the black band may be set to represent, for example, 1 and 0 as data, respectively. In this way, the camera can receive multiple data within one frame. As the image sensor, for example, a CMOS sensor may be used. In this case, the camera may start photographing at any time while the light source is on or off. In this case, it is necessary to distinguish a start frame and a data frame from the captured image. In addition, although the frame rate for capturing the on/off image of the light source of the camera is preset in the present embodiment, accurate data reception may be possible even when the actual frame rate is changed.

The pixel scanner may generate a brightness signal according to a brightness value of an on/off image of a light source photographed for every plurality of rows in the camera. Specifically, as described above, when the light source is turned on or off according to data, a white band and a black band appear, and the brightness values of each of these bands may be different. For example, a color that appears according to on/off of the light source may be displayed, for example, as a brightness value of 0 to 255. In this case, a white band may indicate a brightness value of 255, and a black band may indicate a brightness value of 0. Of course, the range of these brightness values can be changed.

The demodulator may detect the bit sequence from the brightness signal of the on/off image of the light source generated by the OCC transmitter, and may detect the bit sequence by performing a convolution operation on the generated brightness sequence and the additionally generated inversion sequence.

The decoder may extract data from the detected bit sequence. This is to restore the data coded in the on/off image of the light source according to the data to be transmitted from the OCC transmitter. For example, if data 1 to be transmitted by the OCC transmitter corresponds to the on of the light source and data 0 corresponds to the off of the light source, the decoder extracts 1 from the on image of the light source and the off image 0 can be extracted. In this case, in the present embodiment, an output data sequence may be extracted by using the brightness value from the brightness signal of the on/off image of the light source. Specifically, the gradient of the brightness signal, that is, the rising and falling of the brightness signal may be combined and extracted.

Referring to FIG. 1 , the road type determination system may include a road type determination apparatus 100 , a user terminal 200 , a server 300 , and a network 400 .

In this embodiment, users may access an application or website implemented in the user terminal 200 and perform a process such as determining a road type or confirming a vehicle location determination result. The camera of the terminal 200 may be used as an optical camera communication (OCC) receiver.

The user terminal 200 may receive a service through an authentication process after accessing a road type determination application or a road type determination website. The authentication process may include, but is not limited to, authentication for inputting user information such as membership registration, authentication for a user terminal, etc., but is not limited thereto. The authentication process may be performed only by itself.

In this embodiment, the user terminal 200 is a desktop computer, a smartphone, a notebook computer, a tablet PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a GPS (global) operated by the user positioning system) devices, e-book terminals, digital broadcast terminals, navigation devices, kiosks, MP3 players, digital cameras, home appliances, and other mobile or non-mobile computing devices, but are not limited thereto. Also, the user terminal 200 may be a wearable terminal such as a watch, glasses, a hair band, and a ring having a communication function and a data processing function. The user terminal 200 is not limited to the above, and a terminal capable of web browsing may be borrowed without limitation.

In this embodiment, the road type determination system may be implemented by the road type determination apparatus 100 and/or the server 300 , wherein the server 300 is a road type determination system including the road type determination apparatus 100 . It may be a server for operating Also, the server 300 may be a database server that provides data for operating the road type determination apparatus 100 . In addition, the server 300 may include a web server or an application server that enables the road type determination system 1 to be implemented, and may include the above-described servers or may network with these servers.

The network 400 may serve to connect the road type determination apparatus 100 , the server 300 , and the user terminal 200 in the road type determination system. The network 400 is, for example, wired networks such as local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), and integrated service digital networks (ISDNs), wireless LANs, CDMA, Bluetooth, and satellite communication. It may cover a wireless network such as, but the scope of the present invention is not limited thereto. Also, the network 400 may transmit/receive information using short-distance communication and/or long-distance communication. Here, the short-range communication may include Bluetooth (bluetooth), radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), ZigBee, and Wireless fidelity (Wi-Fi) technologies. Communication may include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), single carrier frequency division multiple access (SC-FDMA) technology. can

Network 400 may include connections of network elements such as hubs, bridges, routers, switches, and gateways. Network 400 may include one or more connected networks, eg, multiple network environments, including public networks such as the Internet and private networks such as secure enterprise private networks. Access to network 400 may be provided via one or more wired or wireless access networks. Furthermore, the network 400 may support an Internet of Things (IoT) network and/or 5G communication that exchanges and processes information between distributed components such as things.

3 is a diagram schematically illustrating a change in the tail light of the front vehicle according to the rotation of the front vehicle in the device for determining a road type according to an embodiment of the present disclosure.

이다. Referring to FIG. 3, it can be seen how the shape of the tail light of the vehicle in front can be changed according to the rotation of the vehicle in front. The shape is constant and the length between the plurality of tail lights is

to be.

가 짧아지게 되며, 좌회전의 각도가 클수록 우측 후미등의 크기가 작아지고,

가 짧아질 수 있다.And Fig. 3(b) shows a case in which the front vehicle is turning left, and it can be seen that the shape of the left tail light and the shape of the right tail light are changed, and the size of the right tail light is reduced. And when the vehicle in front turns left, the length between the plurality of tail lights

becomes shorter, and the larger the angle of left turn, the smaller the size of the right tail light.

can be shortened.

가 짧아지게 되며, 우회전의 각도가 클수록 좌측 후미등의 크기가 작아지고,

가 짧아질 수 있다.Also, Fig. 3(c) shows a case in which the vehicle in front is turning right, and it can be seen that the shape of the left tail light and the shape of the right tail light are changed, and the size of the left tail light is reduced. And when the vehicle in front turns right, the length between the plurality of tail lights

becomes shorter, and the larger the right turn angle, the smaller the size of the left tail light.

can be shortened.

That is, in the present embodiment, the bending angle of the road in which the front vehicle is located, that is, the road in front of the host vehicle, can be grasped based on the shape change of the plurality of tail lights of the front vehicle as described above. A more detailed description will be given later.

4 is a block diagram schematically illustrating an apparatus for determining a road type according to an embodiment of the present disclosure.

As shown in FIG. 4 , the road shape determination apparatus 100 may include a single camera 110 , a processor 120 , a memory 130 , and a user interface 140 .

The single camera 110 may mean a camera and/or an image sensor provided in a vehicle. The image sensor may be provided in the camera or may be configured separately. At this time, the vehicle may be provided with a plurality of cameras, but since the reception of optical camera communication (OCC)-based data from the tail light of the vehicle in front is performed with a single camera, it will be described as a single camera in this embodiment. In addition, when the vehicle is provided with a plurality of cameras, the single camera is not fixed to one camera and may be changed according to settings.

In addition, the installation position of the single camera 110 is not limited, and may be installed at a position where it is easy to photograph the front side of the vehicle, and according to an embodiment, the camera of the user terminal 200 may be implemented as a single camera.

The single camera 110 of this embodiment may include an information receiving unit 111 and an image acquiring unit 112, and the information receiving unit 111 may receive optical camera communication (OCC) based data from the tail light of the vehicle in front. Also, the image acquisition unit 112 may acquire an image within a field of view (FOV) of a single camera 110 including the front side of the vehicle. The FOV is a range in which a single camera 110 can acquire an image, and may be a range in which a signal can be recognized. Also, in this embodiment, the information receiving unit 111 may receive data from a plurality of tail lights of the vehicle in front in a rolling shutter method.

The memory 130 may store various types of information necessary for the operation of the road type determination apparatus 100 , and may store control software capable of operating the road type determination apparatus 100 , and includes a volatile or nonvolatile recording medium. can do.

The memory 130 is connected to one or more processors 120, and when executed by the processor 120, causes the processor 120 to control the road type determination apparatus 100. Codes can be stored. have.

Here, the memory 130 may include magnetic storage media or flash storage media, but the scope of the present invention is not limited thereto. The memory 130 may include internal memory and/or external memory, and may include volatile memory such as DRAM, SRAM, or SDRAM, one time programmable ROM (OTPROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, Non-volatile memory, such as NAND flash memory, or NOR flash memory, SSD. It may include a flash drive such as a compact flash (CF) card, an SD card, a Micro-SD card, a Mini-SD card, an Xd card, or a memory stick, or a storage device such as an HDD.

The user interface 140 may include an input interface into which a user request and commands for determining a road type are input. And in this embodiment, the user interface 140, for example, after identifying the front road bending angle, requests and commands such as providing road shape information such as the road bending angle of the front vehicle to the rear or surrounding vehicles are input It may be an input interface that is

In addition, the user interface 140 may include an output interface for outputting a result of the road type determination by the road type determination apparatus 100 or a notification (or warning) message according to the road type determination or the front vehicle position determination.

The input interface and the output interface of the user interface 140 may be implemented in the same interface, for example, a display device such as an AVN system in a vehicle, or an application and/or a website driven in the user terminal 200 . A user interface may be implemented through

The processor 120 may control the overall operation of the road type determining apparatus 100 . Specifically, the processor 120 is connected to the configuration of the road type determination apparatus 100 including the memory 130 , and executes at least one command stored in the memory 130 to operate the road type determination apparatus 100 . can be controlled in general.

The processor 120 may be implemented in various ways. For example, the processor 120 may include an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), or a digital signal processor (Digital Signal). Processor, DSP) may be implemented as at least one of.

The processor 120 is a kind of central processing unit and may control the overall operation of the road type determination apparatus 100 by driving control software mounted on the memory 130 . The processor 120 may include any type of device capable of processing data. Here, the 'processor' may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by, for example, a code or an instruction included in a program. As an example of the data processing apparatus embedded in the hardware as described above, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit) and a processing device such as a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto.

On the other hand, in this embodiment, the processor 120 is to determine the degree of road bending in the front based on optical camera communication (OCC), first, a plurality of tail lights of the vehicle in front using a single camera 110 provided in the vehicle. Interval information and size information of a plurality of tail lights may be received. In this case, the vehicle may be classified according to identification information such as ID. Vehicle ID may be a prerequisite for road shape identification, vehicle location identification and communication based on optical camera communication (OCC).

In addition, the processor 120 may acquire the rear side image of the front vehicle through the single camera 110 , and the rear side image of the front vehicle may have the same meaning as that the front side images of the own vehicle are obtained. However, according to an embodiment, when a vehicle is detected around the own vehicle through a separate sensor of the vehicle, an image may be acquired only for the direction of the corresponding vehicle.

Next, the processor 120 may acquire a plurality of tail light images of the front vehicle through the single camera 110 , and set coordinates of a plurality of tail lights of the front vehicle in the obtained plurality of tail light images. For example, the processor 120 may set the left tail light coordinates (x ₁ , y ₁ ) and the right tail light coordinates (x ₂ , y ₂ ) in the images for the plurality of tail lights.

In addition, the processor 120 calculates the road bending angle with a mathematical algorithm based on the distance information between the plurality of tail lights of the front vehicle and the size information of the plurality of tail lights received from the plurality of tail lights of the front vehicle, and the coordinates of the plurality of tail lights of the front vehicle. can be calculated.

In this case, the processor 120 may utilize prior knowledge about the actual size of the tail light received through optical camera communication (OCC). Considering the road curved to the right, the left tail light appears smaller than the right tail light, and considering the road curved to the left, the right tail light appears smaller than the left tail light. Accordingly, according to the curvature of the road, the road bending angle can be calculated using the ratio of the size of the left tail light and the right tail light, and after determining the coordinates of the tail light, the size can be compared, and the road bending angle can be estimated from the ratio. It will be described in more detail with reference to FIG. 5 .

5 is an exemplary view for explaining a process of calculating a road bending angle based on an equation method of an apparatus for determining a road shape according to an embodiment of the present disclosure. At this time, Fig. 5 (a) shows a case where the front vehicle turns left, and Fig. 5 (b) shows a case where the front vehicle turns right.

Referring to FIG. 5 , the processor 120 may calculate a distance per unit pixel based on the received distance information between the plurality of tail lights of the vehicle ahead and the size information of the plurality of tail lights. In addition, the processor 120 may calculate the distance between the plurality of tail lights from the plurality of tail light images of the vehicle in front obtained based on the distance per unit pixel. At this time, the actual distance in FIG. 5 may mean information on the distance between the plurality of tail lights of the front vehicle received from the plurality of tail lights of the front vehicle, and the projected distance projected to the single camera 110 (Apparent distance) may mean a distance between the plurality of tail lights calculated from the plurality of tail light images.

When the vehicle in front is going straight, the shape of the plurality of tail lights is similar. In this case, the distance between the left and right tail lights can be received from the tail lights of the vehicle in front through optical camera communication (OCC), so that the distance per unit pixel can be calculated. Therefore, in the present embodiment, when the vehicle in front turns left or right, the direction can be detected from different tail light shapes as shown in FIG. 5 . In other words, even if the vehicle in front enters the curved road, the projected distance between the plurality of tail lights (distance obtainable by a single camera) can be calculated. In practice, however, the actual distance of the plurality of tail lights is constant. By analyzing the virtual triangle using these points, the bending angle can be calculated using the cosine function.

That is, the processor 120 generates a virtual triangle based on the received distance between the plurality of tail lights and the calculated distance between the plurality of tail lights, analyzes the virtual triangle, and uses a cosine function as in Equation 1 to obtain a road bending angle can be calculated.

Meanwhile, in the present embodiment, the road bending angle may be predicted using an artificial neural network as well as a mathematical algorithm. That is, the processor 120 is based on the distance information between the plurality of tail lights of the front vehicle received from the plurality of tail lights of the front vehicle, the size information of the plurality of tail lights, and the coordinates of the plurality of tail lights of the front vehicle. It is possible to predict the road bending angle.

At this time, the processor 120 receives the image of a plurality of tail lights collected from a single camera 110 as an input, and based on a machine learning-based learning model trained to output a road bending angle, a road from a plurality of tail light images of the front vehicle. The bending angle can be predicted. The process of learning the trained machine learning-based learning model used here, that is, the pre-trained artificial neural network, will be described in more detail with reference to FIGS. 6 and 7 .

6 is an exemplary diagram for explaining a process of calculating a road bending angle based on an artificial neural network of an apparatus for determining a road type according to an embodiment of the present disclosure, and FIG. 7 is an artificial neural network training phase according to an embodiment of the present disclosure. It is an exemplary diagram for explaining a (phase).

6 is for explaining data set generation for an artificial neural network-based approach, and may represent several situations showing a data set generation concept for a deep learning-based approach. 6(a), 6(b) and 6(c) show states when the vehicle in front moves straight ahead, turns left, and turns right. 6(d) and 6(e) show a situation in which _{the y 1} and y ₂ coordinates of the plurality of tail lights may not match due to the structure of the road, although the vehicle in front is moving straight ahead.

In this embodiment, the artificial neural network is a left tail light coordinates (x ₁ , y ₁ ), right tail light coordinates (x ₂ , y ₂ ) in a plurality of tail light images collected from a single camera, the horizontal coordinate difference between the left tail light and the right tail light (d _{x ), and the vertical coordinate difference (d y} ) of the left tail lamp and the right tail lamp is set as an input vector of the pre-trained artificial neural network, and is an output vector inferred from a machine learning-based learning model by the input vector. It may be trained by a training phase in which the angle is set as a label (L). At this time, in this embodiment, 5000 images may be collected as training data, and 6 inputs and 1 output may be used. That is, the input dimension may be 5000×6 and the output dimension may be 5000×1. However, this is not limited to the embodiment.

And referring to FIG. 7 , the artificial neural network may first acquire an image. In this case, in the present embodiment, unwanted pixels may be excluded from the plurality of tail light images collected by the single camera 110 . In this embodiment, since we are only interested in the tail light area of the vehicle, all pixels are collected as inputs to the neural network, and unwanted pixels are excluded to prevent an increase in the complexity of the neural network.

and allowing a plurality of tail light images of vehicles having different road bending angles to be inferred by at least two or more hidden layers, and allowing the road bending angle to be predicted using the updated weights. it could be That is, in the present embodiment, the input image may be inferred according to the order shown in FIG. 7 and the road bending angle may be output as a label using the updated weight. For example, in this embodiment, a plurality of tail lights of the vehicle in front are first detected, and image warping is performed and then convolution is performed. You can then use max pooling to reduce the size of the current image so that no functionality is lost. These values can be flattened (flattened) and used as input to the neural network. And the updated weights can be used to calculate the road bending angle during testing in the neural network. As such, the structure of the artificial neural network shown in FIG. 7 is not limited and may be changed according to embodiments.

On the other hand, in this embodiment, the processor 120 is based on the size information of the plurality of tail lights received from the plurality of tail lights of the vehicle in front and the sizes of the plurality of tail lights detected by the plurality of tail light images acquired through the single camera 110 Thus, after comparing the sizes of the left and right tail lights of the plurality of tail lights, when the sizes of the left tail lights and the right tail lights are different, the road bending angle can be calculated through a mathematical algorithm and an artificial neural network. This is because, when the size of the plurality of tail lights is not different, since the vehicle in front is going straight, it is not necessary to calculate the road bending angle.

The processor 120 may derive the final road bending angle based on a result of comparing the road bending angle calculated by the mathematical algorithm with the road bending angle predicted through the artificial neural network.

In this case, the processor 120 may compare the difference between the road bending angle calculated by the mathematical algorithm and the road bending angle predicted through the artificial neural network and the threshold value. In this case, the threshold value may be preset or may be changeable.

If the difference between the road bending angle calculated by the mathematical algorithm and the road bending angle predicted through the artificial neural network is less than or equal to the threshold value, the processor 120 may derive the road bending angle predicted through the artificial neural network as the final road bending angle. have. On the other hand, when the difference between the road bending angle calculated by the mathematical algorithm and the road bending angle predicted through the artificial neural network exceeds the threshold value, the processor 120 predicts the road bending angle calculated by the mathematical algorithm and the artificial neural network. The road bending angle data can be deleted.

For example, in this embodiment, a 30 fps (frame rate per second) camera may be used, which may obtain 30 images per second and each image may provide one result. So if you are using a camera with a high frame rate, the number of results you get in 1 second is equal to the frame rate of your camera. That is, deleting some results does not cause significant performance degradation.

In this embodiment, since the deep learning-based method is more accurate than the mathematical approach, the two methods can be compared to provide more accurate results. In this embodiment, since serious performance degradation does not occur even if some results are deleted, if there is a discrepancy in the results between the mathematical method and the deep learning-based method, there is no problem up to a certain level (threshold), the result of the deep learning-based approach can be output as a result value, and if the difference is greater than a threshold value (eg, 5), the corresponding calculation result value can be removed.

Meanwhile, in the present embodiment, it is checked whether the derived final road bending angle is equal to or greater than a set angle, and if the final road bending angle is equal to or greater than the set angle, a warning message may be output. In this case, the warning message may be a message output through the user interface 140 in the own vehicle, or may be a message transmitted to the rear vehicle or surrounding vehicles through optical camera communication (OCC).

In addition, the processor 120 may receive at least one of identification information of the front vehicle and state information of the front vehicle from a plurality of tail lights of the front vehicle using the single camera 110 . That is, the processor 120 may receive identification information for identifying the vehicle in front to determine the shape of the road on which the vehicle is traveling, and may also determine the location of the vehicle. In addition, in this embodiment, status information of the vehicle in front may also be received through optical camera communication (OCC), and the status information may include information about a case in which a problem occurs in the vehicle in front, driving conditions, and the like.

In addition, the processor 120 uses a single camera 110 to obtain identification information of the front vehicle and state information of the front vehicle from a plurality of tail lights of the front vehicle, interval information between the plurality of tail lights of the front vehicle, and sizes of the plurality of tail lights. information can be received at the same time. That is, the processor 120 may simultaneously receive the above information only by capturing a plurality of tail lights based on optical camera communication (OCC).

In addition, the processor 120 includes the identification information of the front vehicle and the status information of the front vehicle received from the front vehicle, the location information of the front vehicle, the bending angle of the front road, the identification information of the own vehicle and the state information of the own vehicle, and the At least one of interval information between the plurality of tail lights of the vehicle and size information of the plurality of tail lights may be transmitted through the tail lights of the own vehicle. That is, in the present embodiment, data may be received from the vehicle ahead through optical camera communication (OCC), and data may be transmitted to the vehicle behind according to circumstances.

In summary, as shown in FIG. 8 , images of a plurality of rear light areas of the vehicle in front may be acquired through a single camera 110 . 8 is a shape of the tail lamp when the vehicle in front is bent at a specific angle at a position relative to the single camera 110, and FIG. 8 (a) shows an image obtained from the single camera 110, and FIG. 8 (b) indicates an image in which the tail light area is detected in the acquired image. And FIG. 8( c ) shows a binary image of the detected tail light area, and FIG. 8 ( d ) shows an image of the right tail light modulated due to the rolling shutter effect of the single camera 110 . Therefore, in this embodiment, based on optical camera communication (OCC), data is received from a plurality of tail lights of the vehicle in front with a single camera 110 of the vehicle, and the road such as the bending angle of the front road through the received data shape can be discerned.

9 is a flowchart illustrating a method for determining a road type according to an embodiment of the present disclosure. In the following description, descriptions of parts overlapping with those of FIGS. 1 to 8 will be omitted.

As shown in Figure 9, in step S100, the road shape determination device 100 using a single camera 110 provided in the vehicle, the distance information between the plurality of tail lights of the front vehicle and the size information of the plurality of tail lights receive

Here, the single camera 110 may mean a camera and/or an image sensor provided in a vehicle. The image sensor may be provided in the camera or may be configured separately. In addition, the single camera 110 of this embodiment may receive optical camera communication (OCC) based data from the tail lights of the vehicle in front, and may receive data from a plurality of tail lights of the vehicle in front in a rolling shutter method.

In step S200, the road shape determination device 100 determines the size of a plurality of tail lights detected by the size information of a plurality of tail lights received from a plurality of tail lights of the vehicle in front and a plurality of tail light images obtained through a single camera 110. Based on the comparison, the sizes of the left tail light and the right tail light of the plurality of tail lights are compared.

In this embodiment, when the size of the left tail light and the right tail light are different as a result of the comparison, the next step of calculating the road bending angle through a mathematical algorithm and an artificial neural network may be performed. This is because it is not necessary to calculate the road bending angle when the size of the left tail light and the right tail light are the same.

In step S300 , the road shape determination apparatus 100 acquires a plurality of tail light images of the front vehicle through the single camera 110 , and sets coordinates of a plurality of tail lights of the front vehicle in the obtained plurality of tail light images.

In this case, the road type determination apparatus 100 may acquire an image within a field of view (FOV) of the single camera 110 including the front side of the vehicle. The FOV is a range in which a single camera 110 can acquire an image, and may be a range in which a signal can be recognized.

And in step S400, the road condition determination device 100 determines the road bending angle with a mathematical algorithm based on the received distance information between the plurality of tail lights of the front vehicle and the size information of the plurality of tail lights and the coordinates of the plurality of tail lights of the front vehicle. Calculate.

In this case, the road condition determination apparatus 100 may calculate the distance per unit pixel based on the received distance information between the plurality of tail lights of the vehicle ahead and the size information of the plurality of tail lights. In addition, the road condition determination apparatus 100 may calculate the distance between the plurality of tail lights from the plurality of tail light images of the vehicle in front obtained based on the distance per unit pixel. When the vehicle in front is going straight, the shape of the plurality of tail lights is similar. In this case, the distance between the left and right tail lights can be received from the tail lights of the vehicle in front through optical camera communication (OCC), so that the distance per unit pixel can be calculated. Even if the vehicle in front enters the curved road, it is possible to calculate the projected distance between the plurality of tail lights (distance obtainable with a single camera). In practice, however, the actual distance of the plurality of tail lights is constant. By analyzing the virtual triangle using these points, the bending angle can be calculated using the cosine function.

That is, the road condition determination apparatus 100 generates a virtual triangle based on the received distance between the plurality of tail lights and the calculated distance between the plurality of tail lights, analyzes the virtual triangle, and calculates the road bending angle using a cosine function can do.

In step S500, the road condition determination device 100 receives the distance information between the plurality of tail lights of the front vehicle and the size information of the plurality of tail lights and the plurality of tail light coordinates of the front vehicle. Predict the bending angle.

At this time, the road condition determination device 100 receives images of a plurality of tail lights collected from a single camera 110 as input and based on a machine learning-based learning model trained to output a road bending angle, a plurality of tail lights of the front vehicle It is possible to predict the road bending angle from the image.

In this embodiment, the pre-trained artificial neural network is a left tail light coordinates (x ₁ , y ₁ ), a right tail light coordinates (x ₂ , y ₂ ), and a left tail light from a plurality of tail light images collected from a single camera 110 . The horizontal coordinate difference (d _x ) of the right tail lamp and the vertical coordinate difference (d _y ) of the left tail lamp and the right tail lamp are set as input vectors of the pre-trained artificial neural network, and inferred from the machine learning-based learning model by the input vectors It may be trained by a training phase in which a road bending angle, which is an output vector, is set as a label (L).

And in this embodiment, the pre-trained artificial neural network excludes unwanted pixels from a plurality of collected tail light images, and a plurality of tail light images of vehicles with different road bending angles are inferred by at least two or more hidden layers. It may be trained by a training phase comprising the steps of: and predicting the road bending angle using the updated weights.

And in step S600, the road condition determination apparatus 100 compares the road bending angle calculated by the mathematical algorithm with the road bending angle predicted through the artificial neural network.

In this case, the road condition determination apparatus 100 may compare the difference between the road bending angle calculated by the mathematical algorithm and the road bending angle predicted through the artificial neural network and the threshold value. In this case, the threshold value may be preset or may be changeable.

In step S700, if the difference between the road bending angle calculated by the mathematical algorithm and the road bending angle predicted through the artificial neural network is less than or equal to the threshold value, the road condition determination apparatus 100 finalizes the road bending angle predicted through the artificial neural network. It is derived as a road bending angle (example of step S600).

On the other hand, in step S800 , the road condition determination apparatus 100 determines the road bending angle calculated by the mathematical algorithm when the difference between the road bending angle calculated by the mathematical algorithm and the road bending angle predicted through the artificial neural network exceeds the threshold value. and deletes the road bending angle data predicted through the artificial neural network (NO in step S600).

For example, in this embodiment, a 30 fps (frame rate per second) camera may be used, which may obtain 30 images per second and each image may provide one result. So if you are using a camera with a high frame rate, the number of results you get in 1 second is equal to the frame rate of your camera. That is, deleting some results does not cause significant performance degradation.

In this embodiment, since the deep learning-based method is more accurate than the mathematical approach, the two methods can be compared to provide more accurate results. In this embodiment, since serious performance degradation does not occur even if some results are deleted, if there is a discrepancy in the results between the mathematical method and the deep learning-based method, there is no problem up to a certain level (threshold), the result of the deep learning-based approach can be output as a result value, and if the difference is greater than a threshold value (eg, 5), the corresponding calculation result value can be removed.

The above-described embodiment according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium includes a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and a ROM. , RAM, flash memory, and the like, and hardware devices specially configured to store and execute program instructions.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and used by those skilled in the computer software field. Examples of the computer program may include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

In the specification of the present invention (especially in the claims), the use of the term "above" and similar referential terms may be used in both the singular and the plural. In addition, when a range is described in the present invention, each individual value constituting the range is described in the detailed description of the invention as including the invention to which individual values belonging to the range are applied (unless there is a description to the contrary). same as

The steps constituting the method according to the present invention may be performed in an appropriate order unless the order is explicitly stated or there is no description to the contrary. The present invention is not necessarily limited to the order in which the steps are described. The use of all examples or exemplary terms (eg, etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is limited by the examples or exemplary terms unless limited by the claims. it is not going to be In addition, those skilled in the art will appreciate that various modifications, combinations, and changes can be made in accordance with design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the spirit of the present invention is not limited to the scope of the scope of the present invention. will be said to belong to

100: road type determination device
110: single camera
111: information receiving unit
112: image acquisition unit
120 : processor
130: memory
140 : user interface
200: user terminal
300 : server
400: network

Claims (20)

As a road type determination method that can determine the degree of bending of the road ahead based on Optical Camera Communication (OCC),
Receiving actual distance information between a plurality of tail lights of a front vehicle and actual size information of the plurality of tail lights by using a monocular camera provided in the vehicle;
acquiring a plurality of tail light images of the front vehicle through the monocular camera, and setting coordinates of a plurality of tail lights of the front vehicle in the obtained plurality of tail light images;
determining a bending direction of the front road through size comparison of the acquired plurality of tail light images; and
Based on the received actual distance information between the plurality of tail lights of the front vehicle, the actual size information of the plurality of tail lights, and the distance between the image-based tail lights calculated from the coordinates of the plurality of tail lights of the front vehicle, a predetermined mathematical algorithm is used. calculating a road bending angle of the road ahead;
Calculating the road bending angle of the front road with the preset mathematical algorithm comprises:
calculating a distance per unit pixel based on the received information on the actual distance between the plurality of tail lights of the vehicle ahead and the actual size information of the plurality of tail lights;
calculating a distance between the plurality of tail lights in the obtained plurality of tail light images of the front vehicle based on the distance per unit pixel;
generating a virtual triangle based on the received actual distance between the plurality of tail lights of the vehicle ahead and the calculated distance between the plurality of image-based tail lights; and
Analyzing the virtual triangle and calculating the road bending angle of the road ahead using a cosine function,
How to determine the shape of a road.
The method of claim 1,
Based on the received actual distance information between the plurality of tail lights of the vehicle in front, the actual size information of the plurality of tail lights, and the distance between the image-based tail lights calculated from the coordinates of the plurality of tail lights of the front vehicle, a pre-learned artificial neural network predicting a road bending angle of the road ahead; and
Deriving the final road bending angle of the front road based on the result of comparing the road bending angle of the front road calculated by the preset mathematical algorithm with the road bending angle of the front road predicted through the artificial neural network more containing,
How to determine the type of road.
The method of claim 1,
Creating the virtual triangle comprises:
setting one straight line among the virtual triangles based on a plurality of tail light coordinates projected on a plurality of tail light images of the front vehicle acquired through the monocular camera;
setting another straight line among the virtual triangles based on the actual distance information between the plurality of tail lights of the front vehicle received using the monocular camera, based on the straight line;
comparing sizes of left tail light areas and right tail light areas of the plurality of tail lights based on the sizes of the plurality of tail lights projected on the plurality of tail light images of the front vehicle; and
As a result of the comparison, generating an imaginary triangle in which the angle on the side of the tail light area with the small size among the plurality of tail lights is a right angle, and the angle on the side of the tail light area with the large size among the plurality of tail lights is the angle between the two straight lines.
How to determine the type of road.
3. The method of claim 2,
Predicting the road bending angle of the front road through the pre-learned artificial neural network comprises:
Based on a machine learning-based learning model trained to output the road bending angle of the front road by inputting images of a plurality of tail lights collected from the monocular camera, the road of the front road in the plurality of tail light images of the front vehicle predicting the bending angle;
How to determine the type of road.
5. The method of claim 4,
The pre-learned artificial neural network,
In the plurality of tail light images collected from the monocular camera, the left tail light coordinates, the right tail light coordinates, the horizontal coordinate difference between the left tail light and the right tail light, and the vertical coordinate difference between the left tail light and the right tail light are the input of the previously learned artificial neural network. set to a vector; and
Trained by a training phase comprising the step of setting the road bending angle of the road ahead, which is an output vector inferred from the machine learning-based learning model by the input vector, as a label,
How to determine the type of road.
5. The method of claim 4,
The pre-learned artificial neural network,
excluding unwanted pixels from a plurality of tail light images collected by the monocular camera;
allowing a plurality of tail light images of vehicles having different road bending angles of the front road to be inferred by at least two or more hidden layers; and
trained by a training phase comprising the step of allowing the road bending angle of the road ahead to be predicted using the updated weights,
How to determine the type of road.
3. The method of claim 2,
The step of deriving the final road bending angle of the front road is,
Comprising the step of comparing the difference between the road bending angle of the road ahead calculated by the mathematical algorithm and the road bending angle of the front road predicted through the artificial neural network and a threshold value,
Comparing the difference value and the threshold value,
If the difference value is less than or equal to the threshold value, the road bending angle of the forward road predicted through the artificial neural network is derived as the final road bending angle of the forward road, and when the difference value exceeds the threshold value, the mathematical algorithm Comprising the step of deleting the calculated road bending angle of the road ahead and the road bending angle data predicted through the artificial neural network,
How to determine the type of road.
3. The method of claim 2,
Comparing the sizes of the left and right tail lights of the plurality of tail lights based on the received actual size information of the plurality of tail lights and the sizes of the plurality of tail lights detected by the obtained plurality of tail light images. ,
When the size of the left tail light and the right tail light are different, calculating the road bending angle of the front road through the mathematical algorithm and the artificial neural network,
How to determine the type of road.
The method of claim 1,
Receiving at least one of identification information of the front vehicle and status information of the front vehicle from a plurality of tail lights of the front vehicle by using the monocular camera,
How to determine the type of road.
The method of claim 1,
Identification information of the front vehicle and state information of the front vehicle from the plurality of tail lights of the front vehicle using the monocular camera, actual distance information between the plurality of tail lights of the front vehicle, and actual size information of the plurality of tail lights Further comprising the step of receiving at the same time,
How to determine the type of road.
The method of claim 1,
Receiving the information comprises:
Receiving data from a plurality of tail lights of the front vehicle in a rolling shutter manner through the monocular camera,
How to determine the type of road.
3. The method of claim 2,
checking whether the derived final road bending angle of the front road is equal to or greater than a set angle; and
If the final road bending angle of the front road is greater than or equal to a set angle, further comprising the step of outputting a warning message,
How to determine the type of road.
As a road type determination device that can determine the degree of bending of the road ahead based on Optical Camera Communication (OCC),
Memory; and
at least one processor coupled to the memory and configured to execute computer readable instructions contained in the memory;
the at least one processor,
receiving information on actual distances between a plurality of tail lights of a front vehicle and information on actual sizes of the plurality of tail lights by using a monocular camera provided in the vehicle;
obtaining a plurality of tail light images of the front vehicle through the monocular camera, and setting coordinates of a plurality of tail light lights of the front vehicle in the obtained plurality of tail light images;
determining a bending direction of the road ahead through size comparison of the plurality of acquired tail light images; and
Based on the received actual distance information between the plurality of tail lights of the vehicle in front, the actual size information of the plurality of tail lights, and the distance between the image-based tail lights calculated from the coordinates of the plurality of tail lights of the vehicle in front, a preset mathematical algorithm is used. configured to perform an operation of calculating a road bending angle of the road;
The operation of calculating the road bending angle of the road ahead by the predetermined mathematical algorithm is,
calculating a distance per unit pixel based on the received information on the actual distance between the plurality of tail lights of the vehicle ahead and the actual size information of the plurality of tail lights;
calculating a distance between the plurality of tail lights in the obtained plurality of tail light images of the front vehicle based on the distance per unit pixel;
generating a virtual triangle based on the received actual distance between the plurality of tail lights of the vehicle in front and the calculated distance between the plurality of image-based tail lights; and
Analyzing the virtual triangle and calculating the road bending angle of the road ahead using a cosine function,
road type judgment device.
14. The method of claim 13,
the at least one processor,
Based on the received actual distance information between the plurality of tail lights of the vehicle in front, the actual size information of the plurality of tail lights, and the distance between the image-based tail lights calculated from the coordinates of the plurality of tail lights of the front vehicle, a pre-learned artificial neural network predicting a road bending angle of the road ahead through and
The operation of deriving the final road bending angle of the front road based on the result of comparing the road bending angle of the front road calculated by the preset mathematical algorithm with the road bending angle of the front road predicted through the artificial neural network further configured to perform
road type judgment device.
14. The method of claim 13,
The operation of generating the virtual triangle includes:
setting one straight line among the virtual triangles based on a plurality of tail light coordinates projected on a plurality of tail light images of the front vehicle obtained through the monocular camera;
setting the other straight line among the virtual triangles based on the actual distance information between the plurality of tail lights of the front vehicle received using the monocular camera based on the straight line;
Comparing sizes of a left tail light area and a right tail light area of the plurality of tail lights based on the sizes of the plurality of tail lights projected on the plurality of tail light images of the front vehicle; and
As a result of the comparison, an angle on the side of the tail light area with the small size among the plurality of tail lights is a right angle, and the angle on the side of the tail light area with the large size among the plurality of tail lights is the angle between the two straight lines.
road type judgment device.
15. The method of claim 14,
The operation of predicting the road bending angle of the front road through the pre-learned artificial neural network is,
Based on a machine learning-based learning model trained to output the road bending angle of the front road by inputting images of a plurality of tail lights collected from the monocular camera, the road of the front road in the plurality of tail light images of the front vehicle Including the operation of predicting the bending angle,
road type judgment device.
17. The method of claim 16,
The pre-learned artificial neural network,
In the plurality of tail light images collected from the monocular camera, the left tail light coordinates, the right tail light coordinates, the horizontal coordinate difference between the left tail light and the right tail light, and the vertical coordinate difference between the left tail light and the right tail light are the input of the previously learned artificial neural network. set to a vector; and
Trained by a training phase comprising the step of setting the road bending angle of the road ahead, which is an output vector inferred from the machine learning-based learning model by the input vector, as a label,
road type judgment device.
17. The method of claim 16,
The pre-learned artificial neural network,
excluding unwanted pixels from a plurality of tail light images collected by the monocular camera;
allowing a plurality of tail light images of vehicles having different road bending angles of the front road to be inferred by at least two or more hidden layers; and
trained by a training phase comprising the step of allowing the road bending angle of the road ahead to be predicted using the updated weights,
road type judgment device.
15. The method of claim 14,
The operation of deriving the final road bending angle of the front road is,
Comprising the operation of comparing the difference between the road bending angle of the front road calculated by the mathematical algorithm and the road bending angle of the front road predicted through the artificial neural network and a threshold value,
The operation of comparing the difference value and the threshold value is,
If the difference value is less than or equal to the threshold value, the road bending angle of the forward road predicted through the artificial neural network is derived as the final road bending angle of the forward road, and when the difference value exceeds the threshold value, the mathematical algorithm Comprising the operation of deleting the calculated road bending angle of the front road and the road bending angle data of the front road predicted through the artificial neural network,
road type judgment device.
15. The method of claim 14,
Based on the received actual size information of the plurality of tail lights and the sizes of the plurality of tail lights detected by the obtained plurality of tail light images, the operation of comparing the sizes of the left tail lights and the right tail lights of the plurality of tail lights is further performed. is composed,
When the size of the left tail light and the right tail light are different, calculating the road bending angle of the front road through the mathematical algorithm and the artificial neural network,
road type judgment device.

Images