KR102343334B1 - Method and apparatus for detecting a transmitting light source on optical camera communication - Google Patents

Abstract

Disclosed are a method and a device for detecting a light source based on OCC. According to one embodiment of the present disclosure, the method for detecting a light source based on OCC includes the steps of: acquiring an image including at least one light source using a camera; extracting a feature with respect to a light source region in the image by applying the acquired image to an image classification algorithm learned to identify the light source; and determining whether the light source is an OCC-based data transmission light source for each light source in the image acquired by applying the extracted feature to a binary classification algorithm learned to identify a data transmission light source.

Description

Optical camera communication (OCC) based transmission light source detection method and apparatus

The present disclosure relates to a transmission light source detection method and apparatus capable of accurately detecting a transmission light source for optical camera communication (OPTICAL CAMERA COMMUNICATION, OCC) from all light sources in an image based on an image classification algorithm and a binary classification algorithm.

In general, wireless communication is advantageous in terms of cost, practicality and operability, but a bottleneck occurs. In addition, although the RF (Radio Frequency) band below the 10 GHz frequency part has been widely used for wireless communication, it does not meet the capacity and speed required by the bandwidth, and several technologies use the same frequency band (Wi-Fi, Bluetooth, mobile communication network). , and wireless phones), research to solve the bottleneck problem that occurs in wireless communication has been actively conducted.

Accordingly, in recent years, many wireless applications based on RF technology are being replaced by optical wireless technology. Among them, Visible Light Communication (VLC) is an already established technology, which uses the visible light spectrum (400-790THz), provides high data rates, and uses modulated optical signals of LEDs and LDs. .

On the other hand, Optical Camera Communication (OCC) technology is a technology that is being developed and standardized. It is one of the new promising technologies that is part of the Optical Wireless Communication (OWC) family. Here, the OWC technology uses an unlicensed spectrum as a communication medium to solve the bottleneck, so there is no need for a frequency use license, and it has the characteristics of easy security maintenance due to the straightness of radio waves and low object permeability.

In optical camera communication (OCC), light coming from different angles can be projected at various locations on the image sensor plane. In other words, optical camera communication (OCC) is an ideal technology for spatial division multiplexing and MIMO (Multiple Input and Multiple-Output) system imaging because it can separate light from different sources and directions through these features.

Such optical camera communication (OCC) in particular is one of the most promising candidates for vehicle communication. For example, as disclosed in Prior Art 1, for optical camera communication (OCC), a lighting lamp such as a tail lamp of a vehicle may be used as a transmitter, and a camera provided in the vehicle may be used as a receiver. That is, the vehicle's front and rear LEDs can be used as an optical camera communication (OCC) transmitter that can transmit vehicle status or other information to other vehicles, and the camera receiver receives data from images captured by the vehicle's front and rear LEDs. It can be decoded to receive data.

However, in real-time applications such as V2X or robotics, there may be many light sources that are not actual transmitters among all the light sources in the front. This can be a growing problem. In addition, there is a problem in that the complexity of the receiver increases when the captured LED region is processed together with other non-transmitted LED regions.

The above-mentioned background art is technical information that the inventor possessed for the derivation of the present invention or acquired in the process of derivation of the present invention, and cannot necessarily be said to be a known technique disclosed to the general public prior to the filing of the present invention.

Prior Art 1: Korean Patent Publication No. 10-1689252 (Registered on Dec. 19, 2016)

An object of an embodiment of the present disclosure is to accurately detect a transmission light source for optical camera communication (OPTICAL CAMERA COMMUNICATION, OCC) from all light sources in an image through a combination of an image classification algorithm and a binary classification algorithm.

An object of the embodiments of the present disclosure is to propose an optical camera communication (OCC) device-based system applicable to real-time application communication, such as a vehicle, to provide very stable communication for real-time connection and safety.

The object of the embodiment of the present disclosure is not limited to the above-mentioned tasks, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the embodiment of the present invention will be. It will also be appreciated that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations thereof indicated in the claims.

An optical camera communication (OCC)-based transmission light source detection method according to an embodiment of the present disclosure includes acquiring an image including at least one light source by using a camera, and classifying the image learned to identify the light source by using the acquired image OCC-based data transmission for each light source in the image obtained by applying the algorithm to extracting a feature for the light source region in the image, and applying the extracted feature to a binary classification algorithm learned to identify the data transmission light source It may include the step of determining whether the light source.

An optical camera communication (OCC)-based transmission light source detection apparatus according to an embodiment of the present disclosure includes a memory and at least one processor connected to the memory and configured to execute computer-readable instructions included in the memory, and at least one The processor of the operation of acquiring an image including at least one light source using a camera, and applying the acquired image to an image classification algorithm learned to identify the light source to extract a feature from the light source region in the image and applying the extracted features to a binary classification algorithm learned to identify a data transmission light source to determine whether or not it is an OCC-based data transmission light source for each light source in an image obtained.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium storing a computer program for executing the method may be further provided.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to the embodiment of the present disclosure, through the combination of the image classification algorithm and the binary classification algorithm, the transmission light source for optical camera communication (OPTICAL CAMERA COMMUNICATION, OCC) can be accurately detected from all light sources in the image, so that other light sources ( LED) can be eliminated, and the BER (bit error rate) can be mitigated.

In addition, by creating an intelligent receiver that implements a transmission light source detection algorithm combined with an image classification algorithm and a binary classification algorithm, it can accurately recognize the actual data transmission light source, thereby providing very reliable communication for real-time connection and safety. .

Effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a schematic illustration of a transmission light source detection system based on optical camera communication (OCC) according to an embodiment of the present disclosure.
2 is a view for explaining a basic operating principle of optical camera communication (OCC) according to an embodiment of the present disclosure.
3 is a block diagram schematically illustrating an apparatus for detecting a transmission light source according to an embodiment of the present disclosure.
4 is a diagram for explaining in more detail an apparatus for detecting a transmission light source according to an embodiment of the present disclosure.
5 is a diagram for explaining a method of deriving the number of line segments from a light source image according to an embodiment of the present disclosure.
6 is a graph showing the number of line segments and the area of a light source region derived according to an embodiment of the present disclosure for a light source that transmits data and a general light source that does not transmit data.
7 is a flowchart illustrating a method of detecting a transmission light source according to an embodiment of the present disclosure.
8 is a flowchart illustrating a learning process of a method for detecting a transmission light source according to an embodiment of the present disclosure.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the detailed description in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments presented below, it can be implemented in a variety of different forms, and should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. . The embodiments presented below are provided so that the disclosure of the present invention is complete, and to completely inform those of ordinary skill in the art to which the present invention pertains to the scope of the invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings, and in the description with reference to the accompanying drawings, the same or corresponding components are given the same reference numerals, and overlapping descriptions thereof are omitted. decide to do

1 is a schematic illustration of a transmission light source detection system based on optical camera communication (OCC) according to an embodiment of the present disclosure, and FIG. 2 is a basic operating principle of optical camera communication (OCC) according to an embodiment of the present disclosure. It is a drawing for explaining.

Referring to FIG. 1 , the optical camera communication (OCC)-based transmission light source detection system of this embodiment is based on an optical camera communication (OCC) technology, and is optical for communication used in real-time applications such as vehicles. Camera communication (OCC) can be applied. However, in this embodiment, the application of optical camera communication (OCC) to vehicle communication may be described as an embodiment.

Optical camera communication (OCC) uses a light source such as an LED as a transmitter and a camera as a receiver. That is, the lights located at the front and rear of the vehicle can be used as an optical camera communication (OCC) transmitter that can transmit the vehicle status or other information to other vehicles, and the camera receiver decodes data from the captured image of the light to You can check the transmitted information.

Here, optical camera communication (OCC) is a technology that uses a camera image sensor to receive data bits transmitted from a light source. Many promising technologies are being studied. In addition, optical camera communication (OCC) can provide high performance characteristics including good signal-to-interference + noise ratio (SINR), high security, low interference and high stability over various communication distances.

Optical Camera Communication (OCC) differs from Visible Light Communication (VLC) and Light Fidelity (LiFi) because different types of receivers can be used, and photodiodes in Visible Light Communication (VLC) and LiFi (photodiode, PD) is used only for data reception. In an optical camera communication (OCC) system, without the need to modify hardware to support the communication purpose, a typical camera consisting of a lens and a two-dimensional image sensor is advantageous for communication because light is spatially separated from the conventional lens compared to a photodiode, and high image quality Along with the resolution, the camera can simultaneously demodulate multiple spatially separated light sources.

The shutter mechanism of the camera receiver can determine the pixel exposure of the image, and depending on the shutter mechanism, the camera can be classified as a global or rolling shutter class. A rolling shutter camera that sequentially exposes rows of pixels to light must sample fast enough to detect changes in the intensity of the modulated light in the rolling image. A global shutter camera that exposes all pixels simultaneously requires a frame rate fast enough to detect changes in the luminance of each LED in successive images.

On the other hand, since the main purpose of the light source is lighting and communication is secondary, the light source and its modulation must be appropriately selected. Modulation methods include pulse-based transmission, which may include a modulation technique in which data is encoded as a pulse wave rather than a sine wave. And pulse-based transfer modulation can be implemented with a single high-power, high-efficiency, slow-response DC converter and an additional fast-acting power switch to deliver current to the LED at a determined moment, and when the average value varies with the pulse width of the data signal, The same switch that operates the transmission can provide dimming control, greatly simplifying the DC converter. Dimmable modulation may include modulation techniques such as On-Off Keying (OOK), Variable Pulse Position Modulation (VPPM), and Color Shift Keying (CSK).

As mentioned above, in an optical camera communication (OCC) system, the transmit LED is sensed by the camera receiver. However, in applications implemented in environments where conditions change in real time, such as vehicle to everything (V2X) or robotics, there may be many LEDs other than the actual transmitter. In this case, the probability of spurious data bits in the decoded data increases, which may cause a problem that the BER (bit error rate) increases, and the complexity of the receiver is increased by processing the captured LED area together with other non-transmitted LED areas. can be higher

That is, the present embodiment is to intelligently handle these problems, and it is characterized in that the transmission LED (light source) can be accurately detected by combining the image classification algorithm and the binary classification algorithm for a stable optical camera communication (OCC) system. do.

In the present embodiment, as the image classification algorithm, various deep learning (artificial neural network) algorithms for classifying unstructured data such as images and voices may be applied, and in particular, Convolutional Neural Networks (CNN) may be applied.

And the binary classification algorithm is about a model that defines a decision boundary, that is, a baseline for classification, in the classification algorithm. . In this embodiment, in particular, a support vector machine (SVM) may be applied.

SVM can be said to be a non-stochastic binary linear classification model that, when a set of data belonging to one of two categories is given, determines which category the new data belongs to based on the given data set. This classification model is expressed as a boundary in the space where data is mapped, and SVM is an algorithm that finds the boundary with the largest width among them. On the other hand, the support vector of SVM means data points close to the decision boundary, and the margin means the distance between the decision boundary and the support vector, and the optimal decision boundary may mean maximizing the margin. .

Most machine learning supervised learning algorithms train the model using all of the training data, but in SVM, the support vector that defines the decision boundary is the support vector. That is, in the present embodiment, it is possible to obtain a desired classification result (eg, LED region detection) more quickly and accurately by applying a high-performance SVM algorithm to the classification technique without setting a threshold value of the feature vector.

Meanwhile, referring to FIG. 2 , looking at the basic operating principle of optical camera communication (OCC), the optical camera communication (OCC) system may include an OCC transmitter and an OCC receiver.

The OCC transmitter may be implemented by lighting such as a vehicle tail light (LED) in this embodiment, and an encoder, an optical signal modulator, a light source driving circuitry and a light source disclosed in FIG. 2 . (LED) may be included.

The OCC transmitter may code an input data sequence to be transmitted in the OCC system. Such coding may be implemented in various ways. For example, when data to be transmitted is 1, the encoder may correspond to on of the light source, and when data is 0, it may correspond to off of the light source. This example may be set differently according to the pulse frequency of the light source. For example, when data is 1, the light source may correspond to on-on, and when data is 0, the light source may correspond to off-off.

As such, in this embodiment, the OCC transmitter may match the on/off states of the light sources corresponding to the data to transmit data through the on/off of the light sources in the future. In this embodiment, the OCC transmitter may code data using, for example, a Manchester coding technique, a 4B6B coding technique, or the like.

Also, in the present embodiment, the optical signal modulator may configure coded data into data symbols and generate a data packet including the data symbols. Such a data packet may be formed by arranging data composed of digital bits 1 and 0 consecutively.

In addition, the light source driving circuit may drive the light source according to the coded data. For example, the light source may be turned on and off according to bits 1 and 0 of the data. Such a light source driving circuit may turn on/off the light source according to a preset pulse frequency. In this way, the light source driving circuit may output data to be transmitted through on/off control of the light source.

That is, the light source serves as a transmitter in an optical camera communication (OCC) system. The light source may be a light emitting diode (LED), and at least one light source may be provided. Such a light source may be turned on or off with a preset pulse frequency by the light source driving circuit according to the coded data as described above. When a plurality of light sources are provided according to the present embodiment, they may be arranged in 1N, may be arranged in M1, or may be arranged in MN. Of course, it may be arranged in various shapes, such as a circular shape, a radial shape, an oval shape, and the like. If the pulse frequency at which the light source is turned on/off is more than 110 times per second, the human eye cannot distinguish the on/off and recognizes that it is in the on-state continuously. These pulse frequencies are, of course, adjustable.

The OCC receiver may be implemented by a single camera provided in the vehicle in this embodiment, and may include an image sensor, a pixel scanner, a demodulator, and a decoder as shown in FIG. 2 . can

That is, a camera including an image sensor may serve as a receiver in an optical camera communication (OCC) system, and the camera may be a camera that captures an image in a rolling shutter method. Specifically, the camera includes a rolling shutter type image sensor combined in a plurality of rows, and can continuously capture the blinking state of the light source for each row according to a preset frame rate. have. To this end, a rolling shutter type image sensor may be provided therein. Each row of the image sensor is sequentially exposed at regular time intervals for a preset exposure time (integration time). The frame time is the last exposure time in the first column and the last exposure time in the last column, and the sum of the exposure time and the frame time is the capture time. An image captured during such a capture time appears as a white band when the light source is on, and appears as a black band when the light source is turned off. Changes in the on/off state of the light source may be sequentially recorded during the capture time. In this case, the white band and the black band may be set to represent, for example, 1 and 0 as data, respectively. In this way, the camera can receive multiple data within one frame. As the image sensor, for example, a CMOS sensor may be used. In this case, the camera may start photographing at any time while the light source is on or off. In this case, it is necessary to distinguish a start frame and a data frame from the captured image. In addition, although the frame rate for capturing the on/off image of the light source of the camera is preset in the present embodiment, accurate data reception may be possible even when the actual frame rate is changed.

The pixel scanner may generate a brightness signal according to a brightness value of an on/off image of a light source photographed for every plurality of rows in the camera. Specifically, as described above, in a process in which the light source is turned on or off according to data, a white band and a black band appear, and the brightness values of each of these bands may be different. For example, a color that appears according to on/off of the light source may be displayed, for example, as a brightness value of 0 to 255. In this case, a white band may indicate a brightness value of 255, and a black band may indicate a brightness value of 0. Of course, the range of these brightness values can be changed.

The demodulator may detect the bit sequence from the brightness signal of the on/off image of the light source generated by the OCC transmitter, and may detect the bit sequence by performing a convolution operation on the generated brightness sequence and the additionally generated inversion sequence.

The decoder may extract data from the detected bit sequence. This is to restore the data coded in the on/off image of the light source according to the data to be transmitted from the OCC transmitter. For example, if data 1 to be transmitted by the OCC transmitter corresponds to the on of the light source and data 0 corresponds to the off of the light source, the decoder extracts 1 from the on image of the light source and the off image 0 can be extracted. In this case, in the present embodiment, an output data sequence may be extracted by using the brightness value from the brightness signal of the on/off image of the light source. Specifically, the gradient of the brightness signal, that is, the rising and falling of the brightness signal may be combined and extracted.

Meanwhile, referring to FIG. 1 , the transmission light source detection system may include the transmission light source detection apparatus 100 , the user terminal 200 , the server 300 , and the network 400 .

In this embodiment, users may access an application or website implemented in the user terminal 200 and perform a process such as checking a result of detecting a transmission light source or checking received data information according to detection of a transmission light source, According to an embodiment, the camera of the user terminal 200 may be used as an optical camera communication (OCC) receiver.

The user terminal 200 may receive a service through an authentication process after accessing a transmission light source detection application or a transmission light source detection website. The authentication process may include, but is not limited to, authentication for inputting user information such as membership registration, authentication for a user terminal, etc., but is not limited thereto. The authentication process may be performed only by itself.

In this embodiment, the user terminal 200 is a desktop computer, a smartphone, a notebook computer, a tablet PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a GPS (global) operated by the user positioning system) devices, e-book terminals, digital broadcast terminals, navigation devices, kiosks, MP3 players, digital cameras, home appliances, and other mobile or non-mobile computing devices, but are not limited thereto. Also, the user terminal 200 may be a wearable terminal such as a watch, glasses, a hair band, and a ring having a communication function and a data processing function. The user terminal 200 is not limited to the above, and a terminal capable of web browsing may be borrowed without limitation.

In this embodiment, the transmission light source detection system may be implemented by the transmission light source detection apparatus 100 and/or the server 300 , wherein the server 300 is a transmission light source detection system including the transmission light source detection device 100 . It may be a server for operating In addition, the server 300 may be a database server that provides data for operating the transmission light source detection apparatus 100 . In addition, the server 300 may include a big data server required to apply various artificial intelligence algorithms, an AI server that performs calculations of various artificial intelligence algorithms, and a web server or application server that enables a light source detection system to be implemented. and may include the above-mentioned servers or network with these servers.

In this case, when the server 300 is an AI server, it is connected to the components constituting the transmission light source detection system environment through the network 400 , and may help at least a part of AI processing of the connected components. In this case, the server 300 may train the artificial neural network according to a machine learning algorithm instead of the components of the transmission light source detection system, and may directly store the learning model or transmit it to the components. At this time, the server 300 receives input data from the components, infers a result value with respect to the received input data using a learning model, generates a response or control command based on the inferred result value, and transmits it to the components. can Alternatively, the configurations may infer a result value with respect to the input data using the direct learning model, and generate a response or control command based on the inferred result value.

Here, artificial intelligence (AI) is a field of computer science and information technology that studies how computers can do the thinking, learning, and self-development that can be done with human intelligence. It could mean making it possible to imitate intelligent behavior.

Also, artificial intelligence does not exist by itself, but has many direct and indirect connections with other fields of computer science. In particular, in modern times, attempts are being made to introduce artificial intelligence elements in various fields of information technology and use them to solve problems in that field.

Machine learning is a branch of artificial intelligence, which can include fields of study that give computers the ability to learn without explicit programming. Specifically, machine learning can be said to be a technology to study and build a system and algorithms for learning based on empirical data, making predictions, and improving its own performance. Algorithms in machine learning can take the approach of building specific models to make predictions or decisions based on input data, rather than executing rigidly set static program instructions.

The network 400 may serve to connect the transmission light source detection apparatus 100 , the server 300 , and the user terminal 200 in the transmission light source detection system. The network 400 is, for example, wired networks such as local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), and integrated service digital networks (ISDNs), wireless LANs, CDMA, Bluetooth, and satellite communication. It may cover a wireless network such as, but the scope of the present invention is not limited thereto. Also, the network 400 may transmit/receive information using short-distance communication and/or long-distance communication. Here, the short-distance communication may include Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), ZigBee, and wireless fidelity (Wi-Fi) technologies. Communication may include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), single carrier frequency division multiple access (SC-FDMA) technology. can

Network 400 may include connections of network elements such as hubs, bridges, routers, switches, and gateways. Network 400 may include one or more connected networks, eg, multiple network environments, including public networks such as the Internet and private networks such as secure enterprise private networks. Access to network 400 may be provided via one or more wired or wireless access networks. Furthermore, the network 400 may support an Internet of Things (IoT) network and/or 5G communication that exchanges and processes information between distributed components such as things.

3 is a block diagram schematically illustrating an apparatus for detecting a transmission light source according to an embodiment of the present disclosure.

As shown in FIG. 3 , the apparatus 100 for detecting a transmission light source may include a camera 110 , a processor 120 , a memory 130 , and a user interface 140 .

The camera 110 may refer to a camera and/or an image sensor provided in a vehicle. However, as described above, since OCC in a vehicle is used as an embodiment in this embodiment, the camera 110 is provided in the vehicle, and when OCC in other devices is an embodiment, the camera 110 is connected to the device. can be provided. This can be equally applied to the following description.

The image sensor may be provided in the camera or may be configured separately. At this time, the vehicle may be equipped with a plurality of cameras, but receiving optical camera communication (OCC) based data may be performed with a single camera. In addition, when the vehicle is provided with a plurality of cameras, the single camera is not fixed to one camera and may be changed according to settings. In addition, the installation position of the camera 110 is not limited and may be installed in a position where it is easy to photograph the front side of the vehicle (depending on the embodiment, it may be possible to photograph all the surroundings of the vehicle, including the front, depending on the embodiment). The camera of the user terminal 200 may be implemented as a single camera.

That is, in the present embodiment, images of all light sources around the vehicle as well as the transmission light source for optical camera communication (OCC) may be acquired and collected using the camera 110 .

Meanwhile, the camera 110 of this embodiment may include an information receiving unit (not shown) and an image acquisition unit (not shown), and the information receiving unit may receive optical camera communication (OCC) based data from a light source, and image acquisition The unit may acquire an image within a field of view (FOV) of the camera 110 including the front side of the vehicle. The FOV is a range in which the camera 110 can acquire an image, and may be a range in which a signal can be recognized.

The memory 130 may store various types of information necessary for the process of the transmission light source detection apparatus 100 , and may store control software for processing of the transmission light source detection apparatus 100 , and may include a volatile or non-volatile recording medium. can

The memory 130 is connected to one or more processors 120 , and when executed by the processor 120 , may store codes causing the processor 120 to control the transmission light source detection apparatus 100 . have.

Here, the memory 130 may include magnetic storage media or flash storage media, but the scope of the present invention is not limited thereto. The memory 130 may include internal memory and/or external memory, and may include volatile memory such as DRAM, SRAM, or SDRAM, one time programmable ROM (OTPROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, Non-volatile memory, such as NAND flash memory, or NOR flash memory, SSD. It may include a flash drive such as a compact flash (CF) card, an SD card, a Micro-SD card, a Mini-SD card, an Xd card, or a memory stick, or a storage device such as an HDD.

The user interface 140 may include an input interface into which a user request and commands for detecting a transmission light source are input. For example, requests and commands such as receiving data from an optical camera communication (OCC)-based transmission light source and providing information on the received data to a rear or surrounding vehicle may be input.

In addition, the user interface 140 may include an output interface for outputting the OCC transmission light source detection result by the transmission light source detection apparatus 100 , information on data received from the corresponding light source, or a notification (or warning) message.

The input interface and the output interface of the user interface 140 may be implemented in the same interface, for example, a display device such as an AVN system in a vehicle, or an application and/or a website driven in the user terminal 200 . A user interface may be implemented through

The processor 120 may control the overall operation of the transmission light source detection apparatus 100 . Specifically, the processor 120 is connected to the configuration of the transmission light source detection apparatus 100 including the memory 130 , and executes at least one command stored in the memory 130 to operate the transmission light source detection apparatus 100 . can be controlled in general.

The processor 120 may be implemented in various ways. For example, the processor 120 may include an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), or a digital signal processor (Digital Signal). Processor, DSP) may be implemented as at least one of.

The processor 120 is a kind of central processing unit, and may control the entire operation of the transmission light source detection apparatus 100 by driving control software mounted on the memory 130 . The processor 120 may include any type of device capable of processing data. Here, the 'processor' may refer to a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by, for example, a code or an instruction included in a program. As an example of the data processing apparatus embedded in the hardware as described above, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit) and a processing device such as a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto.

4 is a diagram for explaining in more detail an apparatus for detecting a transmission light source according to an embodiment of the present disclosure.

Referring to FIG. 4 , a system structure for detecting an optical camera communication (OCC)-based data transmission light source among all light sources in an image acquired by the transmission light source detection apparatus 100 and receiving data from the detected transmission light source will be described. let's see

The processor 120 may acquire at least one light source image around the vehicle using the camera 110 provided in the vehicle. That is, the camera 110 is a receiver capable of receiving data of optical camera communication (OCC), and may acquire an image including a light source (eg, an LED area) for transmitting optical camera communication (OCC) data, In the acquired image, other general light sources that do not transmit OCC data may be included.

As shown in FIG. 4 , in this embodiment, an image including a light source (general light source) area of a traffic light and a light source (transmission light source) area of a tail light of a vehicle may be acquired through the camera 110 . That is, the present embodiment is to accurately detect the light source region of the tail light of the vehicle in the image in order to receive data transmitted from the tail light of the vehicle. Depending on the situation, since data can also be read from the image of the light source of the traffic light, it is necessary to properly classify and detect the light source area of the traffic light and the light source area of the vehicle's tail light in the acquired image.

That is, the present embodiment is to accurately acquire an image of a transmission light source that transmits optical camera communication (OCC) data among all light sources in an acquired image.

In this case, the processor 120 may acquire an image including at least one light source by using the camera 110 . In addition, the processor 120 may extract a feature from the light source region in the image by applying the acquired image to the image classification algorithm learned to identify the light source.

Here, the learned image classification algorithm, for example, CNN (Convolutional Neural Networks) may be applied. However, the present invention is not limited thereto, and other image classification algorithms may be applied depending on the embodiment, and the specific structure of the CNN may consist of various structures that can accurately infer the image of the light source that transmits optical camera communication (OCC) data. have.

As such, the image classification algorithm identifies light sources including a general light source and a transmission light source in an acquired image, and in this embodiment, features are extracted only from the light source area in the image, and features other than the light source are extracted. Since there is no extraction, there is an advantage in processing simplification after computation simplification.

In addition, the processor 120 may determine whether each light source in an image obtained by applying the extracted feature to a binary classification algorithm learned to identify a data transmission light source is an OCC-based data transmission light source.

In this case, the binary classification algorithm, for example, a support vector machine (SVM) may be applied. And the binary classification algorithm collects an image set including an image of a light source that transmits OCC-based data and an image of a general light source that does not transmit data, and performs morphological image processing on images in the collected image set. Training comprising the steps of performing, extracting features of images of the light source in the morphological image-processed image set into a training set and a test set, and training an initial binary classification algorithm using the training set It can be trained by phase.

And in this embodiment, the image of the OCC-based data transmission light source in the image set and the image of the general light source that does not transmit data are labeled as the transmission light source and the general light source, respectively, and the characteristics of the images in the training set and the test set are also respectively It may be labeled as a transmit light source and a normal light source.

Meanwhile, in this embodiment, in the step of performing morphological image processing of the training phase of the binary classification algorithm, a morphological gradient operation and image filtering process are applied to images in the collected image set to generate a preprocessed image. can For example, in the present embodiment, morphological image processing such as image closing and contour approximation may be performed.

More specifically, morphological image processing can also be expressed as a morphology technique, and it analyzes and processes the shape of an image by reflecting information on a target object whose geometric shape is known in advance and using it to extract only a desired part of the image. technique to study the structure of an object in an image by using it to extract elements necessary to express the shape of the image boundary, skeleton, block, etc. It can be classified as a technique to clearly indicate the area of an object and to clarify the structure of the object in the image. For example, in this embodiment, morphological image processing may be performed in order to clarify the structure of an object in an image.

The most basic operations constituting this morphological image processing (morphology technique) are an erosion operation, a dilation operation, and an opening (removal) operation and closing (closing) operation using the basic operations in combination. There is an arithmetic.

The erosion operation compares the 1 component of the erosion mask with the pixel value of the input image in the mask range. In this case, all positions having a value of 1 in the erosion mask must be 1 in the input image so that the pixel value of the resulting image can also be 1. In addition, the erosion operation holds a commutative law, and the degree of erosion may be determined according to the size of the morpheme. For example, if the size of the morpheme is large, the degree of erosion may be large. And if the same morpheme is repeatedly applied, the object can be completely removed, which can be useful for removing the protrusion of the object. In addition, the erosion operation is useful for removing background noise, but in the case of noise within an object, there is a disadvantage of expanding the internal noise, so that the size of the object is reduced and the background is enlarged.

In the expansion operation, when comparing the expansion mask and the input image pixel in the mask range, if even one input image pixel has a value of 1 at a position having a component of 1 in the expansion mask, the result image may be 1. The dilation operation has the advantage that the noise in the background is enlarged and the noise inside the object is removed (filled in), so the size of the object increases and the background decreases. In addition, the dilation operation is commutative and associative.

The open operation is also called a removal operation, and may refer to an algorithm using an erosion operation followed by an expansion operation. The open operation can soften the outline of the image by removing the protruding part and breaking the narrow connection, and it can be smoother if the operation is repeatedly applied. Also, the open operation can preserve the shape and size of the object. And in the open operation, the result of [A, B open operation] is a subset of A, and if A is a subset of C, [A, B open operation] is a subset of [C, B open operation], open It has the property that there is no effect of repeated application on the operation.

The closing operation is also called a filling operation, and may refer to an algorithm that uses an erosion operation after an expansion operation. The closing operation fills concave recesses or small holes, so that the outline part of the image can be smoothed and the shape and size of the object can be preserved. Also, for closing operation, A is a subset of the result of [A, B closing operation], and if A is a subset of C, [A, B closing operation] is a subset of [C, B closing operation]. It has the property that there is no effect of repeated application.

And in morphological image processing, there is also a morphological gradient method, which is about boundary detection using dilation and erosion, which means inner boundary + outer boundary. The morphological gradient can be obtained by calculating the difference between the results of the dilation operation and the erosion operation.

The morphological image processing as described above can be applied to a binary image, and an output pixel value is determined by performing a logical operation with a matrix called a morpheme on the brightness value of the binary image to find a specific pattern in the binary image. Here, morphemes (Structuring Elements, SE, structural elements) are similar to the mask concepts used in low-pass filters and high-pass filters, and may mean expressing any shape with arbitrary size and structure that can be expressed as a binary image. .

In addition, morphological image processing can be applied to a contrast image, and when the difference between the brightness of an object and the brightness of the background is large, the erosion and dilation calculations of the contrast image can be effectively applied. The brightness can be adjusted by selecting the maximum and minimum values among the values obtained as a result of operation with morphemes (pixel-unit addition), and it can also be used to remove impulse noise.

For example, in the erosion operation in the contrast image, the minimum value can be selected among the values obtained as the result of the operation with the morpheme (pixel-unit addition), and the effect of reducing the brightness by making the bright object appear darker, and the area with the non-uniform brightness can be effectively dealt with in

In addition, the dilation operation in the contrast image can select the maximum value among the values obtained as a result of operation with morphemes (pixel unit addition), and has the effect of making the object appear larger by making the object brighter, and the brightness is not uniform. area can be effectively treated.

And the open operation in the contrast image reduces the brightness of all bright features, so the dark features and background of the image are relatively unaffected. Details and background are relatively unaffected.

In addition, the morphological smoothing operation may mean that the closing operation is performed after the opening operation, and may be used as a morphological filter for image smoothing and noise removal through the morphological smoothing operation.

Meanwhile, in the present embodiment, the processor 120 may identify a region in the image with a light source, and extract features from the identified region of the light source. For example, in the present embodiment, a region in which a light source is located may be identified using a region of interest algorithm (RoI).

In this case, the characteristic may include a change in the number of stripes detected in the light source area and an interval between the stripes in the light source area. And the binary classification algorithm may be configured to determine whether the light source is an OCC-based data transmission light source for each light source, based on the number of stripes and a change in the spacing between the stripes in the light source area.

For example, in the image, an area with a light source of a traffic light, which is a general light source, and an area with a light source of a vehicle tail light, which is a transmission light source for transmitting an OCC signal, may be identified. When the characteristics of the identified light source area are extracted, for example, when the number of stripes of the light source area obtained with a rolling shutter type camera is too small, or the interval between the stripes of the light source area is too constant or regular, the signal in the light source area is It is determined that is not being transmitted, it is possible to extract a feature for the general light source region, and the extracted feature can be labeled as a feature of the general light source. That is, in the case of a transmission light source that transmits an OCC signal, since the signal is being transmitted, the number of stripes in the light source area cannot be too small below the reference value in order to express data, and the distance between the stripes in the light source area is also constant above a certain level. or cannot be regular. A reference value or a certain level for classifying them may be preset, but may also be updated in the course of performing learning.

On the other hand, in the present embodiment, the training phase of the binary classification algorithm further includes, after the step of training the initial binary classification algorithm using the training set, confirming the accuracy of the trained binary classification algorithm using the test set. can do. And the training phase of the binary classification algorithm ends the training when the accuracy exceeds the threshold, and performs image processing by changing the preprocessing method of the step of performing morphological image processing when the accuracy is below the threshold, extracting and may be configured to re-perform the training step.

Also, in this embodiment, the training phase of the binary classification algorithm may further include, after training the initial binary classification algorithm using the training set, confirming the accuracy of the trained binary classification algorithm using the test set. can And the training phase of the binary classification algorithm is configured to terminate the training when the accuracy exceeds the threshold, and re-perform the extraction and training steps by changing the feature extraction method of the step of extracting the feature when the accuracy is below the threshold. can In this case, the image classification algorithm may be generated based on the feature extraction method adopted when the accuracy is greater than or equal to the threshold. For example, in this embodiment, since the pattern of the light source (eg, LED) that transmits actual data is different, in order to accurately classify the light source, there are several geometric features effectively used in the pre-learned binary classification algorithm (eg, classifier). It may be selected and configured to re-perform the extracting and training by changing the feature extraction method of the step of extracting the feature when the accuracy is less than or equal to the threshold.

That is, in the present embodiment, when the accuracy of the binary classification algorithm is less than or equal to the threshold, image processing can be performed by changing the pre-processing method of the step of performing morphological image processing, and image processing is performed by changing the pre-processing method. Even when the accuracy of the previous learning is confirmed later, if the accuracy of the binary classification algorithm is below the threshold, the preprocessing method is not changed and based on the feature extraction method of the transmission light source and the general light source adopted when the accuracy is above the threshold, The feature can be extracted by changing the feature extraction method of the extracting step. However, the present invention is not limited thereto, and both the preprocessing method and the feature extraction method may be configured to be changed.

Here, the feature may include a geometrical feature of the light source, the geometrical feature being the area of each stripe of the light source, the sum of the perimeters of all stripes of the light source, and a line representing the stripe pattern of the light source It may include at least one of the number of segments. Here, the number of line segments of the stripe of the light source may be derived using the Douglas-Peucker algorithm in the light source region identified through the ROI algorithm in the acquired image. The Douglas-Poker algorithm can divide the curve into line segments according to a predetermined precision.

5 is a diagram for explaining a method of deriving the number of line segments from a light source image according to an embodiment of the present disclosure.

The Douglas-Poker algorithm is applied to the light source region identified through the ROI algorithm in the captured image to digitize the original curve of the light source image to determine the dominant point on the curve of the light source image, and to You can create curves.

As shown in FIG. 5 , a main point on the curve of the light source image is determined by the Douglas-Poker algorithm, and when the main points are connected, a line segment is generated, and the number of line segments can be calculated.

6 is a graph showing the number of line segments and the area of a light source region derived according to an embodiment of the present disclosure for a light source that transmits data and a general light source that does not transmit data.

Referring to the graph of FIG. 6 , it can be seen that a light source that transmits data (eg, an LED) and a general light source that does not transmit data can be determined according to the number of line segments and the area of the LED region.

In this embodiment, through the combination of the image classification algorithm and the binary classification algorithm, the transmission light source region is finally extracted from the image including the general light source and the transmission light source, that is, the light source regions in the image are already classified through the classification algorithm. After identification, a region of a transmission light source among the identified light source regions is accurately determined through a binary classification algorithm such as SVM.

For example, in the present embodiment, the characteristic of nonlinear data may be converted into a polynomial form using SVM. That is, since not all data can be classified linearly, the characteristic of nonlinear data is converted into a polynomial form so that it can be classified linearly. To this end, a polynomial object is created, a polynomial is obtained by training the object with nonlinear data, and then SVM classification can be performed using the obtained polynomial.

In SVM, methods widely used to match data to a high-dimensional space include a polynomial kernel method and a Gaussian kernel (RBF kernel to be described later). The polynomial kernel method refers to calculating all possible combinations of original features up to a specified order, and the Gaussian kernel (RBF kernel) refers to mapping to a feature space with infinite dimensions. In this embodiment, an RBF kernel may be applied, and a detailed description will be given later.

In this embodiment, the binary classification algorithm applies a radial basis function (RBF) kernel that extends data characteristics to all polynomials of all infinite degrees, and maps data to a feature space with infinite dimensions. can do.

The radial basis function (RBF) is also called a Gaussian RBF kernel function and can be expressed as Equation (1).

)는 파라미터로 커널의 흩어짐을 세팅할 수 있다. 즉 가우시안 분포에서 보면, 분산에 해당하는 부분일 수 있다. 수학식 1은 벡터 l 이라는 랜드마크와 벡터 x가 얼마나 가까운지에 따라 0에서 1사이의 값을 보이는데, 가장 가까울 때 1, 멀 때 0의 값을 가진다. 가우시안 RBF는 비 선형적인(non-linear) 분류기를 생성할 때 사용하기 좋으며 그 이유는 다음과 같다.Here, gamma (

) can set the kernel scatter as a parameter. That is, when viewed from a Gaussian distribution, it may be a part corresponding to the variance. Equation 1 shows a value between 0 and 1 depending on how close the landmark of vector l is to vector x, and has a value of 1 when closest and 0 when far away. Gaussian RBF is good for creating non-linear classifiers, for the following reasons.

For example, if there are n points in an m-dimensional vector space, one point is taken as l , and a Gaussian RBF kernel function is applied to all points x, then the distance n from l to all points is characterized. ) can be created as a vector. That is, it is possible to transform a vector from m-dimensional to n-dimensional. If this mapping is performed for all n points, the previously close points will be similarly close in the new space due to the nature of this function. And assuming that m < n, the dimension will also increase in this process. That is, through the above process, the dimension is increased, but the points that were previously close are similarly close, so that the separation may be easier.

As such, the RBF kernel can be projected into infinite dimensions, and a network based on such a radial structure, for example, has one hidden layer, uses Euclidean distances, uses a backpropagation algorithm, and is stable. discrimination may be possible.

)에 의해 생성될 수 있다. 이때 작은 감마(

) 값은 가우시안 커널의 반경을 크게 하여 많은 포인트들이 가까이 있는 것으로 고려할 수 있으며, 결정 경계를 천천히 바뀌게 하므로 모델의 복잡도를 낮출 수 있다. 반면 감마(

) 값이 커짐에 따라 결정 경계는 하나의 포인트에 더 민감해지고 더 복잡한 모델을 생성하게 된다. Meanwhile, looking briefly at how the SVM using the RBF kernel makes decisions, first, to predict a new data point, the distance from each support vector (data point located on the boundary between two classes) is measured. The classification decision is based on the distance to the support vector, and the importance can be learned during training. An SVM boundary is created using the data set, and this boundary is defined by two parameters: C (regulatory parameter) and gamma (

) can be created by At this time, a small gamma (

) value increases the radius of the Gaussian kernel so that many points can be considered close, and the complexity of the model can be lowered because the decision boundary is changed slowly. On the other hand, gamma (

), the decision boundary becomes more sensitive to one point and produces more complex models.

And when the value of C is small, a model with large constraints is created and the influence of each point is small. As the value of C increases, each point has a large influence on the model, and it can be accurately classified by bending the decision boundary.

On the other hand, in this embodiment, the binary classification algorithm tests the non-stochastic binary linear classification model, and when the accuracy exceeds the threshold, in a training phase further comprising the step of completing the learning of the binary classification algorithm. It may be a learning model trained by On the other hand, in the present embodiment, by testing the non-stochastic binary linear classification model, if the accuracy is less than or equal to a threshold, image processing based on the image classification algorithm may be performed again. In other words, if the binary classification algorithm determines that classification is performed accurately with an accuracy exceeding the threshold, learning is completed, and if the accuracy is low, image processing is performed again and repeated until the accuracy exceeds the threshold. can do.

7 is a flowchart illustrating a method of detecting a transmission light source according to an embodiment of the present disclosure, and FIG. 8 is a flowchart illustrating a learning process of a method of detecting a transmission light source according to an embodiment of the present disclosure. In the following description, descriptions of parts overlapping with those of FIGS. 1 to 4 will be omitted.

7 , in step S100 , the transmission light source detection apparatus 100 acquires an image including at least one light source using the camera 110 .

That is, the camera 110 is a receiver capable of receiving data of optical camera communication (OCC), and may acquire an image including a light source (eg, an LED area) for transmitting optical camera communication (OCC) data, In the acquired image, other general light sources that do not transmit OCC data may be included.

In step S200, the transmission light source detection apparatus 100 applies the acquired image to the image classification algorithm learned to identify the light source to extract a feature from the light source region in the image.

Here, the learned image classification algorithm, for example, CNN (Convolutional Neural Networks) may be applied. However, the present invention is not limited thereto, and other image classification algorithms may be applied depending on the embodiment, and the specific structure of the CNN may consist of various structures that can accurately infer the image of the light source that transmits optical camera communication (OCC) data. have.

As such, the image classification algorithm identifies light sources including a general light source and a transmission light source in an acquired image, and in this embodiment, features are extracted only from the light source area in the image, and features other than the light source are extracted. Since there is no extraction, there is an advantage in processing simplification after computation simplification.

And in step S300, the transmission light source detection apparatus 100 determines whether the OCC-based data transmission light source for each light source in the image obtained by applying the extracted features to the learned binary classification algorithm to identify the data transmission light source.

In this case, the binary classification algorithm, for example, a support vector machine (SVM) may be applied.

On the other hand, in the acquired image, when there are a plurality of light source regions identified through the image classification algorithm, the transmission light source detection apparatus 100 applies the light source regions identified through the image classification algorithm to the binary classification algorithm to obtain a plurality of light source regions. Among them, an optical camera communication (OCC)-based data transmission light source can be detected. Also, in the acquired image, when there is one light source region identified through the image classification algorithm, it is possible to identify whether the corresponding light source region is a general light source or a transmission light source through a binary classification algorithm.

That is, in this embodiment, the optical camera communication (OCC)-based data transmission light source can be accurately detected through the light source detection algorithm in which the image classification algorithm and the binary classification algorithm are combined. The learning method of this transmission light source detection algorithm is shown in FIG. can be explained based on

Referring to FIG. 8 , in step S10 , the transmission light source detection apparatus 100 collects an image set including an image of an OCC-based data transmission light source and an image of a general light source that does not transmit data.

For example, the OCC-based data transmission light source may mean a vehicle tail light capable of transmitting vehicle information, etc., and a general light source that does not transmit data may mean a traffic light or a fluorescent lamp.

In step S20 , the transmission light source detecting apparatus 100 performs morphological image processing on images in the collected image set.

In this embodiment, a preprocessed image may be generated by applying a morphological gradient operation and an image filtering process to images in the collected image set. For example, in the present embodiment, morphological image processing such as image closing and contour approximation may be performed.

Here, the morphological image processing can also be expressed as a morphology technique, and in this embodiment, an ambiguous object region that occurs in the binarization process of an image, such as hole filling and noise removal, is clearly indicated, so that the structure of the object in the image is clearly shown. To do this, morphological image processing can be performed. Also, for example, in the present embodiment, morphological image processing may be performed through a closing operation, a morphological gradient operation, and the like.

And in step S30, the transmission light source detecting apparatus 100 extracts features of the images of the light source in the morphological image-processed image set as a training set and a test set.

In this case, the features of the images of the light source may be arbitrarily divided into a training set and a test set and extracted, but the present invention is not limited thereto, and various methods of being divided into a training set and a test set and extracted may be applied.

And in step S40, the transmission light source detection apparatus 100 trains the initial binary classification algorithm using the training set.

Thereafter, in step S50 , the transmission light source detection apparatus 100 checks the accuracy of the trained binary classification algorithm using the test set.

In step S60, when the accuracy of the trained binary classification algorithm exceeds the threshold (Yes), in step S70, the apparatus 100 for detecting a transmission light source may end the training.

On the other hand, if the accuracy of the trained binary classification algorithm is less than the threshold (No), changing the pre-processing method of the morphological image processing step (S20) to perform image processing (S20), extracting (S30) ) and the training phase of the binary classification algorithm can be configured to re-perform the training step ( S40 ). In this case, the image classification algorithm may be generated based on a feature extraction method adopted when the accuracy is greater than or equal to a threshold.

And according to an embodiment, when the accuracy of the trained binary classification algorithm is less than or equal to the threshold, the extracting step (S30) and the training step (S40) are re-performed by changing the feature extraction method of the step (S30) of extracting the feature. The training phase of the binary classification algorithm can be configured.

That is, in the present embodiment, when the accuracy of the binary classification algorithm is less than or equal to the threshold, image processing can be performed by changing the pre-processing method of the step of performing morphological image processing, and image processing is performed by changing the pre-processing method. Even when the accuracy of the previous learning is confirmed later, if the accuracy of the binary classification algorithm is below the threshold, the preprocessing method is not changed and based on the feature extraction method of the transmission light source and the general light source adopted when the accuracy is above the threshold, The feature can be extracted by changing the feature extraction method of the extracting step. However, the present invention is not limited thereto, and both the preprocessing method and the feature extraction method may be configured to be changed.

Here, the characteristic may include a geometrical characteristic of the light source, the geometric characteristic being the area of each stripe of the light source, the sum of the perimeters of all stripes of the light source, and the number of line segments representing the stripe pattern of the light source. may include at least one of

The above-described embodiment according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium includes a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and a ROM. , RAM, flash memory, and the like, and hardware devices specially configured to store and execute program instructions.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and used by those skilled in the computer software field. Examples of the computer program may include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

In the specification of the present invention (especially in the claims), the use of the term "above" and similar referential terms may be used in both the singular and the plural. In addition, when a range is described in the present invention, each individual value constituting the range is described in the detailed description of the invention as including the invention to which individual values belonging to the range are applied (unless there is a description to the contrary). same as

The steps constituting the method according to the present invention may be performed in an appropriate order unless the order is explicitly stated or there is no description to the contrary. The present invention is not necessarily limited to the order in which the steps are described. The use of all examples or exemplary terms (eg, etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is limited by the examples or exemplary terms unless limited by the claims. it's not going to be In addition, those skilled in the art will appreciate that various modifications, combinations, and changes may be made in accordance with design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the spirit of the present invention is not limited to the scope of the scope of the present invention. will be said to belong to

100: transmission light source detection device
110: camera
120 : processor
130: memory
140 : user interface
200: user terminal
300 : server
400: network

Claims (19)

As a transmission light source detection method for detecting a light source that transmits data for optical camera communication (OPTICAL CAMERA COMMUNICATION, OCC),
acquiring an image including at least one light source using a camera;
extracting a feature from a light source region in the image by applying the acquired image to an image classification algorithm learned to identify a light source; and
Comprising the step of applying the extracted features to a binary classification algorithm learned to identify a data transmission light source, determining whether each light source in the acquired image is an OCC-based data transmission light source,
The binary classification algorithm is
collecting an image set including an image of an OCC-based data transmission light source and an image of a normal light source that does not transmit data;
performing morphological image processing on images in the collected image set;
extracting features of images of a light source in the morphological image-processed image set into a training set and a test set; and
trained by a training phase comprising training an initial binary classification algorithm using the training set;
The image of the OCC-based data transmission light source in the image set and the image of the general light source that does not transmit data are labeled as the transmission light source and the normal light source, respectively,
Features of the images in the training set and the test set are also labeled as transmit light and normal light, respectively.
How to detect a transmitting light source.
delete The method of claim 1,
The step of performing the morphological image processing,
Comprising the step of generating a pre-processed image by applying a morphological gradient operation and image filtering process to the images in the collected image set,
How to detect a transmitting light source.
The method of claim 1,
The training phase is
After the training step,
Confirming the accuracy of the trained binary classification algorithm using the test set,
In the training phase, when the accuracy exceeds a threshold, training is terminated, and when the accuracy is less than or equal to a threshold, changing the pre-processing method of performing the morphological image processing to perform the image processing, the step of extracting and re-performing the training step.
How to detect a transmitting light source.
The method of claim 1,
The training phase is
After the training step,
Confirming the accuracy of the trained binary classification algorithm using the test set,
The training phase is configured to end training when the accuracy exceeds a threshold, and to re-perform the extraction and training by changing the feature extraction method of the step of extracting the feature when the accuracy is less than or equal to the threshold felled,
How to detect a transmitting light source.
6. The method of claim 5,
The image classification algorithm is
generated based on the feature extraction method adopted when the accuracy is greater than or equal to a threshold,
How to detect a transmitting light source.
The method of claim 1,
The characteristic includes a geometric characteristic of the light source,
wherein the geometrical characteristic comprises at least one of the area of each stripe of the light source, the sum of the perimeters of all stripes of the light source, and the number of line segments representing the stripe pattern of the light source;
The number of line segments is derived by applying the Douglas-Poker algorithm to the image of the light source region,
How to detect a transmitting light source.
The method of claim 1,
The step of extracting the feature is
identifying an area within the image with a light source; and
extracting features for the identified light source region;
The characteristic includes a change in the number of stripes detected in the light source area and a spacing between the stripes in the light source area,
the binary classification algorithm is configured to determine whether it is an OCC-based data transmission light source for each of the light sources based on the number of stripes and a change in spacing between stripes within the light source area;
How to detect a transmitting light source.
The method of claim 1,
The learned binary classification algorithm is
By applying a Radial Basis Function (RBF) kernel that extends the characteristics of the data to all polynomials of all infinite degrees, mapping the data to the feature space with infinite dimensions.
How to detect a transmitting light source.
A computer-readable recording medium storing a computer program for executing the method of any one of claims 1 and 3 to 9 using a computer.
As a transmission light source detection device for detecting a light source that transmits data for optical camera communication (OPTICAL CAMERA COMMUNICATION, OCC),
Memory; and
at least one processor coupled to the memory and configured to execute computer readable instructions contained in the memory;
the at least one processor,
An operation of acquiring an image including at least one light source using a camera,
extracting a feature from a light source region in the image by applying the acquired image to an image classification algorithm learned to identify a light source; and
configured to apply the extracted feature to a binary classification algorithm learned to identify a data transmission light source to perform an operation of determining whether it is an OCC-based data transmission light source for each light source in the acquired image,
The binary classification algorithm is
collecting an image set including an image of an OCC-based data transmission light source and an image of a normal light source that does not transmit data;
performing morphological image processing on images in the collected image set;
extracting features of images of a light source in the morphological image-processed image set into a training set and a test set; and
trained by a training phase comprising training an initial binary classification algorithm using the training set;
The image of the OCC-based data transmission light source in the image set and the image of the general light source that does not transmit data are labeled as the transmission light source and the normal light source, respectively,
Features of the images in the training set and the test set are also labeled as transmit light and normal light, respectively.
Transmitting light source detection device.
delete 12. The method of claim 11,
The binary classification algorithm is
configured to perform an operation of generating a pre-processed image by applying a morphological gradient operation and an image filtering process to images in the collected image set in the step of performing the morphological image processing,
Transmitting light source detection device.
12. The method of claim 11,
The training phase is
After the training step,
Confirming the accuracy of the trained binary classification algorithm using the test set,
In the training phase, when the accuracy exceeds a threshold, training is terminated, and when the accuracy is less than or equal to a threshold, changing the pre-processing method of performing the morphological image processing to perform the image processing, the step of extracting and re-performing the training step.
Transmitting light source detection device.
12. The method of claim 11,
The training phase is
After the training step,
Confirming the accuracy of the trained binary classification algorithm using the test set,
The training phase is configured to end training when the accuracy exceeds a threshold, and to re-perform the extraction and training by changing the feature extraction method of the step of extracting the feature when the accuracy is less than or equal to the threshold felled,
Transmitting light source detection device.
16. The method of claim 15,
The image classification algorithm is generated based on the feature extraction method adopted when the accuracy is greater than or equal to a threshold,
Transmitting light source detection device.
12. The method of claim 11,
The characteristic includes a geometric characteristic of the light source,
wherein the geometrical characteristic comprises at least one of the area of each stripe of the light source, the sum of the perimeters of all stripes of the light source, and the number of line segments representing the stripe pattern of the light source;
The number of line segments is derived by applying the Douglas-Poker algorithm to the image of the light source region,
Transmitting light source detection device.
12. The method of claim 11,
The operation of extracting the feature is
identifying an area within the image in which the light source is located; and
configured to further perform an operation of extracting features for the identified light source region,
The characteristic includes a change in the number of stripes detected in the light source area and a spacing between the stripes in the light source area,
the binary classification algorithm is configured to determine whether it is an OCC-based data transmission light source for each of the light sources based on the number of stripes and a change in spacing between stripes within the light source area;
Transmitting light source detection device.
12. The method of claim 11,
The learned binary classification algorithm is
By applying a Radial Basis Function (RBF) kernel that extends the characteristics of the data to all polynomials of all infinite degrees, mapping the data to the feature space with infinite dimensions.
Transmitting light source detection device.

Images