KR102118145B1 - Optical camera communication device and method using smart device - Google Patents

Abstract

The present invention relates to an optical camera communication device and method using a smart device, and more particularly, to designate data to be transmitted to at least one of an R (Red) signal, a G (Green) signal, and a B (Blue) signal, respectively. Modulator; A driver for outputting a control signal for the emission ratios of the R signals, G signals, and B signals within a light emission cycle; And a light emitting unit including at least one light emitting diode (LED) emitting light according to the control signal, wherein the light emitting unit includes at least one anchor as a reference for allocating a region for each of the at least one LED. ) Includes LED.

Description

OPTICAL CAMERA COMMUNICATION DEVICE AND METHOD USING SMART DEVICE

The present invention relates to an optical camera communication device and method using a smart device, and more specifically, to apply data to the image transmitted by the light emitting diode by applying automatic zone allocation to an image obtained by photographing the light emitting diode with a camera provided in the smart device. It relates to an optical camera communication device and method using a smart device for faster recognition.

With the development of information and communication technology, various and convenient communication services that can meet the needs of users regardless of time and place have been actively researched and emerged from the past. As a typical example, short-range wireless communication technologies utilizing radio frequency (RF) and image recognition technologies such as Wi-Fi, Zigbee, Bluetooth, and QR codes are commercially used.

The technology of short-range wireless communication mainly operates on a radio frequency basis, and congestion is gradually increasing as many limited frequency resources are used. Optical wireless communication using light-emitting diodes has the advantage of being harmless to the human body, since there is no problem of frequency permission, and it is possible to directly check the communication path because it uses light to communicate, unlike conventional wireless communication means.

Optical camera communication, which is a camera-based visible light wireless communication, receives data transmitted using a LED (Light Emitting Diode) lamp using a camera and an image sensor mounted on a smart device such as a smartphone or a webcam. to be.

Optical camera communication uses the color change according to the blinking of the RGB LED for data transmission, and uses the camera of the smart device as the receiver to send data according to the color of the blinking LED, and the color condition value of the image obtained through the camera According to the data is restored.

Therefore, in the image processing method, a technique for reducing unnecessary image processing operations by automatically allocating and processing an area for recognizing data and increasing the amount of data that can be transmitted on one screen by using the margins of the resulting image need to be developed. There is.

Therefore, the present invention has been proposed to solve the above problems, and the installation cost by allowing data transmitted through the light emitting diode module to be recognized through a camera provided in a smart device under an indoor environment without a separate device. It is to provide an optical camera communication device and method using a smart device to reduce the.

In addition, the present invention provides an optical camera communication apparatus and method using a smart device that can improve the speed of recognizing data of an LED by reducing unnecessary image processing operations by automatically allocating and processing an area for recognizing data. Is in

The objects of the present invention are not limited to those mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

Optical camera communication device using a smart device according to the present invention for achieving the above object, the data to be transmitted is assigned to at least one of the R (Red) signal, G (Green) signal and B (Blue) signal, respectively Modulator; A driver for outputting a control signal for the emission ratios of the R signals, G signals, and B signals within a light emission cycle; And a light emitting unit including at least one light emitting diode (LED) emitting light according to the control signal, wherein the light emitting unit includes at least one anchor as a reference for allocating a region for each of the at least one LED. ) Includes LED.

In addition, the optical camera communication apparatus using a smart device according to the present invention, a sensor unit including an image sensor for receiving light from at least one LED; Zones for at least one LED are allocated based on at least one anchor LED in the image data obtained from the image sensor, and R (Red), G (Green), and B (Blue) for the LEDs in each of the allocated zones ) A signal processing unit for determining a signal; A demodulation unit for demodulating predetermined data using the determination signal of the image processing unit; And an output unit for outputting the demodulated data.

In addition, an optical camera communication method using a smart device according to the present invention comprises the steps of: acquiring image data through an image sensor that receives light from at least one LED of a sensor unit; An image processing unit identifying at least one anchor LED from the acquired image data, and allocating zones for at least one LED based on the identified anchor LED; Determining, by the image processing unit, R (Red), G (Green), and B (Blue) signals for the LEDs in each of the allocated zones; A demodulator demodulating predetermined data using a determination signal; And outputting the demodulated data by an output unit.

According to the present invention, it is possible to reduce the installation cost by allowing data transmitted through the light emitting diode module to be recognized through a camera provided in a smart device under an indoor environment without a separate device.

In addition, according to the present invention, by automatically allocating and processing an area for recognizing data, unnecessary image processing operations can be reduced to improve the speed of recognizing data of the LED.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1A is a diagram showing an example of zone allocation according to the prior art,
1B is a diagram showing an example of zone allocation according to an embodiment of the present invention;
2 is a block diagram schematically showing the configuration of an optical camera communication system according to an embodiment of the present invention;
3 is a flow chart showing a method of optical camera communication in a receiving device according to an embodiment of the present invention.

The objectives and effects of the present invention, and technical configurations for achieving them, will be clarified with reference to embodiments described below in detail together with the accompanying drawings. In describing the present invention, when it is determined that a detailed description of known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of donation in the present invention, which may vary according to a user's or operator's intention or practice.

However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. Only the present examples are provided to make the disclosure of the present invention complete, and to fully inform the person of ordinary skill in the art to which the present invention pertains, the scope of the invention being defined by the scope of the claims. It just works. Therefore, the definition should be made based on the contents throughout this specification.

On the other hand, in the specification, when a part is "connected" with another part, it is not only "directly connected" but also "indirectly connected" with another member in between. Includes. Also, when a part “includes” a certain component, this means that other components may be further provided instead of excluding other components, unless specifically stated otherwise.

Hereinafter, an optical camera communication apparatus and method using a smart device according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1A is a diagram illustrating an example of zone allocation according to the prior art, and FIG. 1B is a diagram illustrating an example of zone allocation according to an embodiment of the present invention.

In general, when an image is obtained by photographing an LED with a camera provided in a smart device, the entire acquired image becomes a zone, and the color of the entire screen is analyzed to demodulate the data. However, since the speed of the camera provided in the ordinary smart device is about 30 fps, there is a problem in that the data processing speed becomes slow.

In order to overcome this problem, in the related art, as shown in FIG. 1A, the entire camera screen, that is, the entire image is allocated to 21 zones to process 21 data in parallel.

That is, the image acquired from the camera is allocated to 21 zones and processed in parallel. Since this is the entire screen of the camera divided, the entire image is read and the data is processed. It has the problem that it takes some time to process the image.

To this end, the present invention processes an image by automatically allocating an area only for an LED portion where data is present in an image obtained from the camera as shown in FIG. 1B. That is, the operation is performed only on the necessary portion without the operation on the unnecessary portion.

This is possible by providing anchor LEDs 11a and 11b at the upper left and lower right, respectively, and these anchor LEDs 11a and 11b serve as criteria for identifying the LED portion where data is present. To this end, at least one anchor LED is installed adjacent to one upper end and the other lower end of a region in which at least one LED is formed, respectively.

In this way, in the image obtained from the camera, an anchor LED can be used to dynamically allocate a zone to the LED of the transmitting device.

2 is a block diagram schematically showing the configuration of an optical camera communication system according to an embodiment of the present invention.

2, the optical camera communication system includes a transmitting device 110 and a receiving device 130, wherein the transmitting device 110 includes an input unit 111, a modulation unit 113, a driving unit 115 and It includes a light emitting unit 117, the receiving device 130 includes a sensor unit 131, an image processing unit 133, a demodulation unit 135 and an output unit 137. Here, the receiving device 130 may be a smart device.

First, when looking at the transmission device 110, the input unit 111 is a wired/wireless port (eg, USB connector, infrared light receiving unit, Bluetooth communication unit, RF communication, etc.) for inputting information from the outside. Micro SD slot, etc.). Data to be transmitted is input through the input unit 111.

The modulator 113 designates digitized information from the input unit 111 to at least one of an R (RED) signal, a G (Green) signal, and a B (Blue) signal, respectively.

The modulator 113 generates and includes a start signal and an end signal for a series of data according to signals assigned to each channel, and then outputs them repeatedly.

The driver 115 controls the color of light when they emit light. That is, the emission rate of the R signal, the G signal, and the B signal is controlled within one emission period T.

The light emitting unit 117 includes at least one LED and at least one anchor LED.

Here, the at least one LED indicates the data to be transmitted, and the at least one anchor LED serves as a reference for allowing the receiving device 130 to determine an area requiring calculation, that is, at least one where data is present. It is possible to know the area where the above LED is provided.

On the other hand, looking at the receiving device 130, the sensor unit 131 includes an image sensor, the image sensor photographs the light emitting unit 117 of the transmitting device.

The image processing unit 133 allocates an area for at least one or more LEDs having data based on the anchor LED in the image taken by the sensor unit 131, that is, the acquired image.

For example, the image processing unit 133 checks whether the data value of the anchor LED satisfies the condition while moving from the upper portion of one side to the lower portion of the other side of the acquired image, and stores coordinates of the portion that satisfies the condition. At the same time, the image processing unit 133 checks whether the data value of the anchor LED meets the condition while moving from the other lower portion of the acquired image to the upper portion of one side, and stores the coordinates of the portion that meets the condition. From the coordinates of the two parts stored in this way, an area equipped with at least one LED having data is relatively allocated.

In addition, the image processor 133 extracts digitized data included in each signal by determining R, G, and B signals using RGB filters from light emitted from the LEDs in each allocated area.

The demodulator 135 demodulates the data before modulation by using the data for each signal extracted from the image processor 133.

The output unit 137 displays, reproduces, or stores the demodulated data from the demodulator 135. At this time, the output unit 137 may be a recording medium (flash memory) of a smart device, a speaker, a display, a GPS, or a vibration device.

3 is a flowchart illustrating an optical camera communication method in a receiving device according to an embodiment of the present invention.

First, the sensor unit 131 acquires image data by photographing the light emitting unit 117 of the transmitting device 110 through the image sensor (S301), and the image processing unit 133 checks the anchor LED in the acquired image data (S303).

Subsequently, the image processing unit 133 automatically determines an area provided with at least one LED having data based on the identified anchor LED and allocates the area relatively (S305), and the LED of each assigned area. The digitized data is extracted by determining the R, G, and B signals from the light emitted from (S307).

The demodulator 135 demodulates the extracted digitized data in the form of data before modulation, and the output unit 137 displays, reproduces or stores the demodulated data (S309).

Experimental Example

After setting the communication distance between the transmitting device 110 and the receiving device 130 to 20cm, the experiment was conducted by changing the color of the LED and transmitting 200 frames to check the result.

The transmitting device 110 was provided with 64 LEDs in the light emitting unit 117, and the receiving device 130 used the SM-G850 model of Samsung Electronics.

The image obtained by photographing the light emitting unit 117 of the transmitting device through the camera provided in the receiving device 130 is automatically allocated based on the anchor LED, and the RGB color data obtained in each area is a constant demodulation algorithm. The data was restored through. Here, the demodulation algorithm determines red when R>200, G<50, B<50, and green when R<50, G>200, B<50, and R<50, G<50, B If it is >200, it is set to judge in blue, but this is only an example.

Table 1 below shows the results of the experiment by dividing the transmission device 110 into a case where the anchor LED is not provided and a case where the anchor LED is provided under these experimental conditions. At this time, the average time required for color processing before and after applying the anchor LED is recorded every 30 frames.

Before applying anchor LED After applying the anchor LED 7.266667 ms 3.4333334 ms 7.8333 335 ms 3.3 ms 6.4 ms 3.3666666ms 6.9333334ms 2.9 ms 6.9666667ms 2.6 ms 7.266667 ms 2.9 ms 6.8 ms 3.2333333ms 7.0 ms 2.3333 333 ms 6.866667ms 3.6333334 ms 7.1666665 ms 3.1666667 ms 7.5666666ms 3.4666667 ms 6.866667ms 3.1 ms 7.366667 ms 3.1666667 ms 7.5 ms 2.9 ms 7.366667 ms 3.3 ms 6.8333 335 ms 3.2666667 ms 7.233333 ms 3.2 ms 7.4333334 ms 3.3666666ms 6.5666 666 ms 3.1666667 ms

When the anchor LED is not provided in the transmitting device 110, it allocates a zone for 64 LEDs, and it takes about 6 to 8 ms as time is recognized by recognizing the color of the LED by a total irradiation method in each shot. Confirmed.

On the other hand, when four anchor LEDs are provided in the transmitting device 110, about 64 zones are dynamically allocated using the distances between the four anchor LEDs, and the color of the LEDs is recognized by the total irradiation method again. It was confirmed that it took about 5 ms.

That is, since four anchor LEDs are provided in the transmitting device 110, it is possible to reduce the number of pixels to be read in order to recognize the color of the LEDs, so it takes less time to process the image because the calculation throughput is reduced.

According to the experimental results, it can be seen that the image processing speed is about 1.5 to 2 times faster.

Therefore, according to the present invention, not only can the unnecessary image processing operation time be reduced to improve the speed of recognizing the data of the LED, but also the transmission data can be increased because the LED can be added using the margin of the screen. Data rates can also be improved.

In the present specification and drawings, preferred embodiments of the present invention have been disclosed, and although specific terms are used, they are merely used in a general sense to easily describe the technical contents of the present invention and to understand the present invention. It is not intended to limit the scope. It is apparent to those skilled in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

11a: First anchor LED 11b: Second anchor LED
100: optical camera communication system 110: transmitting device
130: receiving device 111: input unit
113: modulator 115: drive unit
117: light emitting unit 131: sensor unit
133: image processing unit 135: demodulation unit
137: output

Claims (6)

In the optical camera communication device,
A modulator configured to designate data to be transmitted to at least one of an R (Red) signal, a G (Green) signal, and a B (Blue) signal;
A driving unit for outputting a control signal for the emission ratios of the R signals, G signals and B signals within a light emission cycle; And
At least one LED (Light Emitting Diode) that displays data by emitting light according to the control signal, and at least one anchor (Anchor) LED that serves as a reference for automatically allocating an area for an area where the at least one LED is formed It includes a light emitting unit that includes,
The at least one anchor LED is a transmission device, characterized in that formed at one side of the upper end and the other lower end of the area where the at least one LED is formed.
delete In the optical camera communication device,
A sensor unit including an image sensor that receives light from at least one LED;
In the image data obtained from the image sensor, one or more anchor LEDs located at one upper end and the other lower end are identified, and each zone for at least one LED displaying data based on the identified anchor LED is automatically An image processing unit to determine R (Red), G (Green), and B (Blue) signals for the LEDs in each of the automatically allocated zones after allocation;
A demodulation unit for demodulating predetermined data using the determination signal of the image processing unit; And
And an output unit for outputting the demodulated data.
According to claim 3,
The image processing unit,
Receiving device, characterized in that for checking the coordinates of the at least one anchor LED, and relative to the zone for the at least one LED based on the identified coordinates.
In the optical camera communication method,
Obtaining image data through an image sensor that receives light from at least one LED of the sensor unit;
The image processing unit checks at least one anchor LED located at one upper end and the other lower end from the acquired image data, and automatically performs each zone for at least one LED displaying data based on the identified anchor LED. Assigning;
Determining, by the image processing unit, R (Red), G (Green), and B (Blue) signals for the LEDs of the automatically allocated zones;
A demodulator demodulating predetermined data using the determination signal; And
And an output unit outputting the demodulated data.
The method of claim 5,
The step of allocating the zone,
Identifying coordinates of the at least one anchor LED; And
And relatively allocating zones for the at least one LED based on the identified coordinates.

Images