KR20220131806A - Apparatus and method for 2d color code optical wireless communication considering signal circumstances - Google Patents

Abstract

According to an embodiment of the present invention, a two-dimensional color code optical signal transmission method, which is an optical signal transmission method by which a processor performs at least a part of each step to transmit a two-dimensional color code, comprises the following steps of: receiving a data stream; generating a color code data signal generated based on the data stream and a reference color signal for decoding the color code data signal; and adjusting light sources of a two-dimensional color light source panel according to the color code data signal and the reference color signal. A light source adjusted according to the reference color signal among the light sources of the two-dimensional color light source panel may be based on the number of colors in a preset bit-color mapping table.

Description

2D color code optical wireless communication method and apparatus considering signal environment

The present disclosure relates to an apparatus and method for transmitting a 2D color code based on optical wireless communication, and more particularly, to an apparatus and method for accurately transmitting a 2D color code in consideration of ambient illuminance or movement of a signal receiving device when receiving a signal is about

The content described below is only provided for the purpose of providing background information related to the embodiment of the present invention, and the content described does not naturally constitute the prior art.

Recently, Visible Light Communication (VLC) technology, a wireless communication technology that adds communication functions to visible light wavelengths, is being actively researched using infrastructure in which lighting such as incandescent light bulbs and fluorescent lamps are replaced with semiconductor LED (Light Emitting Diode) lighting.

In addition, an optical camera communication (OCC) technology for demodulating a visible light communication signal received using a camera mounted on a user device such as a general smart phone or a car camera is also being developed.

A camera mounted on the user device may photograph an object using a global shutter method or a rolling shutter method.

There is a conventional technique for transmitting a two-dimensional color code to a two-dimensional light source panel, receiving it, and demodulating it (Prior Art 1).

However, when the user device receives and demodulates an optical signal, particularly a two-dimensional color code, in a moving or bright environment, demodulation is simply based on a color pixel value of a photographed image and a preset threshold. The demodulated value has a high probability of having an error. This is because, due to the influence of the surrounding environment (including speed), the color pixel value of the captured image is more likely to be captured brighter or darker than the original pixel value. Therefore, there is a need for a technology capable of accurately demodulating a two-dimensional color code regardless of the surrounding environment.

Prior Art 1: Korean Patent Publication No. 10-2163977 (Notice on Oct. 12, 2020)

An embodiment of the present disclosure provides an apparatus and method capable of demodulation in consideration of the possibility of erroneous recognition of a 2D color code signal due to movement of a signal receiving device or ambient illumination in 2D color code optical camera communication.

An embodiment of the present disclosure provides an apparatus and method for differently applying a demodulation method of a two-dimensional color code signal according to a movement of a signal receiving apparatus or an ambient illuminance in optical camera communication.

The object of the present invention is not limited to the above-mentioned problems, and other objects and advantages of the present invention that are not mentioned may be understood by the following description, and will be more clearly understood by the embodiments of the present invention. It will also be appreciated that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations thereof indicated in the claims.

A two-dimensional color code optical signal transmission method according to an embodiment of the present disclosure is an optical signal transmission method in which a processor performs at least a part of each step to transmit a two-dimensional color code, the method comprising: receiving a data stream; generating a color coded data signal and a reference color signal for decoding the generated color coded data signal based on the generated color coded data signal, and adjusting the light sources of the two-dimensional color light source panel according to the color coded data signal and the reference color signal, Among the light sources of the 2D color light source panel, the light source adjusted according to the reference color signal may be based on a preset number of colors in the bit-color mapping table.

An apparatus for transmitting a two-dimensional color code optical signal according to an embodiment of the present disclosure is an optical signal transmitting apparatus for transmitting a two-dimensional color code. a control unit for transmitting a transmission signal, wherein the modulator is configured to generate a color-coded data signal based on the received data stream and to generate a reference color signal for decoding the color-coded data signal, wherein the control unit is a color-coded data signal and adjusting the light sources of the two-dimensional color light source panel according to the reference color signal, wherein among the light sources of the two-dimensional color light source panel, the light source adjusted according to the reference color signal is based on the number of colors in the preset bit-color mapping table. can

A two-dimensional color code optical signal receiving method according to an embodiment of the present disclosure is an optical signal receiving method in which a processor of a signal receiving apparatus including a camera performs at least a part of each step to receive a two-dimensional color code, wherein the camera generating an image frame photographed by a two-dimensional color light source panel, identifying a color coded data block related to received data in the image frame and a reference color block for decoding the color coded data block, and a reference color block and preset bits - decoding the color code data block based on the color mapping table, wherein the reference color block may be based on a preset number of colors in the bit-color mapping table.

An apparatus for receiving a two-dimensional color code optical signal according to an embodiment of the present disclosure is electrically connected to a camera generating an image by receiving an optical signal, at least one processor, and at least one code executed by the processor ) is stored, wherein the memory, when executed through the processor, causes the processor to control the camera to generate an image frame photographing a two-dimensional color light source panel, and color code data blocks and colors associated with the received data in the image frame Identifies a reference color block for decoding the code data block, and stores a code causing decoding of the color code data block based on the reference color block and a preset bit-color mapping table, wherein the reference color block is a preset bit- It may be based on the number of colors in the color mapping table.

The signal transmission/reception apparatus and method according to an embodiment of the present disclosure perform demodulation in consideration of the surrounding environment of the signal reception apparatus, so that the received signal can be demodulated without error.

The apparatus and method for transmitting and receiving a signal according to an embodiment of the present disclosure perform demodulation in consideration of movement of the signal receiving apparatus or ambient illumination, so that a received signal can be demodulated without errors and accuracy can be improved.

Effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a diagram schematically illustrating communication between a signal transmission apparatus and a signal reception apparatus according to an embodiment of the present disclosure.
2 is a block diagram illustrating a configuration of a signal transmission apparatus according to an embodiment of the present disclosure.
3 is a diagram for describing a packet structure of a 2D color code signal according to an embodiment of the present disclosure.
4 is a block diagram illustrating a configuration of a signal receiving apparatus according to an embodiment of the present disclosure.
5 is a view for explaining a payload cell in different environments according to an embodiment of the present disclosure.
6 is a diagram for describing a method for a signal receiving apparatus to perform decoding based on a reference color block according to an embodiment of the present disclosure.
7 is a diagram for describing a method of decoding by a signal receiving apparatus based on an RNN sequence model according to an embodiment of the present disclosure.
8 is a diagram for describing a method of decoding by a signal receiving apparatus based on an LSTM sequence model according to an embodiment of the present disclosure.
9 is a flowchart illustrating a signal transmission method according to an embodiment of the present disclosure.
10 is a flowchart illustrating a signal reception method according to an embodiment of the present disclosure.
11 is a flowchart illustrating a signal reception method according to an embodiment of the present disclosure.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

Communication between the signal transmission apparatus and the signal reception apparatus according to an embodiment of the present disclosure will be described with reference to FIG. 1 .

Referring to FIG. 1 , the signal transmitting apparatus 100 modulates input data based on 2D color code mapping and transmits it by controlling the light sources of the 2D light source panel 130 according to the signal modulated by the controller 120 . may be configured to transmit a signal.

In embodiments of the present disclosure, the transmission signal is obtained after modulating a data stream input according to an optical camera communication (OCC) technology using a 2D color code mapping method based on a bit-color mapping table as shown in Table 1 , is a signal transmitted by controlling ON/OFF of each light source with a color suitable for the bit code.

The light source may not necessarily be one individual light source, but may also be a set of a plurality of light sources. For example, when an LCD or LED display device is used as a light source panel, the screen may be divided into blocks composed of predetermined pixels or a set of predetermined light sources, and each block may be used as one light source.

Also, the number of certain pixels or certain light sources constituting the block may be changed according to the resolution setting of the display device or the light source panel.

In embodiments of the present disclosure, optical signals displayed at once on the two-dimensional light source panel are referred to as packets.

A configuration of the signal transmission apparatus 100 according to an embodiment of the present disclosure will be described with reference to FIG. 2 .

The signal transmission apparatus 100 may include an encoder 110 , a controller 120 , and a two-dimensional light source panel 130 , and may include a clock generator that generates a clock signal.

The encoder 110 corresponds to each information bit using a preset bit-color mapping table for information bits of a Forward Error Correction (FEC) encoder, a preamble insertion unit, a color reference block insertion unit, and modulated data. It may include a bit-color mapping unit for allocating a color to be used.

The encoder 110 receives a binary data signal (or may modulate a binary data signal after receiving general data), generates a color code data signal (payload) based on the received binary data signal, and preambles and a reference color block may be inserted to generate a packet. The controller 120 controls ON/OFF of each light source of the 2D light source panel 130 with a color suitable for the bit code of the packet according to the generated packet.

In an embodiment, the signal transmission apparatus 100 may generate a reference color signal for decoding a color code data signal, and the reference color signal may be based on the number of colors in a bit-color mapping table.

For example, when a two-dimensional color signal is transmitted using eight colors as shown in Table 1, the reference color signal may be a threshold value for determining ON/OFF of the eight colors.

The signal transmission apparatus may include a preamble corresponding to a header and a payload corresponding to data in a packet.

In an embodiment, the signal transmission apparatus may include a sequence number in a packet, the sequence number may be assigned as a continuous number to consecutive data packets, and the sequence number may be a predetermined number (the number of bits). ) can be used repeatedly in order. For example, the sequence number may be 00 for the first packet, 01 for the second packet, and 00 for the third packet. The signal receiving apparatus may determine whether packets are duplicated through the sequence number.

A configuration of a packet transmitted by the signal transmission apparatus 100 through the 2D light source panel according to an embodiment of the present disclosure will be described with reference to FIG. 3 .

The packet generated by the signal transmission apparatus 100 may have a structure as shown in FIG. 3 .

In an embodiment, the packet may include a clock symbol indicating a clock of a color code data signal. The clock symbol can be configured to be displayed on the four corners of the two-dimensional packet.

In an embodiment, the packet may transmit a clock symbol by blinking one of the four clock symbols in phase opposite to the other in order to support the 360 degree rotation decoding of the signal receiving apparatus 200 .

A packet may contain a preamble, a reference color block, a sequence number and a color code data signal (payload), which may be included in any packet.

The reference color signal of the reference color block may be based on the number of colors in the bit-color mapping table, and when black and white are included in the bit-color mapping table, the sequence number may be used as the reference color signal of the corresponding color.

For example, when modulating using the bit-color mapping table as shown in Table 1 and using two sequence numbers as black and white reference color signals, the reference color signal of the reference color block is the reference color signal of the remaining 6 colors. symbol can be displayed. The controller 120 may control the six light sources to display a symbol of six colors. If the sequence number is not used as a part of the reference color signal, the controller 120 may control the 8 light sources to display the symbols of 8 colors in the case of a bit-color mapping table as shown in Table 1.

In an embodiment, the controller 120 may control the 2D light source panel 130 to transmit a color code data signal and a preamble (or a reference color block) with different resolutions.

A configuration of the signal receiving apparatus 200 according to an embodiment of the present disclosure will be described with reference to FIG. 4 .

The signal receiving apparatus 200 includes an area detection unit 220 for detecting a two-dimensional color code signal area from an image obtained by an image sensor of a rolling shutter camera or a global shutter camera 210, and a data stream from the detected area. and a two-dimensional color code decoder 230 that demodulates .

The signal receiving device 200 may include a sensor 240 for measuring ambient illuminance or measuring a moving speed of the signal receiving device 200 .

When the sensor 240 measures illuminance, it may be an illuminance sensor, and when measuring speed, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor ), a motion sensor, and a GPS sensor.

The signal receiving apparatus 200 may detect a two-dimensional color code signal region from an image frame obtained from an image sensor of the camera 210 using a machine learning learning model 220 based on a convolution neural network (CNN).

The machine learning learning model 220 based on a convolutional neural network (CNN) may be a convolutional neural network-based learning model trained to recognize a region of a two-dimensional color light source panel from an input image.

Machine learning-based learning models are CNN or R-CNN (Region based CNN), C-RNN (Convolutional Recursive Neural Network), Fast R-CNN, Faster R-CNN, R-FCN (Region based Fully Convolutional Network) ), YOLO (You Only Look Once), or SSD (Single Shot Multibox Detector) structure may include a neural network.

The learning model may be implemented in hardware, software, or a combination of hardware and software, and when a part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in a memory.

The 2D color code decoder 230 may identify a preamble, a color code data block related to received data (payload), and a reference color block for decoding the color code data block in the 2D color code signal region.

The signal receiving apparatus 200 may demodulate the color code data block based on the reference color block and the preset bit-color mapping table as shown in Table 1. As described above for the reference color block of the signal transmitting apparatus 100, the reference color block checked by the signal receiving apparatus 200 includes a reference color signal equal to the number of colors in the bit-color mapping table as shown in Table 1 or based on the sequence number. When used as a part of a color signal, a reference color signal equal to the number of remaining colors may be included.

The signal receiving apparatus 200 may apply a different demodulation method of the color code data block according to the surrounding illuminance or the moving speed of the signal receiving apparatus 200 .

There is a possibility that the signal receiving device 200 may recognize a two-dimensional color code data block of an image frame captured by the signal receiving device 200 while moving or in a bright environment as a different color from the signal transmitted by the signal transmitting device 100 . have.

For example, although the signal transmitting apparatus 100 controls the light source to display the first color 510 of FIG. 5 , the signal receiving apparatus 200 receives the light from the corresponding light source due to ambient illuminance or distance or movement from the light source. It can be recognized as the second color 520 of 5, and an error may occur if it is demodulated.

The signal receiving apparatus 200 compares the color threshold value determined based on the reference color block and the color code data block with the color code data block when receiving the two-dimensional color code signal in a state where ambient illumination is low or in a stationary state. can be demodulated.

For example, a value recognized by the blue color reference color signal 610 among the reference color blocks is set as a threshold value and the blue channel values of the color code data blocks are compared. 1 can be demodulated from the blue channel of , and a low value 630 can demodulate 0 from the blue channel of the corresponding color code data block.

The 2D color code decoder 230 may check the channel values by applying blue, green, and red filters to the color code data blocks in order to recognize the respective color channel values of the color code data blocks.

If there are 8 reference color blocks (which may include sequence number blocks), blue, green, and red filters are applied to each reference color block, and 4 thresholds for the ON state of blue, green, and red and the OFF state It is possible to obtain four threshold values for . The signal receiving apparatus 200 may compare by averaging threshold values of an ON state or an OFF state for each color. In addition, four thresholds for ON states of blue, green, and red in a Recurrent Neural Networks (RNN)-based sequence model as shown in FIG. 7 or a Long Short Term Memory (LSTM) based sequence model as shown in FIG. 8 . Four thresholds for value and OFF state can be entered.

When the signal receiving apparatus 200 receives a two-dimensional color code signal in a state in which ambient illumination is high or in motion, the color threshold value of the previous reference color block of the previously generated previous image frame and the reference color block of the current image frame The current color threshold may be obtained by inputting the color threshold determined based on .

When the moving speed is higher according to the moving speed, the signal receiving apparatus 200 may control the 2D color code decoder 230 to demodulate the color code data block based on the LSTM-based sequence model. Therefore, since the gradient loss problem due to repetition does not occur in one layer of the RNN-based sequence model, it is possible to more accurately demodulate the color code data block even at a higher moving speed.

When the color code data block is demodulated based on the sequence model, the signal receiving apparatus 200 has four thresholds for the ON state for each red, blue, and green channel obtained from the reference color block of the 2D color code signal currently being demodulated. In addition to the four thresholds for values and OFF states, the thresholds for ON and OFF states for each red, blue, and green channel obtained from the reference color block of the previous image frame are input to the sequence model to present the current A color threshold may be obtained.

A signal transmission method of a signal transmission apparatus according to an embodiment of the present disclosure will be described with reference to FIG. 9 .

The signal transmission apparatus receives a data stream and generates a binary data signal, or receives a data stream composed of a binary data signal (S110).

The signal transmission apparatus generates a color code data signal (payload) based on the binary data signal, and inserts a reference color block composed of a preamble and reference color signals to generate a packet ( S120 ).

The signal transmission apparatus ON/OFF controls each light source of the 2D light source panel to a color suitable for the bit code of the packet according to the generated packet (S130).

The reference color signal may be based on the number of colors in the bit-color mapping table. As shown in Table 1, when a two-dimensional color signal is transmitted using 8 colors, the reference color signal determines ON/OFF of the 8 colors. It may be a threshold value for

The signal transmission apparatus may include the sequence number in the packet, and when black and white are included in the bit-color mapping table, the sequence number may be used as a reference color signal of the corresponding color.

A signal reception method of a signal reception apparatus according to an embodiment of the present disclosure will be described with reference to FIGS. 10 and 11 .

The signal receiving apparatus obtains an image frame generated by a camera photographing the two-dimensional light source panel (S210).

The signal receiving device detects a two-dimensional color code signal region using a CNN-based machine learning learning model from an image frame, and a preamble, a color code data block and a color code data block related to the received data (payload) in the detected region. Check the reference color block for decoding (S220).

The signal receiving device demodulates the color coded data block based on the reference color block and the preset bit-color mapping table as shown in Table 1 (S230), and the color coded data block according to the surrounding illuminance or the moving speed of the signal receiving device. A different demodulation method may be applied.

The signal receiving apparatus may determine the moving speed or ambient illuminance of the signal receiving apparatus based on the output value of the sensor (S231).

In an embodiment, when the signal receiving apparatus receives a two-dimensional color code signal in a stopped state, the color code data block is demodulated by comparing the color threshold value determined based on the reference color block with the color code data block ( S234), if it is determined that it is moving, if it is in a low-speed moving state, the current image frame from the color threshold value of the previous reference color block of the previous image frame and the color threshold value determined based on the reference color block of the current image frame based on the RNN-based sequence model A color threshold is obtained (S235), and in the case of high-speed movement, the color threshold value of the previous reference color block of the previous image frame and the color threshold value determined based on the reference color block of the current image frame based on the LSTM-based sequence model A current color threshold may be obtained from ( S236 ).

The present disclosure described above can be implemented as computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is this. In addition, the computer may include a processor of each device.

Meanwhile, the program may be specially designed and configured for the present disclosure or may be known and available to those skilled in the art of computer software. Examples of the program may include not only machine language codes such as those generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like.

In the specification of the present disclosure (especially in the claims), the use of the term "above" and similar referential terms may be used in both the singular and the plural. In addition, when a range is described in the present disclosure, each individual value constituting the range is described in the detailed description of the invention as including the invention to which individual values belonging to the range are applied (unless there is a description to the contrary). same as

Steps constituting the method according to the present disclosure may be performed in an appropriate order unless there is an explicit order or description to the contrary. The present disclosure is not necessarily limited to the order in which the steps are described. The use of all examples or exemplary terminology (eg, etc.) in the present disclosure is merely for the purpose of describing the present disclosure in detail, and the scope of the present disclosure is limited by the examples or exemplary terms unless defined by the claims. it's not going to be In addition, those skilled in the art will appreciate that various modifications, combinations, and changes may be made according to design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present disclosure should not be limited to the above-described embodiments, and the scope of the spirit of the present disclosure is not limited to the scope of the present disclosure. will be said to belong to

100: signal transmitting device
200: signal receiving device
510: first color
520: second color
610: reference color signal

Claims (8)

An optical signal transmission method in which a processor performs at least a part of each step to transmit a two-dimensional color code, comprising:
receiving a data stream to be transmitted to a signal receiving device;
generating a color code data signal generated based on the data stream and a reference color signal used for decoding the color code data signal; and
Transmitting the color code data signal and the reference color signal by adjusting the light sources of a two-dimensional color light source panel according to the color code data signal and the reference color signal,
The light source controlled according to the color code data signal and the light source controlled according to the reference color signal are different light sources,
A two-dimensional color code optical signal transmission method.
The method of claim 1,
The number of light sources controlled according to the reference color signal among the light sources of the two-dimensional color light source panel is based on the number of colors set in the bit-color mapping table,
A two-dimensional color code optical signal transmission method.
An optical signal transmission device for transmitting a two-dimensional color code, comprising:
a modulator for modulating an input signal; and
A control unit for transmitting a modulated transmission signal through the light sources of the two-dimensional color light source panel,
The modulator is configured to generate a color code data signal based on the received data stream and to generate a reference color signal used for decoding the color code data signal,
the control unit is configured to control the light sources of the two-dimensional color light source panel according to the color code data signal and the reference color signal to transmit the color code data signal and the reference color signal;
The light source controlled according to the color code data signal and the light source controlled according to the reference color signal are different light sources,
Two-dimensional color code optical signal transmission device.
4. The method of claim 3,
The control unit is
The number of light sources controlled according to the reference color signal among the light sources of the two-dimensional color light source panel is based on the number of colors set in the bit-color mapping table,
Two-dimensional color code optical signal transmission device.
An optical signal receiving method in which a processor of a signal receiving device including a camera performs at least a part of each step to receive a two-dimensional color code,
generating, by the camera, an image frame obtained by photographing the two-dimensional color light source panel;
identifying a color coded data block associated with received data in the image frame and a reference color block for decoding the color coded data block; and
Decoding the color code data block based on the reference color block, which is information received from the two-dimensional color light source panel, and a bit-color mapping table preset in the signal receiving apparatus,
wherein the color code data block and the reference color block are photographed from different light sources of the two-dimensional color light source panel and recorded in the image frame,
A method of receiving a two-dimensional color-coded optical signal.
6. The method of claim 5,
The number of the reference color blocks is based on the number of colors in the bit-color mapping table preset in the signal receiving apparatus,
A method of receiving a two-dimensional color-coded optical signal.
a camera that receives an optical signal and generates an image;
at least one processor; and
It is electrically connected to the processor and includes a memory in which at least one code (code) executed by the processor is stored,
The memory, when executed by the processor, causes the processor to:
A camera is controlled to generate an image frame photographing a two-dimensional color light source panel, and a color code data block related to received data and a reference color block for decoding the color code data block are identified in the image frame, and the two-dimensional storing the reference color block as information transmitted from the color light source panel and a code causing to decode the color code data block based on a preset bit-color mapping table in the memory;
wherein the color code data block and the reference color block are photographed from different light sources of the two-dimensional color light source panel and recorded in the image frame,
Two-dimensional color code optical signal receiving device.
8. The method of claim 7,
the number of reference color blocks is based on the number of colors in the bit-color mapping table preset in the memory;
Two-dimensional color code optical signal receiving device.

Images