KR20230103993A - Method and apparatus for receiving c-ook optical signal based on deep-learning in mobile circumstance - Google Patents

Abstract

A signal receiving device according to one embodiment of the present invention comprises: a camera generating images in a rolling-shutter method where optical signals are received; at least one processor; and a memory electrically linked to the processor to store at least one code executed by the processor, wherein the memory can store a code that, when executed by the processor, enables the processor to control the camera such that the light source is photographed at different times and generate an image frame, input the image frame into a deep learning-based first learning model to recognize an area where the light source is photographed, generate an on-off keying (OOK) modulation signal or an OOK-based one-dimensional signal based on the area where the light source is photographed, input the modulation signal into a deep learning-based second learning model to determine the position of a preamble in the modulation signal, and demodulate a payload in an OOK demodulation method from the modulation signal based on the preamble. Accordingly, data can be stably demodulated in optical communication where a camera-based OOK modulation signal is transmitted.

Description

Method and apparatus for receiving C-OOK optical signals based on deep learning in a mobile environment

The present invention relates to a technology for receiving an optical signal based on C-OOK (Camera On-Off Keying), and more particularly, when a signal modulated based on C-OOK is received in a moving environment, based on deep learning It relates to a technology for stably demodulating by doing so.

The contents described below are only described for the purpose of providing background information related to an embodiment of the present invention, and the contents described do not naturally constitute prior art.

Recently, Visible Light Communication (VLC) technology, which is a wireless communication technology in which a communication function is added to visible light wavelengths using an infrastructure in which lights such as incandescent bulbs and fluorescent lights are replaced with semiconductor Light Emitting Diode (LED) lights, has been actively researched.

The prior art discloses a technology for communicating by OOK (On-Off Keying) and multiplexing method based on a photo detector, but a method based on a photo detector requires a separate receiving device unlike the present invention based on a camera, This is a technique of spreading and multiplexing the OOK-modulated data itself.

However, it is not easy to receive and demodulate the OOK-modulated signal in a moving environment due to the intensity change of the transmitted optical signal.

Referring to FIG. 1, FIG. 1 (a) shows a change in intensity of an optical signal received when the optical signal transmission device and the reception device are stationary, and FIG. 1 (b) shows the optical signal transmission device and the reception device moving. It shows the change in intensity of the received optical signals in a situation where the distance between them changes.

The zero-crossing filter disclosed in Prior Art 2 uses 0 as a threshold level to distinguish the ON/OFF level of an optical signal in a receiving device, and this method obtains good results in a high SNR environment as shown in FIG. 1 (a). can do. However, in the environment as shown in FIG. 1 (b), it is difficult to distinguish ON/OFF states of optical signals.

In addition, the matched filter disclosed in Prior Art 2 demodulates the OOK-modulated optical signal by comparing the template signal with the template signal to determine the template signal of the real signal. A matched filter also cannot obtain good results in a mobile environment. In particular, since the transmitter and receiver move relative to each other in a mobile environment, a blur effect is highly likely to occur, and as a result, inter-symbol interference transmitted as an optical signal occurs, resulting in system performance deterioration.

Prior art 1: Korean Patent Registration No. 10-0899031 (Announced on May 21, 2009)

Prior Art 2: Nguyen, V.H.; Thieu, M.D.; Nguyen, H.; Jang, Y.M. Design and Implementation of the MIMO-COOK Scheme Using an Image Sensor for Long-Range Communication. Sensors 2019, 20, 2258

An embodiment of the present disclosure provides an apparatus and method capable of stably demodulating data in optical communication for transmitting a camera-based OOK modulation signal in a mobile environment.

The object of the present invention is not limited to the above-mentioned tasks, and other objects and advantages of the present invention not mentioned above can be understood by the following description and will be more clearly understood by the embodiments of the present invention. It will also be seen that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations indicated in the claims.

A signal receiving apparatus according to an embodiment of the present disclosure includes a camera generating an image in a rolling-shutter method for receiving an optical signal, at least one processor, and at least one processor electrically connected to the processor and performing at least one processor. Includes a memory in which the code of is stored, and when the memory is executed through a processor, the processor controls the camera to take a light source multiple times at different times to create an image frame, and the image frame is created based on deep learning. Recognizes the area where the light source is photographed by inputting it to the first learning model, generates an OOK modulation signal, which is a one-dimensional signal based on OOK (On-Off Keying) based on the area where the light source is photographed, and modulates the modulated signal based on deep learning It is input to the second learning model of to determine the position of the preamble in the modulated signal, and stores a code that causes a payload to be demodulated from the modulated signal according to the OOK demodulation method based on the preamble.

An optical signal receiving method according to an embodiment of the present disclosure is a signal receiving method in which a processor of an optical signal receiving apparatus including a rolling shutter-based camera performs at least a part of each step, wherein the camera sets a plurality of light sources at different times. Step of generating an image frame by photographing once, inputting the image frame to a first learning model based on deep learning, recognizing the area where the light source is photographed, based on OOK (On-Off Keying) based on the area where the light source is photographed Generating an OOK modulated signal, which is a one-dimensional signal of , inputting the modulated signal to a deep learning-based second learning model to determine the position of a preamble in the modulated signal, and OOK demodulation from the modulated signal based on the preamble Demodulating a payload according to a scheme may be included.

The method and apparatus for receiving a signal according to an embodiment of the present disclosure can stably demodulate an OOK modulated signal received based on a camera in a mobile environment by determining the location of a preamble based on deep learning and demodulating a payload.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram showing a case in which a C-OOK modulated signal is optically received in a stationary environment and a moving environment.
2 is a diagram schematically illustrating communication between an optical signal transmission device and an optical signal reception device according to an embodiment of the present disclosure.
3 is a block diagram showing the configuration of an optical signal transmission device according to an embodiment of the present disclosure.
4 is a diagram illustrating a data packet structure for generating a C-OOK modulated signal according to an embodiment of the present disclosure.
5 is a block diagram showing the configuration of an optical signal receiving apparatus according to an embodiment of the present disclosure.
6 is a diagram showing a result of detecting an image frame generated by an optical signal receiving apparatus and a region of a light source according to an embodiment of the present disclosure.
7 is a diagram showing a modulated signal generated from an image frame by an optical signal receiving apparatus according to an embodiment of the present disclosure.
8 is a diagram explaining a method of detecting a position of a preamble based on deep learning by an optical signal receiving apparatus according to an embodiment of the present disclosure.
9 is a diagram illustrating a method of demodulating a payload based on deep learning by an optical signal receiving apparatus according to an embodiment of the present disclosure.
10 is a diagram illustrating an input of a deep learning-based learning model in which an optical signal receiving apparatus demodulates a payload according to an embodiment of the present disclosure.
11 is a flowchart illustrating a signal reception method according to an embodiment of the present disclosure.
12 is an experiment result comparing a method for receiving a signal according to an embodiment of the present disclosure with a method based on a conventional matched filter.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Communication between an optical signal transmission device and an optical signal reception device according to an embodiment of the present disclosure will be described with reference to FIG. 1 . In this specification, a signal transmission device and a signal reception device refer to a device that transmits and receives an optical signal that transmits visible light-based light unless otherwise specified.

Referring to FIG. 1, the signal transmission apparatus 100 modulates data based on C-OOK (Camera On/Off Keying), and a control signal (voltage or After generating an electrical signal such as current or a message signal instructing the light source to blink), the optical signal may be transmitted by turning the light source 130 on and off based on a control signal.

In an embodiment of the present disclosure, the transmission signal is an OOK modulated signal, which is a pulse wave signal generated by an OOK method based on an optical camera communication (OCC) technology, and the signal transmission apparatus 100 transmits the OOK modulated signal It is transmitted based on the light source 130, and the signal receiving device 200 receives the light-based OOK modulated signal with a camera.

The signal transmission apparatus 100 generates a plurality of different data as different OOK modulated signals, and then transmits the OOK based on the light source corresponding to each OOK modulated signal, using the MIMO (Multiple-Input Multiple-Output) method. A modulated signal can be transmitted.

In one embodiment, when two different data are analog data, the signal transmission device 100 may digitally modulate them and then input them to the OOK encoder 110, respectively.

In one embodiment, the two different data may be low-speed data and high-speed data, respectively.

In one embodiment, the LED light source included in the light source 130 may include at least one or more LED light sources and may be spaced apart from each other. The LED light source may be implemented as an automobile rear indicator LED or brake LED.

The signal receiving device 200 receives the optical signal transmitted by the signal transmitting device 100 with a rolling-shutter-based camera 210, and transmits signals generated by each column or row of an image sensor of the camera. Original data may be extracted by generating an image frame, generating an OOK modulated signal from the image frame, and then demodulating the OOK modulated signal in the decoder 220 .

A configuration of a signal transmission apparatus 100 according to an embodiment of the present disclosure will be described with reference to FIG. 2 .

The signal transmission device 100 may include an OOK encoder 110, a controller 120, and a light source 130 including an LED light source, and generates a clock signal to transmit the OOK encoder 110 and/or the light source (130) may include a clock generator.

The OOK encoder 110 and the controller 120 may be part of a processor, implemented as a single semiconductor, or configured on a single printed circuit board (PCB). Alternatively, the OOK encoder 110 and the controller 120 may be implemented as software executable by a computer based on a processor and may be stored in a recording medium such as a memory or a storage unit. Alternatively, the OOK encoder 110 and the controller 120 may be implemented with a programmable logic controller (PLC), a field-programmable gate array (FPGA), or the like.

The OOK encoder 110 may modulate data (analog data may be digitally modulated signals) into pulse waves based on OOK, and the pulse waves consist of high duty and low duty Both duty and low duty modes can have positive amplitude values or only high duty can have positive amplitude values.

Referring to FIG. 3, the OOK encoder 110 converts information bits included in FEC (Forward Error Correction) encoder, Ab bit (asynchronous bits) inserter, and data into 0 or 1 corresponding to each information bit based on OOK. It may include an OOK mapping unit that allocates bits.

In addition, the OOK encoder 110 may generate packets by dividing data according to a packet structure for transmitting FEC-encoded data, for example, according to a preset size, and inserting a preamble of each packet. In this case, the OOK encoder 110 may insert a sequence number according to the order of packets, which will be described in detail below.

Referring to FIG. 4, the structure of a packet converted by the OOK encoder 110 according to an embodiment of the present disclosure will be described.

Packets obtained by converting data may include a plurality of data packets (i−1, i, i+1).

Each of the plurality of data packets (i-1, i, i+1) includes a plurality of data sub-packets (eg, data packet (i) includes data sub-packets (i1, i2, i3), Each data subpacket may include a payload composed of information bits corresponding to an allocated portion of data.

In an embodiment, in order to prevent packet loss due to a variable frame rate of a receiving camera, a plurality of data subpackets included in one data packet may include the same payload composed of identical information bits.

As will be described in detail in the description of the signal receiving device below, a rolling shutter camera continuously photographs the flickering of an LED light source multiple times at different times, and transmits each photographed signal to one column or row of an image sensor. Save to generate an image frame including a strip pattern corresponding to the on and off of the light source (off of the light source in this specification is the complete off of the light source or in preparation for the on level of the light source) It is a concept that includes turning on at a low level of illumination). In this case, the frame rate of the camera may be variable according to device settings or may be lower than the data packet transmission rate. Therefore, the signal transmission apparatus may configure data packets to include data subpackets including the same payload in duplicate in order to prevent packet reception loss due to frame rate limitation of the rolling camera. That is, the data subpackets i1, i2, and i3 of FIG. 4 may include the same payload.

In one embodiment, the OOK encoder 110 may assign a sequence number to each data packet or data subpacket in order to detect packet omission or distinguish duplicate packets in the signal receiving apparatus, and the sequence number may be assigned as consecutive numbers for consecutive data packets.

In one embodiment, each data sub-packet (i1, i2, i3) is identical to the sequence number of the corresponding data packet (i) and the information bits corresponding to the portion of the first information assigned to the corresponding data packet (i). may contain loads.

In one embodiment, the sequence number may be inserted at the front of the packet, or in another embodiment, at both the front and back of the packet. 4 illustrates an embodiment in which sequence numbers are inserted at both the front and rear ends of a packet. If the sequence number is inserted at both the front and rear ends of the packet, when the signal receiving device finds one preamble in one image frame, forward decoding and backward decoding are performed in consideration of the sequence numbers before and after the preamble. decoding) to compose packets.

Data subpackets included in one data packet may have the same sequence number. Therefore, even if a part of a specific data subpacket is missed, information loss can be prevented by a plurality of data subpackets in which information is copied, thereby preventing an information transfer error. In addition, redundantly received signals may be removed through the sequence number, and it may be determined whether there is an omission in the received signal.

In one embodiment, a packet of an OOK modulated signal or a data subpacket may include a header portion that is a preamble including meta information such as a size of the packet.

In another embodiment, the preamble is a bit code notifying the start frame of a packet, and may be a bit code previously known to the signal transmission device and the signal reception device.

In one embodiment, a packet or data subpacket of the OOK modulated signal may include identification information (id) of the light source 130 through which the OOK modulated signal is transmitted. For example, when a plurality of LEDs spaced apart from each other exist in the light source 130 and information transmitted from the plurality of LEDs is different, the plurality of OOK modulated signals may include different light source identification information (LED ID). can Accordingly, the plurality of modulated signals generated from the plurality of light sources recognized by the signal receiving apparatus 200 through the camera can be stably generated based on the light source identification information.

A signal receiving apparatus 200 according to an embodiment of the present disclosure will be described with reference to FIG. 5 .

The signal receiving device 200 includes an LED detector 230 recognizing an area where a light source is photographed in an image (image frame) acquired by an image sensor included in a rolling shutter-based camera 210, and a light source captured in the image frame. It may include a down sampler 220 that generates a one-dimensional OOK modulated signal from the domain and an OOK decoder 220 that demodulates data from the OOK modulated signal based on the OOK demodulation method.

Referring to FIG. 6, the camera 210 is a rolling shutter-based camera that continuously captures the flickering (or lighting with different luminous intensity) of an LED light source multiple times at different times, and transmits each captured signal to one image sensor. Create image frames by storing them in columns or rows. When the signal transmission apparatus 100 transmits a plurality of different data to a plurality of different light sources, the image frame may separately include optical signals transmitted from the plurality of light sources in the form of a strip, as shown in FIG. 6 .

The LED detector 230 is a first learning model based on deep learning, and can determine the area of the light source as shown in FIG. 6 (a) by receiving an image frame in which the light source is photographed. The first learning model may be a convolutional neural-network (CNN)-based learning model in which an image frame in which a light source is photographed is trained with training data labeled as a region of the light source. The first learning model may be a learning model in which a conventionally known segmentation network or object detection network is trained with training data in which an image frame in which a light source is photographed is labeled as a region of a light source. The LED detector 230 may be utilized as an LED tracker by determining the area of a light source in a plurality of image frames.

The downsampler 240 may generate a one-dimensional signal having an amplitude of a pixel intensity value for each line of the image frame from an area where a light source of the image frame is photographed. The one-dimensional signal may be an OOK modulated signal. OOK modulation signals generated from regions where the light sources of the image frames of FIG. 6 are photographed may be the same as those of FIG. 7 .

The OOK decoder 220 determines the position of a preamble in the OOK modulated signal by inputting the OOK modulated signal to the second learning model based on deep learning.

Referring to FIG. 8, a method for the second learning model to receive an OOK modulated signal and determine the position of a preamble will be described.

The OOK modulation signal 820 may be generated from a change in pixel intensity of a strip pattern included in the region 810 of the light source determined by the LED detector 230 in the image frame.

In one embodiment, the off-duty value of the OOK modulated signal 820 (the same applies to the on-duty) may constantly decrease or increase (red plot) according to a change in distance. The signal receiving apparatus 200 may determine the off-duty directionality and modify the generated OOK modulated signal by reflecting the off-duty directionality (blue plot). That is, when the OOK modulation signal is downward as shown in the red plot of FIG. 8, a rotation transformation matrix is generated and applied so that the straight line connecting the off-duties is horizontally converted, or the difference from the horizontal is applied to each pixel intensity. can do.

The OOK decoder 220 configures each value of the OOK modulated signal 820 into a one-dimensional column vector 830 and inputs it to the preamble detection unit 840, which is a second learning model based on deep learning, to determine the starting position of the preamble 851 or the OOK modulated signal 850 indicating the start position 851 of the preamble can be obtained. The second learning model may be a deep learning-based learning model trained with training data obtained by labeling a one-dimensional OOK modulated signal obtained from an image of a light source as a preamble position.

Referring to FIG. 9, a method in which the third learning model receives an OOK modulated signal based on a preamble and demodulates a payload will be described. In addition to the payload, the sequence number and light source identification information (LED-ID) can also be demodulated in the same way.

In the image frame generated in the moving environment of FIG. 6, the light source peripherals 610 and 620 of the area where the light source is photographed can be confirmed through the peripherals 710 and 720 of the OOK modulation signal in FIG. Since the difference is very minute, accurate demodulation cannot be performed with the conventional zero-crossing filter or threshold-based OOK demodulation method.

In one embodiment, the OOK decoder 220 configures each value of the OOK modulation signal 820 as a one-dimensional column vector, and then converts the backward signal 911 received before receiving the preamble 913 and the preamble ( 913) After reception, the modulation signal 910 included so that the received forward signal 915 continues before and after the preamble 913 may be input to a deep learning-based third learning model to demodulate the payload. The third learning model may be a deep learning-based learning model trained with training data in which the OOK modulation signal is labeled as 0 or 1.

In one embodiment, a preamble may not be included in an image frame or a plurality of preambles may be included in an image frame due to a change in the shutter speed of the camera 21 or a change in the flashing speed of a light source of the signal transmission device 100 or a mobile environment. there is.

10, the first image frame 1010 and the third image frame 1030 among the first image frame 1010, the second image frame 1020, and the third image frame 1030 continuously generated in chronological order. A preamble (indicated by a dark area) is present in , but the preamble may not be present in the second image frame 1020 . When the signal receiving apparatus 200 cannot detect the preamble in the second image frame 1020 based on the first learning model, the OOK modulated signal generated in the part 1021 of the second image frame 1020 is detected. A signal following the forward signal 1013 of one image frame 1010 constitutes an OOK modulation signal, and the backward signal generated from the part 1011 of the first image frame 1010, the preamble and the first image frame 1010 ) together with the forward signal generated from the other part 1013, the payload may be demodulated by inputting the third learning model based on deep learning.

The OOK modulated signal generated in the other part 1023 of the second image frame 1020 is a signal following the backward signal generated in the part 1031 of the third image frame 1030, and the OOK modulated signal may be generated. .

The OOK decoder 210 is a header detector that detects a sequence number from the OOK demodulated signal based on the second learning model and the third learning model, and a header detector that detects a sequence number from the OOK demodulated signal by determining a payload belonging to the same packet based on the sequence number. It may include a forward/backward decoder unit that separates, a packet combiner unit that detects missing packets and combines packets, and an FEC decoder unit that FEC decodes combined packet data.

In one embodiment, the forward/backward decoder unit, when one preamble is detected in one image frame as shown in FIG. (backward payload) may be determined as a payload belonging to the same packet.

In another embodiment, when the sequence number is inserted only at the front end of the packet, the decoder unit stores the most recent sequence number in a variable and compares the found sequence number with the variable to determine whether the packet is a duplicate. Alternatively, when one preamble and one sequence number are detected in one image frame, the front payload and the next payload may be determined as payloads belonging to the same packet by comparing with variables.

In one embodiment, the packet combiner checks the sequence number of the reconstructed packet to detect packet omission, and if there is no omission, the packet combiner may generate final packet data from the OOK demodulation signal, and the FEC decoder may generate final packet data from the OOK demodulation signal Data can be FEC decoded to create a data stream.

The C-OOK decoder 210 may determine a packet reception error when packet omission is found, and may request the signal receiving apparatus 200 to increase the camera's shooting frame rate.

Since a data packet may include a plurality of data subpackets composed of the same payload, the packet combiner of the C-OOK decoder 210 may generate a final data stream by excluding overlapping data subpackets.

A method for receiving an optical signal according to an exemplary embodiment of the present disclosure will be described with reference to FIG. 11 .

The signal receiving device acquires a signal value corresponding to the brightness of the LED light source according to the blinking of the LED light source in one column or row of the image sensor of the rolling camera by sequentially exposing each row or column of the image sensor of the rolling camera, A plurality of image frames are generated (S110).

The signal receiving device determines the region where the light source is captured by inputting the image frame into a CNN-based learning model trained with training data labeled as the region of the light source (S120).

The signal receiving device generates an OOK modulation signal, which is a one-dimensional brightness signal, based on the amount of charge captured by the image sensor according to the brightness value of the light source captured in the area where the light source of the image frame is photographed (S130). The brightness signal may be composed of high duty and low duty both having positive amplitude values, or may be an OOK modulation signal having positive amplitude only on duty.

The signal receiving device transmits the OOK modulated signal composed of column vectors to the second learning model, which is a deep learning-based learning model trained with training data in which the one-dimensional OOK modulated signal obtained from the image of the light source is labeled as the position of the preamble. The position of the preamble is determined by inputting it (S140).

The signal receiving apparatus inputs a modulated signal including a forward signal received after the preamble is received and a backward signal received before the preamble is received consecutively before and after the preamble to a deep learning-based third learning model to obtain a payload. Demodulate (S150). The third learning model may be a deep learning-based learning model trained with training data in which the OOK modulation signal is labeled as 0 or 1.

Referring to FIG. 12 , comparative experimental results of an optical signal receiving method according to an embodiment of the present disclosure and a conventional matched filter-based decoder will be described.

As can be seen in FIG. 12 , it can be confirmed that the bit error rate (BER) according to the embodiment of the present disclosure is lower than that of the matched filter-based decoder in the entire range of the distance between the transmitter and the receiver. In particular, it can be confirmed that the bit error rate difference is maintained up to 20 m, which is a relatively long distance, compared to the conventional matched filter-based decoder.

The present disclosure described above can be implemented as computer readable codes in a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. there is Also, the computer may include a processor of each device.

On the other hand, the program may be specially designed and configured for the present disclosure, or may be known and usable to those skilled in the art in the field of computer software. Examples of programs may include not only machine language codes generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like.

In the specification of the present disclosure (particularly in the claims), the use of the term "above" and similar indicating terms may correspond to both singular and plural. In addition, when a range is described in the present disclosure, as including the invention to which individual values belonging to the range are applied (unless otherwise stated), each individual value constituting the range is described in the detailed description of the invention Same as

Unless an order is explicitly stated or stated to the contrary for steps comprising a method according to the present disclosure, the steps may be performed in any suitable order. The present disclosure is not necessarily limited to the order of description of the steps. The use of all examples or exemplary terms (eg, etc.) in this disclosure is simply to explain the present disclosure in detail, and the scope of the present disclosure is limited due to the examples or exemplary terms unless limited by the claims. it is not going to be In addition, those skilled in the art can appreciate that various modifications, combinations and changes can be made according to design conditions and factors within the scope of the appended claims or equivalents thereof.

Therefore, the spirit of the present disclosure should not be limited to the above-described embodiments, and not only the claims to be described later, but also all ranges equivalent to or equivalent to these claims are within the scope of the spirit of the present disclosure. will be said to belong to

100: optical signal transmission device
200: optical signal receiving device

Claims (16)

A signal receiving method in which a processor of an optical signal receiving device including a rolling shutter-based camera performs at least part of each step,
generating an image frame by photographing the light source multiple times at different times by the camera;
Recognizing an area where the light source is photographed by inputting the image frame to a first learning model based on deep learning;
generating an OOK modulation signal, which is a one-dimensional signal based on On-Off Keying (OOK), based on an area where the light source is photographed;
determining a position of a preamble in the modulated signal by inputting the modulated signal to a second learning model based on deep learning; and
Demodulating a payload according to an OOK demodulation method from the modulated signal based on the preamble,
A signal receiving method of an optical signal receiving device.
According to claim 1,
The second learning model is a learning model trained with training data in which a one-dimensional OOK modulated signal obtained from an image of a light source is labeled as a preamble position,
A signal receiving method of an optical signal receiving device.
According to claim 1,
The OOK modulated signal includes identification information of a light source that generated the OOK modulated signal.
A signal receiving method of an optical signal receiving device.
According to claim 1,
The OOK modulation signal includes a plurality of packets, the packet includes a plurality of subpackets, and the subpacket includes a sequence number indicating order information of the packet or the subpacket,
A signal receiving method of an optical signal receiving device.
According to claim 4,
checking the sequence information of the plurality of subpackets; and
Determining to discard any one of the plurality of subpackets based on the order information or determining a packet reception error when the order information of the consecutive subpackets is not consecutive,
A signal receiving method of an optical signal receiving device.
According to claim 1,
Demodulating a payload according to an OOK demodulation scheme from the modulated signal based on the preamble comprises:
By inputting the modulated signal included so that the backward signal received before the preamble is received and the forward signal received after the preamble is received consecutively before and after the preamble are input to a deep learning-based third learning model, the payload Including the step of demodulating,
A signal receiving method of an optical signal receiving device.
According to claim 6,
The image frame includes a first image frame and a second image frame generated successively by taking pictures of the light source a plurality of times at different times by the camera;
When the position of the preamble in the first image frame is not confirmed, at least a part of the modulated signal generated from the first image frame is converted into the backward signal or the forward signal of the modulated signal generated from the second image frame. Comprising a signal and inputting it to the third learning model,
A signal receiving method of an optical signal receiving device.
According to claim 1,
The image frame is generated by the camera based on a light modulation signal transmitted from the light source provided in the moving object,
The image frame includes a strip pattern based on on and off of the light source.
A signal receiving method of an optical signal receiving device.
A camera generating an image using a rolling shutter method that receives an optical signal;
at least one processor; and
A memory electrically connected to the processor and storing at least one code executed by the processor;
When the memory is executed through the processor, the processor,
By controlling the camera, the light source is photographed multiple times at different times to generate an image frame, the image frame is input to a deep learning-based first learning model to recognize an area where the light source is photographed, and the light source is photographed. An OOK modulated signal, which is a one-dimensional signal based on OOK (On-Off Keying), is generated based on the keyed region, and the modulated signal is input to a deep learning-based second learning model to position a preamble in the modulated signal Determining and storing a code that causes a payload to be demodulated according to an OOK demodulation scheme from the modulated signal based on the preamble,
Optical signal receiving device.
According to claim 9,
The second learning model is a learning model trained with training data in which a one-dimensional OOK modulated signal obtained from an image of a light source is labeled as a preamble position,
Optical signal receiving device.
According to claim 9,
The OOK modulated signal includes identification information of a light source that generated the OOK modulated signal.
Optical signal receiving device.
According to claim 9,
The OOK modulation signal includes a plurality of packets, the packet includes a plurality of subpackets, and the subpacket includes a sequence number indicating order information of the packet or the subpacket,
Optical signal receiving device.
According to claim 12,
The memory causes the processor to:
Checking the sequence information of a plurality of subpackets;
Further storing a code for determining discarding any one of a plurality of subpackets based on the order information or determining a packet reception error when the order information of the consecutive subpackets is not consecutive,
Optical signal receiving device.
According to claim 9,
The memory causes the processor to:
By inputting the modulated signal included so that the backward signal received before the preamble is received and the forward signal received after the preamble is received consecutively before and after the preamble are input to a deep learning-based third learning model, the payload further storing code that causes to demodulate
Optical signal receiving device.
According to claim 14,
The image frame includes a first image frame and a second image frame generated successively by taking pictures of the light source a plurality of times at different times by the camera;
The memory causes the processor to:
When the position of the preamble in the first image frame is not confirmed, at least a part of the first image frame is composed of the backward signal or the forward signal of the second image and inputted to the third learning model. which saves more code,
Optical signal receiving device.
According to claim 9,
The image frame is generated by the camera based on a light modulation signal transmitted from the light source provided in the moving object,
The image frame includes a strip pattern based on on and off of the light source.
Optical signal receiving device.

Images