KR20160137342A - Optical camera communication method using LED and rolling shutter camera - Google Patents

Optical camera communication method using LED and rolling shutter camera Download PDF

Info

Publication number
KR20160137342A
KR20160137342A KR1020160003125A KR20160003125A KR20160137342A KR 20160137342 A KR20160137342 A KR 20160137342A KR 1020160003125 A KR1020160003125 A KR 1020160003125A KR 20160003125 A KR20160003125 A KR 20160003125A KR 20160137342 A KR20160137342 A KR 20160137342A
Authority
KR
South Korea
Prior art keywords
data
frame
led
rolling shutter
image
Prior art date
Application number
KR1020160003125A
Other languages
Korean (ko)
Other versions
KR101952994B1 (en
Inventor
장영민
반 뉴엔 트랑
Original Assignee
국민대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 국민대학교산학협력단 filed Critical 국민대학교산학협력단
Priority to US15/551,561 priority Critical patent/US10560188B2/en
Priority to PCT/KR2016/000482 priority patent/WO2016133285A1/en
Publication of KR20160137342A publication Critical patent/KR20160137342A/en
Application granted granted Critical
Publication of KR101952994B1 publication Critical patent/KR101952994B1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/22—Adaptations for optical transmission
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
    • H04B10/11—Arrangements specific to free-space transmission, i.e. transmission through air or vacuum
    • H04B10/114—Indoor or close-range type systems
    • H04B10/116—Visible light communication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
    • H04B10/50—Transmitters
    • H04B10/501—Structural aspects
    • H04B10/502—LED transmitters
    • H04N5/225—

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Multimedia (AREA)
  • Optical Communication System (AREA)
  • Studio Devices (AREA)

Abstract

The present invention provides an optical camera communication method using a light emitting diode (LED) and a rolling shutter camera. According to the present invention, the optical camera communication method comprises: a step of coding transmission data to be transmitted by a data coding part and forming a data frame including the coded transmission data: a step of turning on/off the LED to correspond to the data frame in accordance with a pulse frequency by an LED driving part; a step of capturing an on/off image of the LED into a continuous frame image per each of multiple rows by a rolling shutter method in accordance with a frame speed by a roller shutter camera; a step of generating a brightness signal in accordance with a brightness value of the on/off image of the LED, which is captured into the continuous frame image per row by an image processing part; and a step of extracting the transmission data from the brightness signal from an image extraction part. The data frame has a plurality of super-frames continuously arranged thereon and the super frames are divided according to each piece of transmission data. Each super-frame includes data sub-frames (DS) of N (N=natural number), which are continuously repeated. Each data sub-frame comprises: a data packet (DP) including the coded transmission data; an asynchronous bit (Ab) added to each of front and rear ends of the data packet, respectively, and a start frame (SF) added to a front bit of the asynchronous bit.

Description

[0001] The present invention relates to an optical camera communication method using an LED and a rolling shutter camera,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an optical camera communication (OCC) method, and more particularly, to an optical camera communication (OCC) method in which an LED which is turned on / off in response to data to be transmitted is photographed by a rolling shutter camera, And an optical camera communication method using a rolling shutter camera.

Visible Light Communication (VLC), which is a typical illumination / communication convergence technology, is a technology for performing wireless communication by transmitting information on illumination of a light source. Conventionally, a light source is received by a photodiode (PD) (digital data 1 or 0) in accordance with the on / off state of the data.

Conventionally, a visible light communication system has been proposed in which a plurality of LEDs are photographed using a camera instead of a photodiode, and data corresponding to ON / OFF of the LEDs acquired for each frame of the camera is extracted. In this way, visible light communication using a camera is also called an optical camera communication (OCC) system in that a camera is used instead of a photodiode as an optical receiver, and work for standardization in the IEEE 802.15.7a research group is attempted have.

Recently, there has been an attempt to apply a rolling shutter camera as a camera to such an OCC system. Such a rolling shutter camera captures an on / off image of an LED for each row in an image sensor combined with a plurality of rows to acquire an image for each frame.

However, in the related art, there is not yet a technique for extracting data corresponding to on / off images of a light source for each row in a rolling shutter camera. In addition, although the frame rate of a general rolling shutter camera is fixed at 30 fps, it varies from 20 to 35 fps depending on the actual product. This causes a change in the frame rate of the camera when the pulse rate of the LED is constant, resulting in data loss. For example, when the camera operates in a situation where a change in the frame rate can not be expected, there is a problem in that data loss occurs because the camera can not shoot an image when the LED is turned on / off between two image frames.

Further, in the related art, there is a problem that it is difficult to accurately extract data due to asynchronization of the on / off timing of the frame and the light source of the rolling shutter camera since the rolling shutter camera starts to photograph at an arbitrary point in time.

Korea Patent No. 1472583 Korean Patent Publication No. 2009-0016176 Korea Patent Publication No. 2010-0135683

SUMMARY OF THE INVENTION The present invention has been proposed in order to solve the problems of the related art described above, and it is an object of the present invention to provide an optical camera communication method using an LED and a rolling shutter camera that can prevent loss of transmitted data even if the frame rate of a rolling shutter camera changes in an OCC system The purpose is to provide.

In addition, the present invention can prevent the frame omission due to the asynchronism between the frame of the rolling shutter camera and the ON / OFF state of the LED even when the rolling shutter camera starts to shoot at an arbitrary point in time, The present invention provides an optical camera communication method using the same.

An optical camera communication method using an LED and a rolling shutter camera according to an embodiment of the present invention includes a coding step of coding transmission data to be transmitted by a data coding unit and configuring a data frame including the coded transmission data; A driving step of turning on / off the LED to correspond to the data frame in accordance with the pulse frequency in the LED driver; Capturing an on / off image of the LED according to a frame rate in a rolling shutter camera into a continuous frame image for each of a plurality of rows in a rolling shutter manner; A generating step of generating a brightness signal according to a brightness value of the on / off image of the LED captured in the continuous frame image for each row in the image processing unit; And an extracting step of extracting the transmission data from the brightness signal in an image extracting unit; (N = natural number) data sub-frames (DS) in which the super frames are consecutively arranged in succession, and the data frames , Each data sub-frame including a data packet (DP) including the coded transmission data, an asynchronous bit (Ab) added to the front end and a rear end of the data packet, a start frame added to the front end of the front end asynchronous bit SF).

In the present invention, at least one data sub-frame is captured for each frame image.

In the present invention, the number of captured data subframes (Nrepeats) per frame image satisfies the following equation.

Figure pat00001

(tcap is the capture time at which one frame image is exposed in the rolling shutter camera, N is the number of repeats of the data subframe (DS) in the superframe, DSlength is the length of the data subframe)

In the present invention, the asynchronous bit (Ab) is an identifier for identifying consecutive neighboring superframes, in which superframes classified by the transmission data are continuously arranged, and when the index is an odd number or an even number, 1 and 0 1 < / RTI > bits are added to the data sub-frames alternately.

In the present invention, the extracting step may include extracting a start frame (SF) and an asynchronous bit (Ab) at the previous stage from the first frame image captured by the data extracting unit, A first step of extracting a packet (DP); If there is an asynchronous bit Ab at the rear end of the data packet DP in the first frame image, extracts transmission data from the data packet DP, and if not, A second step of extracting an asynchronous bit Ab at a subsequent stage in the second frame image; A third step of determining whether the asynchronous bit Ab of the trailing end extracted from the second frame image coincides with the asynchronous bit Ab of the preceding stage extracted from the first frame image; A fourth step of extracting a data packet (DP) located at the previous stage of the asynchronous bit Ab at the succeeding stage if it is matched; And a fifth step of extracting transmission data by combining the data packet DP extracted from the first frame image and the data packet DP extracted from the second frame image.

In the present invention, the extracting step may include extracting a start frame (SF) and an asynchronous bit (Ab) at the previous stage from the first frame image captured by the data extracting unit, A first step of extracting a data packet DP; If there is a rear end asynchronous bit Ab at the rear end of the data packet DP extracted from the first frame image, the transmission data is extracted from the data packet DP, A second step of determining whether or not there is an asynchronous bit Ab; A third step of extracting a data packet DP located at a preceding stage of the asynchronous bit Ab of the subsequent stage if the next stage asynchronous bit Ab is present; And a fourth step of extracting transmission data by combining the data packet DP extracted in the first step and the data packet DP extracted in the third step.

In the present invention, the pulse frequency is set within a range within the shutter speed of the rolling shutter camera.

In the present invention, the pulse frequency is set within a range of 100 Hz to 8 kHz.

According to the present invention, since a modulation frequency range for driving an LED suitable for optical camera communication (OCC) using an LED and a rolling shutter camera can be set and efficient data recovery can be performed, even if the frame rate of a rolling shutter camera changes, Loss can be prevented.

In addition, according to the present invention, accuracy of data transmission can be improved even when the rolling shutter camera starts to photograph at an arbitrary point in time.

1 is an overview of an optical camera communication (OCC) system using an LED and a rolling shutter camera according to an embodiment of the present invention;
2 is a structure diagram of a data packet transmitted and received in an optical camera communication (OCC) system according to an embodiment of the present invention;
3 is a schematic diagram illustrating a process of extracting transmission data from a rolling shutter camera of an optical camera communication (OCC) system according to an embodiment of the present invention;
4 is a graph of the amplitude response pattern of a rolling shutter camera with respect to the pulse frequency of an LED in an optical camera communication (OCC) system according to an embodiment of the present invention,
5 is a flowchart illustrating an optical camera communication method using an LED and a rolling shutter camera according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating a process of extracting transmission data from a data extraction unit in an optical camera communication using an LED and a rolling shutter camera according to an exemplary embodiment of the present invention;
7 is a flowchart illustrating a process of extracting transmission data in an optical camera communication using an LED and a rolling shutter camera according to another embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

1 is a schematic diagram of an optical camera communication (OCC) system using an LED and a rolling shutter camera according to an embodiment of the present invention.

1, an optical camera communication (OCC) system 100 using an LED and a rolling shutter camera according to the present invention includes a data coding unit 110, an LED driving unit 120, an LED A camera 130, a rolling shutter camera 140, an image processing unit 150, and a data extraction unit 160.

The data coding unit 110 codes transmission data to be transmitted in the OCC system 100. Such coding can be implemented in various ways. For example, when the transmission data to be transmitted is 1, it corresponds to the ON state of the LED 130, and when the transmission data is 0, it corresponds to the OFF state of the LED 130. Such an example may be set differently depending on the pulse frequency of the LED 120. [ For example, when the transmission data is 1, the LED 130 may correspond to ON-ON, and when the transmission data is 0, the LED 130 may correspond to OFF-OFF. In this way, in the present invention, the data coding unit 110 matches the on / off states of the LED 130 corresponding to the transmission data with each other, and transmits the transmission data through on / off of the LED 130 in the future. In this embodiment, the data coding unit 110 can code data using, for example, a Manchester coding scheme, a 4B6B coding scheme, an 8B10B coding scheme, or the like. In addition, the data coding unit 110 constructs the coded transmission data as a data packet and generates a data frame including the data packet. These data packets are configured by successively arranging transmission data composed of digital bits 1 and 0. The structure of such a data frame will be described in detail in Fig.

The LED driving unit 120 drives the LED 130 according to the transmission data coded as described above. For example, the LED 130 is turned on and off according to bits 1 and 0 of transmission data. The LED driving unit 120 turns on / off the LED 130 according to a predetermined pulse frequency. In this way, the LED driving unit 120 outputs the transmission data to be transmitted through on / off control of the LED 130. [

The LED 130 serves as a transmitter in the present OCC system 100. At least one such LED 130 is provided and is turned on or off at a predetermined pulse rate by the LED driving unit 120 according to transmission data coded in the data coding unit 110 . According to the present exemplary embodiment, when the light sources 120 are provided in a plurality of the light sources 120, they may be arranged in 1 × N, M × 1, and preferably M × N. Of course, it can be arranged in various forms such as circular, radial, elliptical and the like. The LED 130 recognizes that the on / off state of the LED 130 can not be distinguished from the on / off state by the human eye if the pulse rate is 110 or more per second. This pulse rate can of course be adjusted.

The rolling shutter camera 140 serves as a receiver in the present OCC system 100. The rolling shutter camera 130 captures an on / off image of a light source according to a predetermined frame rate for each of a plurality of rows by using a rolling shutter method. For this purpose, a rolling shutter type image sensor is provided inside. Each row of the image sensor is sequentially exposed at predetermined time intervals during a predetermined integration time. The last exposure time of the first column and the last exposure time of the last column are the frame time, and the sum of the exposure time and the frame time is the capture time. The captured image during this capture time appears as a white band when the LED 130 is on and appears as a black band when it is off. Changes in the on / off state of the LEDs are sequentially recorded during the capture time. At this time, the white band and the black band may be set to represent, for example, 1 and 0 as data, respectively. Thus, the rolling shutter camera 140 can receive multiple data within one frame. As the image sensor, for example, a CMOS sensor can be used. At this time, the rolling shutter camera 140 can start shooting at an arbitrary point of time while the LED 130 is turned on or off. In this case, it is necessary to distinguish the start frame and the data frame from the captured image. In addition, although the frame rate for photographing the on / off image of the LED 130 of the rolling shutter camera 140 is preset in the present embodiment, there is a need for a technique for accurately receiving data even when the actual frame rate varies Do. This will be described in detail below. In this embodiment, the rolling shutter camera 140 may include a digital camera, a camera mounted on a mobile phone, a smart device, or the like.

The image processing unit 150 generates a brightness signal corresponding to the brightness value of the on / off image of the LED 130 photographed for each of a plurality of rows in the rolling shutter camera 140. Specifically, as described above, the LED 130 appears as a white band and a black band in the process of being turned on or off according to the transmission data, and the brightness values of the respective bands may be different. That is, the color that is displayed according to the ON / OFF state of the LED 130 may be displayed as a brightness value of 0 to 255, for example. For example, the white band may represent a brightness value of 255, and the black band may represent a brightness value of 0. [ Of course, the range of brightness values can be changed.

The data extraction unit 160 extracts transmission data from the brightness signal of the on / off image of the LED 130 generated in the image processing unit 150. [ This restores the transmission data coded in the on / off image of the LED 130 according to the data to be transmitted by the data coding unit 110. For example, when the transmission data 1 to be transmitted in the data coding unit 110 corresponds to the ON state of the LED 130 and the transmission data 0 corresponds to the OFF state of the LED 130, 150, 1 is extracted in the ON image of the LED 130, and 0 is extracted in the OFF image. At this time, in the present invention, transmission data is extracted by using the brightness value in the brightness signal of the on / off image of the LED 130. Specifically, the inclination of the brightness signal, that is, the rise and fall of the brightness signal are combined and extracted.

2 is a structural diagram of a data frame according to transmission data in an optical camera communication (OCC) system according to an embodiment of the present invention.

Referring to FIG. 2, a plurality of super-frames 20, each of which is divided according to transmission data to be transmitted, are continuously implemented in a data frame of transmission data according to the present invention. That is, when each of the super frames 20 has different transmission data, for example, when it is desired to transmit other transmission data from the LED 130, each super frame 20 includes these respective transmission data. Therefore, each superframe 20 includes different transmission data.

Each of these super frames 20 includes data sub-frames (DS: Data Sbuframe) 21 repeated N times (N = natural number). Here, the transmission data to be transmitted in the OCC system 100 is included in the data sub-frame (DS) 21. As described above, according to the present invention, since each data sub-frame 21 including transmission data to be transmitted is repeatedly transmitted N times, data can be extracted even if the rolling shutter camera 130 photographs at an arbitrary point in time, The frame rate of the shutter camera 130 can be varied to extract the entire data even if the entire transmission data is not captured at one time in the captured frame image.

Each data subframe 21 includes a start frame (SF), two asynchronous bits (Ab), and a data packet (DP). The start frame SF and the asynchronous bit Ab are preferably set to 1 bit in consideration of the capacity of the data frame. The asynchronous bit Ab serves to distinguish the super frame 20 and the data packet DP in the data sub-frame 21. Also, the data packet DP includes transmission data to be transmitted by the OCC system 100. In the present embodiment, the asynchronous bit Ab is preferably alternately inserted into 1 bit and 1 bit. For example, if the index of the superframe 20 classified by transmission data is an odd number, the asynchronous bit is set to 1 and the asynchronous bit is set to 0 when it is an even number. Of course, it is also possible to alternately insert 0s and 1s. As a result, the asynchronous bit Ab serves as an identifier for distinguishing between neighboring super frames 20.

As described above, in the present invention, the transmission data to be transmitted is included in the data sub-frame (DS) (21), and such a data sub-frame (21) is repeated N times to continuously configure one super frame . Thus, superframes are repeated N times for each transmission data, and each superframe includes different transmission data.

The structure of such a data frame is to code transmission data to be transmitted by the data coding unit 110 into a data frame. That is, according to the data frame structure proposed in the present embodiment, the data coding unit 110 transmits the transmission data to be transmitted to the super-duplexer using a data packet DP, a start frame SF, and an asynchronous bit Ab. Frame. This is proposed to efficiently extract transmission data from the on / off image of the LED 130 photographed by the rolling shutter camera 140. The process of extracting transmission data from the data extracting unit using the data frame structure will be described in detail with reference to FIG.

3 is a schematic diagram illustrating a process of extracting transmission data from a rolling shutter camera of an optical camera communication (OCC) system according to an embodiment of the present invention.

In FIG. 3, two super frames 20a, 20b and 20c are shown for convenience of explanation and two data sub frames 21a, 21b and 21c are shown in each of the super frames 20a, 20b and 20c. The data subframes 21a, 21b and 21c are repeated twice (i.e., N = 2) in the super frames 20a, 20b and 20c. The example of FIG. 3 is only one example for illustrating the present invention, and the present invention is not limited thereto.

3 (a) and 3 (b), the LED driving unit 120 turns on / off the LED 130 corresponding to the structure of the super frames 20a, 20b, and 20c coded according to the transmission data to be transmitted And the rolling shutter camera 140 photographs the frame image at an arbitrary point in time according to the frame rate.

3 (a) illustrates a process of extracting a data sub-frame by combining two frame images. In the rolling shutter camera 140, the first through third frame images (image # 1 through image # 3) And captures the data sub-frame DS with the data sub-frame DS. Specifically, the first data sub-frame 201a of the first super frame 20a is captured in the first frame image (image # 1), the first super frame 20a is captured in the second frame image (image # 2) The second data sub-frame 202a of the first super frame 20b and the first data sub-frame 201b of the second super frame 20b are captured in the third frame image (image # 3) And captures the data sub-frame 202b and the first data sub-frame 201c of the third super frame 20c.

At this time, in order to extract a data packet included in each of the super frames 20a, 20b and 20c, the data extracting unit 160 first checks the start frame SF and the asynchronous bit Ab in the data subframe of the superframe do. Since N (N = 2) repeated data subframes within one superframe are added with one start frame SF and two asynchronous bits Ab, the start frame SF and the data packet DP When the asynchronous bit Ab added to the front end and the back end is checked, the data packet DP can be extracted. Here, in this embodiment, since the rolling shutter camera 140 acquires the frame image at an arbitrary point in time, it may happen that the start frame and the asynchronous bit Ab of the data subframe can not be confirmed in one frame image.

In order to solve this problem, in the present embodiment, the start frame SF, the asynchronous bit Ab at the previous stage, and the first super frame 20a in the first frame image (image # 1) captured as shown in FIG. The first data frame 201a is combined with the second data frame 201b of the first super frame 20a captured in the second frame image (image # 2) And extracts the data subframe 21a. At this time, the data sub-frame 21a can be identified by the asynchronous bit Ab at the rear end of the data packet DP. That is, since the same asynchronous bits Ab are inserted into the front and rear ends of the plurality of data sub-frames 21a included in the first super frame 20a, if it is confirmed that they are the same asynchronous bits Ab, It means that the captured superframe and the data subframe contain the same transmission data. Therefore, it is possible to confirm whether the same data packet is the same through checking the asynchronous bits Ab captured in each frame image.

The same applies to the second super frame 20b. That is, after confirming the start frame SF and the asynchronous bit Ab at the previous stage in the captured second frame image (image # 2), the second data sub-frame 21b of the second super frame 20b, Frame 21b of the second super frame 20b by combining the first data sub-frame 21c of the second super frame 20b in the third super-frame image (image # 3). Of course, this data sub-frame 21b is also confirmed by the asynchronous bit Ab added to the rear end of the data packet DP.

3 (a) illustrates an example in which the rolling shutter camera 140 captures an on / off image in a frame image by photographing the LED 130 at an arbitrary point in time, The data subframe can be extracted on the basis of the principle. That is, the data subframe can be extracted by a combination of two frame images according to the shooting time and frame rate.

FIG. 3 (b) illustrates a process of extracting a data sub-frame by one frame image. Specifically, the rolling shutter camera 140 captures the first data sub-frame 201a and the second data sub-frame 202a of the first super frame 20a in the first frame image (image # 1) And the first data sub-frame 201b and the second data sub-frame 202b of the second super frame 20b are captured in the frame image (image # 2).

At this time, the data extracting unit 160 first checks the start frame SF and the asynchronous bit Ab in the data sub-frame of the super frame captured in the first frame image (image # 1). That is, the start frame SF of the data sub-frame 201a of the first super frame 20a captured in the first frame image (image # 1) and the asynchronous bit Ab of the preceding stage are checked, The data sub-frame 21a of the first super frame 20a is extracted by combining the data sub-frame 201a at the front end of the first super frame 20a and the data sub-frame 202a after the asynchronous bit Ab at the previous stage. At this time, this data sub-frame 21a can be confirmed by the asynchronous bit Ab at the rear end of the data packet DP.

If there is an asynchronous bit Ab at the previous stage and an asynchronous bit Ab at the subsequent stage in the same one frame image in FIGS. 3A and 3B, the transmission data is extracted from the data packet DP located therebetween . However, according to the present invention, when the photographing time of the rolling shutter camera 140 is arbitrarily determined and the frame rate of the rolling shutter camera 140 is variable, the data packet DP and the non- (Ab) and the asynchronous bit (Ab) at the subsequent stage are not all present. In order to solve such a problem, a method of extracting transmission data by combining two data packets in two frame images as shown in FIG. 3 (a) and a method of extracting transmission data by combining two data And a method of extracting transmission data by combining packets.

As described above, in the present invention, the rolling shutter camera 140 combines the data sub-frames of the data frames captured in the respective frame images to extract the data packets therein, thereby extracting transmission data . Such data extraction can extract data by two frame images as shown in FIG. 3A and extract data by one frame image as shown in FIG. 3B. In order to extract data in this manner, one or more data sub-frames must be captured for each frame image in the present invention. For this, the number of data frames (Nrepeats) captured per frame image should satisfy the following equation (1).

Figure pat00002

tcap is the capture time at which one frame image is exposed in the rolling shutter camera, N is the number of repetitions of the data subframe (DS) in the superframe, and DSlength is the length of the data subframe. Here, when Nrepeats = 1, the data rate in the unidirectional communication exhibits the maximum performance.

4 is an amplitude response pattern of a rolling shutter camera with respect to a pulse frequency of an LED in an optical camera communication (OCC) system according to an embodiment of the present invention.

Referring to FIG. 4, in the OCC system according to the present invention, the LED 130 is driven in accordance with the pulse frequency set in the LED driver 120. At this time, the on / off driving of the LED 130 should be avoided to flicker to be safe for human eyes. For this purpose, the pulse frequency for turning on / off the LED 130 must be sufficiently high. For example, the pulse frequency of the LED 130 should be at least 100 Hz. However, the frame rate of the rolling shutter camera 140 applied to the OCC system is approximately 30 fps, which is a significantly lower rate than the pulse frequency of the LED 130 in kHz units. Since the frame rate of the rolling shutter camera 140, which is a receiver, is much lower than the on / off flicker speed of the LED 130, the on / off image of the LED 130 is captured in the frame image due to the speed difference. Which can lead to data loss. Accordingly, in the present invention, the sampling rate of the frame image captured when the rolling shutter camera 140 is applied is set to the shutter speed instead of the frame rate. That is, the shutter speed of the rolling shutter camera 140 is in the unit of kHz, and is high enough to easily record even the state change of the LED 130 according to the data transfer.

4 is a graph of experimental results as to how the rolling shutter camera 140 responds to the flashing LED 130 to extract a suitable pulse frequency range for the rolling shutter camera 140 as a receiver, And shows the amplitude response pattern according to the pulse frequency while keeping the distance d to the rolling shutter camera 140 constant. It can be seen in FIG. 4 that the response of the rolling shutter camera 140 at the high pulse frequency is small, and that at the pulse frequency higher than the shutter speed, the rolling shutter camera 140 can not record the signal. Therefore, it is preferable that the ON / OFF pulse frequency of the LED 130 according to the present invention is set within a range of the shutter speed of the rolling shutter camera 140. [ Since the general rolling shutter camera 140 has a shutter speed of 4 kHz, it is preferable to set it in the range of 100 Hz to 4 kHz in the present embodiment. Of course, this is merely an example, and it is natural that the pulse frequency of the LED 130 can be further extended in the case of the rolling shutter camera 140 having a shutter speed higher than that.

5 is a flowchart illustrating an optical camera communication method using an LED and a rolling shutter camera according to an embodiment of the present invention.

5, in the optical camera communication method using the LED and the rolling shutter camera according to the present invention, transmission data to be transmitted by the data coding unit 110 is encoded and a data frame including the coded transmission data is configured (S101 ). In the present embodiment, a plurality of superframes 20 are sequentially arranged in the data frame according to a plurality of transmission data to be transmitted. Each of the superframes 20 includes N (N = natural number) And a data sub-frame (DS) 21 of FIG. Each of the data sub-frames 21 includes a data packet DP including transmission data coded as described above, an asynchronous bit Ab added to the front end and a rear end of the data packet DP, And a start frame (SF) added to the previous stage.

Then, the LED 130 is turned on / off to correspond to the data frame according to the pulse frequency set by the LED driver 120 (S103). Thereby, the LED 130 is turned on / off to correspond to the data frame including the transmission data. Specifically, it is turned on / off to correspond to the start frame SF, the front-end asynchronous bit Ab, the data packet DP, and the rear-end asynchronous bit Ab included in the data sub-frame 21. Of course, the data subframe 21 is turned on / off so as to be repeated N times predetermined times, and further, it is turned on / off so as to be divided into superframes.

Subsequently, an on / off image of the LED 130 is captured in a continuous frame image for each of a plurality of rows by a rolling shutter method according to a frame rate set by the rolling shutter camera 140 (S105). The rolling shutter camera 140 captures and captures an image for each of a plurality of rows during one capture time 10. [ At this time, image capturing for each row is performed by a nonlinear scanning method at predetermined time intervals. That is, each row of an image sensor (not shown) provided in the camera is sequentially exposed for a predetermined integration time, and each row is exposed at a predetermined time interval. The last exposure time of the first row and the last exposure time of the last row are called frame time, and the exposure time and frame time are the capturing time.

Thereafter, the image processor 150 generates a brightness signal corresponding to the brightness value of the on / off image of the LED 130 captured in the continuous frame image for each row (S107) The transmission data is extracted from the brightness signal (S109).

6 is a flowchart illustrating a process of extracting transmission data from a data extracting unit in an optical camera communication using an LED and a rolling shutter camera according to an embodiment of the present invention.

Referring to FIG. 6, in order to extract transmission data according to an embodiment of the present invention, a start frame (SF) and an asynchronous bit Ab at the previous stage are extracted from a first frame image captured by the data extraction unit 160, (S201). Next, the data packet DP located at the rear end of the asynchronous bit Ab of the preceding stage is extracted (S203). Subsequently, it is determined whether there is a rear-end asynchronous bit Ab at the rear end of the data packet DP in the first frame image (S205). If there is any, the transmission data is extracted from the data packet DP (S207) The asynchronous bit Ab of the succeeding stage is extracted from the second frame image of the neighbor captured continuously in one frame image (S209). In this manner, it is determined whether the asynchronous bit Ab of the trailing end extracted from the second frame image coincides with the asynchronous bit Ab of the preceding stage extracted from the first frame image (S211). If they match, the data packet DP located at the previous stage of the asynchronous bit Ab at the subsequent stage is extracted (S213). This is because the asynchronous bit Ab of the data packet DP and the backward asynchronous bit Ab of the data packet DP are identical because the asynchronous bits Ab inserted in the consecutive data sub- ) Are the same, it implies that the data packet DP contained therein is the same. Subsequently, transmission data is extracted by combining the data packet DP extracted from the first frame image and the data packet DP extracted from the second frame image (S215). As a result, transmission data is extracted from two neighboring frame images.

7 is a flowchart illustrating a process of extracting transmission data in an optical camera communication using an LED and a rolling shutter camera according to another embodiment of the present invention.

Referring to FIG. 7, in another embodiment of the present invention, the start frame SF and the asynchronous bit Ab at the previous stage are extracted from one first frame image captured by the data extracting unit 160 (S301) , The data packet DP located at the rear end of the asynchronous bit Ab of the preceding stage is extracted (S303). Subsequently, it is determined whether there is a rear end asynchronous bit Ab at the rear end of the data packet DP in the first frame image (S305). If YES, the transmission data is extracted from the data packet DP (S307) It is determined whether or not there is a rear end asynchronous bit Ab in the preceding stage of the asynchronous bit Ab at step S309. If there is the rear end asynchronous bit Ab, The data packet DP extracted in step S303 is combined to extract transmission data (S311). This confirms the start frame (SF) and the asynchronous bit (Ab) at the previous stage in one frame image and stores the data packet (DP) at the previous stage of the start frame (SF) Extracts transmission data from the frame DP, and extracts transmission data by one frame image.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. Furthermore, the terms "comprises", "comprising", or "having" described above mean that a component can be implanted unless otherwise specifically stated, But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

110: Data coding unit 120: LED driving unit
130: LED 140: Rolling shutter camera
150: Image processing unit 160: Data extraction unit

Claims (8)

A coding step of coding transmission data to be transmitted by the data coding unit and constituting a data frame including the coded transmission data;
A driving step of turning on / off the LED to correspond to the data frame in accordance with the pulse frequency in the LED driver;
Capturing an on / off image of the LED according to a frame rate in a rolling shutter camera into a continuous frame image for each of a plurality of rows in a rolling shutter manner;
A generating step of generating a brightness signal according to a brightness value of the on / off image of the LED captured in the continuous frame image for each row in the image processing unit; And
An extracting step of extracting the transmission data from the brightness signal by an image extracting unit; Lt; / RTI >
Wherein each of the superframes includes N consecutive repeated data sub-frames (N is a natural number), and each of the super frames includes a plurality of data sub- The subframe includes a data packet DP including the coded transmission data, an asynchronous bit Ab added to the front end and a rear end of the data packet, and a start frame SF added to the front end of the front end asynchronous bit And a method of communicating an optical camera using a rolling shutter camera. The method according to claim 1,
And an LED and a rolling shutter camera capturing at least one data sub-frame for each frame image. 3. The method of claim 2,
Wherein the number Nrepeats of data subframes captured per each frame image satisfies the following equation.
Figure pat00003

(tcap is the capture time at which one frame image is exposed in the rolling shutter camera, N is the number of repeats of the data subframe (DS) in the superframe, DSlength is the length of the data subframe) The method according to claim 1,
The asynchronous bit Ab is an identifier for identifying consecutive neighboring superframes. When the index is odd and the number of superframes divided by the transmission data is continuously arranged, and a plurality of bits are alternately added to the data sub-frame, and a rolling shutter camera. 5. The method according to claim 4,
(SF) and the asynchronous bit (Ab) at the previous stage in the first frame image captured by the data extracting unit and extracts a data packet (DP) positioned at the rear end of the previous asynchronous bit (Ab) step;
If there is an asynchronous bit Ab at the rear end of the data packet DP in the first frame image, extracts transmission data from the data packet DP, and if not, A second step of extracting an asynchronous bit Ab at a subsequent stage in the second frame image;
A third step of determining whether the asynchronous bit Ab of the trailing end extracted from the second frame image coincides with the asynchronous bit Ab of the preceding stage extracted from the first frame image;
A fourth step of extracting a data packet (DP) located at the previous stage of the asynchronous bit (Ab) of the subsequent stage, if it is matched;
A fifth step of extracting transmission data by combining a data packet DP extracted from the first frame image and a data packet DP extracted from the second frame image; And an optical camera communication method using a rolling shutter camera. 5. The method according to claim 4,
(SF) and the asynchronous bit (Ab) at the previous stage in the first frame image captured by the data extracting unit and extracts a data packet (DP) positioned at the rear end of the asynchronous bit Ab at the previous stage Stage 1;
If there is a rear end asynchronous bit Ab at the rear end of the data packet DP extracted from the first frame image, the transmission data is extracted from the data packet DP, A second step of determining whether or not there is an asynchronous bit Ab;
A third step of extracting a data packet DP located at a preceding stage of the asynchronous bit Ab of the subsequent stage if the next stage asynchronous bit Ab is present; And
A fourth step of extracting transmission data by combining the data packet DP extracted in the first step and the data packet DP extracted in the third step; And an optical camera communication method using a rolling shutter camera. The method according to claim 1,
Wherein the pulse frequency is set within a range of a shutter speed of the rolling shutter camera, and a rolling shutter camera. 8. The method of claim 7,
Wherein the pulse frequency is set within a range of 100 Hz to 8 kHz.
KR1020160003125A 2015-02-17 2016-01-11 Optical camera communication method using LED and rolling shutter camera KR101952994B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/551,561 US10560188B2 (en) 2015-02-17 2016-01-15 Image sensor communication system and communication method using rolling shutter modulation
PCT/KR2016/000482 WO2016133285A1 (en) 2015-02-17 2016-01-15 Image sensor communication system and communication method using rolling shutter modulation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR20150070608 2015-05-20
KR1020150070608 2015-05-20

Publications (2)

Publication Number Publication Date
KR20160137342A true KR20160137342A (en) 2016-11-30
KR101952994B1 KR101952994B1 (en) 2019-02-27

Family

ID=57707793

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160003125A KR101952994B1 (en) 2015-02-17 2016-01-11 Optical camera communication method using LED and rolling shutter camera

Country Status (1)

Country Link
KR (1) KR101952994B1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019235870A1 (en) * 2018-06-08 2019-12-12 숭실대학교산학협력단 Optical signal transmission device and optical signal communication device
KR20190139748A (en) * 2018-06-08 2019-12-18 숭실대학교산학협력단 Optical signal transmitter and apparatus of optical signal communication
KR20200064866A (en) 2018-11-29 2020-06-08 주식회사 케이티앤씨 Apparatus and method for object speed detection using rolling shutter
KR102136497B1 (en) * 2019-10-24 2020-07-21 국민대학교산학협력단 Method and apparatus for modulating and demodulating optical camera communication signal
US11700461B2 (en) 2020-12-28 2023-07-11 Kookmin University Industry Academy Cooperation Foundation Method and apparatus for processing image based on optical communication, and computer-readable recording medium with program therefor
US11979193B2 (en) 2019-07-08 2024-05-07 Kookmin University Industry Academy Cooperation Foundation Communication method and device in optical camera communication system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090016176A (en) 2007-08-10 2009-02-13 삼성전자주식회사 Method and apparatus for processing signals of photodetector in visible light communication
JP2009539300A (en) * 2006-06-05 2009-11-12 サムスン エレクトロニクス カンパニー リミテッド Channel assignment management method for asynchronous data transmission, asynchronous data transmission method, and apparatus using the method
KR20100135683A (en) 2009-06-17 2010-12-27 호야 가부시키가이샤 Imager that photographs an image using a rolling shutter
JP2014533030A (en) * 2011-11-04 2014-12-08 ザ ユニバーシティ コート オブ ザ ユニバーシティ オブ エジンバラ Communication apparatus and method
KR101472583B1 (en) 2014-01-28 2014-12-16 국민대학교산학협력단 Camera communication system, camera communication method and camera device therefor

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009539300A (en) * 2006-06-05 2009-11-12 サムスン エレクトロニクス カンパニー リミテッド Channel assignment management method for asynchronous data transmission, asynchronous data transmission method, and apparatus using the method
KR20090016176A (en) 2007-08-10 2009-02-13 삼성전자주식회사 Method and apparatus for processing signals of photodetector in visible light communication
KR20100135683A (en) 2009-06-17 2010-12-27 호야 가부시키가이샤 Imager that photographs an image using a rolling shutter
JP2014533030A (en) * 2011-11-04 2014-12-08 ザ ユニバーシティ コート オブ ザ ユニバーシティ オブ エジンバラ Communication apparatus and method
KR101472583B1 (en) 2014-01-28 2014-12-16 국민대학교산학협력단 Camera communication system, camera communication method and camera device therefor

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019235870A1 (en) * 2018-06-08 2019-12-12 숭실대학교산학협력단 Optical signal transmission device and optical signal communication device
KR20190139748A (en) * 2018-06-08 2019-12-18 숭실대학교산학협력단 Optical signal transmitter and apparatus of optical signal communication
KR20200075802A (en) * 2018-06-08 2020-06-26 숭실대학교산학협력단 Optical signal transmitter and apparatus of optical signal communication
KR20200064866A (en) 2018-11-29 2020-06-08 주식회사 케이티앤씨 Apparatus and method for object speed detection using rolling shutter
US11979193B2 (en) 2019-07-08 2024-05-07 Kookmin University Industry Academy Cooperation Foundation Communication method and device in optical camera communication system
KR102136497B1 (en) * 2019-10-24 2020-07-21 국민대학교산학협력단 Method and apparatus for modulating and demodulating optical camera communication signal
US11700461B2 (en) 2020-12-28 2023-07-11 Kookmin University Industry Academy Cooperation Foundation Method and apparatus for processing image based on optical communication, and computer-readable recording medium with program therefor

Also Published As

Publication number Publication date
KR101952994B1 (en) 2019-02-27

Similar Documents

Publication Publication Date Title
KR101952994B1 (en) Optical camera communication method using LED and rolling shutter camera
JP5936902B2 (en) Transmission system, transmission device and reception device
KR101651584B1 (en) Image Sensor Communication between LED-to-Rolling Shutter Camera using Multiple Frequency Shift Keying
KR101472583B1 (en) Camera communication system, camera communication method and camera device therefor
US10560188B2 (en) Image sensor communication system and communication method using rolling shutter modulation
CN102006120B (en) Method and apparatus for visible light communication
Nguyen et al. High-speed asynchronous Optical Camera Communication using LED and rolling shutter camera
US10432215B2 (en) Optical camera communication apparatus and method
JP7090233B2 (en) Visible light communication system
KR101937560B1 (en) Image sensor communication system based on dimmable M-PSK
US10659172B2 (en) Capsule endoscope transmitter and capsule endoscope receiver configured to perform human body communication and human body communication method using the same
WO2015121135A1 (en) Signaling using idle period for coded light
US9929807B2 (en) Visible light receiving method
US20210126713A1 (en) Roi-based optical wireless communication method and device
KR101706849B1 (en) Apparatus and method for transceiving data using a visible light communication system
KR101550375B1 (en) Optical camera communication system
KR101466954B1 (en) LED Visual Light Communication System and the Communication Method thereof
CN106452583A (en) Device and method for parallel video transmission based on visible light
KR101625534B1 (en) Optical Camera Communication System using Rolling Shutter Camera
CN109586791B (en) Visible light communication method and device
KR102136497B1 (en) Method and apparatus for modulating and demodulating optical camera communication signal
KR102076393B1 (en) Method and apparatus for lighting control using optical camera communication
KR101550376B1 (en) Dimming controling method for optical camera commnication system
CN108156364A (en) Visible Light Communication System And Method
Marcu et al. Flicker free VLC system with enhanced transmitter and low frame rate camera

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant