KR101625534B1 - Optical Camera Communication System using Rolling Shutter Camera - Google Patents

Abstract

The present invention relates to an optical camera communication (OCC) system which extracts data from an on-image and an off-image of a light source shot by a rolling shutter camera. According to an embodiment of the present invention, the OCC system using the rolling shutter camera comprises: a coding unit which codes transmission data to transmit; the light source turned on and off depending on the transmission data coded by the coding unit; the rolling shutter camera which continuously shoots the on-image and the off-image depending on the on state and the off state of the light source for each plurality of rows in the rolling shutter mode; an image processing unit which generates a luminance signal in accordance with the luminance value of the on-image and the off-image of the light source shot for each of the rows by the rolling shutter camera; and a data extracting unit which extracts the transmission data form the luminance signal of the on-image and the off-image of the light source generated by the image processing unit.

Description

Technical Field [0001] The present invention relates to an optical camera communication system using a rolling shutter camera,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an optical camera communication (OCC) system, and more particularly, to an optical camera communication (OCC) system using a rolling shutter camera that extracts data from on / off images of a light source photographed using a rolling shutter camera Communication system.

Visible Light Communication (VLC), which is a typical illumination / communication convergence technology, is a technology for carrying out wireless communication by transmitting information on illumination of a light source. Conventionally, a light source is received by a photodiode (PD) (digital data 1 or 0) in accordance with the on / off state of the data.

Conventionally, a visible light communication system has been proposed in which a plurality of LEDs are photographed using a camera instead of a photodiode, and data corresponding to ON / OFF of the LEDs acquired for each frame of the camera is extracted. In this way, visible light communication using a camera is also called an optical camera communication (OCC) system in that a camera is used instead of a photodiode as an optical receiver, and work for standardization in the IEEE 802.15.7a research group is attempted have.

Recently, there has been an attempt to apply a rolling shutter camera to such an optical camera communication system (OCC). The rolling shutter camera is an electronic shutter used in most CMOS image sensors. The rolling shutter camera captures images for each row of image sensors combined in a plurality of rows and combines them to obtain an image for each frame.

However, in the related art, there is not yet a technique for extracting data corresponding to on / off images of a light source for each row in a rolling shutter camera. In addition, when an on / off image of a light source is photographed using a rolling shutter camera in an optical camera communication system, since the photographing starts at an arbitrary point in time, the frame of the rolling shutter camera can not be precisely aligned with the on / off timing of the light source do. In this case, it is difficult to extract accurate data.

Korea Patent No. 1472583 Korean Patent Publication No. 2009-0016176 Korea Patent Publication No. 2010-0135683

An object of the present invention is to provide an optical camera communication system using a light source and a rolling shutter camera that photographs a light source using a rolling shutter camera and extracts data from on / off images of the light source.

An optical camera communication system using a rolling shutter camera according to an embodiment of the present invention includes a coding unit for coding transmission data to be transmitted; A light source that is turned on / off according to transmission data coded in the coding unit; A rolling shutter camera for continuously photographing on / off images according to ON / OFF of the light source in a plurality of rows by a rolling shutter system; An image processor for generating a brightness signal according to a brightness value of an on / off image of the light source photographed for each of a plurality of rows in the rolling shutter camera; And a data extracting unit for extracting the transmission data from the brightness signal of the on / off image of the light source generated by the image processing unit, wherein the light source includes a time interval of on or off for indicating a start frame (SF) And the time interval of on or off for indicating transmission data are set to be different from each other.

delete

In the present invention, the data extracting unit distinguishes the start frame and the transmission data from the on / off time interval, and extracts the transmission data from the brightness signal of the transmission data existing between the brightness signals of the two start frames.

In the present invention, the data extracting unit extracts the transmission data using the slope of the brightness signal at a predetermined brightness value for each of the plurality of rows in the brightness signal generated for each of the plurality of rows.

In the present invention, the data extracting unit extracts the transmission data by a combination of a portion where the brightness signal rises at the predetermined brightness value and a portion that falls after the rising portion.

In the present invention, the data extracting unit extracts the transmission data by a combination of a maximum brightness value and a minimum brightness value for each of the plurality of rows in the brightness signal generated for each of the plurality of rows.

According to the present invention, a rolling shutter camera can be applied to an optical camera communication system to extract data corresponding to an on / off image according to on / off of a light source.

In addition, according to the present invention, when data is transmitted using a rolling shutter camera, the reliability of data transmission can be improved by accurately detecting a start frame when the camera is photographed at an arbitrary point in time.

1 is a view for explaining a process of capturing on / off images of a light source in a rolling shutter camera according to the present invention;
FIG. 2 is an illustration of an on / off image of a light source captured by a rolling shutter camera according to the present invention,
3 is a configuration diagram of an optical camera communication system using a rolling shutter camera according to an embodiment of the present invention,
4 is a diagram for explaining a principle of generating a brightness signal in an image processing unit according to an embodiment of the present invention,
5 is a diagram illustrating a process of extracting transmission data from a brightness signal for an on / off image of a light source according to an embodiment of the present invention;
6 illustrates a process of extracting transmission data from a brightness signal for an on / off image of a light source according to another embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference numerals whenever possible, even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

FIG. 1 is a view for explaining a process of capturing on / off images of a light source in a rolling shutter camera according to the present invention.

Referring to FIG. 1, the roll shutter camera according to the present invention captures and captures an image for each of a plurality of rows during one capture time 10. At this time, image capturing for each row is performed by a nonlinear scanning method at predetermined time intervals. This exposes each row of an image sensor (not shown) provided in the camera for a predetermined integration time 11, and exposes a predetermined time interval for each row. The last exposure time of the first row and the last exposure time of the last row are called the frame time 12 and the exposure time and frame time are the capturing time 10.

2 is an exemplary view of an on / off image of a light source captured by a rolling shutter camera according to the present invention.

Referring to FIG. 2, a rolling shutter camera photographs a light source while the light source according to the present invention is turned on / off. The figure shows an example of capturing an on / off image for each of a plurality of rows in a rolling shutter camera while one light source is on and off, for example. Referring to the drawing, an image captured during a capture time is displayed as white (W) when the light source is on, and displayed as black (B) when the light source is off. Note that the brightness values of white (W) and black (B) may be different in the process that the light source is turned on or off. For example, since the rolling shutter camera captures images at an arbitrary point in time, when the light source is captured while the light source changes from the off state to the on state, the brightness value may be an intermediate color between white (W) and black (B). The captured image thus appears as shown in FIG. 2 (b).

3 is a configuration diagram of an optical camera communication system using a rolling shutter camera according to an embodiment of the present invention.

3, an optical camera communication system 100 using a rolling shutter camera according to the present invention includes a coding unit 110, a light source 120, a rolling shutter camera 130, an image processing unit 140, (150).

The coding unit 110 codes transmission data to be transmitted in the optical camera communication system. Such coding can be implemented in various ways. For example, when the transmission data to be transmitted is 1, the light source 120 is turned on, and when the transmission data is 0, the light source 120 is turned off. Such an example may be set differently according to the frequency pulse of the light source 120. [ For example, when the transmission data is 1, the light source is made to correspond to on-on, and when the transmission data is 0, the light source can be made to correspond to off-off. As described above, in the present invention, the coding unit 110 matches the ON / OFF states of the light sources corresponding to the transmission data and transmits the transmission data through on / off of the light source in the future.

The light source 120 serves as a transmitter in an optical camera communication system. At least one such light source 120 is provided and is turned on or off at a predetermined pulse rate according to transmission data coded in the coding unit 110. According to the embodiment, when the light sources 120 are provided in a plurality of the light sources 120, they may be arranged in 1 × N, M × 1, and preferably M × N. Of course, it can be arranged in various forms such as circular, radial, elliptical and the like. If the pulse rate at which the light source 110 is turned on / off is 110 times or more per second, it is recognized that the on / off state of the light source 110 can not be distinguished by the human eye. This pulse rate can of course be adjusted.

The rolling shutter camera 130 serves as a receiver in an optical camera communication system. The rolling shutter camera 130 captures an on / off image of a light source for each of a plurality of rows by a rolling shutter method using the light source 120 to be turned on and off. To this end, an image sensor for capturing an image by a rolling shutter method is provided, and sequentially exposes each row of the image sensor. When the light source 120 is turned on, the output of the exposed row appears as a white band, and when the light source is turned off, black appears as a black band . At this time, the white band and the black band are set to represent 1 and 0 as data, respectively. Thus, multiple data can be received within one frame. As the image sensor, for example, a CMOS sensor can be used. At this time, the rolling shutter camera 130 can start shooting at an arbitrary point in time when the light source 120 is turned on or off. In this case, it is necessary to distinguish the start frame and the data frame from the captured image. This will be described in detail below. In this embodiment, the rolling shutter camera 130 may include a digital camera, a camera mounted on a mobile phone, a smart device, or the like.

The image processing unit 140 generates a brightness signal corresponding to the brightness value of the on / off image of the light source 120 photographed for each of a plurality of rows in the rolling shutter camera 130. Specifically, as described above, a white band and a black band appear in the process of turning on or off the light source 120. The brightness values of the respective bands may be different. That is, the hue that appears on / off of the light source 120 may be displayed as a brightness value of 0 to 255, for example. For example, a white band may represent a brightness value of 255, and a black band may represent a brightness value of 0. [ Of course, the range of brightness values can be changed. In addition, as described above, since the rolling shutter camera 130 can capture images at an arbitrary point in time, when capturing while the light source 120 is changed from the off state to the on state, the brightness values of the white band and the black band It is possible to obtain a neutral color band. This means that the brightness value may be between 0 and 255. The image processor 140 generates a brightness signal corresponding to the brightness value of each band according to the on / off image of the light source 120 photographed for each row for each row. At this time, since the light source 120 is continuously turned on or off according to a predetermined frequency pulse, the brightness signal of the on / off image has a continuous value for each row.

The data extraction unit 150 extracts transmission data from the brightness signal of the on / off image of the light source 120 generated by the image processing unit 140. This is to restore the transmission data coded in the on / off image of the light source 120 in the coding unit 110. For example, when the transmission data to be transmitted by the coding unit 110 is 1, the light source 120 is turned on, and when the transmission data is 0, the light source 120 is turned off, The data extracting unit 150 extracts 1 from the on image of the light source 120 and extracts 0 from the off image. At this time, in the present invention, transmission data is extracted using the brightness value in the brightness signal of the on / off image of the light source 120. Specifically, the inclination of the brightness signal, that is, the rise and fall of the brightness signal are combined and extracted.

4 is a diagram for explaining a principle of generating a brightness signal in an image processing unit according to an embodiment of the present invention.

4 is an example of a case where the entire screen 20 of the rolling shutter camera 130 is smaller than the light source 120. [ It goes without saying that the entire screen 20 may be larger than the light source 120. In the example shown in FIG. 4, the rolling shutter camera 130 captures an image with a white band W and a black band B for each of a plurality of rows while the light source 10 is turned on or off. In the figure, for example, five columns are shown, and examples of capturing bands of W, B, W, W, and B are shown. At this time, since W is a white band, a brightness value is, for example, 255, and B is a black band. These brightness values may be set to other values. Thus, a brightness signal for a brightness value such as the one shown on the right side can be generated. Here, the brightness signal shown on the right side illustrates an ideal example, and the brightness value has an arbitrary value between 0 and 255 while the light source 120 is on and off. For example, when shooting is performed in an arbitrary column while the light source 120 changes from the on state to the off state, it may have a brightness value of 175, for example. It appears to be halfway between white and black. The brightness signal according to the brightness value may be expressed as shown in FIG. 5 and FIG. 6 when it is expressed for each row.

5 is a view for explaining a process of extracting transmission data from a brightness signal for on / off images of a light source according to an embodiment of the present invention. And extracting transmission data from the brightness signal for the image.

5 and 6, the horizontal axis represents a row, and the vertical axis represents a relative value of a brightness value. For example, when the white band is set to 255 and the black band is set to 0, the relative value on the vertical axis is closer to the white band, that is, to 255, and the lower the relative value, the closer to the black band. When the brightness value of the on / off image of the light source 120 is determined for each row as described above, the brightness signal according to each row is calculated from the brightness value in the image processing unit 140 as shown in FIGS. 5 and 6 . 5, in order to distinguish the start frame (SF) from the transmission data by using the on / off time of the light source 120 while the light source 120 is on / off, The ON or OFF time interval and the ON or OFF time interval of the light source 120 for the transmission data are set to be different from each other. For example, in Fig. 5, the start frame A implements a relatively long ON time and the transmission data B implements a relatively short ON time. Of course, these relative time lengths may be reversed. The difference in the time interval between the start frame A and the transmission data B is to distinguish which portion is transmitted data while the light source 120 is continuously turned on or off. Accordingly, in FIG. 5, it is determined that the brightness signal between the first start frame A and the second start frame A 'corresponds to the transmission data, and transmission data is extracted from the brightness signal. At this time, the data extracting unit 140 extracts transmission data using the slope of the brightness signal at the same brightness value preset for a plurality of rows in the brightness signal generated for each of a plurality of rows. Specifically, transmission data is extracted by a combination of rising and falling of a brightness signal at the same brightness value set in advance. For example, in FIG. 5, a brightness value at which a red dot and a green dot are located is set, and the slope of the brightness signal at the corresponding brightness value, that is, the slope value at the red point has a positive value, ) Value. At this time, if the (+) slope value at the red point at the predetermined brightness value is combined with the (-) slope value at the next green point, transmission data in a certain row can be extracted. For example, a brightness signal in B is formed in a white band in about 140th to 160th columns. It can be seen that the light source 120 is turned on in the 140th to 160th columns. In the 160th to 175th columns, since the brightness signal falls in the period, the light source 120 is photographed in the corresponding row while the light source 120 changes from on to off. In this case, if there is a portion where the slope of the brightness signal at a predetermined brightness value rises and a falling portion that follows the rising portion of the brightness signal of the on / off image of the light source 120, It can be seen that there is a case where the light source 120 is turned on by combining the parts so that the transmission data can be extracted. At this time, the predetermined brightness value is determined as a brightness value in the brightness signal. That is, if the brightness value is set too high or too low, there may be a portion where the brightness value exists. It is preferable to set a brightness value that can correspond to all the brightness signals when the red dot and the green dot shown in FIG. 5 correspond to the brightness signal.

6, the data extracting unit 150 extracts transmission data (hereinafter referred to as transmission data) by combining a maximum brightness value Max and a minimum brightness value Min for each of a plurality of rows in a brightness signal generated for each of a plurality of rows. . That is, transmission data may be extracted by combining a portion Max having a maximum brightness value and a portion Min having a minimum brightness value appearing after the start frame. It is possible to extract, for example, transmission data 1 from these two values when the maximum value Max of the brightness value and the minimum value Min thereafter appear. Conversely, if the maximum value (Max) appears after the minimum value (Min) of the brightness value, the transmission data 0 can be extracted.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. Furthermore, the terms "comprises", "comprising", or "having" described above mean that a component can be implanted unless otherwise specifically stated, But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

110: coding unit 120: light source
130: Rolling shutter camera 140: Image processing unit
150:

Claims (6)

A coding unit for coding transmission data to be transmitted;
A light source that is turned on / off according to transmission data coded in the coding unit;
A rolling shutter camera for continuously capturing an on / off image according to on / off of the light source in a plurality of rows in a rolling shutter manner;
An image processor for generating a brightness signal according to a brightness value of an on / off image of the light source photographed for each of a plurality of rows in the rolling shutter camera; And
A data extracting unit for extracting the transmission data from the brightness signal of the on / off image of the light source generated by the image processing unit; Lt; / RTI >
Wherein the light source sets a time interval of ON or OFF for indicating a start frame (SF) and a time interval of ON or OFF for indicating transmission data to be different from each other. delete The method according to claim 1,
Wherein the data extracting unit separates the start frame and the transmission data from the time interval of the on or off and extracts the transmission data from the brightness signal of the transmission data existing between the brightness signals of the two start frames, system. The method according to claim 1,
Wherein the data extracting unit extracts the transmission data using a slope of a brightness signal at a predetermined brightness value for each of the plurality of rows in the brightness signal generated for each of the plurality of rows, Camera communication system. 5. The method of claim 4,
Wherein the data extracting unit extracts the transmission data by a combination of a portion where the brightness signal rises at the predetermined brightness value and a portion that falls after the rising portion. The method according to claim 1,
Wherein the data extracting unit extracts the transmission data using a combination of a maximum brightness value and a minimum brightness value for each of the plurality of rows in the brightness signal generated for each of the plurality of rows, Optical camera communication system.

Images