KR101759249B1

KR101759249B1 - Data transferring method based on emitting light and sound wave

Info

Publication number: KR101759249B1
Application number: KR1020160003110A
Authority: KR
Inventors: 홍광석; 오병훈
Original assignee: 성균관대학교산학협력단
Priority date: 2016-01-11
Filing date: 2016-01-11
Publication date: 2017-07-18

Abstract

A method of transmitting data using light emission and sound waves includes the steps of determining the degree of image noise and the degree of acoustic noise of the transmitter, the color code or the color code used by the transmitter to transmit the source data according to the degree of the image noise, Converting the binary bit string corresponding to the source data to be transmitted into the color code when the transmitter transmits the source data using the color code, and transmitting the color code to the display device Converting the binary bit string corresponding to the source data to be transmitted into the sound wave signal when the transmitter transmits the source data using the sound wave signal and outputting the sound wave signal through the speaker, The color code or image output to the display device If the case of the step and the receiver for acquiring at least one of a sound wave signal output to the speaker obtain the color code decoding the obtained color code, and obtains the sound wave signal comprises decoding the sound wave signal.

Description

TECHNICAL FIELD [0001] The present invention relates to a data transmission method using light emission and a sound wave,

The techniques described below relate to techniques for transmitting data using light emission and sound waves.

Recently, near field communication (near field communication) technology has been attracting attention as a technology designed to exchange communication between devices at a near distance. Conventional short-range wireless communication technology has been developed for the purpose of exchanging information at a close distance using electromagnetic waves, but a separate NFC hardware is required to use this technology. Furthermore, research on data transmission techniques using other media (visible light, etc.) than RF signals is under way.

Korean Patent Publication No. 10-2012-0060475

The technique described below is intended to provide a technique of transmitting data using light (light emission) and sound waves.

A method of transmitting light using a light source and a sound wave includes the steps of: converting a source data to a source binary bit string, which is a binary bit string; transmitting a first segment to be transmitted in a color code among the source binary bit string; Determining a second segment to transmit as a signal, the transmitter converting the first segment to the color code, and outputting the color code to a display device, the transmitter converting the second segment to the sonic signal And outputting the sound wave signal through a speaker; acquiring the color code outputted by the receiver to the display device using a camera; acquiring the sound wave signal using a microphone device; A binary bit string decoded from the code and a binary bit decoded from the sound wave signal A combination of a step of decrypting the source data.

In another aspect, a method of transmitting data using light emission and sound waves includes the steps of determining the degree of image noise and the degree of acoustic noise of the transmitter, and the step of transmitting the source data according to the degree of the image noise and the degree of acoustic noise The method comprising the steps of: determining at least one of a color code and an acoustic wave signal; converting the binary bit string corresponding to the source data to be transmitted into the color code when the transmitter transmits the source data using the color code; Converting the binary bit string corresponding to the source data to be transmitted into the sound wave signal when the transmitter transmits the source data using the sound wave signal and outputting the sound wave signal through the speaker , And the receiver outputs the color Decoding a color code obtained when the receiver obtains the color code, and decoding the sound wave signal when the sound wave signal is obtained, .

The technique described below can transmit data more accurately even in a noisy environment using light emission and sound waves. The techniques described below do not require additional hardware and can easily transfer data using a device such as a smart phone.

1 is an example of a system for transferring data using a color code and an acoustic signal.
2 is an example of bidirectional communication using a smart phone.
3 is an example of a process of transmitting data using a color code.
4 is an example of a region for outputting a color code on a display screen.
5 shows an example of outputting a color code on a display screen.
6 is another example of outputting a color code on the display screen.
7 is another example of outputting a color code on the display screen.
8 shows an example of outputting a binary code corresponding to text data using a color code on a display screen.
9 shows an example of a frame structure in which a color code is transmitted.
10 is an example of a flowchart of a process in which a code decoding apparatus extracts a color code.
11 is an example of a process of transmitting data using a sound wave signal.
12 is an example of a flow chart of a data transmission method using light emission and sound waves.
13 is another example of a flow chart of a data transmission method using light emission and sound waves.
14 is an example of a table in which a transmitter determines a noisy environment.
15 is an example of a method by which a transmitter transmits data in consideration of a noisy environment.

The following description is intended to illustrate and describe specific embodiments in the drawings, since various changes may be made and the embodiments may have various embodiments. However, it should be understood that the following description does not limit the specific embodiments, but includes all changes, equivalents, and alternatives falling within the spirit and scope of the following description.

The terms first, second, A, B, etc., may be used to describe various components, but the components are not limited by the terms, but may be used to distinguish one component from another . For example, without departing from the scope of the following description, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

As used herein, the singular " include "should be understood to include a plurality of representations unless the context clearly dictates otherwise, and the terms" comprises & , Parts or combinations thereof, and does not preclude the presence or addition of one or more other features, integers, steps, components, components, or combinations thereof.

Before describing the drawings in detail, it is to be clarified that the division of constituent parts in this specification is merely a division by main functions of each constituent part. That is, two or more constituent parts to be described below may be combined into one constituent part, or one constituent part may be divided into two or more functions according to functions that are more subdivided. In addition, each of the constituent units described below may additionally perform some or all of the functions of other constituent units in addition to the main functions of the constituent units themselves, and that some of the main functions, And may be carried out in a dedicated manner.

Also, in performing a method or an operation method, each of the processes constituting the above method may occur in a different order than that described in the context without explicitly specifying a specific order in the context. That is, each process may occur in the same order as described, may be performed substantially concurrently, or may be performed in the opposite order.

A computer device recognizes various types of data in binary form. The technique described below relates to a technique for transmitting data using two types of media. The medium used for transmitting data is a specific color in which the display device emits light and a sound wave signal output from the speaker. That is, one computer device converts data to be transmitted into color and / or sound wave signals and outputs them, and decodes color and / or sound wave signals received by the other computer device. A computer device refers to a computer device such as a PC, a notebook, a smart phone, a tablet PC, etc. connected to a monitor.

Hereinafter, a computer apparatus that codes (converts) data and outputs the data constantly is called a transmitter, and a computer apparatus that receives a signal output from a transmitter and decodes data is called a receiver. The transmitter may transmit various types of source data in color and / or sound wave signals. Source data is meant to include all types of data represented as digital data. The source data includes text data, image data, voice data, and the like. For convenience of explanation, text data among source data will be mainly described.

1 is an example of a system for transferring data using a color code and an acoustic signal. A system for transmitting data using a color code and an acoustic signal includes a transmitter for outputting a color code and an acoustic signal, and a receiver for acquiring a color code and an acoustic signal output from the transmitter to decode the data. 1 shows an example in which a smart phone is used as a transmitter and a PC is used as a receiver.

Describe the process of data transmission using color codes. The transmitter converts the source data into a binary bit string and outputs a color having a specific pattern corresponding to the binary bit string to the display device. Hereinafter, a color having a specific pattern corresponding to data is referred to as a color code. The transmitter outputs a color code to the display device (screen). The display device outputs R (Red), G (Green), and B (Blue) colors having a predetermined shape (pattern). Details about the color codes will be described later. A receiver including a camera acquires an image output from a display device, extracts a color code, and decodes data from a color code.

The data transmission process using the sound wave signal will be described. The transmitter converts the source data into a binary bit string, converts the binary bit string into a sound wave signal, and outputs it to the speaker. The receiver receives the microwave signal, converts the sound wave signal into a binary bit stream, and then decodes the data represented by the binary bit stream.

2 is an example of bidirectional communication using a smart phone. Figure 2 shows two terminals (smartphone). The first terminal and the second terminal can exchange data using the color code and the sound wave signal, respectively. Some examples of performing bi-directional communication are described. (1) The first terminal emits data requesting a telephone number to the second terminal using a color code, and the second terminal decodes the color code transmitted by the first terminal to extract data. The second terminal modulates and outputs its telephone number to a sound wave signal, and the first terminal can decode the data from the sound wave signal. (2) The first terminal modulates the password request for the cash transfer to the sound wave signal and outputs the modulated signal, and the second terminal outputs the requested password to the display device in the color code. The first terminal and the second terminal can exchange various data using a color code and an audio signal at a short distance.

Hereinafter, the binary code representing the source data and the color code representing the binary code will be described first. In a computer device, data is represented by a specific code. For example, the text data is represented by a code such as Unicode, ASCII code, or the like. The computer device finally recognizes an ASCII code or the like as a binary number bit string. A bit string representing specific data in a computer device is called a binary code. A color code represents a binary code in a specific color pattern.

Taking an ASCII code as an example, the text data of 'hello' is represented by 01101000 (h), 01100101 (e), 01101100 (l), 01101100 (l), 01101111 (o). Text data is composed of one character, and each character is represented by 8 bits. The color code eventually represents the entire text data.

Just as text data is composed of characters, the color code is also made up of the separating elements that make up the color code. The color code element is called the color code element. A color code element may correspond to an element such as a plurality of characters, a character (8-bit binary number), a binary digit of one digit, a binary digit of two digits, and the like. For example, (1) one color code element may correspond to '01101000 (h)'. In this case, the color code element would require 2 ⁸ = 256 to identify an 8-bit binary number. (2) One color code element may correspond to a two digit binary number. In this case, 2 ² = 4 color code elements are required. In this case, four consecutive color code elements (01/10/10/00) will be needed to express the above-mentioned '01101000 (h)'. (3) Further, one color code may correspond to binary numbers of various digits.

3 is an example of a process of transmitting data using a color code. The transmitter 10 receives the source data (101). The source data may be input directly by the user through the interface device, or may be transmitted by the transmitter 10 via the network. The transmitter 10 converts the source data, which is text data, into an ASCII code (102). The transmitter 10 converts the binary bit string (ASCII code) into a color code in units of a predetermined length (103). The transmitter emits a color code to the display device (104).

The receiver 50 acquires an image including the display screen of the transmitter 10 with the camera (111). The receiver 50 identifies an area corresponding to the color code in the acquired image, and extracts a color code (112). The receiver 50 generates a binary bit string (ASCII code), which is a color code, and converts it into source data which is text data (113). Finally, the receiver 50 can output the decoded source data to a screen or the like (114).

4 is an example of a region for outputting a color code on a display screen. A color code is generated by combining a specific color outputted from a display screen and a shape (pattern) outputting a color. 4 is an example showing a region in which a color code is output.

The color codes are output to a plurality of zones having a predetermined order. As will be described later, the order in which color codes are output in a plurality of zones may have important meaning. A specific example is described in Fig. Although various orders can be used, (1) for ease of illustration, assume that the direction from top to bottom is the basic direction when a plurality of zones are arranged vertically. In this case, the downward direction is the reverse direction. (2) In the case where a plurality of zones are arranged horizontally, it is assumed that the direction from left to right is the basic direction. In this case, the direction from right to left is reversed. (3) On the other hand, a plurality of zones may be arranged in a form having two or more rows or columns. In this case, the basic direction is the direction from top to bottom in the same column, and from the left to the right in the same row. (4) Further, a plurality of zones may be arranged in an area not continuous with the display screen. In this case, the predetermined order may be the basic direction.

Figure 4 (a) shows three columns I, J and K. The three columns represent two-digit binary bit sequences in the same order, respectively. That is, Fig. 4 (a) shows an example in which one color code element represents a two-digit bit stream. In FIG. 4 (a), since three columns are displayed on one screen, one screen can display a six-digit bit stream.

A region representing one color code element such as the columns I, J, and K in Fig. 4 (a) is referred to as a code group region. Each code group region has a plurality of sub-regions according to the number of bits in the bit string indicated by the code group region. A sub-section within a code group section is called a sub-section. Referring to FIG. 4 (a), three columns I, J and K correspond to code group zones, and each code group zone consists of four sub-zones representing 00, 01, 10 and 11 do. Each sub-zone must have a predefined bit that the sub-zone means. The transmitter and the receiver must share in advance a code group zone, a sub zone, information on the binary bits represented by each sub zone, and information on the basic direction (or reverse direction) of each sub zone.

In FIG. 4 (a), the transmitter may indicate a '00' bit if it outputs a specific color in the uppermost sub-zone of the I code group zone. If the transmitter outputs the top sub-zone of the I code group zone, the second sub-zone of the J code group zone and the last sub zone of the K code group zone, the six digit bit may indicate "00/01/11" ('/' Is an indication to distinguish the bit string represented by each code group section).

Fig. 4 (b) shows three code groups each with three columns as shown in Fig. 4 (a), and each code group zone includes four sub-zones representing two-digit bits. However, the two-digit bit indicated by the sub-zone of the J code group zone and the subcode of the K code group zone of Fig. 4 (b) differs from that of Fig. 4 (a). Unlike the J code group zone in FIG. 4 (a), the sub-zone of the J code group zone in FIG. 4 (b) shows 11 -> 10 -> 01 -> 00 from top to bottom. The sub-zone of the K-code group zone of FIG. 4 (b) shows 00 -> 10 -> 11 -> 01 from top to bottom.

Fig. 4 (c) shows a form in which the three code group zones are not vertical columns. Referring to FIG. 4 (c), each of the code group zones I, J, and K has four sub zones in order of 00-> 10 -> 10 -> 11.

4 (d) shows an example of one code group zone I on one screen. Each sub-zone in Figure 4 (d) represents a three digit bit. In FIG. 4 (d), information corresponding to a 3-digit bit can be transmitted on one screen.

4 corresponds to an example of a display screen. The number of code group zones and the shape of the code group zones may vary. Also, the number of sub-zones may vary depending on the total number of bits that each code group zone is intended to represent.

Here are some examples of color codes. 5 to 8 show an example using the code group zone and the sub zone as shown in Fig. 4 (a).

5 shows an example of outputting a color code on a display screen. 5 shows an example of outputting a specific color in the I code group region of FIG. 4 (a). However, Fig. 5 shows three-digit bits instead of two digits using an additional criterion of the order in which the sub-zones are output, unlike Fig. 4 (a). As described above, for convenience of explanation, the direction downward on the screen is referred to as a basic direction, and conversely, the direction upward from the bottom of the screen is referred to as a reverse direction. 5, the arrow indicates the output direction.

FIG. 5A shows an example in which a red color (R) is output to the sub-area located at the beginning of the four sub-areas in the basic direction. FIG. 5 (b) shows an example of outputting red (R) in two sub-regions of four sub-regions in the basic direction. FIG. 5 (c) shows an example in which red (R) is output in three sub-regions of four sub-regions in the basic direction. 5 (d) shows an example of outputting red (R) in all four sub-regions in the basic direction. In FIG. 5, the red color (R) is output in one embodiment, and blue (B) and green (G) other than the red color (R) are also output according to a rule defined by the user.

Figure 5 (a) shows each sub-zone in order of A, B, C and D. The transmitter can output a color in one sub-zone A. When the transmitter outputs colors in the color sub-regions B, C, and D, the color is continuously output starting from the sub-region A. As a result, colors are output in subregions A, A + B, A + B + C, and A + B + C + D as shown in Figs. 5 (a) to 5 (d). The area in which the colors are continuously output is called a continuous zone.

Each sub-zone represents a two-digit bit in the top-down direction, 00 -> 01 -> 10 -> 11. The code color can convey information with one bit added depending on the direction in which the color is output. There is a bit that each sub-region means, and the bit that the last sub-region of the continuous region means is called the reference bit (or reference binary number). For example, in FIG. 5A, the reference bit of A is 00, the reference bit of B is 01, the reference bit of C is 10, and the reference bit of D is 11.

For example, the basic direction may correspond to information that a bit of 0 is added to the front of the reference bit. And the reverse direction may correspond to information indicating that a 1 is added to the front of a reference bit. Of course, it may mean information that the basic direction adds 1 to the bit. Further, it may be added to the middle or the end of the reference bit according to the information of the output direction. Also, the bit to be added may be a plurality of bits rather than a single digit.

5 (a) to 5 (d) show examples in which 0 is added in front of the reference bit when color is output in the basic direction in the continuous area. 5 (a) shows "000", FIG. 5 (b) shows "001", FIG. 5 (c) shows "010" and FIG. 5 (d) shows "011". 5 (e) to 5 (h) show an example in which 1 is added to the front of the reference bit when color is output in the reverse direction in the continuous region. 5 (e) shows "111", FIG. 5 (f) shows "110", and FIG. 5 (g) shows "101". Finally, FIG. 5 (h) outputs blue (B) rather than red (R). If you output the same color, you can not distinguish the output direction if all sub-zones are output. When the red color (R) is output as blue (B) as shown in FIG. 5 (h), the colors to be output in FIGS. 6 and 7 may vary depending on conditions. 5 (h) shows "100" in the case where all the sub-zones are output in the reverse direction.

6 is another example of outputting a color code on the display screen. Fig. 6 is an example of outputting a specific color in the J code group region of Fig. 4 (a). FIG. 6 shows an example of expressing a certain bit in the same manner as FIG. 5 although the code group region is different. The transmitter outputs green (G) in the J code group area. In FIG. 6, green (G) is also output in one embodiment, and red (R) and blue (B) outputs other than green (G) are possible according to a rule defined by a user. 6 (a) to 6 (d) show an example in which 0 is added in front of a reference bit when color is output in a basic direction in a continuous zone. 6 (a) shows "000", FIG. 6 (b) shows "001", FIG. 6 (c) shows "010" and FIG. 6 (d) shows "011". FIGS. 6 (e) to 6 (h) show examples in which 1 is added to the front of the reference bit when color is output in the reverse direction in the continuous region. 6 (e) shows "111", FIG. 6 (f) shows "110", and FIG. 6 (g) shows "101". Finally, FIG. 6 (h) should output a color other than green (G) to distinguish the output direction. 6 (h) shows "100" in the case where all sub-zones are outputted in the reverse direction.

7 is another example of outputting a color code on the display screen. Fig. 7 is an example of outputting a specific color in the K code group region of Fig. 4 (a). The K subgroup shown in FIG. 7 can also represent a 3-digit bit as shown in FIG. 5 to FIG. However, the data represented in a computer device generally has the form of 8 bits, 16 bits, and so on. Therefore, in order to represent 8 bits on one screen, the code group area of FIG. 7 tries to represent two-digit bits. FIG. 7 shows an example in which blue (B) is output to the sub-zone.

In FIG. 7, the output of blue (B) is one embodiment, and red (R) and green (G) outputs other than blue (B) are possible according to a rule defined by a user. In FIG. 7, colors not output in FIGS. 5 and 6 are generally output. For example, if red (R) is outputted in FIG. 5 and blue (B) is outputted in FIG. 6, green (G) is outputted in FIG. In FIG. 7, two types of code regions are output according to the first bit starting in FIG. 5 in order to extract the vertex coordinates of the entire region. The type 1 of FIG. 7 illustrates the output when the first bit is " 0 " in FIG. 7 (a) shows all the code areas in the reverse direction to indicate "00", FIG. 7 (b) shows the reverse code area to the third code area to show "01" Quot; 10 "in the direction of the first code area, and outputs" 11 "in the first code area. In addition, type 2 of FIG. 7 illustrates an output when the first bit is "1 " in FIG. 7 (e) shows the first code zone in the basic direction to indicate "00", FIG. 7 (f) shows the second code zone in the basic direction to indicate "01" Quot; 10 ", and Fig. 7 (h) shows all code areas to indicate "11 ". Of course, two different bits may be represented by various methods different from FIG.

Further, various colors different from those used in Figs. 5 to 7 may be used. You can also use the color itself as a reference to a piece of information. For example, if a red color is output, a bit may be added to the front of the reference bit, and if blue is output, a bit may be added after the reference bit.

8 is an example of outputting a binary code corresponding to text data using the color code described in Figs. 5 to 7 on the display screen. FIG. 8 shows 8 bits (3 bits + 3 bits + 2 bits) by outputting colors in a code group area divided into three columns as described with reference to FIG. 5 to FIG. 8 shows an example of outputting text data "hello " in ASCII code.

Fig. 8A shows an example of outputting a start code for informing source data transmission to a display screen before transmitting a color code corresponding to source data. 8 (h) shows an example of outputting to the display screen an end code indicating that the data transmission is completed at the time when the transmission of the source data ends. The start code of Fig. 8 (a) and the end code of Fig. 8 (h) are only one example, and other colors or patterns may be used. The receiver recognizes the start code and the end code, respectively, and starts decoding and ends decoding.

8 (b) outputs a color code element indicating 011/010/00 in the code group area divided into three columns. This corresponds to the letter "h " in ASCII code. In the same way, Fig. 8 (c) outputs 011/001/01 corresponding to the letter "e", Fig. 8 (d) outputs 011/011/00 corresponding to the letter "l" e outputs 011/011/00 corresponding to the letter "l ", and Fig. 8 (f) outputs 011/011/11 corresponding to the letter" o ". 8 corresponds to an example in which one character is transmitted to one screen.

Fig. 8 (g) shows an example of outputting a screen for the afterimage removal in the middle of transferring the source data. Because the color code basically prints a specific color, a residual image of the previous frame may remain on the display screen. Therefore, it is preferable to output the afterimage removal screen as shown in FIG. 8 (g) at a constant time interval or at a constant frame interval. The afterimage clear screen can output black color.

9 shows an example of a frame structure in which a color code is transmitted. The color code preferably includes an area (start byte) indicating the start of the color code and an area (end byte) indicating the end of the color code. The start byte may correspond to the start code of Fig. 8 (a). The termination byte may correspond to the termination code of Figure 8 (h). In Fig. 9, the data byte represents an area representing the source data.

On the other hand, since the color pattern is transmitted through the liquid crystal light emission of the display device, due to the nature of the liquid crystal, a residual image of a pattern transmitted in the immediately preceding frame remains, which may become a problem of transmission accuracy. For example, if the text is transmitted, a start byte (Byte) is transmitted to transmit the text, and a problem can be solved by including the afterimage removing byte between the 8-bit character transmission and the next 8-bit transmission. The afterimage removal byte causes the display panel to display a black color. The afterimage removal byte corresponds to the afterimage removal screen described above.

As described with reference to FIG. 9, the transmitter can output a black frame for a predetermined period of time between frames transmitting source data. Or the transmitter may not output the image itself for a certain period of time at regular time intervals between the frames transmitting the source data.

10 is an example of a flowchart for a process 200 of a receiver extracting a color code.

The receiver acquires the image displayed on the display screen of the transmitter through the camera (210). The receiver basically normalizes the acquired image (220). Since the color image input to the camera is composed of RGB models, since the general RGB color model is sensitive to the variation of illumination and it is difficult to perform accurate calculation, the component values of r, g and b are divided by the sum of the three component values Normalization process is performed. The normalization process is shown in Equation 1 below.

YCbCgCr color model conversion and HSV color model conversion are performed on the normalized image, respectively.

First, the YCbCgCr color model conversion will be described. The receiver converts the normalized RGB color image into a YCbCgCr image (230). The process of extracting the chrominance information Cb, Cg and Cr for each of B (blue), G (green) and R (red) excluding the brightness component Y in the RGB color model is represented by the following Equations 2 to 4 same. In the following equations, R, G, and B represent the respective component values for r (red), g (green), and b (blue) in which the normalization process is performed in the RGB color model.

After that, the receiver extracts a region having a threshold value or more for each color channel. This process extracts a pixel (a color having a darker color) of a predetermined threshold value or more in the chrominance channels of Cb, Cg, and Cr. The threshold value setting can be changed in consideration of various environment variables such as system performance, image information of an object to be detected, illuminance at which an image is acquired, and the like. The threshold value may be a different value depending on the chrominance channel.

For example, in the blue color channel, the receiver extracts an object having a specific threshold value or more (241), performs binarization on the extracted image (242), and removes noise from the binarized image (243). In the green color channel, the receiver extracts an object having a specific threshold value or more (251), performs binarization on the extracted image (252), and removes noise from the binarized image (253). Finally, in the red color channel, the receiver extracts an object having a specific threshold value or more (261), performs binarization on the extracted image (162), and removes noise from the binarized image (163).

The receiver performs binarization (242, 252, and 262) on each chrominance channel in order to simplify the object having the corresponding color to improve object detection efficiency. The binarization process is shown in Equation (5) below.

"Region of Object" means an area where a specific object is located in the image.

Denotes a threshold value for binarization. That is, the color difference information extracted for each of Cg, Cg, and Cr

(Black) in a case where the number of pixels is equal to or larger than 255,

(White), the value of 0 is obtained.

After performing the binarization step, unnecessary objects

Or removing noise or the like from the input signal (243, 253, and 263). For example, the noisy can be removed through erosion by the following equation (6) and expansion by the following equation (7).

When the erosion operation assumes that A and B are a set of pixels, A

B can be defined as shown in Equation (6). The erosion operation is mainly used to fill a space such as a hole formed in the inside of the object or the background or to connect a short-cut region by reducing the protrusion inside the object and increasing the protrusion of the outside. In the binary image, But there is no change in the area where black and white pixels are present.

Assuming that A and B are a set of pixels,

B can be defined as Equation (7). here

Is the result of moving the morpheme B,

The result of erosion. In other words, a set that is obtained by finding the place where B is completely contained while moving B over A and gathering the points corresponding to the origin in each place can be defined as an expansion operation.

The HSV color model conversion process is now described. The receiver converts the normalized RGB color image HSV image (271).

The receiver extracts pixels in the Saturation (S) channel of the HSV image that are greater than or equal to the first threshold value and pixels less than or equal to the second threshold value in step 272 and extracts pixels in the Value (V) channel in step 273. The receiver combines the image extracted from the S channel and the image extracted from the V channel by an AND operation to generate a mask image (274).

The receiver combines the extracted image and the mask image for each channel in the YCbCgCr color model conversion process. The receiver detects the blue color by combining the image with the noise removed from the Cb channel and the mask image (281), detects the green color by combining the image with the noise removed from the Cg channel and the mask image (282) And the red color is detected (283). Finally, the receiver combines the B, G, and R colors (290). The receiver then decodes the data based on the final image.

11 is an example of a process of transmitting data using a sound wave signal. The transmitter 10 receives the source data (301). The source data may be input directly by the user through the interface device, or may be transmitted by the transmitter 10 via the network. The transmitter 10 converts the source data, which is text data, into an ASCII code (302). The transmitter 10 modulates the binary bit string (ASCII code) into a sound wave signal in the non-audible frequency band (303). The transmitter outputs the sound wave signal to the speaker (304).

The receiver 50 micro-acquires the sound signal output by the transmitter 10 (311). The receiver 50 extracts a desired frequency component from the acquired sound signal (312). The receiver 50 decodes the sound wave signal into a binary bit stream (ACII code) (313). The receiver 50 converts the binary bit stream into source data, which is text data (314). Finally, the receiver 50 can output the decoded source data to a screen or the like (315).

Hereinafter, the process by which the transmitter 10 converts the source data into the sound wave signal will be further described. The transmitter 10 converts the source data (text data assumption) to be transmitted into an ASCII code value (other code method is also acceptable). The transmitter 10 may transmit a series of binary bit streams to a sound wave signal of one frequency band or may transmit a sequence of binary bits in parallel using a plurality of frequency bands. Assuming that all K bits are transmitted, the transmitter 10 converts the binary bit string m [k] (k = 0,1,2, ..., K-1) Lt; / RTI >

In Equation (8), f _cn represents a frequency band. The frequency band can be extended from 1 channel to n channels (f _c1 to f _cn ). The transmitter 10 can send data to one channel or a plurality of channels.

The transmitter 10 basically transmits data using the sound wave signal in the non-audible frequency band. The transmitter 10 can output a sound (music, etc.) of an audio frequency band together with the sound wave signal to hide the data to be transmitted. In this case, if the receiver 50 does not know the promised frequency band in advance, it can not decode the data.

The process of extracting data by the receiver 50 is as follows. The receiver 50 can detect the components of the frequency band adopted by the user using the Goertzel algorithm to extract only a desired frequency component from the sound output from the speaker. On the other hand, various methods of detecting a desired frequency component in a sound wave signal may be used. In this case, the receiver 50 can detect the frequency component using a window smaller than the length of 1 bit indicated by the sound wave signal output from the transmitter 10. For example, if the length of one bit is 882 samples, the receiver 50 can set the window size to 441 samples. The receiver 50 can detect the binary bit string more accurately by collecting the information displayed in the various windows and comparing them.

On the other hand, a constant header can be used in a sound wave signal. The header may include information that the sound wave signal is for data transmission and that it is loading data from a certain position. In this case, when the header is detected from the sound wave signal, the receiver 50 converts the continuous sound wave signal into 0 and 1 based on whether or not the sound wave signal exceeds a predetermined threshold value. When the extraction of the binary bit stream is completed, the receiver 50 decodes it into constant data. For example, in the case of text data, the receiver 50 can decode text data using an ASCII code table.

12 is an example of a flowchart of a data transmission method 400 using light emission and sound waves. FIG. 12 shows a process of transmitting data using a color code and a process of using data using a sound signal. In some cases, the transmitter may transmit data using only the color code, or may transmit data using only the sound wave signal. Further, the transmitter may transmit the data using the color code and the sound wave signal at the same time. When the transmitter uses the color code and the sound wave signal at the same time, they may transmit different data or transmit the same data, respectively.

The transmitter converts the source data into a binary bit stream (410). 11, an example in which data is transmitted using a color code, and a flow in the right is an example in which data is transmitted using a sound signal.

Describe the process using the color code. The transmitter converts the binary bit string into a constant color code (421). The transmitter outputs the converted color code to the display device according to a certain criterion (422). The receiver acquires an image including the display device using the camera (423). The receiver determines an area in which the color code is output from the acquired image, and then extracts a series of color codes (431). The receiver converts the color code into a series of binary bit streams (432). Thereafter, the receiver converts (decodes) the binary bit string into the source data (433).

The process using the sound wave signal will be described. The transmitter converts the binary string of bits into an acoustic wave signal (441). The transmitter outputs the sound wave signal to the speaker (442). The receiver micro-acquires the sound wave signal (451). The receiver converts the sonic signal to a binary bit stream (452). Finally, the receiver converts (decodes) the binary bit string into the source data (453).

13 is another example of a flowchart for the data transmission method 500 using light emission and sound waves. In FIG. 13, the transmitter selects a method of transmitting data according to a noisy environment.

The transmitter determines the noise environment (510). 14 is an example of a table in which a transmitter determines a noisy environment. 14 (a) is an example of a table for judging image noise. Referring to FIG. 14 (a), two types of noise are considered. One is light noise and the other is motion noise. The illumination noise is determined by the illuminance around the transmitter. If the ambient light is too bright or too dark, the specific color that the transmitter outputs can be recognized as a different color at the receiver. The illumination noise can be measured by the transmitter using the illuminance sensor.

The noise caused by the illumination can be determined by determining the illumination chromaticity from the input image, thereby determining whether the noise is due to the threshold value for illumination. If the threshold is greater than the threshold value, the image is damaged due to the influence of illumination. There is no image damage due to illumination. The method of estimating illumination includes illumination chromaticity estimation using cognitive light source and highlight, illumination estimation method using RGB three-dimensional color space, and illumination estimation method using white region, and the algorithm can be changed in consideration of various environments Do.

Motion noise is assumed to be a situation where the transmitter (and possibly the receiver) is constantly moving or vibrating. That is, when the transmitter (or the receiver) moves constantly, the color code output on the screen may not be properly transmitted. Motion noise can be determined using the motion sensor of the transmitter.

The shaking noise caused by the motion can determine whether the noise is due to the threshold value of the shaking by estimating the motion of the input image. If the noise is greater than the threshold value, the shaking noise is relatively large. If the threshold is smaller than the threshold value, . Motion estimation algorithms that determine the accuracy of noise determination for shaking include Block Matching Algorithm (BMA), Edge Detection Algorithm (EDA), Bit Plane Matching (BPM), and Projection Algorithm (PA).

Further, when the transmitter and the receiver are capable of bidirectional communication, a test color code may be transmitted to judge the degree of image noise by whether or not the transmission is correctly performed.

The transmitter determines that there is image noise when the value of the illumination noise exceeds the threshold value alpha and determines that there is image noise when the value of the motion noise exceeds the threshold value beta.

14 (b) is an example of a table for a criterion for determining acoustic noise. The transmitter can determine acoustic noise based on SNR (Signal to Noise Ratio). The transmitter determines that there is acoustic noise if the SNR value exceeds?. The transmitter can measure acoustic noise by collecting sound waves around it using a microphone.

14 (c) is an example of a table dividing the noise environment into four types according to the degree of image noise and the degree of acoustic noise. Case 1 has no image noise or acoustic noise. Case 2 has only acoustic noise. Case 3 shows only image noise. Case 4 has both image noise and acoustic noise.

The transmitter determines the noise environment as shown in FIG. 13 (c) by using the same reference as the table of FIG. 14 (a) and FIG. 14 (b) (510). The transmitter determines a data transmission mode according to the noisy environment (520). The transmitter converts the source data into a binary bit stream (530).

15 is an example of a method by which a transmitter transmits data in consideration of a noisy environment. The method used for data transmission depends on the noise environment. In Case 1 of Fig. 14 (c) (no image noise and no acoustic noise), the transmitter transmits the binary bit string using both the color code and the sound wave signal. However, in order to increase the transmission efficiency, the transmitter can be divided into a portion (first segment) for transmitting a binary bit string in a color code and a portion (second segment) for transmitting a sound signal. The transmitter transmits different data (first and second segments, respectively) using both the color code and the sound wave signal (541). The receiver decodes the color code and the sound wave signal to generate a binary bit string, and combines the two binary bit strings with a certain reference to generate a final binary bit string (542). The receiver converts the last binary bit stream into source data (542). When the transmitter transmits different parts of the source data as color code and sound wave signals, the receiver must know which parts to transmit in each color code and sound wave signal. 15 (a) is an example of a case corresponding to Case 1. And the source data has four units (DATA 1 to DATA 4). The transmitter can transmit DATA 1 and DATA 2 as a color code and DATA 3 and DATA 4 as sound wave signals.

In Case 2 of FIG. 14C (only acoustic noise is present), the transmitter can transmit data using only the color code (551). The receiver extracts the binary bit string from the color code and converts the data therefrom (552). 15 (b) shows an example in which the entire source data is transferred using a color code.

In Case 3 of FIG. 14C (only image noise is present), the transmitter can transmit data using only the sound wave signal (561). The receiver extracts the binary bit stream from the sonic signal and converts the data therefrom (562). 15 (c) shows an example in which the entire source data is transmitted using the sound wave signal.

In Case 4 of FIG. 14 (c) (both image noise and acoustic noise), the transmitter waits for a predetermined time without transmitting data, and can judge the noise environment again. In some cases, the transmitter may transmit data using both the color code and the sound signal (571). In this case, the transmitter can transmit the entire source data using the color code and the sound wave signal, respectively. The receiver extracts the binary bit stream from the color code and the sonic signal and converts the data (572). At this time, the receiver can compare the binary bit string (first bit string) extracted from the color code with the binary bit string (second bit string) extracted from the sound wave signal to determine whether there is an error portion. The part where the error occurs can be restored to the maximum using the first bit string and the second bit string. 15 (d) shows an example in which the entire source data is transmitted using color codes and sound wave signals, respectively.

It should be noted that the present embodiment and the drawings attached hereto are only a part of the technical idea included in the above-described technology, and those skilled in the art will readily understand the technical ideas included in the above- It is to be understood that both variations and specific embodiments which can be deduced are included in the scope of the above-mentioned technical scope.

10: Transmitter
50: receiver

Claims

The transmitter converting the source data into a source binary bit stream that is a binary bit string;
Converting the source binary bit string into a corresponding color code and outputting the color code to a display device;
The receiver using the camera to obtain the color code output to the display device; And
And the receiver decodes the source binary bit stream from the color code to detect the source data,
The outputting of the color code may include dividing the source binary bit stream into L binary bits, dividing the display device into M * N regions, allocating P binary bits to each region, A binary bit of 0 or 1 in accordance with the light emission order of the P binary bits constitutes the L binary bits when the binary bit added before or after the P binary bits constitutes the L binary bits, In a predetermined color sequentially,
Wherein the step of detecting the source data comprises decoding the binary code string according to whether or not each region of the display device emits light and a light emission order from the obtained color code.

delete

The method according to claim 1,
And converting the source binary bit string into a sound wave signal and outputting the converted sound wave string to the transmitter. The method according to claim 1, further comprising: .

The method according to claim 6,
Converting the sound wave signal into the sound wave signal,
And converting the binary bits constituting the source binary bit string into the wavelengths of the non-audible frequency band representing 0 or 1, respectively, and outputting the sound wave signals.

The method according to claim 6,
Further comprising the step of acquiring the micro-acoustic wave signal of the receiver after obtaining the color code.

9. The method of claim 8,
Wherein the step of detecting the source data comprises:
Wherein the decoding unit decodes the source binary bit stream from the sound wave signal to detect the source data and then compares the source binary data with the source data detected from the color code to determine final source data. .

The method according to claim 1,
Wherein the step of outputting the color code outputs the color code to the display device so that the transmitter outputs a screen having a black color for every binary bit string of a predetermined size.

The transmitter determining the degree of image noise and the degree of acoustic noise;
Determining at least one of a color code or an acoustic signal as a medium used by the transmitter to transmit the source data according to the degree of the image noise and the degree of the acoustic noise;
If the degree of the image noise and the degree of the acoustic noise satisfy a first condition, the transmitter divides a binary bit string corresponding to the source data into a first segment and a second segment, Outputting the color code to a display device, converting the second segment to a sound signal, and outputting the sound signal to a speaker;
Converting the binary bit string corresponding to the source data into the color code and outputting the color code to the display device when the degree of the image noise and the degree of the acoustic noise satisfy the second condition ;
When the degree of the image noise and the degree of the acoustic noise satisfy the third condition, the transmitter converts the binary bit string corresponding to the source data into the sound wave signal, and outputs the sound wave signal through the speaker step;
When the degree of the image noise and the degree of the acoustic noise satisfy a fourth condition, the transmitter converts the binary bit string corresponding to the source data into the color code and the sound wave signal, respectively, Respectively;
Obtaining at least one of a color code output from the receiver and a sound signal output to the speaker; And
Decoding the color code obtained when the receiver obtains the color code, and decoding the sound signal to detect the source data when the sound signal is obtained,
The outputting of the color code may include dividing the source binary bit stream into L binary bits, dividing the display device into M * N regions, allocating P binary bits to each region, A binary bit of 0 or 1 in accordance with the light emission order of the P binary bits constitutes the L binary bits when the binary bits added before or after the P binary bits constitute the L binary bits, In a predetermined color sequentially,
Wherein the step of detecting the source data comprises decoding the binary code string according to whether or not each region of the display device emits light and a light emission order from the obtained color code.

12. The method of claim 11,
Wherein the step of detecting the source data comprises:
Wherein when the first condition is satisfied, the source data corresponding to the first segment and the source data corresponding to the second segment are combined to detect one source data. .

12. The method of claim 11,
Wherein the step of detecting the source data comprises:
Wherein when the fourth condition is satisfied, the final source data is determined by comparing the source data detected by decoding the source binary bit stream from the sound wave signal with the source data detected from the color code And a method of transmitting data using a sound wave.

delete