KR20110086680A - Sensor using a smartphone that supports the plurality of n-screen-based multi-user implementation of the controller - Google Patents

Abstract

PURPOSE: A method for realizing a multiple user controller based on N- screen using a plurality of smart phone supporting a sensor is provided to realize a multiple user controller supporting an exact control by using a plurality of smart phone in an N-screen device. CONSTITUTION: A protocol generating module(1130) generates a protocol through a sensing data sampling and an event sensing in a client module. A communication protocol includes inherent identifier of a smart phone. The identifier of the smart phone includes element of inherent identification of the smart phone terminal corresponding as a terminal producing inherent number, a telephone number, a subscriber name.

Description

Sensor using a smartphone that supports the multiple of N-screen-based multi-user implementation of the controller}

The present invention controls contents implemented in an N-screen device by using a wireless communication technology with a plurality of smartphones including a geomagnetic sensor, an acceleration sensor, a gravity sensor and an N-screen device including a PC, a smart TV, and an IP set-top box. It relates to a method for implementing a multi-user controller to perform the.

[Document 1] Kyu-Sung Ahn and 3 others, Mouse Design using Acceleration Sensor of Smartphone, Journal of Korea Multimedia Society, Autumn Conference, 2010.

[Document 2] Choi Ji Hoon and 4 others, PC Game System using Smartphone Sensor, Korean Multimedia Society, Fall Conference 2010, 2010.

The conventional controller implementation method using a smartphone is a method of implementing a single user controller based on a PC as a single user controller, a mouse implementation method using a touch screen of a smartphone and a smartphone as shown in [Documents 1 and 2]. It is being developed as a mouse implementation using the acceleration sensor.

The above method is a method of implementing a single user controller using a single smartphone, and a method of implementing a simple direction change according to the variation of the acceleration sensor based on the current pointer position of the N-screen device using the acceleration sensor. It is not possible to precisely control the angle, direction and rotation of the user. The problem is that only a single user can be controlled by a single smartphone.

The present invention has been devised to achieve the above object, and an object thereof is to implement a multi-user controller that supports accurate control using a plurality of smartphones in an N-screen device.

In particular, an object of the present invention is to realize an accurate multi-user control of the angle, direction, and rotation of the user using the acceleration sensor, geomagnetic sensor, gravity sensor of the smartphone.

The main idea of the present invention for achieving the above object is to implement a multi-user controller using a plurality of smartphones including a unique identifier of the smartphone in the communication protocol header portion between the smartphone and the N-screen device.

In addition, the deeper idea of the present invention is the direction of the initial smartphone and the user using the gravity sensor and geomagnetic sensor of the smart phone for precise control of the angle, direction, rotation of the user when implementing the multi-user controller Accurate control is realized by setting the information, determining the direction and angle of the smartphone with the gravity sensor, detecting the user's rotation through the geomagnetic sensor, and detecting the user's movement through the variance of the acceleration sensor.

Communication between the smartphone and the N-screen device of the present invention for achieving the above object is a protocol consisting of a header portion and a data portion; And a header unit including unique identifiers of the smart phone for supporting multi-users and additional information such as date and time. And a data unit including gravity sensor data, geomagnetic sensor data, acceleration sensor data, and touch screen data to implement accurate control. Characterized in that it comprises a.

Gravity sensor data of the present invention for achieving the above object performs sampling on the basis of ms (milliseconds), characterized in that the X-axis, Y-axis, Z-axis three-axis information according to the angle and direction of the smartphone In this case, the first to third elements of the protocol data portion are allocated.

Geomagnetic sensor data of the present invention for achieving the above object performs sampling on the basis of ms (milliseconds), characterized in that the information on the X-axis, Y-axis, Z-axis three-axis according to the angle and direction of the smartphone And the fourth to sixth elements of the protocol data portion.

Acceleration sensor data of the present invention for achieving the above object performs sampling on the basis of ms (milliseconds), characterized in that the X-axis, Y-axis, Z-axis three-axis information according to the angle and direction of the smartphone In this case, the seventh to ninth elements of the protocol data portion are allocated.

The touch screen data of the present invention for achieving the above object is extracted when a touch event occurs in the touch screen, touch down X coordinate, Y coordinate, touch move X coordinate, Y coordinate, touch-up X coordinate, Y coordinate, touch cancel X coordinate It is characterized by the Y coordinate, and is assigned to the tenth to seventeenth elements of the protocol data portion.

In order to achieve communication between the smart phone and the N-screen device of the present invention, a client module of the smart phone and a server module of the N-screen device are characterized in that they are configured.

Client module of the present invention for achieving the above object is a gravity sensor, geomagnetic sensor, acceleration sensor, event detection function for detecting the event of the touch screen; And a sampling function of sampling data of each sensor at a predetermined time (ms) when an event is detected. A protocol generation function for collecting a unique identifier of a smartphone and assigning it to a header part and assigning data collected from each sensor to a data part; A transmission function for transmitting the combined protocol to the server; Characterized in that it comprises a.

The transmission function of the present invention for achieving the above object is characterized by including Bluetooth and wireless Internet.

The server module of the present invention for achieving the above object comprises a session management function for processing the connection request of the client and manages the connection state; A receiving function for receiving a protocol from a client; A protocol analysis function for dividing the received protocol into a header portion and a data portion and analyzing the protocol; Obtaining and storing the unique identifier of the smartphone extracted from the header unit; Obtaining and storing three-axis data of the gravity sensor extracted from the data unit; Obtaining and storing three-axis data of the geomagnetic sensor extracted from the data unit; Obtaining and storing three-axis data of the acceleration sensor extracted from the data unit; Obtaining and storing data of the touch screen extracted from the data unit; Relationship setting function and relationship information storage portion of the unique identifier of the smartphone and the unique identifier of the object in the content; A coordinate transformation function for controlling movement of an object inside the content by combining three-axis data received from each sensor; A touch processing function for controlling a response of the acquired data on the touch screen and an object inside the content; Characterized in that it comprises a.

The data compression algorithm, encryption, and decryption algorithm may be selectively applied to the transmission function of the client module and the reception function of the server module according to the computational performance of the N-screen device and the efficiency of the transmission bandwidth of the communication medium.

In addition, the three-axis data value of each sensor of the smart phone can be extended to four axes, six axes according to the hardware specification of the sensor, and accordingly, the data element of the protocol data section is also extended to four or six elements Can be implemented.

In addition, according to the type of the touch screen of the smart phone may support a single touch, sliding, multi-touch, according to the data portion of the protocol can be extended.

The present invention enables the implementation of a multi-user controller such as a user's movement, rotation, and strike through communication with an N-screen device using three-axis data values of sensors of each of a plurality of smartphones.

Accordingly, the present invention has the effect of realizing a multi-user implementation in the implementation of a single-user controller using a smartphone in the N-screen device to control the multi-user content, such as multi-player games.

The sensor data of the data part defined in the communication protocol proposed in the present invention enables precise control such as rotation, movement, and hitting of the user in the N-screen device, and can be applied to various areas such as movement and navigation between contents as well as games. Can be applied.

1 is a smart phone client module
Figure 2 N-screen device server module
3 communication protocol
[Figure 4] Relationship Information Matrix
[Figure 5] Smartphone and N-screen based interworking structure diagram

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, embodiments of the present invention may be modified in many different forms, and the scope of the present invention should not be construed as being limited by the following embodiments. Embodiments of the present invention are provided to more fully explain the present invention to those skilled in the art. Therefore, the shape and the like of the elements shown in the drawings means simplified to emphasize more clear description.

1 illustrates a client module installed in a smart phone in which the main idea of the present invention is implemented. The illustrated client module 1100 is installed and executed in a smart phone to perform event detection 1110 of various sensors and touch screens on the smart phone to sample 1120 data of each sensor, and to the communication protocol shown in FIG. 3. The header unit and the data unit are generated 1130 according to the bandwidth and state of the network, and the server module of the N-screen device through the compression module 1140 and the encryption module 1150 according to the computing capability of the N-screen device. Send 1160.

The event detection 1110 module detects that an event occurs in a gravity sensor, a geomagnetic sensor, an acceleration sensor, and a touch screen, and mainly events of touchdown, touchmove, touchup, and touch cancel of the touch screen and corresponding X coordinates, It extracts the Y coordinate.

The sensor data sampling 1120 module mainly serves to provide information for the protocol generation 1130 module to process data by sampling three-axis data information of a gravity sensor, a geomagnetic sensor, and an acceleration sensor.

The protocol generation 1130 module configures various sensor information extracted from the event detection 1110 module and the sensor data sampling 1120 module according to the communication protocol shown in FIG. 3.

The communication protocol 1300 of FIG. 3 includes a header unit 1310 and a data unit 1320, and combines data of sensors extracted from an event detection 1110 module and a sensor data sampling 1120 module. It serves to combine the data to be sent to the server module 1200 in accordance with the protocol.

The header portion 1310 of the communication protocol 1300 is composed of a unique identifier entry of the smart phone, an execution date and time, a processing task classification, and a terminal unique code entry. The smartphone unique identifier entry is for supporting multiple users by recognizing a plurality of smartphones in the server module 1200. The sensor values are changed to individually select movement of a plurality of objects in the content executed in the N-screen device. It is used as a separator to adjust. In addition, the processing work classification entry contains the work classification code for data compression, compression method, encryption, encryption and method, so that the server module 1200 decompresses and decrypts the data through the work classification entry.

The data unit 1320 of the communication protocol 1300 includes three-axis data of the extracted gravity sensor, three-axis data of the geomagnetic sensor, three-axis data of the acceleration sensor, touch-down X, Y coordinate of the touch screen, touch move X, Y coordinates, touch-up X, Y coordinates, touch cancel X, Y coordinates, and the data is changed according to the movement of the smartphone and the user's touch screen touch.

The compression module 1140 of the client module 1100 is a module that performs a function of compressing information of the data unit 1320 of the communication protocol 1300, and may selectively perform compression. By selecting one of Run-Length, Lempel-Ziv, and Huffman Encoding methods, which is a compression algorithm, a method of reducing the data transmission capacity is realized.

The encryption module 1150 of the client module 1100 compresses both the header portion and the data portion of the communication protocol 1300 to realize a function of improving the security of information transmitted from the client module to the server module. It can be selectively performed according to the computing power and security risk of the screen device.

The transmission module 1160 of the client module 1100 transmits data packets generated through the protocol generation module 1130, the compression module 1140, and the encryption module 1150 to the server module 1200. According to the hardware support specification of the N-screen device, which is a server, a method using Bluetooth 1010 networking or a method using Wi-Fi 1020 networking is selected to transmit the data packet to the server module 1200.

The transmission 1160 module can accommodate both TCP and UDP in the case of Wi-Fi 1020 networking, and can connect by specifying an IP address and a PORT number according to the setting of the server.

2 illustrates a server module installed in an N-screen device in which the main idea of the present invention is implemented. The illustrated server module 1200 is installed and executed in the N-screen device to process the connection request session from the client module 1100 of the smartphone (1210), and receives the data packet protocol transmitted for each session (1220) In the case of the encrypted packet, decryption 1230 is performed, and in the case of the compressed packet, the compression module 1240 is used to recover the data packet and then analyze the protocol 1250 to analyze the protocol header part 1310 of the requested session. The unique identifier extracted from the smartphone is stored in the unique identifier store 1251, the 3-axis data of each sensor extracted from the protocol data unit 1320 is stored in the sensor data store 1252, the protocol data unit The touchdown X, Y coordinates, Touch Move X, Y coordinates, Touch Up X, Y coordinates, Touch Cancel X, Y coordinates of the touch screen extracted from (1320) are stored in the touch screen data store 1253 Through Identifier Relationship Management (1260) module And store in the unique identifier relationship information matrix 1261 and 1400 of FIG. In this way, the coordinate processing and the touch processing module 1270 are based on the data stored in the unique identifier store 1251, the sensor data store 1252, and the touch screen data store 1253. Plays a role in controlling the movement of an object inside the content executed on the N-screen device.

The session management 1210 module is a module that manages the connection of each smartphone when a connection is requested from the client module 1100 through Bluetooth 1010 networking or Wi-Fi 1020 networking. It performs core resource management that implements the supporting multi-user controller. The session management module 1210 manages the Bluetooth connection request pool and the Wi-Fi connection request pool, and when a connection request is received from the client module 1100, resources are allocated to the Bluetooth connection request pool and the Wi-Fi connection request pool according to the connection request method. Assign and manage connections for that session. Through this, the data packet transmitted from each session is received, thereby realizing session-specific protocol analysis 1250 and processing 1270.

The protocol receiving module 1220 is a module for receiving data packets transmitted from the client module 1100. The protocol receiving module 1220 receives data packets received from resources allocated by the session management module 1210 and assigns the data packets to temporary storage. The protocol analysis 1250 may be performed through the decoding module 1230 and the compression module 1240.

The decryption module 1230 is a module for decrypting a corresponding content when a data packet received from the client module 1100 is encrypted, and is used when security is required during communication between the client module 1100 and the server module 120. do.

The compression module 1240 is a header when the contents of the communication protocol 1300 data unit 1320 of the data packet received from the client module 1100 are compressed according to the network bandwidth and the computing capability of the N-screen device as a server. A module that restores the contents of the data unit 1320 by checking the compression and compression method derived from the unit 1310. Run-Length, Lempel-Ziv, and Huffman Encoding techniques, which are 'lossless compression' algorithms, are restored. Can optionally be used.

The protocol analysis 1250 module is a module that analyzes the header part 1310 and the data part 1320 of the communication protocol 1300, which has been decoded and restored, and stores them in a storage for processing. Extract the unique identifier from the smartphone unique identifier entry of 1310) and store the unique identifier in the unique identifier store 1251, three-axis data of the gravity sensor, three-axis data of the geomagnetic sensor from the data unit 1320, The 3-axis data of the acceleration sensor is stored in the sensor data storage 1252, and touch-down X, Y-coordinates, touch-move X, Y-coordinates, touch-up X, Y-coordinates, touch-cancellation X, Y-coordinates of the touch screen The data storage 1253 stores data in the coordinate processing and touch processing module 1270 to provide data for performing movements and activities of objects inside the content executed in the N-screen device.

The unique identifier store 1251 stores the smartphone unique identifier received from the client module 1100 based on the session ID of the resource allocated from the session management 1210 module.

The sensor data store 1252 is three-axis data of the gravity sensor received from the client module 1100, three-axis data of the geomagnetic sensor, the acceleration sensor based on the session ID of the resource allocated from the session management 1210 module 3-axis data is stored.

The touch screen data store 1253 is a touchdown X, Y coordinate, touch move X, Y of the touch screen received from the client module 1100 based on the session ID of the resource allocated from the session management 1210 module Coordinates, touch-up X, Y coordinates, touch cancel X, Y coordinates are stored.

The unique identifier relationship management module 1260 acquires a unique identifier of the smartphone present in the unique identifier store 1251 based on the session ID and executes the content in the N-screen device to be controlled by the smartphone of the session. The unique identifier of the object is registered in the entry of the relationship information matrix 1400 to set the relationship between the smartphone and the object in the content to be controlled by the smartphone of the session. Through this, the smartphone corresponding to each session can be realized to control individual objects in the content, thereby enabling the final implementation of a multi-user controller with multiple smartphones.

The coordinate processing and touch processing module 1270 is a module for realizing the movements and activities of the related individual smartphones and the objects in the content through the relationship information matrix 1400 generated through the unique identifier relationship management module 1260. By varying the amount of data generated in each session to the properties and methods of the unique identifier of the object in the content based on the smartphone unique identifier of the session, you can move the objects in the content running on the N-screen device individually. In this way, a multi-user controller is realized by using a plurality of smartphones in one N-screen device.

Although the present invention has been described in detail with reference to the above preferred embodiments, the present invention is not limited to the above embodiments, and various modifications may be made by those skilled in the art within the scope of the technical idea of the present invention. Do.

1010 Communication protocol transmission via Bluetooth networking
Communication protocol transmission via 1020 Wi-Fi networking
1100 client module: client module application that operates on a smartphone
1110 Event Detection: Event Detection of Touch Screen and Sensor in Client Module
1120 Sensor data sampling: event detection and sensor data sampling of various sensors in the client module
1130 Protocol generation: A function that combines the unique identifier of the smartphone and data collected from each sensor to meet the protocol definition.
1140 Compression module: Compression module that compresses and transmits communication protocol when N-screen device has good computing power and network bandwidth is not enough. Run-Length, Lempel-Ziv, Huffman Encoding Can be implemented by introducing a technique
1150 encryption module: module to be applied when the security of the information sent from the smartphone terminal optionally
1160 Transmission: Sends the created communication protocol to the server module. It executes the transmission to the server module through Bluetooth and Wi-Fi networking.
1200 Server Module: Server module application installed in N-screen device
1210 session management: perform connection requests and connection management of multiple smartphones
1220 Receive protocol: Receive protocol received from client module
1230 decryption module: a module that performs decryption when the protocol received from the client module is encrypted
1240 compression module: module that restores communication protocol according to compression algorithm when protocol received from client module is compressed
1250 Protocol analysis: Acquires unique identifier of smartphone in header, extracts sensor data and touch screen data and stores in server side storage
1251 Unique Identifier Storage: Server-side storage that stores the unique identifier of the smartphone received from the client module.
1252 Sensor data storage: Server-side storage that stores information of multiple sensor data
1253 Touch Screen Data Storage: A server-side storage that stores touch screen data.
1260 Unique identifier relationship management: Manages the relationship so that individual smartphones can be involved in the movement of a specific object by managing the relationship between requests from multiple smartphones and the unique identifier of the object inside the content.
1261 Unique identifier relationship matrix: A repository that sets the relationship between the unique identifier of the Smart For and an object inside the content running on the N-screen device.
1270 Coordinate processing / touch processing: A module that executes the movement of an object inside the content defined through unique identifier relationship management for the movement requested from the client module.
1300 Communication Protocol: Communication Protocol between Client Module and Server Module
1310 Header part: The header part of the communication protocol optionally includes a smartphone unique identifier, date, time, processing task classification (compression, compression, encryption, encryption) and terminal unique code as necessary.
1320 Data part: The data part of the communication protocol arranges the data of various sensors of the smartphone and the touch screen in order and transmits the corresponding data.
1400 Relationship Information Matrix Details: Repository structure that manages the relationship information between the unique identifier of a smartphone and the unique identifier of an internal object running on an N-screen device.

Claims (9)

A protocol generating module for generating a protocol through event detection and sensor data sampling in a client module, the communication protocol comprising a unique identifier of a smartphone; The unique identifier of the smartphone includes elements capable of unique identification of the corresponding smartphone terminal, such as a terminal production unique number, telephone number, and subscriber name. A module for compressing a communication protocol through a compression module according to a selection; The compression module includes Run-Length, Lempel-Ziv, and Huffman Encoding, which are 'lossless compression' algorithms, and other functions such as 'lossy compression' algorithms to reduce the capacity of data packets.  The encryption module, characterized in that for encrypting the data packet including the smartphone unique identifier according to the communication protocol through the encryption algorithm through the DES, AES encryption algorithm, derivative algorithm, UTF encoding. Session management module of the server module, characterized in that for managing the connection request of the client module to allocate the resources for the connection requesting the Bluetooth networking and Wi-Fi networking, and to manage the session unit to support multiple access. Compression characterized by restoring the data packet through other types of algorithms such as Run-Length, Lempel-Ziv, Huffman Encoding and 'Loss Compression' algorithms, which are 'lossless compression' algorithms, depending on the communication protocol selected by the server module. module. Decryption module characterized in that the server module decrypts the encrypted data packet of the communication protocol including the smartphone unique identifier through a decryption algorithm through DES, AES decryption algorithm and derivative algorithm, UTF Encoding. Unique identifier storage, characterized in that for storing the smartphone unique identifier of the communication protocol received from the client module in the memory storage on the server side for use in the server module. The relationship information matrix, characterized in that the server module associated with the unique identifier of the received smartphone and the unique identifier of the object in the N-screen content to be controlled. 10. The method of claim 8, wherein the relationship information matrix is a session allocated by requesting from the smartphone to the session management module of the server module in addition to the method of associating the smartphone unique identifier with the unique identifier of the object in the N-screen content to be controlled. A relationship information matrix characterized by relating an ID to a unique identifier of an object in an N-screen content.

Images