KR20090001411A - Haptic data transferring system for networked collaboration and method thereof - Google Patents
Haptic data transferring system for networked collaboration and method thereof Download PDFInfo
- Publication number
- KR20090001411A KR20090001411A KR1020070034025A KR20070034025A KR20090001411A KR 20090001411 A KR20090001411 A KR 20090001411A KR 1020070034025 A KR1020070034025 A KR 1020070034025A KR 20070034025 A KR20070034025 A KR 20070034025A KR 20090001411 A KR20090001411 A KR 20090001411A
- Authority
- KR
- South Korea
- Prior art keywords
- haptic
- packet
- data transmission
- coordinate
- coordinates
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention relates to a haptic data transmission system for network collaboration and a method thereof.
The haptic data transmission system of the present invention comprises: a management device for generating cube coordinates corresponding to the haptic pointer coordinates based on a preset rendering rate, and calculating a buffer time based on delay times received from a plurality of transmission devices; Receiving the haptic pointer coordinates from the haptic device, generating a packetized first packet, and transmitting the packetized first packet to a management device, obtaining a delay time corresponding to receiving a second packet including cube coordinates, and transmitting the received packet to the management device; Receiving a buffer time corresponding to the transmission device for performing a haptic rendering of the second packet based on the buffer time.
According to the present invention, by predicting the lost data, it is possible to expect an effect that can effectively improve the error due to jitter or loss that can occur when transmitting and receiving the pointer coordinate packet. In addition, it is possible to expect an effect that can effectively synchronize the synchronization of the client by solving the asynchronousness to the delay.
Description
1 is a block diagram illustrating a haptic data transmission system according to an embodiment of the present invention.
2 is a block diagram illustrating a packet structure for transmitting and receiving between a haptic data management device and a haptic data transmission device according to an embodiment of the present invention.
3 is a data flow diagram illustrating a haptic data transmission method of a haptic data transmission system according to an embodiment of the present invention.
The present invention relates to a haptic data transmission system and method for network collaboration, and more particularly, to a haptic data transmission system and method for reducing losses due to the network.
Recently, in the virtual environment, the use of tactile data as well as visual and audio data through audio and video is increasing. The environment in which online users play games or perform the same tasks using these haptic devices (also referred to as "haptic devices") is called the Haptic Collaborative Virtual Environment (hereafter referred to as "HCVE"). .
Since human touch is more sensitive than sight and hearing, when tactile data is transmitted, it is quite sensitive to network conditions. In particular, tactile data used in online games or media is more sensitive to network conditions.
In a collaborative virtual environment (hereinafter referred to as "CVE"), haptic data may refer to virtual objects shared by all clients or may be 3D coordinates pointed at by the haptic device. The transmission of these coordinates is similar to the existing network game or video and audio transmission method.
In the case of a client sending conventional coordinates, it transmits approximately 60 update packets per second in accordance with the graphics rendering cycle.
However, in order to transmit haptic data, the haptic data must be transmitted in accordance with the haptic rendering period of 1 kHz. Therefore, haptic data transmission is very sensitive to the actual network environment, and thus collaboration cannot be properly implemented due to delay, loss, and jitter in the collaboration environment.
Currently, various researches are being conducted for the transmission of haptic data. Representative studies include improving the virtual-time rendering (VTR) algorithms used for video and audio transmissions to perform haptic group synchronization, and using Queue Monitoring (QM). There is research to increase efficiency.
However, these existing studies do not provide an integrated solution first, and there is a problem in that synchronization between clients is not achieved due to delay due to network conditions.
Accordingly, an object of the present invention is to provide a haptic data transmission system and method using a new synchronization technique for reducing losses due to a network.
According to an aspect of the present invention for solving the above technical problem, a method for transmitting haptic data to a management device for managing a client,
Receiving haptic pointer coordinates from the haptic device; Transmitting a haptic pointer coordinate packet to which the haptic pointer coordinates are packetized in a preset format to the management device; A first coordinate packet corresponding to the haptic pointer coordinate packet from the management device based on a preset period, wherein the first coordinate packet is based on a preset rendering rate at which object coordinates corresponding to the haptic pointer coordinate are stored; A packet generated corresponding to the pointer coordinate packet; receiving and storing; Predicting packet loss for the received first coordinate packet; When a packet loss occurs according to the prediction, generating a lost packet based on a previously received first coordinate packet; And performing haptic rendering on the first coordinate packet based on a previously stored first buffer time.
Here, after receiving and storing the first coordinate packet,
Calculating a delay time based on the stored first coordinate packet and transmitting the delay time to the management device; Receiving a second buffer time corresponding to the delay time from the management device; And resetting the first buffer time based on the second buffer time.
According to a feature of the invention, the haptic data transmission method of the management device for transmitting and receiving haptic data with a plurality of haptic data transmission device,
Receiving haptic pointer coordinates from the haptic data transmission device; Generating first coordinates corresponding to the received haptic pointer coordinates based on a preset rendering rate at which object coordinates corresponding to the haptic pointer coordinates are stored; Transmitting the first coordinates to the haptic data transmission device; Receiving a delay time corresponding to the first coordinate transmission from the haptic data transmission device; And calculating a buffer time based on the received delay time and transmitting the buffer time to the haptic data transmission device.
Here, the step of receiving haptic pointer coordinates from the haptic data transmission device,
Predicting packet loss for the received haptic pointer coordinate packet; And
If packet loss occurs in accordance with the prediction, generating lost haptic pointer coordinates based on previously received haptic pointer coordinates.
According to an aspect of the present invention, the haptic data transmission system generates a first coordinate corresponding to the haptic pointer coordinates based on a preset rendering rate in which object coordinates corresponding to the haptic pointer coordinates are stored, and receives delays received from the plurality of transmission devices. A management device for calculating a buffer time based on time; And receiving the haptic pointer coordinates from the haptic device and transmitting the packetized haptic pointer coordinate packets to the management apparatus, obtaining the delay time corresponding to the reception of the first packet including the first coordinates, and transmitting the packet to the management apparatus. And a transmission device that receives the buffer time corresponding to the delay time and performs haptic rendering of the first packet based on the buffer time.
DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.
Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise. In addition, the terms “… unit”, “… unit”, “module”, “block”, etc. described in the specification mean a unit that processes at least one function or operation, which is hardware or software or a combination of hardware and software. It can be implemented as.
A haptic data transmission system and method thereof according to an embodiment of the present invention will now be described in detail with reference to the drawings.
1 is a block diagram illustrating a haptic data transmission system according to an embodiment of the present invention.
As shown in FIG. 1, a haptic data processing system according to an exemplary embodiment of the present invention has a haptic
Here, the network includes a wired network such as Ethernet, and a wireless network such as a wireless local area network (WLAN).
First, the haptic
The haptic
The haptic
Calculation Module (hereinafter, also referred to as "CM") 202 has an execution period equal to the rendering time of each haptic data transmission device 100-1, 100-n, and every execution period. Receive a haptic pointer coordinate packet from packet
The
In detail, the
Here, the haptic pointer coordinate packet is a packet including haptic pointer coordinates received from a haptic device (omitted), and the cube coordinate packet is a packet including cube coordinates generated based on a preset rendering rate corresponding to the haptic pointer coordinates. .
Here, the rendering rate includes cube coordinates that are coordinates of a plurality of virtual objects.
Meanwhile, in the embodiment of the present invention, the coordinates of the virtual objects are used as the cube coordinates, but other coordinates that may be used as the coordinates of the virtual objects may be used in addition to the cube coordinates.
The packet loss prediction module (hereinafter referred to as “PM”) 204 may be configured to lose haptic pointer coordinate packets received from the respective haptic data transmission devices 100-1 and 100-n through the
At this time, the packet
Here, the packet
Vn-1 = Xn-1-Xn-2
Where Xn is the predicted packet coordinate, Xn-1 is the last arrived packet coordinate, and Vn-1 is the difference between the last arrived packet coordinate and the previously arrived packet coordinate.
In addition, the packet
The
At this time, the
Rn = M-Dn
Here, Dn is a delay time received from each of the haptic data transmission apparatuses 100-1, 100-n, M is the largest value of the delay time, and Rn is a buffer time.
The
Next, the haptic data transmission apparatus 100-1 will be described in detail with reference to FIG. 1.
The haptic data transmission device 100-1 includes a
The
The haptic input module (HIM) 104 receives haptic pointer coordinates from the haptic device (omitted), packetizes the received haptic pointer coordinates, and transmits the haptic
Here, the
The
In addition, the
Specifically, when the buffer time is 0, the
Here, the
The
At this time, the
Here, the
In addition, the
The storage module 110 (hereinafter, also referred to as “BF”) 110 temporarily stores a cube coordinate packet received from the haptic
The
In this case, when the buffering time exceeds 0, the rendering module calculates coordinates used for haptic rendering based on Equation 3 below.
CP = BF [P]
Here, C is a current time (ms), R is a buffer time, CP is a coordinate to be used in the
Here, the haptic rendering is a rendering technique for showing the realistic interaction between the physical world and the virtual environment. The haptic rendering measures the distance between the object in the 3D virtual environment and the pointer indicating the location of the hapting device. It is a rendering technique that is calculated through.
Meanwhile, the
Such a haptic data transmission device according to an embodiment of the present invention has a great advantage of improving an error caused by jitter or loss.
In addition, the haptic data transmission device has an advantage that can be efficiently synchronized between each haptic data transmission device through the buffer time received from the haptic data management device.
2 is a block diagram illustrating a packet structure for transmitting and receiving between a haptic data management device and a haptic data transmission device according to an embodiment of the present invention.
As shown in FIG. 2, the
3 is a data flow diagram illustrating a haptic data transmission method of a haptic data transmission system according to an embodiment of the present invention.
As shown in FIG. 3, in the haptic data transmission method according to an embodiment of the present invention, the haptic data transmission apparatuses 100-1 and 100-n receive haptic pointer coordinates from a haptic device (not shown).
In this case, the haptic device is a physical device for transmitting a tactile sense, and can input 3D coordinates through manipulation of the haptic device and provide force feedback to provide a tactile feel.
The haptic data transmission apparatus 100-1 or 100-n receives the haptic pointer coordinates from the haptic device (omitted) (S100), and generates a haptic pointer coordinate packet by converting the received haptic pointer coordinates into a preset packet format. The haptic
The haptic
At this time, the haptic
In addition, the haptic
The haptic data transmission apparatus 100-1 or 100-n stores the received cube coordinate packet (S112), and performs packet loss prediction and jitter prediction on the received packet (S114).
In this case, the haptic data transmission apparatus 100-1 or 100-n checks the packet stored in the
In addition, the
The haptic data transmission apparatuses 100-1 and 100-n calculate the delay time based on the received packet, and then transmit the calculated delay time to the haptic
Thereafter, the haptic data transmission apparatuses 100-1 and 100-n perform haptic rendering based on the pre-stored buffer time (S120).
The haptic
The haptic
The haptic data transmission devices 100-1 and 100-n reset the buffer time in the haptic data transmission device based on the buffer time received from the haptic data management device 200 (S126).
The embodiments of the present invention described above are not only implemented through the apparatus and the method, but may also be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded. Implementations can be easily implemented by those skilled in the art from the description of the above-described embodiments.
Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.
By the above-described configuration, the haptic data transmission system can anticipate the lost data, and thus can expect an effect of efficiently improving the error caused by jitter or loss that may occur when transmitting and receiving pointer coordinate packets.
In addition, it is possible to expect an effect that can effectively synchronize the synchronization of the client by solving the asynchronous with delay.
Claims (10)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR20070025081 | 2007-03-14 | ||
KR1020070025081 | 2007-03-14 |
Publications (1)
Publication Number | Publication Date |
---|---|
KR20090001411A true KR20090001411A (en) | 2009-01-09 |
Family
ID=40484491
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
KR1020070034025A KR20090001411A (en) | 2007-03-14 | 2007-04-06 | Haptic data transferring system for networked collaboration and method thereof |
Country Status (1)
Country | Link |
---|---|
KR (1) | KR20090001411A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111949135A (en) * | 2020-08-31 | 2020-11-17 | 福州大学 | Touch communication fault-tolerant method and system based on hybrid prediction |
-
2007
- 2007-04-06 KR KR1020070034025A patent/KR20090001411A/en not_active Application Discontinuation
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111949135A (en) * | 2020-08-31 | 2020-11-17 | 福州大学 | Touch communication fault-tolerant method and system based on hybrid prediction |
CN111949135B (en) * | 2020-08-31 | 2022-06-17 | 福州大学 | Touch communication fault tolerance method and system based on hybrid prediction |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10666371B2 (en) | System for establishing and maintaining a clock reference indicating one-way latency in a data network | |
Zats et al. | DeTail: Reducing the flow completion time tail in datacenter networks | |
US8171123B2 (en) | Network bandwidth detection and distribution | |
Lee et al. | Dx: Latency-based congestion control for datacenters | |
Strowes | Passively measuring TCP round-trip times | |
US8266317B2 (en) | Reducing idle time due to acknowledgement packet delay | |
CN104581188A (en) | Method and apparatus for IP video signal synchronization | |
Boukerche et al. | An efficient synchronization scheme of multimedia streams in wireless and mobile systems | |
Gokhale et al. | Congestion control for network-aware telehaptic communication | |
JP2009055114A (en) | Communication device, communication system, transfer efficiency improvement method, and transfer efficiency improvement program | |
KR101983088B1 (en) | Method for UDP Packet Processing in the Multi-path Environment | |
JP5675703B2 (en) | Communication device, control device, and program | |
US11755057B2 (en) | Method, system, and computer program product for producing accurate IEEE 1588 PTP timestamps in a system with variable PHY latency | |
Gokhale et al. | HoIP: A point-to-point haptic data communication protocol and its evaluation | |
JP4798495B2 (en) | Video distribution quality measurement system, apparatus and method | |
Al Osman et al. | Alphan: Application layer protocol for haptic networking | |
US11088960B2 (en) | Information processing apparatus and verification system | |
KR20090001411A (en) | Haptic data transferring system for networked collaboration and method thereof | |
US20150156261A1 (en) | Methods and apparatus for cycle accurate time stamping at line rate throughput | |
Ishibashi et al. | Causality and media synchronization control for networked multimedia games: Centralized versus distributed | |
Al Osman et al. | Evaluating ALPHAN: A communication protocol for haptic interaction | |
JP2011182115A (en) | Communication method, communication system and server | |
JP2004135065A (en) | Transmission terminal, reception terminal and data transmission system | |
KR20190073058A (en) | Method and apparatus of multichannel media synchronization based on MMT protocol | |
KR20130009671A (en) | Packet transmission apparatus and method, and packet reception apparatus and method in mmt system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A201 | Request for examination | ||
E902 | Notification of reason for refusal | ||
E601 | Decision to refuse application | ||
J201 | Request for trial against refusal decision | ||
J501 | Disposition of invalidation of trial |