WO2001080573A2 - Apparatus and method for remote troubleshooting - Google Patents
Apparatus and method for remote troubleshooting Download PDFInfo
- Publication number
- WO2001080573A2 WO2001080573A2 PCT/SG2000/000050 SG0000050W WO0180573A2 WO 2001080573 A2 WO2001080573 A2 WO 2001080573A2 SG 0000050 W SG0000050 W SG 0000050W WO 0180573 A2 WO0180573 A2 WO 0180573A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- site
- customer service
- data
- video
- cursor
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/5003—Managing SLA; Interaction between SLA and QoS
- H04L41/5009—Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/508—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement
- H04L41/509—Network service management, e.g. ensuring proper service fulfilment according to agreements based on type of value added network service under agreement wherein the managed service relates to media content delivery, e.g. audio, video or TV
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
- H04L65/4053—Arrangements for multi-party communication, e.g. for conferences without floor control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
- H04N7/185—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q3/00—Selecting arrangements
- H04Q3/0016—Arrangements providing connection between exchanges
- H04Q3/0062—Provisions for network management
- H04Q3/0087—Network testing or monitoring arrangements
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/31—From computer integrated manufacturing till monitoring
- G05B2219/31479—Operator select part of process he wants to see, video image is displayed
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/35—Nc in input of data, input till input file format
- G05B2219/35495—Messages to operator in multimedia, voice and image and text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/131—Protocols for games, networked simulations or virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/1305—Software aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13092—Scanning of subscriber lines, monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13103—Memory
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13166—Fault prevention
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13292—Time division multiplexing, TDM
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13337—Picturephone, videotelephony
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13349—Network management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
A remote troubleshooting system that involves video, audio and data communications is disclosed. The system can be used for performing remote troubleshooting with the support of distant customer service experts. In troubleshooting applications, the video component of the system originating at factory premises is the main point of interest. For the identification of problems during troubleshooting, good quality video is usually required. If this is not achievable using real-time video transmission, video clips can be transmitted instead and this need not be done in real-time. After the video clip is transmitted, means of synchronisation is incorporated to enable synchronised playback of the video at the troubleshooting site and the customer service centre, thus allowing a site service technician and the customer service expert to view identical videos simultaneously. In addition, visual interactivity is achieved by means of overlaying cursor objects, text and graphics onto the video. Two cursor objects, one from the site service technician and the other from the customer service expert, is displayed on the video or images. The descriptions on the locations as well as the types of objects to be displayed are embedded in a set of data. Thus, the system allows a basis for further troubleshooting sessions and discussions with the aid of video and visual interactivity on top of the audio communication channel. In such an implementation, real-time audio and control data communications is easily achievable, as the bandwidth requirement is not significant.
Description
APPARATUS AND METHOD FOR REMOTE TROUBLESHOOTING
Field of invention
This invention relates to remote troubleshooting systems. In particular, it relates to remote troubleshooting systems using video, audio and data communications.
Background
There are prior developments in video conferencing systems for industrial process monitoring and troubleshooting. The major benefit of such systems is to reduce production downtime that results in costly lost of revenue, hi addition, customer support personnel need not be sent to sites if problems can be handled through video conferencing sessions, thus resulting in savings in manpower resources. Another benefit is when site service technicians are performing the troubleshooting with the support of a distant customer service expert, these technicians gain working knowledge for handling similar problems in the future. However, most video conferencing proposals are usually hardwired and located at predefined areas or rooms within factory buildings that may be a distance away from machines being repaired. Moving or installing video conferencing devices near the machines may not be feasible or practical. Cables linking audio and video devices may impede the movements of the service technicians.
Real-time video conferencing proposals require significant transmission bandwidth, thus limiting the choice for the selecting suitable communications channels. The ITU-T H.324 Video Conferencing Standard covers both digital video and audio communications over telephone networks for video conferencing systems. Such
systems work with conventional telephone networks but are not suitable for mobile phone and low-speed Internet applications.
Video conferencing is usually done via two-way video and audio communications. The video commumcations portion uses most of the available transmission bandwidth, i troubleshooting applications, the video component of video conferencing originating from factory premises forms the main point of interest. Thus, one-way video communication is usually sufficient for assisting troubleshooting, thereby resulting in reducing the video transmission bandwidth. Two-way audio communications occupy less transmission bandwidth and real-time audio communications are easily achievable. In remote troubleshooting, sending repetitive still or motion pictures at a slower rate can be an option. However, in such an approach, there is no visual interactivity for pinpointing the area of focus on the images. Furthermore, voice communication over the audio channel for identifying and locating the problematic areas can be confusing and may cause wrong interpretation.
There is therefore a need to provide for portable, wearable and hands-free remote troubleshooting operation. There is also a need to address the limitation in video transmission bandwidth and provide one-way video communications and visual interactivity for implementing an efficient, as well as cost-effective remote troubleshooting solution.
Summary
Embodiments of the invention involve a method of implementing remote troubleshooting whereby a site service technician is assisted by a distant or multiple customer service experts in identifying and locating the problematic areas using video, audio and data communications. Such embodiments enable portable, wearable and hands-free remote troubleshooting operation and address the limitation in transmission
bandwidth as well as real-time issues. Visual interactivity is added to provide an efficient, as well as cost-effective solution to remote troubleshooting.
In accordance with a first aspect of the invention, there is provided an apparatus for remote troubleshooting enabling video, audio and data communications between a site service technician and a distant customer service expert, the apparatus including: means for the video communications from the site service technician to the distant customer service expert, where the site video can be viewed by the site service technician and the distant customer service expert simultaneously; means for the audio communications between the site service technician and the distant customer service expert; means for the data communications between the site service technician and the distant customer service expert; means for enabling visual interactivity by placing cursor objects onto the video, where the cursor objects descriptions are carried in the data communications; and means for enabling visual communications by placing text and graphics onto the video, where the text and the graphics descriptions are carried in the data communications.
hi accordance with a second aspect of the invention, there is provided an apparatus for remote troubleshooting enabling video, audio and data communications between a site service technician and a distant customer service expert, the apparatus including: means for the audio communications between the site service technician and the distant customer service expert; means for the data communications between the site service technician and the distant customer service expert; means for carrying video in the data communications, where the video is made up of short video clips; means for synchronising playback of the short video clips, where the short video clips can be viewed by the site service technician and the distant customer service expert simultaneously;
means for enabling visual interactivity by placing cursor objects onto the video, where the cursor objects descriptions are carried in the data communications; and means for enabling visual commumcations by placing text and graphics onto the video, where the text and the graphics descriptions are carried in the data communications.
In accordance with a third aspect of the invention, there is provided an apparatus for remote troubleshooting enabling still images, audio and data communications between a site service technician and a distant customer service expert, the apparatus including: means for the audio commumcations between the site service technician and the distant customer service expert; means for the data communications between the site service technician and the distant customer service expert; means for carrying still images in the data communications; means for sequencing and displaying the still images to the site service technician and the distant customer service expert simultaneously; means for enabling visual interactivity by placing cursor objects onto the still images, where the cursor objects descriptions are carried in the data communications; and means for enabling visual communications by placing text and graphics onto the still images, where the text and the graphics descriptions are carried in the data communications .
hi accordance with a fourth aspect of the invention, there is provided an apparatus for remote troubleshooting enabling video, audio and data communications between a site service technician and multiple customer service experts, the apparatus including: means for the audio communications between the site service technician and the multiple customer service experts; means for the data communications between the site service technician and the multiple customer service experts;
means for carrying video in the data commumcations, where the video is made up of short video clips; means for synchronising playback of the short video clips, where the short video clips can be viewed by the site service technician and the multiple customer service experts simultaneously; means for enabling visual interactivity by placing cursor objects onto the video, where the cursor objects descriptions are carried in the data communications; and means for enabling visual communications by placing text and graphics onto the video, where the text and the graphics descriptions are carried in the data communications.
In accordance with a fifth aspect of the invention, there is provided an apparatus for remote troubleshooting enabling still images, audio and data communications between a site service techmcian and multiple customer service experts, the apparatus including: means for the audio communications between the site service techmcian and the multiple customer service experts; means for the data commumcations between the site service technician and the multiple customer service experts; means for carrying still images in the data communications; means for sequencing and displaying the still images to the site service technician and the multiple customer service experts simultaneously; means for enabling visual interactivity by placing cursor objects onto the still images, where the cursor objects descriptions are carried in the data communications; and means for enabling visual communications by placing text and graphics onto the still images, where the text and the graphics descriptions are carried in the data communications .
In accordance with a sixth aspect of the invention, there is provided an apparatus for remote troubleshooting, enabling real-time visual interactivity between a site service technician and a distant customer service expert, the apparatus including:
means for generating a site cursor, where the site cursor is controlled by the site service technician; means for generating a customer service cursor, where the customer service cursor is controlled by the distant customer service expert; means for embedding the site cursor's information and the customer service cursor's information in data transmissions; means for overlaying the site cursor and the customer service cursor onto the video that is viewed by the site service technician and the distant customer service expert simultaneously; and means for overlaying text and graphics onto the video that is viewed by the site service technician and the distant customer service expert simultaneously.
hi accordance with a seventh aspect of the invention, there is provided an apparatus for remote troubleshooting, enabling real-time visual interactivity between a site service techmcian and multiple customer service experts, the apparatus including: means for generating a site cursor, where the site cursor is controlled by the site service technician; means for generating a customer service cursors, where the customer service cursors are controlled by the customer service experts; means for embedding the site cursor's information and the customer service cursors' information in data transmissions; means for overlaying the site cursor and the customer service cursors onto the video that is viewed by the site service technician and the customer service experts simultaneously; and means for overlaying text and graphics onto the video that is viewed by the site service technician and the customer service experts simultaneously.
hi accordance with an eighth aspect of the invention, there is provided a method for providing synchronised playback of short video clips with visual interactivity for remote troubleshooting between a site service technician and a distant customer service expert, the method including the steps of:
sending video clips from the site service technician to the distant customer service expert; sending control data initiating synchronised playback of the video clips, where the video clips are viewed by the site service technician and the distant customer service expert simultaneously; sending site cursor data from the site service technician to the distant customer service expert; sending customer service cursor data from the distant customer service expert to the site service technician; overlaying site cursor and customer service cursor onto the video clips using the site cursor data and the customer service cursor data respectively; sending site text and graphics data from the site service technician to the distant customer service expert; sending customer service text and graphics data from the distant customer service expert to the site service technician; and overlaying text and graphics onto the video clips using the site text and graphics data and the customer service text and graphics data respectively.
hi accordance with a ninth aspect of the invention, there is provided a method for displaying still images with visual interactivity for remote troubleshooting between a site service technician and a distant customer service expert, the method including the steps of: sending still images from the site service technician to the distant customer service expert; sending control data for sequencing still images for display, where the still images are viewed by the site service technician and the distant customer service expert simultaneously; sending site cursor data from the site service technician to the distant customer service expert; sending customer service cursor data from the distant customer service expert to the site service technician;
overlaying site cursor and customer service cursor onto the still images using the site cursor data and the customer service cursor data respectively; sending site text and graphics data from the site service technician to the distant customer service expert; sending customer service text and graphics data from the distant customer service expert to the site service technician; and overlaying text and graphics onto the still images using the site text and graphics data and the customer service text and graphics data respectively.
Brief description of the drawings
Figure 1 illustrates the functional modules of a remote troubleshooting system at a troubleshooting site in accordance with an embodiment of the invention.
Figure 2 illustrates the functional modules of the remote troubleshooting system of Figure 1 at the customer service centre.
Figure 3 provides an illustration on how a site service technician and a customer service expert communicate using the remote troubleshooting system of Figure 1.
Table 1 provides a definition of a private_data_section.
Table 2 provides definitions of data__type fields.
Table 3 provides a definition of a control_data field.
Table 4 provides definitions of cursor_type fields.
Detailed description
Real-time video conferencing uses significant transmission bandwidth, especially the video communications portion. Thus, the main problem to be addressed is to reduce the overall transmission bandwidth requirements, and yet provide an efficient means of remote troubleshooting. This problem is addressed by providing a remote troubleshooting system according to an embodiment of the invention described hereinafter. In troubleshooting applications, the video component of video conferencing originating from the factory premises forms the main point of interest. Thus, one-way video communication is usually sufficient for assisting troubleshooting, thereby resulting in reducing the video transmission bandwidth. Still images or video clips are sent if the transmission channel is not able to accommodate real-time video transmission. Acknowledgement is sent back when the still images or video clips are received.
Besides the audio channel, data is also transmitted to enable a site service technician and a distant customer service expert to conduct effective troubleshooting sessions. Visual interactivity is achieved by means of overlaying cursor objects into the video component. Two cursor objects, one relating to site service technician and the other relating to the customer service expert, are displayed on videos or images. The descriptions on the locations as well as the types of objects to be displayed are embedded in the data. Besides the cursor objects, text and graphics can be added for providing visual information. Real-time audio and data communications are easily achievable, as the bandwidth requirements are not significant.
For the identification of problems during troubleshooting, good quality video is usually required and this is not achievable using real-time video transmission at low transmission bandwidth. To overcome this limitation, high quality video clips can be transmitted instead and this need not be done in real-time. After a video clip is transmitted, a means of synchronisation is incorporated to enable synchronised playback of the video clip at the troubleshooting site and the customer service centre,
thus allowing the site service technician and the customer service expert to view identical video clips simultaneously. Together with the text, graphics and cursor objects, the system allows a basis for further troubleshooting sessions and discussions with the aid of video and visual interactivity in addition to the audio communications channel.
Operation of the system
During a remote troubleshooting session, the site service technician communicates with the distant customer service expert using video, audio and data communications over a suitable transmission channel such as the Internet, a telephone line, or a hand-phone or other wireless systems. The site service technician captures a video of the point of interest and transmits the video shot to the distant customer service expert. Still images or video clips are sent if the transmission channel is not able to accommodate real-time video transmission. When the video or still image is being played, the site technician highlights the subject in focus using a pointer device. The pointer device overlays a cursor object onto the video. At customer service centre, the cursor object that is created by the site service technician also appears on the video displayed to the customer service expert. The customer service expert can look into the problem relying on the video that is transmitted, the cursor object overlaid onto the video, as well as the audio communication established with the site service technician. The customer service expert in turn can request to zoom into certain areas by highlighting the desired portion of the video using another cursor object, hi response to the request, the site technician zooms in and captures the video and the troubleshooting process continues as such. Thus there are two cursor objects overlaying the video, one being controlled by the site service technician and the other by the customer service expert. The movements and shape of the cursor objects are displayed on the video viewed concurrently by the site service technician and the customer service expert. In addition to the cursor objects, text and graphics can be added for visual communications. Thus, visual interactivity is achieved by means of overlaying text, graphics and cursor objects into the video, assisting in highlighting the area of focus or pinpointing the problematic areas.
Description of the system
Functional modules of the remote troubleshooting system in according to an embodiment of the invention for use at the troubleshooting site are shown in Figure 1. A pointer device 100 includes a trackball for providing positioning control, as well as left and right buttons for signalling user commands. The pointer device 100 allows the site service technician to insert and overlay a troubleshooting site cursor onto the video or image that is displayed on an LCD (liquid crystal display) display 111. Positioning and user signals 101 from the pointer device 100 are processed by a data processor 106. The data processor 106 also receives customer service centre data 102 from a data decoder 103. A data processor 106 relays cursor data 113 comprising of information in relation to the troubleshooting site cursor and the customer service centre cursor to a video overlay module 114. Data for the troubleshooting site cursor is derived from the pointer device 100 and data for the customer service centre cursor is derived from the data decoder 103. Using such cursor information, the video overlay module 114 generates and overlays the cursor objects onto incoming video signal 120 originating from a video capture module 121. In addition to the cursor information, text and graphics can be added for providing visual information. Resultant video output 112 is displayed on the LCD display 111. There are two cursor objects overlaying onto the video, one being confrolled by the site service technician at the troubleshooting site and the other by the distant customer service expert over at a distant customer service centre. The video capture module 121 captures video data 119 using a troubleshooting site camera 118 and relays digital video signals 115 & 120 to the video overlay module 114 and a video encoder module 116. The captured video data 119 can be stored in memory 124 for subsequent retrieval. The video encoder module 116 incorporates a video compression technique for reducing transmission bandwidth. Compressed video data 117 is sent to a multiplexer 110 that multiplexes data from the video encoder 116, an audio encoder 122 and a data encoder 108 into packetised data stream 135. The packetised data stream 135 is sent to a transmitter 134 for transmission to the distant customer service centre.
The site service technician can send machine data 105 over to the distant customer service expert for checking and validating purposes. The machine data 105 is uploaded though a serial port 104 and sent to the data processor 106 which relays the machine data 107 to the data encoder module 108. The data encoder module 108 incorporates a lossless compression technique. Compressed machine data 109 is sent to the multiplexer 110, then to the transmitter 134 for transmission to the distant customer service centre. In addition to the cursor information, the distant customer service expert can also send a set of machine, document and short AV (audio/video) clip data to the site service technician. A receiver 136 receives received data 137 from the customer service centre and transfers the received data 137 to a demultiplexer 133. After demultiplexing, the set of machine, document and the short AV clip data, as well as cursor data 138, are sent to the data decoder 103. The decompressed data 102 is transferred to the data processor 106, which determines the subsequent course of action for the data 102. Document data can be processed, routed to the video overlay module 114 for displaying on the LCD display 111 while the machine data 105 is routed to the serial port 104. The short AV clip data is archived in a memory 124 for subsequent synchronised playback.
The audio section of the system includes a headset consisting of a microphone 125 and a headphone 127. An audio encoder 122 encodes a voice signal 126 from the microphone 125 into compressed audio data packets 123. Compression is performed to reduce the audio transmission bandwidth. The packetised compressed audio data 123 is sent to the multiplexer 110 for transmission. The receiver 136 receives audio data from the distant customer service expert. After de-multiplexing, audio data 132 is sent to an audio decoder 131. Audio output 130 is mix with the voice signal 126 in an audio mixer 129 and the resultant audio output 128 is sent to the headphone 127.
Functional modules of the remote troubleshooting system at the customer service centre are shown in Figure 2. A pointer device 205 generates positioning and user signals 204 that are sent to a data processor 209. The data processor 209 also receives
troubleshooting site data 208 from a data decoder 207. The data processor 209 relays cursor data 215 comprising of information relating to the troubleshooting site cursor and the customer service centre cursor to a video overlay module 216. Data relating to customer service cursor is derived from the pointer device 205 and data relating to the troubleshooting site cursor data is derived from the cursor information embedded in troubleshooting site data 208 provided by the data decoder 207. Using such cursor information, the video overlay module 216 generates and overlays the cursor objects onto a video signal 214 provided by a video decoder 213 which decodes a digital video signal 212 provided by a demultiplexer 200. The digital video data 212 can be stored in a memory 219 for subsequent retrieval. The resultant video output 217 is displayed on a video display 218.
Besides the cursor information, the customer service expert can also send a set of machine, document and short AV clip data 210 through a serial port 211. The set of machine, document and the short AV clip data 210, as well as customer service cursor data 203, are sent to a data encoder 202, and subsequently encoded data 201 is routed to a multiplexer 226. Multiplexed output 234 is sent to a transmitter 233 for transmission to the site service technician. On the other hand, a receiver 231 receives the troubleshooting site data and received data 232 is transferred to a demultiplexer 200. After de-multiplexing, data relating to the site cursor, as well as site machine, document and short AV clip data 206 are sent to the data decoder 207. The decompressed data 208 is transferred to the data processor 209. Document data can be processed, routed to the video overlay module 216 for displaying on the display 218 while the machine data 210 is routed to the serial port 211. The short AV clip data is archived in memory 219 for subsequent synchronised playback.
The audio section of the system includes a headset consisting of a microphone 230 and a headphone 225. An audio encoder 228 encodes a voice signal 229 from the microphone 230 into compressed audio data packets 227. The packetised compressed audio data 227 is sent to a multiplexer 226 for transmission to the troubleshooting site. The receiver 232 receives audio data from the site service technician. After de-
multiplexing, audio data 220 is sent to an audio decoder 221. Audio output 222 is mix with the voice signal 229 in an audio mixer 223 and the resultant audio output 224 is sent to the headphone 225.
Figure 3 shows the communication set-up between the site service technician and the customer service expert. At the troubleshooting site, the system includes a wearable site console 310. The site console 310 interconnects video input 303 to a video camera 301, video output 309 to a video display unit 302, audio signals 305 to a headset 300, and a positional and control signal 306 to a pointer device 304. The site console 310 is constructed from the data encoder 108, the data decoder 103, the data processor 106, the video overlay 114, the video encoder 116, the video capture 121, the memory 124, the audio encoder 122, the audio mixer 129, the audio decoder 131, the multiplexer 110 and the demultiplexer 133 functional modules as shown in Figure 1. The site console 310 relays video data, audio data, as well as cursor information, machine or document data 311 to a site communicator 312. The site communicator 312 is formed from the transmitter 134 and the receiver 136 functional modules as shown in Figure 1. The site communicator 312 provides communication of digital data between the troubleshooting site and the customer service centre.
Over at the customer service centre, the system includes a customer service console 315, interconnecting a video signal 320 to a video display unit 321, audio signals 316 to a headset 317, and a positional and control signal 318 to apointer device 319. The customer service console 315 is constructed from the data encoder 202, the data decoder 207, the data processor 209, the video overlay 216, the video decoder 213, the memory 219, the audio encoder 228, the audio mixer 223, the audio decoder 221, the multiplexer 226 and the demultiplexer 200 functional modules as shown in Figure 2. The site console 315 relays audio data as well as cursor information, machine, document or AV clips data 314 to a customer service communicator 313. The customer service communicator 313 is formed from the transmitter 233 and the receiver 231 functional modules as shown in Figure 2.
Two cursor objects, one relating to the site service technician and the other relating to the customer service expert, are displayed on the video or images. At the troubleshooting site, the site service technician views the video with the troubleshooting site cursor 307 and the customer service centre cursor 308. Over at the customer service centre, the customer service expert views the same video with the troubleshooting site cursor 322 and the customer service centre cursor 323. The descriptions on the locations as well as the types of cursor objects to be displayed are embedded in the transmission of the digital data between the troubleshooting site and the customer service centre. Data commumcation for real-time audio and cursor easily achievable, as the bandwidth requirement is not significant. The positions and movements of the cursors appearing over the displays at troubleshooting site and the customer service centre are in synchronisation. The viewer can toggle the cursors on or off.
In the embodiment, visual interactivity is achieved using cursor information embedded in data transmission in real-time. Machine data, document and short AV clip data can also be transmitted but need not be done in real-time. Information relating to the cursor, machine, document and short AV clip data is described in the private data section table as shown in Table 1. The semantic definitions are:
• data_id -- A 13 -bit unique identification for private data section carrying cursor, machine and document data. This is used for filtering the private data section from the data packets.
• data ype — This 3-bit field identifies the type of data that follows. The description of the datajype is tabulated in Table 2.
• confrol_data — This 32-bit field contains the control data. The description of the control_data is tabulated in Table 3.
• cursor_type ~ This 4-bit field describe the type of cursor. The description of the cursorjype is tabulated in Table 4.
• cursor_size — This 4-bit field indicates the preferred size of the cursor.
• cursor_style ~ This 4-bit field indicates the preferred style of the cursor.
x_axis ~ This 10-bit field contains the horizontal co-ordinates of the cursor. y_axis — This 10-bit field contains the vertical co-ordinates of the cursor. x_size ~ This 10-bit field contains the horizontal size of the rectangular cursor box. y_size ~ This 10-bit field contains the vertical size of the rectangular cursor box. radius — This 8-bit field contains the radius of the circular cursor box. text_data_length ~ This 12-bit field specifies the length of bytes of the following text data description. text_data ~ This 8-bit field contains the text data. machine_id -- A 16-bit unique identification for the machine model. machine_data_type ~ This 6-bit field identifies the machine data type, sectionjnumber ~ This is a 16-bit field that gives the number of this private_data_section. The private data are retrieved starting from section 0 to the last_section_number. The numbers are running sequentially. last_section_number — This 16-bit field specifies the last section number of the private_data_section. machine_data_length — This 12-bit field specifies the length of bytes of the following machine data description. machine_data ~ This 8-bit field contains the machine data. document_type ~ This 6-bit field identifies the document type. document_data_length — This 12-bit field specifies the length of bytes of the following document data description. document_data ~ This 8-bit field contains the document data. AV_type — This 6-bit field identifies the AV data type.
AV_version_number— This 8-bit field contains the version number of the AV data. It provides an identification tag for retrieval of the archived short video clip or still images for synchronised playback.
AV_data_length — This 12-bit field specifies the length of bytes of the following AV data description. AV_data - This 8-bit field contains the short AV clip data.
• CRC — This 32-bit field contains CRC value. It can be used to check the correctness of the data in this section.
For the identification of problems during remote troubleshooting, good quality video is usually required. If this is not achievable using real-time video transmission, video clips can be transmitted instead and this need not be done in real-time. The short video clips or still images can be delivered using the private_data_section with datajype value equals 5. After a video is transmitted, a means of synchronisation is incorporated to enable synchronise playback of the video at the troubleshooting site and the customer service centre, thus allowing the site service technician and the customer service expert to view identical videos simultaneously. The synchronisation information is embedded in the control_data carried in the private_data_section. Thus, together with the cursor objects, the system allows a basis for further troubleshooting sessions and discussions with the aid of video and visual interactivity on top of the audio communication channel. Multiple communication stations can be set up for communication involving three or more parties.
The system provides an enhancement to existing methods of industrial process monitoring and troubleshooting. The major benefit of such a system is to provide a cost-effective remote troubleshooting solution, enabling video, audio and data communications between the site service technician and the distant customer service expert.
Claims
1. An apparatus for remote troubleshooting enabling video, audio and data communications between a site service technician and a distant customer service expert, said apparatus including: means for said video communications from said site service technician to said distant customer service expert, where the site video can be viewed by said site service techmcian and said distant customer service expert simultaneously; means for said audio communications between said site service technician and said distant customer service expert; means for said data communications between said site service technician and said distant customer service expert; means for enabling visual interactivity by placing cursor objects onto said video, where said cursor objects descriptions are carried in said data communications; and means for enabling visual communications by placing text and graphics onto said video, where said text and said graphics descriptions are carried in said data communications .
2. An apparatus for remote troubleshooting enabling video, audio and data communications between a site service technician and a distant customer service expert, said apparatus including: means for said audio communications between said site service technician and said distant customer service expert; means for said data communications between said site service technician and said distant customer service expert; means for carrying video in said data communications, where said video is made up of short video clips; means for synchronising playback of said short video clips, where said short video clips can be viewed by said site service technician and said distant customer service expert simultaneously; means for enabling visual interactivity by placing cursor objects onto said video, where said cursor objects descriptions are carried in said data communications; and means for enabling visual communications by placing text and graphics onto said video, where said text and said graphics descriptions are carried in said data communications.
3. An apparatus for remote troubleshooting enabling still images, audio and data communications between a site service technician and a distant customer service expert, said apparatus including: means for said audio communications between said site service technician and said distant customer service expert; means for said data communications between said site service technician and said distant customer service expert; means for carrying still images in said data communications; means for sequencing and displaying said still images to said site service technician and said distant customer service expert simultaneously; means for enabling visual interactivity by placing cursor objects onto said still images, where said cursor objects descriptions are carried in said data communications; and means for enabling visual communications by placing text and graphics onto said still images, where said text and said graphics descriptions are carried in said data communications.
4. An apparatus for remote troubleshooting enabling video, audio and data communications between a site service technician and multiple customer service experts, said apparatus including: means for said audio communications between said site service technician and said multiple customer service experts; means for said data commumcations between said site service technician and said multiple customer service experts; means for carrying video in said data communications, where said video is made up of short video clips; means for synchronising playback of said short video clips, where said short video clips can be viewed by said site service technician and said multiple customer service experts simultaneously; means for enabling visual interactivity by placing cursor objects onto said video, where said cursor objects descriptions are carried in said data communications; and means for enabling visual communications by placing text and graphics onto said video, where said text and said graphics descriptions are carried in said data communications .
5. An apparatus for remote troubleshooting enabling still images, audio and data commumcations between a site service technician and multiple customer service experts, said apparatus including: means for said audio communications between said site service technician and said multiple customer service experts; means for said data commumcations between said site service technician and said multiple customer service experts; means for carrying still images in said data communications; means for sequencing and displaying said still images to said site service technician and said multiple customer service experts simultaneously; means for enabling visual interactivity by placing cursor objects onto said still images, where said cursor objects descriptions are carried in said data communications; and means for enabling visual communications by placing text and graphics onto said still images, where said text and said graphics descriptions are carried in said data communications.
6. An apparatus for remote troubleshooting, enabling real-time visual interactivity between a site service technician and a distant customer service expert, said apparatus including: means for generating a site cursor, where said site cursor is controlled by said site service technician; means for generating a customer service cursor, where said customer service cursor is controlled by said distant customer service expert; means for embedding said site cursor's information and said customer service cursor's information in data transmissions; means for overlaying said site cursor and said customer service cursor onto the video that is viewed by said site service technician and said distant customer service expert simultaneously; and means for overlaying text and graphics onto said video that is viewed by said site service technician and said distant customer service expert simultaneously.
7. An apparatus for remote troubleshooting, enabling real-time visual interactivity between a site service technician and multiple customer service experts, said apparatus including: means for generating a site cursor, where said site cursor is controlled by said site service technician; means for generating a customer service cursors, where said customer service cursors are controlled by said customer service experts; means for embedding said site cursor's information and said customer service cursors' information in data transmissions; means for overlaying said site cursor and said customer service cursors onto the video that is viewed by said site service technician and said customer service experts simultaneously; and means for overlaying text and graphics onto said video that is viewed by said site service technician and said customer service experts simultaneously.
8. A method for providing synchronised playback of short video clips with visual interactivity for remote troubleshooting between a site service technician and a distant customer service expert, said method including the steps of: sending video clips from said site service technician to said distant customer service expert; sending control data initiating synchronised playback of said video clips, where said video clips are viewed by said site service technician and said distant customer service expert simultaneously; sending site cursor data from said site service technician to said distant customer service expert; sending customer service cursor data from said distant customer service expert to said site service technician; overlaying site cursor and customer service cursor onto said video clips using said site cursor data and said customer service cursor data respectively; sending site text and graphics data from said site service technician to said distant customer service expert; sending customer service text and graphics data from said distant customer service expert to said site service technician; and overlaying text and graphics onto said video clips using said site text and graphics data and said customer service text and graphics data respectively.
9. A method for displaying still images with visual interactivity for remote troubleshooting between a site service technician and a distant customer service expert, said method including the steps of: sending still images from said site service technician to said distant customer service expert; sending control data for sequencing still images for display, where said still images are viewed by said site service technician and said distant customer service expert simultaneously; sending site cursor data from said site service technician to said distant customer service expert; sending customer service cursor data from said distant customer service expert to said site service technician; overlaying site cursor and customer service cursor onto said still images using said site cursor data and said customer service cursor data respectively; sending site text and graphics data from said site service technician to said distant customer service expert; sending customer service text and graphics data from said distant customer service expert to said site service technician; and overlaying text and graphics onto said still images using said site text and graphics data and said customer service text and graphics data respectively.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| AU41627/00A AU4162700A (en) | 2000-04-15 | 2000-04-15 | Apparatus and method for remote troubleshooting |
| PCT/SG2000/000050 WO2001080573A2 (en) | 2000-04-15 | 2000-04-15 | Apparatus and method for remote troubleshooting |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/SG2000/000050 WO2001080573A2 (en) | 2000-04-15 | 2000-04-15 | Apparatus and method for remote troubleshooting |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| WO2001080573A2 true WO2001080573A2 (en) | 2001-10-25 |
| WO2001080573A3 WO2001080573A3 (en) | 2002-02-07 |
Family
ID=20428804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/SG2000/000050 WO2001080573A2 (en) | 2000-04-15 | 2000-04-15 | Apparatus and method for remote troubleshooting |
Country Status (2)
| Country | Link |
|---|---|
| AU (1) | AU4162700A (en) |
| WO (1) | WO2001080573A2 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1445686A2 (en) * | 2003-02-05 | 2004-08-11 | Siemens Aktiengesellschaft | HMI device with a computer user station with an interface for transmitting communication data |
| EP1548529A1 (en) * | 2003-12-25 | 2005-06-29 | Fanuc Ltd | Numerical controller |
| US20090185792A1 (en) * | 2008-01-18 | 2009-07-23 | Rutan & Tucker, LLP | Digital video camcorder with wireless transmission built-in |
| US10901032B1 (en) | 2019-09-13 | 2021-01-26 | Raytheon Company | System and method for remote intelligent troubleshooting |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2688643A1 (en) * | 1992-03-13 | 1993-09-17 | Bali Dev | System for remote maintenance of industrial equipment, especially of woodworking machines, and method employed in this system |
| JPH08235479A (en) * | 1995-02-28 | 1996-09-13 | Hitachi Zosen Corp | Remote monitoring device |
| WO1999041649A1 (en) * | 1998-02-13 | 1999-08-19 | Sierra Concepts Corporation | Interactive remote control system of manufacturing equipment |
-
2000
- 2000-04-15 AU AU41627/00A patent/AU4162700A/en not_active Abandoned
- 2000-04-15 WO PCT/SG2000/000050 patent/WO2001080573A2/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2688643A1 (en) * | 1992-03-13 | 1993-09-17 | Bali Dev | System for remote maintenance of industrial equipment, especially of woodworking machines, and method employed in this system |
| JPH08235479A (en) * | 1995-02-28 | 1996-09-13 | Hitachi Zosen Corp | Remote monitoring device |
| WO1999041649A1 (en) * | 1998-02-13 | 1999-08-19 | Sierra Concepts Corporation | Interactive remote control system of manufacturing equipment |
Non-Patent Citations (1)
| Title |
|---|
| PATENT ABSTRACTS OF JAPAN vol. 97, no. 1 31 July 1997 & JP 08 235 479 A (HITACHI) 13 September 1996 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1445686A2 (en) * | 2003-02-05 | 2004-08-11 | Siemens Aktiengesellschaft | HMI device with a computer user station with an interface for transmitting communication data |
| EP1445686A3 (en) * | 2003-02-05 | 2008-07-02 | Siemens Aktiengesellschaft | HMI device with a computer user station with an interface for transmitting communication data |
| EP1548529A1 (en) * | 2003-12-25 | 2005-06-29 | Fanuc Ltd | Numerical controller |
| US20090185792A1 (en) * | 2008-01-18 | 2009-07-23 | Rutan & Tucker, LLP | Digital video camcorder with wireless transmission built-in |
| US10901032B1 (en) | 2019-09-13 | 2021-01-26 | Raytheon Company | System and method for remote intelligent troubleshooting |
Also Published As
| Publication number | Publication date |
|---|---|
| AU4162700A (en) | 2001-10-30 |
| WO2001080573A3 (en) | 2002-02-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6839079B2 (en) | Video-telephony system | |
| US7634731B2 (en) | System and method for remotely controlling character avatar image using mobile phone | |
| JP3125552B2 (en) | Video conference system | |
| US20050208962A1 (en) | Mobile phone, multimedia chatting system and method thereof | |
| EP2154885A1 (en) | A caption display method and a video communication system, apparatus | |
| US5898457A (en) | TV conference communication device and method of controlling the same | |
| CN101500127A (en) | Method for synchronously displaying subtitle in video telephone call | |
| CN103384319B (en) | The image-scaling method and system of double-current video conference terminal | |
| JP4605028B2 (en) | Remote control system, remote control communication device, and controlled communication device | |
| WO2001080573A2 (en) | Apparatus and method for remote troubleshooting | |
| KR100358166B1 (en) | Image hand-phone | |
| KR20040081370A (en) | Control method and system for a remote video chain | |
| JPH10290448A (en) | Monitoring system | |
| KR20070013067A (en) | Method and terminal for image sharing | |
| JP2002290940A (en) | Video conference system | |
| KR100897897B1 (en) | Apparatus and Method for Providing Video Call Service | |
| CN109889787A (en) | A kind of video transmission method, apparatus and system | |
| JP2769195B2 (en) | Multipoint video conference still image display system | |
| CN113179426B (en) | High-security audio and video command and scheduling system | |
| CN101783922B (en) | Video conference device and method thereof | |
| JP2009171272A (en) | Video telephone terminal device | |
| JPH08294120A (en) | Moving image communication system | |
| JP4429837B2 (en) | Switching device for remote monitoring | |
| KR20050001929A (en) | Service device and the method for multi streaming of mobile phone | |
| JP3982059B2 (en) | Data processing method for videophone device and videophone device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
| AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
| AK | Designated states |
Kind code of ref document: A3 Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
| AL | Designated countries for regional patents |
Kind code of ref document: A3 Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
| REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
| 122 | Ep: pct application non-entry in european phase | ||
| NENP | Non-entry into the national phase |
Ref country code: JP |