NL2014695B1

NL2014695B1 - A method and system for two-way A/V-communication.

Info

Publication number: NL2014695B1
Application number: NL2014695A
Authority: NL
Inventors: De Andrade Diogo
Original assignee: Divitel B V
Priority date: 2015-04-22
Filing date: 2015-04-22
Publication date: 2017-01-18
Also published as: NL2014695A

Abstract

The invention relates a method for two-way AN-communication between a first user at a first end (A) and a second user at a second end (B). A webcam (A10) and a set-top box (A30) at the first end and a webcam (B10) and a set-top box (B30) at the second end poll a server (1 00) for commands in corresponding command register. In response to an accept message from the set-top box (B30) at the second end, the server (1 00) stores a stream command in a camera command register of the webcam at the first end. The webcam (A 1 0) goes to a recording state after retrieving the stream command and stream recorded AN-data over the communication link. In this way, existing set-top boxes to which a camera cannot be connected, can be used to provide a user interface for two-way AN communication and decoding facilities for decoding the AN stream.

Description

A method and system for two-way A/V-communication

TECHNICAL FIELD

The invention relates to a method for two-way A/V-communication between a first user at a first end and a second user at a second end. The method further relates to a method and system for two-way A/V-communication over an IP-network between a first user at a first end and a second user at a second end.

BACKGROUND

Two-way A/V-communication is known from GB2463110. The disclosed method may be performed by using a set-top box or television set. A set-top box (STB) is used in the connection. A webcam is connected to the settop box or built in the set-top box. US2013312021 discloses systems, methods and computer products for querying for an interactive assistant for communication over a network via the set-top box. EP2537324 discloses a client application configured to allow a local user to participate in bidirectional communication with other remote users via a network interface and packet based network. A television set comprises a webcam and an audio transceiver with speaker and microphone, or is connected or communicable with such components. Furthermore, W02012/083606A1 discloses also a method for audio and video conversation which utilizes existing devices such as a set-top box.

There are several different systems for two-way video communication present in the market. The most important ones are Skype© and WebRTC. The issue with all of the known systems is the fact that they are unsuitable for deployment on a normal digital TV environment on a set-top box, without extensive changes on the existing software and/or hardware. A set-top box environment on digital TV systems usually comprises a hardware platform and a software environment. The hardware platform comprises custom hardware, which is normally not very powerful and aided by a H264 hardware video decoder chip. The software environment is often a Linux-based operating system running a modified/simple web browser.

These restrictions of the set-top box environment make it unsuitable for the commercially available two-way video communication systems. Skype/WebRTC use their video compression formats. The compressed video cannot be decoded on the hardware in normal set-top boxes and these set-top boxes usually do not have the processing power to decode the compressed HD video in software. Furthermore, normal webcams cannot be connected to set-top boxes, either by lack of connectivity ports (USB for example) or by lack of software drivers. The Linux based operating system on a set-top box usually is a minimal configuration without any ancillary drivers. Additionally, the common middlewares, i.e. the software that runs on top of the web browser that provides the user interface, do not have support for two-way AA/-communication. Finally, the software platforms present in the set-top boxes usually do not support the transport layers necessary for real-time and interactive video. Most of the existing set-top boxes are designed for a TV/movie watching scenario, in which delays of 2 - 10 seconds are not noticed by the users of the set-top box. Furthermore, they mostly implement protocols like HLS (HTTP live Streaming), which have similar delays.

SUMMARY

It is an object of the invention to provide a method for two-way ΑΛ/ communication which allows using most of the existing commercially available settop boxes (set-top boxes) in a two-way A/V-communication system without changing the hardware of the existing set-top boxes.

According to the invention, this object is achieved by a method having the features of Claim 1. Advantageous embodiments and further ways of carrying out the invention may be attained by the measures mentioned in the dependent claims.

According to a first aspect of the invention, there is provided a method for two-way AA/-communication between a first user at a first end and a second user at a second end. A set-top box at the first end processes AA/-data recorded at the second end. The AA/-data is reproduced at a display device at the first end. A communication arrangement at the second end comprises a camera device and a display device. The communication arrangement processes AN-data recorded by a camera device at the first end to be reproduced with the display device at the second end. The camera device at the first end is a webcam. The webcam polls a server for a camera command stored in a camera command register associated with the webcam. The set-top box sends a command to the server requesting a communication link with the communication arrangement at the second end. In response to the command the server sets an indication for the communication arrangement that there is a request for a communication link. The communication arrangement polls the server for an indication that there is a request for a communication link. In response to input of the second user, the communication arrangement sends an accept message to the server. In response to the accept message, the server stores a stream command in the camera command register. The webcam goes to a recording state after retrieving from the server the stream command stored in the camera command register and streams recorded AA/-data over the communication link to the second end.

The basic idea of the present application is to provide a two-way communication system that can be easily used by elderly people. Most elderly people are familiar with the user interface of a set-top box. In this application the set-top box will be part of the communication system. However, most of the existing set-top boxes do not have suitable hardware to connect a camera device to the set-top box. Furthermore, the hardware is normally aided by a H264 hardware video decoder chip. This requires that the AN- content received by the set-top box from the other end should preferably be in the H264-format. In the present application, a smart camera in combination with an existing set-top box is used to provide all necessary components at one end of a two-way A/V communication link. A smart camera is an active webcam that actively requests a server what to do. Normal webcams are passive components that wait for instructions transmitted by devices that would like to receive AN content from a webcam. A user has to know the internet address of the webcam to control the webcam and to receive the AN content. Elderly would need assistance from others to connect a normal webcam to the internet and to provide the necessary information to other people who would communicate with the elderly people via a two-way communication link.

The smart camera at one end records the audio and video content and submits the A/V content to the other end for reproduction on a display device. The set-top box at one end receives the A/V content from the other end for reproduction on a display device. As the smart camera is not directly coupled with the set-top box, a method for a server is developed to setup a communication link between two or more users and to control the components necessary for the two-way A/V communication. The use of a server has security advantages. The server will determine when a smart camera should generate A/V content or not. A caller requesting a call will not be able to receive A/V content from the called person, without permission. Furthermore, the caller and the called person will not be aware of the internet addresses used by the devices. The server is a kind a switchboard that directs the A/V content from one end to the other end and provides the necessary control information for the devices that take part of the two-way A/V communication. After the user has initiated a setup of the webcam to connect with the server once, the webcam could obtain all necessary control information from the server. To make the existing set-top box suitable for two-way communication, the software environment has to be adapted to comprise content, for example a web page, to select a person to call. The caller does not have to know or to receive address information of the devices of the called person, for example IP-address or MAC-address to setup the two-way communication link. As a well-known user interface is used on the set-top box, elderly people familiar with the user interface could easily call another person. The smart camera used by the caller knows where to retrieve its control information by polling a register at the server assigned to the smart camera. In an advantageous embodiment, the polling actions are performed by sending HTTP-requests to the server. HTTP-requests are a simple way of communication with a proved concept.

In an embodiment, the camera device generates an A/V stream according to a first protocol and the communication device is configured to process an A/V stream according to a second protocol. The webcam streams the A/V-data to the server in an A/V stream according to the first protocol. The server transforms the A/V-stream according to the first protocol to an AN stream according to the second protocol and streams the NM stream according to the second protocol to the communication arrangement. These features prevent that a two-way communication link is not limited to devices that use the same streaming protocol. Furthermore, these features allow using smart cameras that generate NM streams with different streaming protocols in a communication network.

In an embodiment, the stream command comprises a first parameter indicating a destination address. The webcam obtains the first parameter from the server and streams the AA/-data to an address indicated by the first parameter of the stream command. These features allow using more than one server that transforms the NM stream generated by the webcam in an NM stream that can be processed by the receiving reproducing device. The server that maintains the content of the camera command registers of the webcams in the network, monitors the load of the servers that transform NM streams and assigns when a communication link is set-up a destination address available on a server that has sufficient processing power available to transform the NM stream. These features further allow bypassing a server in case both ends of the communication link use the same protocol for both generating and processing an NM stream. It might further be possible that the NM stream is transferred entirely via a local network.

In an embodiment, the stream command comprises a second parameter indicating the protocol of the AA/-stream to be generated by the webcam associated with the web command register. The webcam obtains the second parameter and generates an AA/-stream according to a protocol indicated by the second parameter of the stream command. In case a webcam can generate an NM stream according to different protocols, these features allow instructing the webcam to generate an NM stream according to a protocol that could be processed by the communication arrangement receiving the NM stream.

In an embodiment, the stream command comprises a third parameter indicating a desired bitrate of the AA/-stream to be generated by the webcam. The webcam obtains the third parameter from the server and generates an AA/-stream having a bitrate in accordance of the third parameter. These features allow adapting the bitrate to the traffic load on the network. A high traffic load could result in unacceptable delays to transmit the AN stream over the communication link. By reducing the bitrate, the image quality will degrade but the delay time over the communication link will shorten.

In an embodiment, the set-top box polls the server for an indication that the communication arrangement sends an accept message to the server. In an embodiment, the set-top box polls the server for an indication that there is an incoming call from a communication arrangement. In response to the indication, the set-top box processes data from the server related to the incoming call to be displayed on the display device at the first end. In a further embodiment, the settop box sends an accept message to the server to establish an AA/-connection between the first end and the second end. In response to the accept message the server sets values in the camera command register indicating that the webcam has to start streaming AA/-data. These embodiments have the advantage that the server can be located outside of the internet service provider (ISP) network without increasing the risk that the ISP network will be hacked via the port used by the settop box. There is only outbound initiated communication via the registers at the server and there is no risky inbound initiated communication that could change the status of a set-top box or webcam.

In an embodiment, the set-top box polls the server for a video data source address. The server controls the process of forwarding an AV-stream from the communication arrangement to the video data source address. The set-top box retrieves the AV-stream from the communication arrangement via the video data source address. These features allow increasing the flexibility of using the available sources and servers in the network. For example, it might be possible that the set-top box receives the AN stream directly from the camera device at the other end. In that case the AN stream does not pass through a server that transforms AN streams.

In an embodiment, the set-top box or communication arrangement sends a termination request to the server to terminate the communication link. In response to the termination request, the server stores a termination indication in the camera command register. The webcam goes to an idle state after retrieving the termination indication stored in the camera command register from the server and stops generating the A/V stream.

According to a second aspect, a communication system is provided for two-way A/V-communication over an IP-network between a first user at a first end and a second user at a second end arranged to perform any of the methods described above.

According to a third aspect, a webcam is provided comprising a processor, an Input/Output device to connect to a network system and a data storage comprising instructions, which when executed by the processor cause the computer implemented system to perform the actions of a webcam in any of the methods described above.

According to a fourth aspect, a set-top box is provided comprising a processor, an Input/Output device to connect to a network system and a data storage comprising instructions, which when executed by the processor cause the computer implemented system to perform the actions of a set-top box in any of the methods described above.

According to a fifth aspect, there is provided computer program product comprising instructions that can be loaded by a computer arrangement, causing said computer arrangement to perform all actions of a set-top box in any of the methods described above. A processor readable medium can be provided with the computer program product.

Other features and advantages will become apparent from the following detailed description, taken in conjunction with the accompanying drawings which illustrate, by way of example, various features of embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects, properties and advantages will be explained hereinafter based on the following description with reference to the drawings, wherein like reference numerals denote like or comparable parts, and in which:

Fig. 1 illustrates the components and connectivity in a two-way communication system;

Fig. 2 illustrates an example of an A/V connection using a set-top box and webcam at each end;

Fig. 3 illustrates a time-diagram of setting up an A/V connection using a set-top box and webcam at each end;

Fig. 4 is a block diagram illustrating a server;

Fig. 5 is a block diagram illustrating a set-top box; and,

Fig. 6 is a block diagram illustrating a smart webcam.

DETAILED DESCRIPTION

Prior to describing exemplary embodiments in detail, terms to be used herein will be defined.

The term "webcam" is a digital camera designed to take digital photographs and transmit them over the internet. An IP camera is an example of a webcam.

The term “smart TV” relates to a television set with integrated Internet and “social network” features.

Fig. 1 illustrates the components and connectivity in a two-way AN-communication system according to the present application. The system is controlled by a server 100. In Fig. 1 one server block is shown, however the block may represent a cluster of servers, each performing a particular function in the two-way communication system. A devices for recording audio and video signals and generating an A/V-stream could be and is not limited to a webcam 10, camera of a personal computer 82 (laptop camera or camera connected to a desktop computer), camera of a tablet 81, camera of a smartphone 80, a camera build in a smart TV, and could be a combination of a USB camera and dedicated electronics to connect the USB camera to the internet. Devices for reproducing audio and video signals from an A/V-stream could be a set-top box 30 combined with a TV (not shown), a smart TV 35, a smartphone 80, a tablet 81 and a personal computer 82.

All devices communicate via the internet with a call manager running on the server 100. For the control (login, getting contact lists, activating/deactivating cameras), HTTP queries to the server are used. The reason for this is that most of the existing networks do not block HTTP traffic, and a proxy is easy to develop/add into a system that actually does that sort of blocking. All communication is done from the devices (cameras and reproducing devices) to the call manager of the server. This has the advantage that the server does not have to know the hierarchy of the hidden private network to contact a recording device or a reproducing device.

For streaming AA/-data over the internet a variety of protocols are available. For example, iOS uses the QuickTime based formats (e.g. Real-Time Steaming Protocol (RTSP), and/or HLS (HTTP Live Streaming). However, iOS does not support WebRTC (Web Real-Time Communication). Systems running Android support MPEG-DASH (MPEG Dynamic Adaptive Streaming over HTTP) and/or WebRTC. Personal computers support WebRTC on Chrome and Firefox browsers. Most smart-TVs support RTSP and some WebRTC. RTSP is the only available low latency video format in most set-top boxes. Adobe AIR supports the RTMP (Real Time Messaging Protocol) for both sending and receiving A/V data. Adobe AIR is available on iOS, Android and PC systems. There is currently no A/V streaming protocol that could be used on all recording and reproducing devices. IP cameras usually support both MJPEG (Motion JPEG), streamed over HTTP, as well as H.264 streamed over the RTSP-protocol. An IP Decoder module of an IP camera supports streams using the RTSP or RTMP protocols. In an alternative embodiment of a webcam a specific protocol is used. In this protocol a MPEG-2 transport stream is used on top of a UDP (User Datagram Protocol) unicast, abbreviated as UDP-TS. However it might also be MPEG2 over TCP or RTMP. An advantage of UDP is that it uses a simple connectionless transmission model with a minimum of protocol mechanism. UDP has no handshaking dialogues. With UDP, the webcam can send messages, in this case referred to as datagrams, to other hosts on an Internet Protocol (IP) network without prior communications to set up special transmission channels or data paths. For streaming audio, preferably the compression codec AAC (Advanced Audio Coding) is used. However, other audio codecs might be used.

The major difference between the protocols is not the actual video payload (which is H264 for RTMP/RTSP, and VP8 for WebRTC), but the actual container and negotiating layer. For this reason, the two-way communication system comprises a server that allows video to come in, in either UDP-TS/RTMP/RTMP and to go out in any of these formats, through the use of a stream handler module. In Fig. 1, is only shown that the webcam 10 records AN data and transmit the AN data in a stream (UDP Video (UP)) to the server 100. It might be clear from the description above, that the webcam might transmit the AN data as MPEG2 over TCP or RTMP. Similarly, the AN data recorded by the camera of the smartphone 80, tablet 81 and personal computer 82 might be transmitted as AN stream (UDP-TS/RTMP/RTSP) to the server depending on the implementation.

Normally, a set-top box 30 or a smart-TV 35 with build-in set-top box communicate via a private network 75 with the services running on a Middleware server of an ISP (Internet Service Provider). A proxy server of the ISP facilitates the set-top box or smart-TV access to content on the World Wide Web and provides anonymity.

According to the present application the streaming of AN data from a recording device to a reproducing device can be as follows. The recording device transmits the generated AN stream according to a first protocol to a destination address provided by the server. In case the reproducing device requires an A/V stream according to a second protocol which differs from the first protocol, the destination address is an input address of the stream handler of the server 100. The stream handler forwards the AN stream according to the first protocol to at least one of the following servers: RTMP server, RTSP server and UDP server. The RTMP server, RTSP server and UDP server transform the AN stream according to the first protocol in respectively an AN stream according to RTSP-, RTMP- and UDP-TS protocol. Depending on the implementation, the server generates only an AA/-steam with a protocol that could be processed by the reproducing device. However, in case of a conference call wherein the AN content is reproduced at several reproducing devices, the server 100 transforms the incoming A/V stream into A/V streams in all possible protocols. The ΑΛ/ streams could be retrieved by the reproducing device via video data source addresses, one for each protocol, from the server 100. The reproducing device obtains the video data source address for the desired ΑΛ/ stream from the server 100. The server 100 further comprises a call manager unit. The call manager unit handles call requests, provides contact lists, informs the called device that there is an incoming call and informs the calling device that the call request is accepted. After a call is accepted, the call manager provides the destination addresses to the recording devices and the video data source addresses to the reproducing devices. This will be described in more detail below.

Fig. 2 illustrates an example of an ΑΛ/ connection using a set-top box and webcam at each end of a communication link. At a first end A are present a webcam A10, a display device A20, e.g. television, a set-top box A30, a modem A40 and a remote control A50 to control the set-top box. The webcam A10 is connected via modem A40 to the network A45 of an ISP. The modem could comprise a firewall. The network A45 of the ISP is coupled via the internet 70 to server 100. The display device A20 is connected to set-top box A30. set-top box A30 is connected via the network A60 of an ISP and the internet 70 to the server 100. The remote control A50 enables a user at the first end A to control the settop box. At a second end B are a webcam B10, a display device B20, e.g. television, a set-top box B30, a modem B40 and a remote control B50 to control the set-top box. The webcam B10 is connected via modem B40 to the network B45 of an ISP. The modem could comprise a firewall. The network B45 of the ISP is coupled via the internet 70 to server 100. The display device B20 is connected to set-top box B30. set-top box B30 is connected via the network A60 of an ISP and the internet 70 to the server 100. The remote control B50 enables a user at the second end B to control the set-top box. The networks of the ISPs A45, A60 might be different networks or the same network. The connections between the components could be wired or wireless. The set-top boxes A30 and B30 are connected to the private network of the operator (ISP), with outbound access to the internet. The webcams A10, B10 are connected to a local home network, with internet access. At the server 100, webcam A10 is logically linked to set-top box A30. Similarly, webcam B10 is logically linked to set-top box B30. A set comprising a webcam or camera device with internet access, a set-top box and TV forms a communication arrangement. The three components could be integrated into one device.

It should be noted that an HTTP query from a device in a local home network results in outbound traffic. Normally by default a firewall allows outbound traffic and blocks most inbound traffic. As a result of this, a user can simply add a webcam adapted for two-way ΑΛ/ communication according to the present application to his local home network. The user does not have to change settings in his firewall to enable communication between the webcam and the server, as the network traffic generated by the webcam is outbound traffic in view of the local home network.

Fig. 3 illustrates a time-diagram of setting up an ΑΛ/ connection using a set-top box and webcam at each end of the communication link. On top of Fig. 3 are shown webcam A10 and set-top box A30 at the first end of the communication link and webcam B10 and set-top box B30 at the second end of the communication link. Furthermore, a camera command register associated with webcam A10, a set-top box command register associated with set-top box A30, the call manager, a camera command register associated with webcam B10 and a set-top box command register associated with set-top box B30 are indicated.

The webcams have dedicated hardware/software to communicate over the internet with the server. The webcams are polling the server periodically, asking if there are any commands for them. Via an HTTP-request a webcam retrieves the content of its associated camera command register. A polling interval, i.e. the time period between two subsequent requests, is customizable and is in the range of 1 - 30 seconds. A polling interval of 15 seconds has been found to give acceptable delay to setup a communication link. Similarly, the settop boxes are polling the server periodically, asking if there is an incoming call or a call has been accepted. A two-way communication starts with a user at a first end requesting the two-way communication system to set up a two-way communication link with the devices at the second end. The user uses a remote control to communicate with the set-top box. The set-top box runs a browser-like application to indicate by the user that he would like to have a two-way communication link with another user at the second end. With use of the browser-like application, a user could retrieve a list with contacts. By selecting a name from the list with contacts, the user indicates that he would like to have a two-way communication link with the devices associated with the selected name. Reference 300 refers the action that the set-top box sends a command to the server indicating that the user would like to have a call with the user at the second end. In other words, set-top box A30 at the first end sends a command to the server 100 requesting a communication link with the set-top box B30 and webcam B10 at the second end. Reference 302 refers to the action that the call manager indicates in the set-top box command register associated with the user at the second end that the user at the first end requests a communication link with the user at the second end. By polling the settop box command register, the set-top box B30 at the second end retrieves information about the caller and shows the information of the caller on the connected display device to indicate that there is a request for a communication link. Reference 304 refers to the receipt of the content of the set-top box command register at the set-top box B30 at the second end. Then, the information is shown on the display device at the second end enabling the second user to accept or refuse the call. Reference 306 corresponds to an accept message, wherein the set-top box answers the server that the user at the second end accepts the call. In response to the accept message, the server sets 308 the content of the camera command registers and the set-top box command registers of the devices that take part in the communication link. A stream command could be stored in the camera command register. The webcam polls the camera command register and detects that it should start recording ΑΛ/ data and streaming the AA/-data to a destination address. The vertical bars in Fig. 3 indicate that the associated device is active and the associated register indicates that the corresponding device should be active.

Reference 310 refers to the action that the webcam at the first end retrieves from the server the stream command stored in the camera command register. Subsequently, the webcam starts recording A/V-data and streaming the A/V-data over the communication link in the direction of the set-top box at the other end. The camera command register comprises a first parameter indicating a destination address. The destination address is an address of the stream handler in case the streaming protocol used by the webcam differs from the streaming protocol used by the set-top boxes. In that case, the server will transform the AN stream according to a first protocol from the webcam into an AA/-stream according to a second protocol which will be fetched by the set-top box. In case the streaming protocol of the webcam and the set-top box are the same, the destination address could be of any intermediate device that could receive and buffer the AA/-stream from the webcam and where the set-top box could fetch the AA/-stream. Optionally the camera command register comprises a second parameter indicating the protocol of the AN stream to be generated by the webcam. This parameter is advantageous when the webcam is capable of generating AN streams according to more than one protocol. After retrieving the second parameter, the webcam starts generating an AA/-stream according to the desired protocol. Furthermore, the camera command register optionally comprises a third parameter indicating the desired bit-rate of the A/V-stream. If the delay time is too large due to high traffic load of the network, reduction of the bitrate reduces the traffic load and consequently enables that the delay time is shortened. However, other methods are possible to adapt during streaming the bitrate of the generated AA/-stream.

Reference 312 refers to the action that the webcam at the second end retrieves the stream command stored in its associated camera command register. In response to this the webcam at the second end starts streaming AN-content to the server or intermediate device.

Reference 314 refers to the action that the set-top box at the first end retrieves, from its associated set-top box command register at the server, an indication that it could fetch an AA/-stream from a video data source address. Depending on the implementation, the presence of the video data source address could be an indication that the set-top box should start reproducing the AN content of an A/V-stream. The absence of a video data source address (for example a value NIL) could be an indication that the set-top box at the first end should stop retrieving and displaying the AN stream. Reference 316 refers to a corresponding action but now for the set-top box at the second end. The video data source address allows for load balancing, using multiple servers and even different protocols. In an implementation, the set-top box command register comprises a video data source address for each possible streaming protocol to which the server transforms an incoming AA/-stream from a webcam. For example, the first video source data address refers to an address where an A/V-stream according to the RTSP protocol could be fetched and the second video source data address refers to an address where an AA/-stream according to the RTMP-protocol could be fetched.

Reference 318 refers to the action that set-top box at the first end informs the call manager that the user at the first end wants to stop the call. This could be done with a call termination request send by set-top box A30. In response to the call termination request, the call manager stores a terminate indication in the camera command registers of webcams A and B and set-top boxes A and B. Reference 320 refers to this action.

The webcams A10 and B10 and the set-top boxes A30 and B30 continuously poll the associated registers at the server for a change. After receipt of the termination indication stored in the register, the webcams stop generating an A/V-stream and the set-top boxes stop fetching and reproducing the AN stream. References 322 and 326 refer to the action that the webcams receive the termination indication. References 324 and 328 refer to the action that the set-top boxes receive the termination indication.

For polling an open-ended HTTP request could be sent. This means that a webcam sends a request like http://server/get_command, and if there is no command, there is no response whatsoever. The webcam enters a loop and waits for an answer. This is asynchronous in the camera, so it does not upset other tasks the webcam is running, like video streaming. This means that as soon as there is a new command stored in the register, the response is sent. This mechanism significantly reduces the latency of command reaction.

It might be possible that a set-top box has a Picture in Picture capability. In that case, the set-top box command register also comprises one or more video data source addresses indicating where the set-top box can fetch the (transformed) AV-stream generated by the webcam located at the same end as the set-top box.

In another implementation, the server performs the Picture in Picture capability by combining the AA/-streams from the webcams into one AA/-stream that can be fetched at the corresponding video data source address(es). It might be clear, that the AA/-stream for set-top box at the first end differs from the ΑΛ/-stream for the set-top box at the second end.

In case the server 100 comprises a number of server units to perform all functions, each server unit performs at least one function. Some functions performed by server 100 are: calling manager, stream handling RTSP server and RTMP server. One server unit might be configured to monitor the AA/-streams coming from the webcams and camera devices and processed by the server running the stream handler. The server unit might further be configured to monitor the A/V-streams fetched by the set-top boxes and the other communication arrangements. The call manager will use the thus obtained information to set the video data source addresses in the set-top box command register as soon as an AA/-stream from a corresponding webcam or camera device is coming in. In this way, black screens can be avoided. Furthermore, the server unit might be configured to determine the latency of the ΑΛ/ data from transmittal of the webcam or camera device to receipt of the ΑΛ/ data at the reproducing device. If the latency is too high, the call manager can decide to decrease the bitrate of the ΑΛ/-stream generated by the webcam or camera device and the server or to decrease the bitrate of the AA/-stream from the server to the set-top box or communication arrangement receiving the AA/-stream. In this way, the total latency can be controlled.

Referring to Fig. 4, there is illustrated a block diagram of exemplary components of a server 1400. The server 1400 can be any type of computer having sufficient memory and computing resources to perform the actions of a server according to the methods described before. As illustrated in Fig. 4, the processing unit 1400 comprises a processor 1410, data storage 1420, an Input/Output unit 1430 and a database 1440. The I/O unit 1430 is configured to support at least all communications over the network with the recording devices and reproducing devices. The data storage 1420, which could be any suitable memory to store data, comprises instructions that, when executed by the processor 1410, cause the computer implemented system 1400 to perform the actions of the server corresponding to any of the methods described in the present application. The database 1440 could be used to store the video data command registers of the reproducing devices in the network and to store the camera command registers of the camera devices in the network.

Referring to Fig. 5, there is illustrated a block diagram of exemplary components of a set-top box. The set-top box 1500 can be any type of computing device having sufficient memory and computing resources to perform the actions of a set-top box according to the methods described before. As illustrated in Fig. 5, the processing unit 1500 comprises a processor 1510, data storage 1520 and an Input/Output unit 1530. The I/O unit 1530 is configured to support at least all communications over the network with the server. The data storage 1520, which could be any suitable memory to store data, comprises instructions that, when executed by the processor 1510, cause the set-top box 1500 to perform the actions of the set-top box corresponding to any of the methods described in the present application. The instructions include web browser functionality which forms the user interface of the set-top box.

Referring to Fig. 6, there is illustrated a block diagram of exemplary components of a smart camera having the functionality described above. The smart camera 1600 can be any type of digital camera having sufficient memory and computing resources to perform the actions of a webcam according to the methods described before. As illustrated in Fig. 6, the processing unit 1600 comprises a processor 1610, data storage 1620, an Input/Output unit 1530, an image sensor 1640 and a microphone 1650. The I/O unit 1630 is configured to support at least all communications over the network with the server. The data storage 1620, which could be any suitable memory to store data, comprises instructions that, when executed by the processor 1610, cause the smart camera 1600 to perform the actions of the webcam corresponding to any of the methods described in the present application.

The method could be embodied as a computer program product comprising instructions that can be loaded by a computer arrangement, such as a set-top box, tablet, smartphone and smart TV. The instructions cause said computer arrangement to perform any of the methods of a set-top box described above. The computer program product could be stored on a computer readable medium. The computer program product can be in the form of an app that can be downloaded from an internet store.

The two-way communication system described above could be used to provide A/V-communication between medical professionals and patients. The professional and patient can see each other which provides additional visual information of the patient to the professional. In a lot of cases, a call with a two-way communication system can replace a visit in which the professional meets the patient. This reduces travel time. Furthermore, the professional can use his time more effectively and the patient can wait at home doing other things when the agreed time cannot be met. As elderly people are familiar with the user interface of a set-top box, it will not be a hurdle to use the disclosed two-way communication system.

While the invention has been described in terms of several embodiments, it is contemplated that alternatives, modifications, permutations and equivalents thereof will become apparent to those skilled in the art upon reading the specification and upon study of the drawings. The invention is not limited to the illustrated embodiments. Changes can be made without departing from the scope and spirit of the appended claims. *******

Claims

A method for two-way A / V communication between a first user at a first end (A) and a second user at a second end (B), which method comprises: - a set-top box (A30) at the first end processing AA / data recorded at the second end to be displayed by a display device (A20) at the first end; and, a communication device (B10, B20, B30) at the second end comprising a camera device (B10) and a display device (B20), AA / data processing the communication device which is transmitted by a camera device (A10 ) is registered at the first end to be displayed with the display device (B20) at the second end; characterized in that the camera device (A10) is a webcam at the first end, the method further comprising: - the webcam (A10) polls a server (100) for a camera command stored in a camera command register associated with the webcam; - the set-top box (A30) sends a command to the server (100) to request a communication connection with the communication device; - in response to the command, the server sets an indication to the communication device that there is a request for a communication connection; - the communication device polls the server for an indication that there is a request for a communication connection; - in response to input from the second user, the communication device (B10, B20, B30) sends an acceptance message to the server (100); - in response to the acceptance message, the server (100) stores a streaming command in the camera command register; and, the webcam (A10) goes to a recording state after retrieving the server from the streaming command stored in the camera command register and streams recorded AA / data via the communication link.

The method of claim 1, wherein the camera device generates an A / V stream according to a first protocol and the communication device is adapted to process an A / V stream according to a second protocol, the method further comprising: - the webcam (A10) streams the A / V data to the server in an A / V stream according to the first protocol; and the server (100) transcodes the A / V stream according to the first protocol into an A / V stream according to the second protocol and streams the A / V stream according to the second protocol to the communication device.

The method of any one of claims 1 to 2, wherein the streaming command comprises a first parameter indicating a destination address; and the method further comprises: - the webcam (A10) streams the A / V data to an address indicated by the first parameter of the streaming command.

The method of any one of claims 1 to 3, wherein the streaming command comprises a second parameter indicating the protocol of the A / V stream to be generated; and the method further comprises: - the webcam (A10) generates an A / V stream according to a protocol indicated by the second parameter of the streaming command.

The method of any one of claims 1 to 4, wherein the streaming command comprises a third parameter indicating a desired bit rate of the A / V stream generated by the webcam; and the method further comprises: - the webcam (A10) generates an A / V stream with a bit rate that is in accordance with the third parameter.

The method of any one of claims 1 to 5, wherein the method further comprises: - the set-top box (A30) polls the server (100) for an indication that the communication device has sent an acceptance message to the server.

The method of any one of claims 1 to 6, wherein the method further comprises: - the set-top box (A30) polls the server for a video data source address; - the server (100) controls that an AV stream from the communication device is forwarded to the video data source address; and, the set-top box (A30) retrieves the AV stream from the communication device via the video data source address.

The method of any one of claims 1 to 7, wherein the method further comprises: - the set-top box (A30) polls the server for an indication that there is an incoming call from a communication device; and, in response to the indication, the set-top box (A30) processes the data from the server regarding the incoming call to be displayed on the display device at the first end.

The method of claim 8, wherein the method further comprises: - the set-top box (A30) sending an acceptance message to the server to establish an AA / connection between the first end and the second end; - in response to the acceptance message, the server (100) sets values in the camera command register that indicate that the webcam (A10) should start streaming AA / data.

The method of any one of claims 1 to 9, wherein the method further comprises: the set-top box (A30) or communication device sends a termination request to the server (100) to terminate the communication connection; - in response to the termination request, the server stores a termination indication in the camera command register; and, - the webcam (A10) goes to sleep after retrieving from the server the termination indication stored in the camera command register.

The method of any one of claims 1-10, wherein the wrist actions are performed by sending HTTP requests to the server.

12. A communication system for two-way A / V communication over an IP network between a first user at a first end (A) and a second user at a second end (B), the communication system comprises a set-top box (A30 ), a display device (A20) and a webcam (A10) at the first end and a communication device (B10, B20, B30) at the second end; the communication device comprises a camera device (B10), a processing device (B30) and a display device (B20); characterized in that the communication system further comprises a server (100); - the server (100) comprises a processor (1410), an input / output device (1430) for connection to the network, a database (1440) and a data storage (1420) comprising instructions which, when executed by the processor (1410) causing the server: - in response to a command from the set-top box, to indicate to the communication device that there is a request for a communication connection in the database; storing a streaming command in a camera command register associated with the webcam in the database in response to an acceptance message from the communication device; - a set-top box (A30) at the first end comprises a processor (1510), an input / output device (1530) for connection to the network and a data storage (1520) comprising instructions which, when executed by the processor (1510) cause the set-top box to: - process AA / data registered at the second end to be displayed by the display device at the first end; and - sends a command to the server for requesting a communication link with the communication device at the second end; - a webcam (A10) at the first end comprises a processor (1610), an input / output device (1630) for connection to the network system and a data storage (1620) comprising instructions which, when executed by the processor (1610), causing the webcam to: - poll the server (100) for a camera command stored on the server in a camera command register associated with the webcam; - goes to a recording state after retrieving the server from the streaming command in the camera command register and recorded A / V data flows via the communication link to the communication device.

A webcam (A10) comprising a processor (1610), an input / output device (1630) for connection to a network system and a data storage (1620) comprising instructions which, when executed by the processor (1610), cause the webcam: - polls a server (100) for a camera command stored on the server in a camera command register associated with the webcam; - goes to a recording state after retrieving the server from the streaming command in the camera command register and streams recorded A / V data via a communication link.

The webcam of claim 13, wherein the streaming command comprises a first parameter indicating a destination address; and the instructions further cause the webcam to obtain the first parameter via the streaming command and to stream the A / V data to an address indicated by the first parameter of the streaming command.

The webcam of claim 13 or 14, wherein the streaming command comprises a second parameter indicating a protocol of the A / V stream to be generated; and the instructions further cause the webcam to obtain the second parameter of the streaming command and generate an A / V current according to a protocol indicated by the second parameter of the streaming command.

The webcam according to any of claims 13-15, wherein the streaming command comprises a third parameter indicating a desired bit rate of the A / V stream to be generated by the webcam; and the instructions further cause the webcam to obtain the third parameter of the streaming command and generate an A / V stream with a bit rate that is in accordance with the third parameter.

A computer program comprising instructions that can be loaded by a computer device that cause the computer device to perform all actions of a set-top box in any of the methods of claims 1-11.

A processor-readable medium provided with a computer program comprising instructions that can be loaded by a computer device that cause said computer device to perform all actions of a set-top box in any of the methods claims 1-11. *******