NO20150189A1 - Methods of upgrading professional digital display monitors for use as full endpoints and systems for video conferencing and telepresence without the use of a separate communication device - Google Patents

Description

Technical area

The invention relates to methods for upgrading professional digital signage screens so that they can also be used as full-fledged systems for video conferencing and telepresence, among other things for meeting rooms and teaching, where the connection of one or more video conference cameras is the only necessary additional equipment so that an almost unlimited number of virtual meeting participants can call in from all types of machine devices, and all types of communication protocols, without the connection of additional communication devices being necessary.

Background

While communication protocols and platforms have changed significantly, the actual structure of video conferencing systems for meeting rooms has remained practically unchanged since the 1990s. For large meeting rooms, separate units (codecs) are used for coding and decoding of video, audio and data before and after transmission over the Internet or virtual private networks. The codecs are mounted in a cabinet under or behind one or more large screens, which together with one or more cameras and speakers form part of the system. Operation takes place with a separate remote control with operating buttons or touch screen.

For medium-sized and smaller meeting rooms, the "Set Top" systems introduced in the late 1990s are still popular. The solution consists in the video camera and codec being built together in a box that is placed on top of a standard large screen that the customer acquires separately for connecting the Set Top and the device with a cable. A remote control with operating buttons is included in the system.

For organizations that want to save money, it is common to install a permanent PC in the meeting room and download a software client. A screen, a standard video conference camera and standard keyboard with mouse are connected with a USB cable, where the latter is used to operate the system.

The analysis company Wainhouse Research, Boston specializes in following the video conferencing market worldwide, and publishes comprehensive reports every year on trends and developments in the video conferencing market with statements and wishes from users and user groups. The reports show that the number of installed meeting room systems showed a sharp decline from 2010 to 2013, when they began to increase again. This coincides with a sharp increase in portable devices used for visual meetings, and around 70% of the respondents in the Wainhouse study believe that the increased prevalence of visual applications on mobile devices spills over into meeting rooms. Many mention that the meeting room solutions should be made simpler and more user-friendly, that loose cables in meeting rooms are problematic and that the systems must be better adapted to be used by many different people with varying interests and skills to operate a video conference system.

In the present invention, this is the basis.

US 20120293602 (application of 16 February 2012) describes a telepresence system with unified communication for audio and video conferences in an Entreprise Network (intranet) system. The system consists of a separate control system and user interface that is connected to one or more separate display screens, i.e. a conventional structure of a video conference system as described above. An executive telepresence system for use in audio and video conferencing uses a unified communications

system in an enterprise network system. The executive telepresence system includes a control system that further comprises a user interface, a mixing system, and a dialing system. In addition, the executive telepresence system includes one or more video display devices that couple to the control system where the video display devices display a video signal, and may further include one or more video cameras that couple to the control system where the video cameras capture video information and turn it into the video signal. Further, the executive telepresence system includes one or more microphones that couple to the control system where the microphones capture audio information and turn it into an audio signal, and may include one or more speakers that couple to the control system where the speakers play back the audio signal.

WO 2013056756 (application) describes a method for providing visual information about the participants in a video conference and contains requirements in connection with the identification of the meeting participants, a method for displaying visual information about participants in a teleconference comprises mixing of audio signals originating from participants in the teleconference , providing an automatic identification of a participant currently speaking and displaying at least one static digital image associated with the identified participant currently speaking at least during part of the time while this participant is speaking.

KR 20130030066 describes a system to remedy video conferences in remote locations. An image conference system is provided to efficiently progress a conference in a remote place by adding microphone composition to a remote

US 2014354761 describes methods for users of computers, tablets and smartphones to enable "pairing" of their devices against applications for such devices. According to a general embodiment of the present invention, a computer, tablet or smart phone executable application is provided, allowing a user of the application to activate paihng of the dé*i"se;.and;ai*ideQiCoMerénfe séssiorirfbmiVid"^

US 2014/0241216 includes a portable system with concrete functions, which may be to organize contact persons. A portable communication, comprises a control unit, a display coupled to the control unit; a contact book coupled to the control unit to store a list of contact person's phone number, e-mail address, account information, or the combination US 2014/184725 Shows a telephone with a video function and projection, network, video-audio capture and conferencing module and a processor configured to capture an image and the voice of a local user to generate local data. A telephone with a video function is provided, including a projection module, a network module, a video-audio capturing module, a conference module and a processor. The network module provides a network interface. The video-audio capturing module is configured to capture an image and voice of a local user, and accordingly generates local video-audio data. The conference module provides the video conference function. The processor is electrically connected to the projection module, the network module, the video-audio capturing module and the conference module. When the processor enables the video conference function of the conference module, the telephone establishes a connection with a remote apparatus through the network interface, in order to receive a remote video-audio data of a remote user from the remote apparatus, and transmit the local video-audio data to the remote apparatus. The projection module projects an image corresponding to the remote video-audio data onto a projection surface.

US 2014/0092199 shows a television with a video function and method for video conferencing using a telephone, a telephone with a video function is provided, including a projection module, a network module, a video-audio capturing module, a conference module and a processor . The network module provides a network interface. The video-audio capturing module is configured to capture an image and voice of a local user, and accordingly generates local video-audio data. The conference module provides the video conference function.

US 2014/0176664 Shows a projector with a function for video conferencing and a method for video conferencing using an image projector. A projection apparatus includes a projection module, a network module, a video-audio capturing module, a conference module and a processor. The network module provides a network interface. The video-audio capturing module is configured to capture an image and voice of a local user, and accordingly generates local video-audio data. The conference module provides the video conference function. The processor is electrically connected to the projection module, the network module, the video-audio capturing module and the conference module.

US 2002/0071026 Shows an apparatus for video conferencing between several networks where a processor assesses the user platform of a meeting participant and obtains from a database the necessary prerequisites for participation in the video meeting to take place. The present invention provides an apparatus and method for conducting a video conference. The video conference apparatus is connected to at least one network. The video conference apparatus includes an MCU, an environment processor, a user database interface and an environment database interface. When users log onto the video conference apparatus, it is determined whether each user has designated an alternative environment from that normally detected by the camera device during the video conference. If the user has designated an alternative environment, the environment processor obtains the environment from the environment database and the video conference apparatus uses the designated environment during the video conference.

US 2008/0088693 Method for content transmission and device that uses video conferencing and that supports the transmission of content between a majority of video devices connected to the network, and in particular in relation to the transmission of content as well as video and audio signals in a VolP-based video conferencing device. A content transmission method and apparatus using video calls is provided. The content transmission method includes performing an encoding operation on reproduced content; and transmitting the encoded content to at least one of video call apparatuses connected to a network, wherein the content is distinguished from video and the voice data. The content is reproduced by the content transmission apparatus or an external content providing server. Therefore, the content transmission apparatus can share content with another type of apparatus that supports VolP, such as a digital signage. US 2013/0027504 concerns the preview of video data for a video conference, in one exampie implementation, a request from a first endpoint is identified for a second endpoint to join a video communication session. Particular video data captured at the first endpoint is displayed at the second endpoint prior to the second endpoint joining the video communication session. In certain embodiments, the particular video data captured at the first endpoint can be displayed at the second endpoint without the user of the first endpoint being made aware that the particular video data was displayed at the second endpoint.

None of the above-mentioned patents, applications or products launched on the market touch the same area as the present invention.

Summary

The invention relates to methods for upgrading professional digital signage screens so that they can also be used as full-fledged systems for video conferencing and telepresence, among other things for meeting rooms and teaching, where the connection of one or more video conference cameras is the only additional equipment necessary so that an almost unlimited number of virtual meeting participants can call in from all types of machine devices, and all types of communication protocols. No connection of additional communication devices is required.

The invention relates to methods and a system as stated in the claims.

Brief description of the drawings

In order to make the invention easier to understand, the following discussion will refer to the accompanying drawings, in which

Figure 1 illustrates the layout of a System on Chip (SoC) for a digital display screen that can also function as an endpoint for professional video conferencing / telepresence. Figure 2 illustrates the parts of a digital display screen upgraded to professional video conferencing / telepresence. Figure 3 shows an arrangement of the functionality in the display device, online cloud and video-as-a-service provider. Figure 4 shows the interoperability possibilities for a fully upgraded screen connected to the internet or a virtual private network.

Detailed description

In the following, the present invention is reviewed by describing preferred embodiments together with reference to the attached drawings. Those skilled in the art will be able to find other applications and modifications within the scope of the invention as defined in the independent claims.

The purpose of the invention is to configure and upgrade digital display screens in production, so that they can function as both conventional digital display screens or independent, professional point-to-point video conferencing systems and endpoints for advanced video conferencing in the cloud by only connecting to one or more video conference cameras, without connecting external devices or boxes. The choice of what the screens are to be used for therefore takes place after the screens leave the production site.

If a user wants to use their screen for video conferencing, the area of use will include the following main functions: i) Point-to-point HD video conferencing with a minimum of 720p resolution (HD) and at least 30 fps (frames per second) against any type of video conferencing system, PC, Mac or Android-based device,

ii) Multi-party video conferences in HD, where an unlimited number of participants see and hear

each other at the same time,

iii) Point-to-point and multi-party video conferences (such as i) and ii)) above between participants using different types of equipment and different communication platforms, such as a. Conventional video conference systems available on the market, both H.323 and

SIP-based,

b. PC, Mac and Android portable devices (full video, audio and

interaction functionality),

c. Microsoft Lyne (full video, audio and interaction functionality),

d. Microsoft SKYPE (full video, audio and interaction functionality),

e. Landlines (audio only),

f. Smartphones (full video, audio and interaction functionality)

iv) Standard HD (720 and 1080p) video conference cameras for USB or other standard

connection can be connected,

v) The invention enables the screens, when connected to the network, to automatically seek connection to preferred networks, including through a video client with routing options,

vi) Additional functions for interaction in video meetings and teaching, both local and remote, including where people can transfer content from their handhelds

devices to the big screen in the central room, where it can be edited and sent back to the participants in the local meeting or classroom and to the participants over the web.

vii) Transmission of a video call that is in progress on a handheld device of the type PC, Mac,

Android or other, to the system screen, camera and audio without being interrupted

viii) Other functions not mentioned or invented will be able to be upgraded in software over the Internet.

For a person skilled in the art who will practice the invention, it is valuable to note that Digital Signage screens are designed to receive information over the Internet to be displayed locally on the screen (in hotel receptions, at airports in shops and the like). Therefore, digital display screens are designed to be left on 24/7 without being damaged, and they can be remotely controlled, configured and upgraded over the network with RS23 or other interfaces. These monitors also have relatively powerful processors built in, which run Linux or Pro Linux operating systems, which makes them suitable for the necessary coding for professional video conferencing. They also support HTML5, a language for structuring and presenting content on the Word Wide Web, and they can be upgraded to "touch screen" function by plugging in a touch plate on one of the available USB connectors.

On the other hand, digital display screens do not support two-way communication with a video camera, which is a basic feature of a video conferencing system.

The digital display displays the System on Chip / SoC in its processing part, and the practice of the invention places the following requirements on it:

a) Structure as illustrated in Figure 1

b) Support for the necessary codecs and standards for both video and audio

c) Sufficient processing power

d) Support for camera

Figure 1 shows the layout of the SoC that will work for both advanced video conferencing and digital signage.

Necessary standards and codecs for video and audio that must be supported are:

H.264/AVC which is a modern standard for video compression identical to MPEG-4 Part 10. It must be supported in both 720p and 1080p and at least 30 fps (frames per second). Current audio codecs such as OGG Vorpis and OPUS should be used. WebRTC uses the two audio codecs G.711 and OPUS. SIP and H.323 based videoconferencing most often use G.711 (telephone audio), and one of either G722, G.722.1, G.722.1C, Siren or AAC-LD should be used to achieve the best possible sound.

Audio codecs are characterized by the fact that they are not particularly resource-intensive for CPU power and therefore these can and should be implemented in the system's software to enable and simplify later upgrading over the network after the systems have been put into use by the customer.

In order for the invention to be implemented, the SoC must contain a hardware-accelerated VP8 encoder/decoder because it works interactively with the browsers Google Chrome and/or Mozilla Firefox, which in turn support the cloud-based functions that the invention must be able to perform. The invention can be implemented in two ways: 1. The WebRTC specifications in http://ww.w3org/TRwebrtc/ are fully implemented in the SoC, so that the video signals from the camera can be encoded in the display unit (SoC). WebRTC corresponds to H.264/5.

WebRTC supports browser-to-browser applications voice calling video chat and Point-to-Point file sharing without having to use either internal or external plug-ins.

2. Google Chrome is supported by the display platform in the display unit's SoC.

A professional for digital display screens will be able to deal with this and relate the information to the production of own systems.

In order for a high-quality HD camera with PTZ (Pan Tilt Zoom) with USB connection to an advanced video conferencing system to work together with the SoC, it must support two-way communication between the software in the system and the camera as follows:

• Take in and process the video signals from the camera and audio from the system's microphone

• Encode video and audio in a known format, such as VP8 or H.264, either in the hardware or the software - preferably in the software because this simplifies remote upgrades of the systems over time at the user site,

• Send encoded video and audio over the network

• Receive encoded video and audio from the network

• Decode video and audio

• Show the video on the screen and play back the audio in the speakers

Claims (1)

Requirements 1. Methods for upgrading professional digital signage screens (hereafter Methods) to also be used as a full-fledged and professional video conference and Telepresence system, where a video conference camera that is connected via a USB plug or other interface is the only necessary additional equipment to be able to function in Point-to-point and multi-party video conferences, where an unlimited number of participants see and hear each other at the same time, requirement 2. Methods also include that the participants can use different types of equipment, such as: a. Conventional video conference systems with full video, audio and interaction functionality , b. PC, Mac, Android and other desktop and portable devices with audio only, full video and audio and full interaction functionality, c. Soft clients, d. Microsoft Lyne (now called "Skype for Business") with full video, audio and interaction functionality, e. Microsoft SKYPE with full video, audio and interaction functionality, f. Landlines with audio only, g. Sma rtphones that can participate with full video, audio and interaction functionality. and any communication platform, such as: i) H.323, HTML5 and SIP, ii) SVC, H.264, H.263, VP8, H.265 iii) G.7xx, OPUS, AAC Requirement 3. Methods include the possibility that the screens, when connected to the Internet or virtual networks, by cable or wireless, can automatically seek connection to preferred networks, both through a video client with routing capabilities, or in some other way. Requirement 4. Methods include additional functions for interaction in video meetings and teaching, both local and remote, so that people can transfer and share content from handheld devices (PCs, tablets and smart phones) for display on the big screen, and back to such handheld devices, both locally in a meeting or classroom and over the Internet. Claim 5. Methods include the transmission of a video call that is in progress on a handheld device of the type PC, Mac, Android or other, to a large screen with a camera and other additional equipment, without the video call being interrupted. Claim 6. Methods include related additional functions that can be upgraded in software over the Internet, even if these are not mentioned above. Summary Methods for upgrading professional digital signage screens, where video conference cameras connected via USB plug or other connection are the only necessary additional equipment so that digital signage screens and others can also be used as full-fledged systems for video conferencing and Telepresence, both "point-to-point " and in multi-user meetings, with participants regardless of the type of system or communication platform used.