JP2005516557A - Video conferencing system and operation method - Google Patents

Abstract

A method for relaying a video image in a multimedia video conference between a plurality of multimedia user devices (550, 560, 570, 580) is provided by a certain number of multi-user devices. Transmitting a layered video image by a media user equipment, the layered video image comprising a base layer (552, 562, 572, 582) and one or more enhancement layers (555, 565). , 575, 585). The transmitted layered video image is received at the multipoint controller (520), where there is a certain number of base layer video images and the most active of a certain number of active speakers (535). One or more enhancement layers (540) of the middle speakers are selected. The multi-point controller (520) includes a base layer video image and one or more enhancement layers (540) of the most active speakers in a plurality of multimedia user devices (550, 560, 570). , 580) to one or more multimedia user devices. Instead of a single full quality video stream, it becomes possible to share one available bandwidth and send one enhancement layer and several base layers so that speaker identification Greatly improved compared to video conferencing system.

Description

[Field of the Invention]
The present invention relates to video conferencing. The present invention uses layered video coding and uses H.264. The present invention is applicable to a video switching mechanism in a H.323 and / or SIP-based centralized video conference, but is not limited thereto.

[Background of the invention]
As the pace of business accelerates and relationships expand around the world, the need to quickly and economically fill communication distances has become a major challenge. Efficiently bringing customers and staff together is essential to succeed in an increasingly more competitive market. Businesses are looking for flexible solutions that support sharing real-time information across countries and continents using a variety of communication methods such as voice, video, image data, and any combination thereof.

  In particular, multinational organizations are increasingly wishing to eliminate costly movements and links in multiple locations in order to communicate more efficiently and effectively to groups within the organization. Multipoint conferencing systems that operate over Internet Protocol (IP) networks strive to address this need. In the field of the invention, it is known that terminal devices exchange audio and video streams in real time in a multipoint video conference. A conventional method for setting up a multipoint conference via an IP network is a method using a multipoint control unit (MCU). An MCU is an endpoint on the network that provides the capability for three or more terminal devices and / or communication gateways to participate in a multipoint conference. The MCU also connects two end devices in a point-to-point conference, so that the point-to-point conference has the ability to evolve into a multipoint conference.

  Referring initially to FIG. 1, a known centralized conference model (hereinafter referred to as a “centralized conference model”) 100 is shown. Centralized conferences utilize MCU-based conference bridges. All terminal devices (endpoints) 120, 122, 125 send media information in the form of audio, video and / or data signals and a control information stream 140 to and from the MCU 110. These transmissions are performed in a point-to-point manner. This is illustrated in FIG.

  The MCU 110 consists of a multipoint controller (MC) and zero or more multipoint processors (MP). The MC handles call setup and call signaling negotiations between all terminal devices to determine common capabilities for audio and video processing. MCU 110 does not process any media stream directly. This is up to the MP, which mixes, switches and processes audio, video and / or data bits.

  In this way, the MCU provides the ability to host multi-location (same at multiple points) seminars, sales conferences, group conferences, and other “face-to-face” communications. . In addition, it is known that the multipoint conference can be used for various applications.

  (I) Executives and managers in multiple locations can meet “face-to-face”, share real-time information, and make faster decisions without any loss of time, expense and travel needs. .

  (Ii) Project teams and knowledge workers can coordinate their work and view and revise shared documents, presentations, designs and files in real time.

(Iii) Students, trainees and employees at remote locations can access shared education / training resources across any distance or time zone.
As a result, MCU-based systems are expected to play an important role in the future in multimedia communications over IP-based networks.

Such multimedia communications often employ video transmission. In such transmission, a sequence of images (often called a frame) is transmitted between a transmitting device and a receiving device. The multipoint multimedia conference system is, for example, H.264. It can be set using various methods, as specified by H.323 and SIP Session Layer Protocol standards. SIP reference materials
http://www.ietf.org/rfc/rfc2543.txt, and
http://www.cs.columbia.edu/~hgs/sip
Can be found in

  Further, for example, ITU H.264. H.263 video compression [ITU-T recommendation, H.264 263, “Video Coding for Low Bit Rate Communication”], the first frame of the video sequence is a large amount of image code, commonly referred to as intra coded information. Contains data. An intra-coded frame gives a substantial portion of the image to be displayed because it is the first frame. This intra-coded frame is followed by inter-coded (predicted) information, and the inter-coded (predicted) information is generally transmitted. Contains data related to image changes in it. Thus, the predicted and intercoded information contains much less information than the intra-coded information.

  In conventional multimedia conferencing systems, users need to identify themselves when they speak, so that the receiving terminal knows who is talking. Obviously, if the sending terminal fails to identify itself, the listening user must guess who is talking.

  Known techniques solve this problem by analyzing the audio stream and then sending the name of the active speaker and the video stream to all participants. In centralized conference systems, MCUs often perform this function. The MCU can then send the speaker's name and corresponding video and audio streams to all participants by switching the appropriate input multimedia stream to the output port / path.

Video switching is a well-known technique aimed at providing a single video stream to each endpoint, and is equivalent to arranging multiple point-to-point sessions. Video switching
(I) voice activated switching, in which the MCU transmits the video of the active speaker;
(Ii) timed and activated switching, where each participant's video is transmitted one by one at a predetermined time interval;
(Iii) Individual video selection switching, where each endpoint can request a video stream of the participant that the participant wishes to receive. Can be.

  Referring now to FIG. 2, a block diagram of a conventional video switching mechanism 200 is shown. In a conventional centralized conference system, video switching is performed as follows. For example, MCU 220 located within Internet Protocol (IP) based network 210 includes a switch 230. The MCU 220 receives the video streams 255, 265, 275, 285 of all participants (user devices) 250, 260, 270, 280. The MCU may also receive a combined (multiplexed) audio stream 290 separately from the speaking participant. MCU 220 then selects one of those video streams and sends this video stream 240 to all participants 250, 260, 270, 280.

  Such conventional systems have the disadvantage of transmitting only the video stream of the active speaker. Users still have problems identifying video stream speakers when several speakers are speaking at the same time or when active speakers are constantly changing. This is especially the case for large video conferences.

Alternatively, each participant's video can be sent to all participants. However, this approach is exacerbated with wireless-based conferencing due to bandwidth limitations.
In the field of video technology, it is known that video is transmitted as a series of still images / pictures. Since the quality of the video signal may be adversely affected during the encoding or compression of the video signal, additional information “layers” based on the difference between the video signal and the encoded video bit stream. Is known to include. By including additional layers, it is possible to enhance the quality of the received signal followed by decoding and / or reconstruction. Thus, a layered video bit stream is generated using a hierarchical structure of pictures and augmented pictures partitioned into one or more layers.

In a layered (scalable) video bitstream, enhancements to the video signal can be added to the base layer by either:
(I) increase picture resolution (spatial scalability); or (ii) include error information to improve the signal-to-noise ratio (SNR scalability) of the picture; or (iii) include extra pictures Increase the rate (temporal scalability).

  Such enhancement can be applied to the entire picture or to any shaped object in the picture, which is referred to as object-based scalability. To preserve the disposable nature of the temporal enhancement layer, The H.263 standard requires that the pictures included in the temporal scalability mode should be bi-predicted (B) pictures as shown in the video stream of FIG.

FIG. 3 shows a schematic diagram of a scalable video structure 300 that illustrates B picture prediction dependencies, as is known in the video coding art. The initial intra-coded frame (I ₁ ) 310 is followed by a bi-predicted frame (B ₂ ) 320. This is then followed by a (unidirectional) predicted frame (P ₃ ) 330 and again a second bidirectional predicted frame (B ₄ ) 340. This is then followed again by a (unidirectional) predicted frame (P ₅ ) 350, and so on.

  FIG. 4 is a schematic diagram of a layered video structure as known in the field of video coding technology. The layered video bit stream includes a base layer 405 and one or more enhancement layers 435.

  The base layer (layer 1) includes one or more intra-coded pictures (I pictures) sampled, coded and / or compressed from the original video signal picture. In addition, the base layer will include a plurality of predicted intra-coded pictures (P pictures) predicted from the intra-coded picture (s) 410.

In the enhancement layer (layer 2 or 3 or 4 or more) 435, three types of pictures may be used. That is,
(I) bidirectionally predicted (B) picture (not shown),
(Ii) enhanced intra (EI) picture 440 based on the base layer 405 intra-coded picture (s),
(Iii) Enhanced predicted (EP) pictures 450, 460 based on base layer 405 intra-coded predicted pictures 420, 430.

A vertical arrow from the lower layer indicates that a picture in the enhancement layer is predicted from a reconstructed approximation of that picture in the reference (lower) layer.
In summary, scalable video coding has been used only in multicast multimedia conferencing and in the context of point-to-point or multicast video communications. However, wireless networks currently do not support multicasting. Furthermore, in the case of multicasting, each layer is sent in a separate multicast session, in which case the receiver itself decides whether to record one or more sessions.

  Accordingly, there is a need for an improved video conferencing configuration and method of operation that can ameliorate the aforementioned drawbacks.

[Statement of invention]
A method for relaying a video image in a multimedia videoconference according to claim 1, according to the present invention, a videoconferencing device for relaying a video image according to claim 7, and a video according to claim 11. A wireless device participating in a conference, a multipoint processor according to claim 12, a video communication system according to claim 16, a media resource function device according to claim 18, and a media resource function device according to claim 19 or 20. A video communication device and a storage medium according to claim 23 are provided. Further aspects of the invention are described in the so-called dependent claims.

In summary, the inventive concept of the present invention addresses the shortcomings of the prior art arrangements by providing a video switching method to improve the identification of participants and speakers in a video conference. The present invention utilizes layered video coding in order to better use the bandwidth available to each user.
Exemplary embodiments of the invention will now be described with reference to the accompanying drawings.

[Description of Preferred Embodiment]
In summary, the preferred embodiment of the present invention proposes a new video switching mechanism for multimedia conferencing that utilizes layered video coding. Previously, layered video coding partitioned the video bitstream into two or more layers, namely a base layer and one or several enhancement layers as described above in connection with FIG. (partition). These known techniques for scalable video communication are described in H.264. It is described in detail in standards such as H.263 and MPEG-4.

  However, the inventors of the present invention recognize that it should benefit from adapting the concept of layered video coding and applying the adapted concept to multimedia video conferencing applications. did. In this way, the present invention defines various types of scalable video coding focused on the use of multimedia conferencing as opposed to point-to-point or multicast video communications.

  Referring now to FIG. 5, a functional block diagram 500 of a video switching mechanism is shown according to a preferred embodiment of the present invention. In contrast to conventional centralized conferencing systems, video switching is performed as follows. For example, MCU 520 located within Internet Protocol (IP) based network 510 includes a switch 530.

  The MCU 520 includes a base layer 552, 562, 572, 582 and one or more enhancement layer streams 555, 565, 575, 585 of all participants (user equipment) 550, 560, 570, 580. A containing “layered” video stream is received. For clarity only, only one enhancement layer video stream is shown for one participant.

  The MCU 520 may also receive a combined (multiplexed) audio stream 590 separately from the participants. The MCU 520 then selects a number of active speaker 535 base layer video streams and the most active speaker enhancement layer 540 using the switch 530. The MCU 520 then sends these video streams 535, 540 to all participants 550, 560, 570, 580.

  The selection process for determining the most active speaker is preferably performed by the MCU 520 first analyzing the audio stream 590 to determine who is all active speakers. The The most active speaker is then preferably determined at the multipoint processor unit, as described with reference to FIG. One or more base layers and one augmentation layer are preferably transmitted to the participants according to a priority level based on each participant's activity.

  In order to effectively operate the improved but more complex video switching mechanism shown in FIG. 5, a multipoint processing unit (MP) 600 is in accordance with a preferred embodiment of the present invention and as shown in FIG. Adapted to facilitate a new video switching mechanism.

The MP 600 still receives the audio stream 590 from the participant's video / multimedia communication device via the packet filtering module 610 and routes this audio stream to the packet routing module 630. However, the audio stream is again routed to the speaker identification module 620, which analyzes the audio stream 590 to determine who is the active speaker. The speaker identification module 620 assigns priority levels based on each participant's activity and determines: That is,
(I) the most active speaker 662,
(Ii) any other active speaker 625, and this is by default,
(Iii) Any remaining inactive speakers.

  The speaker identification module 620 then sends priority level information to a switching module 640 adapted to process the speaker priority level in accordance with a preferred embodiment of the present invention. In addition, the switching module 640 may send the layered video streams including video base layer streams 552, 562, 572 and 582, and video enhancement layer streams 555, 565, 575 and 585 to the video communication of the participant. It is adapted to receive from a device via a packet filtering module 610. The switching module 640 uses this speaker information to use the video base layer of the secondary (lower level) active speaker and the most active speaker and the video of the most active speaker. Only the enhancement layer is transmitted to all participants via the packet routing module 630.

  Thus, one or more receive ports of the multipoint processor are layered video including base layer video streams 552, 562, 572 and 582 and enhancement layer streams 555, 565, 575 and 585. It is adapted to receive streams from multiple user devices 550, 560, 570 and 580. If it is determined that there is only one active speaker, the switching module 640 may only select one base layer video image and the corresponding one or more enhancement layers. Within the spirit of the present invention. This speaker is then automatically designated as the most active speaker to send to one or more user devices 550, 560, 570 and 580.

  The enhancement layer is constantly switching when the most active speaker is constantly changing, as may occur in video conferencing. The inventor of the present invention has recognized the potential problems associated with such constant and rapid switching. Under such circumstances, if the first frame is actually the predicted frame (EP) from a speaker that was previously simply the second active speaker, the first frame Needs to be converted to intra frames (EI).

  To address this potential problem, video base layer streams 552, 562, 572, and 582 and video enhancement layer streams 555, 565, 575, and 585 from packet filtering module 610 are depacketized. It is preferably input to the (de-packetation) function unit 680. The packet release function unit 680 demultiplexes the video stream, and supplies the demultiplexed video stream to the video decoder and buffer function unit 670.

  To synchronize and co-ordinate video decoding, video decoder and buffer function 670 receives an indication of the most active speaker 622. After extracting the video stream information for the most active speaker, the video decoder and buffer function 670 may provide the bi-predicted (BP) video stream data of the most active speaker 622 and The predicted (EP) video stream data is provided to an “EP frame / EI frame conversion module” 660. An “EP frame / EI frame conversion module” 660 processes the input video stream and provides the primary speaker enhancement layer video stream as intra-coded (EI) frames.

  The main speaker enhancement layer video stream is then input to the packetization function 650 where the main speaker enhancement layer video stream is packetized and input to the switching module 640. The switching module 640 then combines the primary speaker enhancement layer video stream with the video base layer streams 552, 562, 572 and 582 of the secondary active speaker and the combined multi Route the media stream to the packet routing module 630. The packet routing module 630 then routes the information to the participants according to the method of FIG.

  In a preferred embodiment of the present invention, the video switching module 640 uses the output of the “EP frame / EI frame conversion module” 660 when the “EP frame / EI frame conversion module” 660 determines that the main speaker has changed. .

  One or more modules similar to the EP frame / EI frame conversion module 660 are also included in the MP 600 to perform the same function for the secondary speaker when the secondary speaker is considered to have changed. It is within the spirit of the invention that it could be included. Otherwise, in embodiments where a single “EP frame / EI frame conversion module” 660 is used to convert the primary speaker-only video stream, for example, an inactive speaker is secondary active. Speaker identification module 620 (or switching module 640) may request a new intra frame. Alternatively, the switching module 640 may send the new secondary active speaker's new intra frame before sending the new secondary active speaker's video base layer stream to all participants. You can wait.

  In addition to the preferred embodiment of the present invention, it is within the intent of the present invention that more classes of speakers can be used if more than one enhancement layer is available for use. . By using more classes of speakers, finer scalability of multimedia messages can be achieved because speaker identification is improved especially for large video conferences.

  It is also within the intent of the present invention that transforming a predicted frame into an intra frame could be added for one or more of the base layer streams. It is. In this way, the switching module 640 can quickly switch between base layers without having to wait for a new intra frame.

  FIG. 7 illustrates a video display 710 of a wireless device 700 that participates in a video conference using the preferred embodiment of the present invention. By implementing the inventive concepts described so far, improved video communication is achieved. In particular, for a given bandwidth, the participant can now lower the lower level (secondary) active speaker 730 and not provide video of the inactive speaker. Can receive better video quality of the most active speakers. In order to provide such improved video conferencing, the video communication device receives the enhancement layer and base layer of the most active speaker 720 and the base layer of the secondary active speaker 730. And do not receive video from inactive speakers.

  In that way, the video communication device can provide a constantly updated video image of the most active speaker on a larger and higher resolution display, while the smaller display is secondary ( Active speakers (lower level) can be displayed.

  Wireless device 700 includes a primary video display that displays a higher quality video image of the most active speaker, and one or more second speakers that display each lower level active speaker. Preferably with a separate display. The operation of displaying each video image on each display is preferably performed by a processor (not shown) operably coupled to the video display. The processor receives an indication of the most active speaker 720 and the lower level active speaker, and which received video image is to be displayed on the first display and the lower level Determine which video image (s) received from the active speaker 730 should be displayed on the second display. The second display is advantageously configured to save costs by providing a lower quality video image of the lower active speaker.

  It is expected that MCU-based systems will facilitate multimedia communication over IP-based networks in the future. Accordingly, the inventor of the present invention is aware that any H.264 technology in which the techniques described herein utilize an MCU. It is envisioned that it could be incorporated into a H.323 / SIP based multipoint multimedia conference or system.

  A preferred application of the foregoing invention is in the Third Generation Partnership Project (3GPP) specification for the Wideband Code Division Multiple Access (WCDMA) standard. In particular, the present invention can be applied to the IP multimedia domain (described in the 3G TS 25.xxx series of specifications). Planning to incorporate H.323 / SIP MCU into 3GPP network. The MCU will be hosted by the media resource function 890A (see FIG. 8).

  FIG. 8 shows a 3GPP (UMTS) communication system / network 800 in a hierarchical format, which 3GPP (UMTS) communication system / network 800 can be adapted according to a preferred embodiment of the present invention. The communication system 800 is compliant with and includes network components that can operate over a UMTS and / or GPRS air interface.

The network may be convenient to have:
(I) User device area 810 made from:
(A) User SIM (USIM) area 820; and (b) Mobile device area 830.
(Ii) Infrastructure area 840 made from:
(C) an access network area 850, and (d) a core network area 860 made up of:
(Di) a service providing network area 870, and
(Dii) the migration network area 880, and
(Iii) IP multimedia area 890 (note that multimedia is provided by SIP (ETF RFC2543)).

  In the mobile device area 830, the UE 830A receives data from the user SIM 820A in the USIM area 820 via the wired Cu interface. UE 830A communicates data with Node B 850A in network access area 850 via a wireless Uu interface. Within the network access area 850, the Node B 850A includes one or more transceiver devices, and through the remaining components of the cell-based system infrastructure, eg, the RNC 850B and Iub interface, Communicate as defined in

  The RNC 850B communicates with other RNCs (not shown) via the Iur interface. The RNC 850B communicates with the SGSN 870A in the service providing network area 870 via the Iu interface. Within the service providing network area 870, SGSN 870A communicates with GGSN 870B via the Gn interface, and SGSN 870A communicates with VLR server 870C via the Gs interface. In accordance with a preferred embodiment of the present invention, SGSN 870A communicates with an MCU (not shown) residing in a media resource function (890A) in IP multimedia domain 890. The communication is performed via the Gi interface.

  GGSN 870B (and / or SSGN) is responsible for interfacing UMTS (GPRS) with a public switched data network (PSDN) 880A, such as the Internet or the public switched telephone network (PSTN). SGSN 870A, for example, performs routing and tunneling functions for traffic within the UMTS core network, while GGSN 870B links to an external packet network in this case where a person accesses the system's UMTS mode.

  RNC 850B is a UTRAN component responsible for controlling and allocating resources for multiple Node B 850A. Typically, 50 to 100 Node Bs can be controlled by one RNC 850B. The RNC 850B also provides a reliable supply of user traffic over the air interface. The RNCs communicate with each other (via interface Iur) to support handover and macro diversity.

  SGSN 870A is the UMTS core network component responsible for session control and interface to location registers (HLR and VLR). SGSN is a large centralized controller for many RNCs.

  GGSN 870B is a UMTS core network component responsible for concentrating and tunneling user data in the core packet network to the final destination (eg, Internet Service Provider (ISP)). Such user data includes multimedia and associated signaling data to and / or from the IP multimedia area 890. Within the IP multimedia area 890, the MRF is divided into a multimedia resource function controller (MRFC) 892A and a multimedia resource function processor (MRFP) 891A. As described above, the MRFC 892A provides a multipoint controller (MC) function, while the MRFP 891A provides a multipoint processor (MP) function.

  The protocol used between Mr reference point / interface 893A is SIP (defined by RFC2543). The call state control function (CSCF) 895A acts as a call server and handles multimedia call signaling.

  Thus, in accordance with the preferred embodiment of the present invention, all components in components SGSN 870A, GGSN 870B, and MRF 890A are adapted to facilitate multimedia messages as described above. Further, UE 830A, Node B 850A and RNC 850B are also adapted to facilitate improved multimedia messages as described above.

  More generally, the adaptation may be performed in any suitable manner at each communication device. For example, a new device can be added to a conventional communication device, or alternatively, an existing component of a conventional communication device is adapted, for example, by reprogramming one or more processors therein. obtain. As such, the requested adaptation is a processor stored on a storage medium such as a floppy disk, hard disk, PROM, RAM, or any combination thereof, or other storage multimedia. It can be executed in the form of executable instructions.

  Also, such adaptation of multimedia messages may alternatively be controlled and performed in whole or in part by adapting any other suitable component of communication system 800.

  The above components are typically as separate and separate devices divided across mobile device area 830, access network area 850, and service providing network area 870 (with their respective software and / or (Or on a hardware platform), but other configurations are envisioned.

  Further, for other network infrastructures such as GSM networks, the implementation of the processing operation may be any other suitable type of base station, base station controller, mobile switching center, or operation and management controller, etc. Can be implemented at any suitable node. Alternatively, the foregoing steps may be performed by various components distributed at different locations or entities within any suitable network or system.

A video conferencing method using layered video coding, preferably when applied to the centralized video conferencing described above, provides the following advantages.
(I) Speaker identification is greatly improved compared to conventional systems. That is because instead of only one full quality video stream, it is possible to share bandwidth and transmit one or more enhancement layers and several base layers.

  (Ii) Video switching when the active speaker changes is very smooth using the inventive concepts described herein. This is because the inventive concept defines several states of active speakers, second most active speakers, and inactive speakers.

(Iii) The video quality of the most active speaker is improved.
(Iv) The improved video communication device displays various speakers, and each displayed image is dependent on the wired level associated with the transmission of the respective video communication device.

  A method for relaying video images in a multimedia video conference between multiple multimedia user devices has been described. The method includes transmitting a layered video image including a base layer and one or more enhancement layers by a number of user devices out of a plurality of user devices and the transmitted Receiving a layered video image at a multipoint controller. A number of base layer video images of a number of active speakers are selected, and one or more enhancement layers of the most active speakers are selected. The multipoint controller can transfer a number of base layer video images of a number of active speakers and one or more enhancement layers of the most active speakers to multiple multimedia Send to one or more of the user equipment multimedia user equipment.

  In addition, a video conferencing device that relays video images between a plurality of user devices has been described. In addition, a wireless device for participating in a video conference has been described in which a certain number of participants transmit video images.

FIG. 1 shows a known centralized conference model. FIG. 2 shows a functional diagram of a conventional video switching mechanism. FIG. 3 is a schematic diagram of a video structure showing picture prediction dependencies as known in the field of video coding technology. FIG. 4 is a schematic diagram of a layered video configuration known in the field of video coding technology. FIG. 5 shows a functional diagram of a video switching mechanism according to a preferred embodiment of the present invention. FIG. 6 shows a functional block diagram / flow chart of a multipoint processing device according to a preferred embodiment of the present invention. FIG. 7 shows a video display of a wireless device participating in a video conference using the preferred embodiment of the present invention. FIG. 8 illustrates a UMTS (3GPP) communication system adapted according to a preferred embodiment of the present invention.

Claims (23)

A method for relaying a video image in a multimedia video conference between a plurality of multimedia user devices (550, 560, 570, 580), comprising:
Transmitting a layered video image by a number of multimedia user devices of the plurality of multimedia user devices, wherein the layered video image is a base layer (552, 562, 572). 582) and one or more enhancement layers (555, 565, 575, 585), and
Receiving the transmitted layered video image at a multipoint controller (520);
Selecting a number of base layer video images of a number of active speakers (535) and one or more enhancement layers (540) of the most active speakers; ,
The multipoint controller (520) allows the number of active speakers (535) of the number of base layer video images and one or more of the most active speakers. Transmitting an enhancement layer (540) to one or more multimedia user devices of the plurality of multimedia user devices (550, 560, 570, 580). A method of relaying video images in a conference. The step of selecting further comprises:
A certain number transmitted by the plurality of multimedia user devices (550, 560, 570, 580) to determine the certain number of active speakers and / or the most active speakers; The method of relaying video images in a multimedia videoconference according to claim 1, comprising the step of analyzing an audio data stream (590). Assigning a priority level to each layered video image and / or said audio data stream transmitted by a respective user equipment;
A number of base layer video images (535) for transmission to the one or more multimedia user devices of the plurality of multimedia user devices (550, 560, 570, 580). And selecting one or more enhancement layers (540) based on the assigned priority level. 3. A multimedia video conference according to claim 1, further comprising: To relay video images in   Converting (660) a first predicted frame of the video image of the most active speaker to an intra-coded frame to enhance the video quality of the most active speaker. A method for relaying a video image in a multimedia video conference according to any one of claims 1 to 3.   When more than one enhancement layer is available, the multi-class indication of the one or more speakers with each layered video image transmission is provided to provide finer scalability of the video image. 5. A method for relaying a video image in a multimedia videoconference according to any one of claims 1 to 4, further comprising the step of receiving by a point controller (520).   6. The method of any one of claims 1-5, further comprising converting predicted frames to intra-coded frames for one or more base layer video streams. A method of relaying a video image in the multimedia video conference described in the paragraph. A video conferencing device for relaying a video image between a plurality of user devices (550, 560, 570, 580),
A multipoint controller (520) adapted to receive a number of layered video images transmitted by a number of multimedia user devices of the plurality of multimedia user devices. The multipoint controller (520), wherein the layered video image includes a base layer (552, 562, 572, 582) and one or more enhancement layers (555, 565, 575, 585); ,
A number of base layer video images of a number of active speakers (535) and one of the most active speakers operatively coupled to the multipoint controller (520) A video switching module (530) adapted to select more enhancement layers (540);
The multipoint controller (520) further includes the number of base layer video images of a number of active speakers and one or more enhancement layers of the most active speakers. (540) is adapted to transmit to one or more of the plurality of multimedia user devices (550, 560, 570, 580).   A prediction frame / intra coded frame conversion module (660) for converting a prediction frame operably coupled to the video switching module (530) into an intra-coded frame, wherein the multipoint controller (520) is first A predictive frame / intra-coded frame conversion module (660) that provides the enhancement layer video stream of the most active speaker as an intra-coded frame when receiving the frame as a predicted frame 8. The video conference apparatus according to claim 7, further comprising:   Further comprising a speaker identification module (620) that analyzes a number of active speakers and / or a number of audio streams (590) to determine the most active speakers. The video conferencing apparatus according to claim 7 or 8.   The speaker identification module (620) assigns a priority level based on each participant's determined activity to determine the most active speaker (622), any other active speaker (625), and The video conferencing apparatus of claim 9, wherein one or more of the non-active speakers are determined. A wireless device (700) participating in a video conference in which a plurality of participants transmit video images,
A video display (710) having a first display for displaying each participant (720, 730) from the plurality of participants and one or more second individual displays;
Operatively coupled to the video display to receive indications of the least active speaker (720) and less active speaker (730), and from the most active speaker (720) The received video image should be displayed on the first display providing a higher quality video image and received from a number of the less active speakers (730). A wireless device (700) comprising: a processor for determining that the video image should be displayed on the one or more second displays that provide a lower quality video image. A layered video stream, including a base layer video stream (552, 562, 572, 582) and an augmented layer video stream (555, 565, 575, 585), is transmitted to a plurality of user devices (550, 560, 570). , 580) one or more receiving ports adapted to receive from
A number of active speakers (535) operatively coupled to the one or more receiving ports and transmitting to one or more user devices (550, 560, 570, 580). Multipoint processor comprising a switching module (640) for selecting a certain number of base layer video images and one or more enhancement layers (540) of the most active speakers.   One of the plurality of user devices operatively coupled to the one or more receiving ports to determine a number of active speakers and / or the most active speakers. The multipoint processor of claim 12, further comprising a speaker identification module (620) for analyzing a number of audio streams (590) received from a number of user devices.   The speaker identification module (620) assigns a priority level based on a determined activity of a certain number of participants, so that the most active speaker (622), any other active speaker ( 625) and the multipoint processor of claim 12 or 13, wherein one or more of any inactive speakers are determined.   A prediction frame / intra-coded frame conversion module (660) operably coupled to the switching module (640), wherein the enhancement layer video stream of the most active speaker is a prediction frame at each port; 13. A prediction frame / intra coded frame conversion module (660) for converting the enhancement layer video stream of the most active speaker into an intra coded frame when received as 15. The multipoint processor according to any one of 1 to 14.   12. Adapted to perform the steps of the method according to any one of claims 1 to 6, or adapted to incorporate the video conferencing device according to any one of claims 7 to 10, or claim 12. A video communication system adapted to incorporate a multipoint processor according to any one of 1 to 15.   The video communication system of claim 16, wherein the video communication system is compatible with a UMTS communication standard (800) having an internet protocol multimedia domain (890) to facilitate video conferencing communications.   12. Adapted to perform the steps of the method according to any one of claims 1 to 6, or adapted to incorporate the video conferencing device according to any one of claims 7 to 10, or claim 12. A media resource function device (890A) adapted to incorporate a multipoint processor according to any one of 1 to 15.   A video communications apparatus (700) adapted to receive a layered video conference image generated according to the method of any one of claims 1-6.   7. A layer adapted to generate a layered video conference image for use in a method according to any one of claims 1 to 6, or generated according to a method according to any one of claims 1 to 6. Video communication device adapted to transmit a structured video conference image.   The video communication device according to claim 19, wherein the video communication device is one of Node B (850A), RNC (850B), SGSN (870A), GGSN (870B) and MRF (890A).   Video conferencing image 11. A method for relaying a video image in a multimedia conference according to claim 1 or 6 adapted to facilitate based on the H.323 standard or SIP standard, or according to any one of claims 7 to 10. 19. A video conferencing device, or a multipoint processor according to any one of claims 12 to 15, or a multipoint processor according to claim 16 or 17, or a media resource function device (890A) according to claim 18. The video communication device according to any one of claims 19 to 21.   A storage medium storing processor-executable instructions for controlling a processor to perform the method according to any one of claims 1-6.

Images