Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/14—Systems for two-way working
  • H04N7/15—Conference systems
  • H04N7/152—Multipoint control units therefor
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/14—Systems for two-way working
  • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
  • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/14—Systems for two-way working
  • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
  • H04N7/148—Interfacing a video terminal to a particular transmission medium, e.g. ISDN

Definitions

• This invention relates generally to the field of video communication and, more particularly, to providing an efficient and easy way of controlling video communication. Discussion of Prior Art
• An endpoint is a terminal on a network capable of providing real-time one way or two way audio and/or visual communication with other terminals or with a control unit.
• a Multipoint Communication Unit (MCU) is used to connect all the endpoints together.
• Controlling video communications may be complicated by options to change the layout, add or subtract participants, and request statistical information, for example.
• Common control systems of video communications use external communication paths such as a phone call to a human operator, web controlling with a web browser at the user's site, and a web server that connects to the MCU, the MGW, or an Interactive Voice Response (IVR) unit.
• the IVR unit provides vocal messaging to the user and receives the user's response by using Dual Tone Multi-Frequency (DTMF) by, for example, a touch- tone telephone.
• DTMF Dual Tone Multi-Frequency
• None of the current video communication control systems provides a method or a system that enables the user to control the video communication, via a terminal (e.g., an endpoint) of the video communication.
• none of the current video communication control systems provides visual information that can be viewed on the user's video display.
• a visual display that presents various control options and features that are enabled by the MCU or the MGW and are selectable by the user may improve the ease of use and the user friendliness of the system.
• the prior art may not recognize that a visual display that displays the results of the user's request may improve the ease of use. [0005] Therefore, there is a need for a system or a method that enables control of the communication via an endpoint, such as a terminal, and or displays control information of a video communication on the user's video monitor at the endpoint.
• the present invention may include, but is not limited to, a new way to control a conference that does not require external communication paths.
• an output video stream may be used to display menu options
• an input audio stream may be used to obtain commands and feedback from a user (i.e., an end user, conferee, or a conferencee).
• the end user may be enabled to (1) control the conference by providing the end user visual menus that are sent as part of the video streams, and (2) obtain responses from the end user using a common signaling convention such as, but not limited to DTMF.
• interactive control of the conference is provided thereby facilitating, for example, changing the layout by pointing with a keypad and/or a pointing device (such as a mouse, trackball, touch sensitive pad, touch sensitive screen, or other such device for pointing to, selecting with, and or moving the cursor) to the part that needs to be changed, or defining an area of interest on the screen that will get a special treatment that is different from the rest of the frame.
• a pointing device such as a mouse, trackball, touch sensitive pad, touch sensitive screen, or other such device for pointing to, selecting with, and or moving the cursor
• FIG. la is a block diagram showing a general description of a conferencing system
• FIG. lb is a flowchart showing the operation of the conferencing system of FIG. la;
• FIG. 2a and FIG. 2b are timing diagrams showing a screen of an endpoint operating according to an exemplary method of the present invention
• FIGs. 2c-f show different types of menus displayed on a screen of an endpoint that operate according to an exemplary method of the present invention
• FIG. 3 is a flowchart showing a process carried out by a control unit that operates according to an exemplary embodiment of the present invention
• FIG. 4 is a block diagram showing a Menu Generator (MG) that operates according to an exemplary embodiment of the present invention
• FIG. 5 is a flowchart showing the operation of the exemplary MG.
• FIG. la is a block diagram of a conferencing system 100 having a plurality of endpoints HOa-n with speakers llla-n, user control devices 112a-n, microphones 114a-n, cameras 116a-n, displays 118a-n, and multimedia lines 120a-n, respectively.
• the conferencing system further comprises a multimedia line 120q, a network 130, and a Video Control Unit (VCU) 140.
• VCU Video Control Unit
• the VCU 140 further includes a Network Interface (NT) 142, an audio module 160 having a DTMF module 165, a control module 170 with control lines 172, 174, 176, 178, and 180, video lines 182 and 184, an audio line 190, a Menu Generator (MG) 192, and a video module 194 having input modules 195, output modules 196, and common interface 198.
• NT Network Interface
• MG Menu Generator
• Each of the output modules 196 may include an editor 199.
• FIG. la is a general description of a typical environment in which the VCU 140 may be used. Alternative embodiments of the conference system may have other components and/or may not include all of the components shown in FIG. la. [0019] In this application the words "unit” and "module” are used interchangeably.
• the control lines 172, 174, 176, 178, and 180; the video lines 182 and 184; the audio line 190; and the multimedia lines 120a-n and 120q may be lines specially designed for, and dedicated to, carrying control signals, video signals, audio signals, and multimedia signals, respectively.
• control lines 172, 174, 176, 178, and 180; the video lines 182 and 184; the audio line 190; and multimedia lines 120a-n and 120q may be constructed from generic cables for carrying signals.
• the control lines 172, 174, 176, 178, and 180; the video lines 182 and 184; the audio line 190; and the multimedia lines 120a-n and 120q may carry other signals in addition to signals described in this specification.
• the control lines 172, 174, 176, 178, and 180; the video lines 182 and 184; the audio line 190; and the multimedia lines 120a-n and 120q may carry optical and/or electrical signals, may be paths of radio waves, or a combination thereof, for example.
• Each of these lines may actually be a pathway through a complex array of circuitry and/or components through which the corresponding signal travels, or may be a common memory buffer used for exchanging information. In embodiments in which the modules connected by any of these lines are software components, these lines may just represent an exchange of information.
• the information communicated between the endpoints HOa-n and the VCU 140 includes indicators, audio information, video information, and data.
• the multimedia lines 120a-n convey information to the network 130, and the multimedia line 120q conveys the information to the VCU 140.
• any combination of endpoints HOa-n may participate in any given conference.
• the endpoints HOa-n may provide speech, data, video, or any combination thereof.
• the endpoints HOa-n may include speakers llla-n, user control devices 112a-n, microphones 114 a-n, cameras 116a-n, and displays 118a-n, respectively.
• Each of the endpoints 1 lOa-n may contain a different combination of these components.
• the user control devices 112a-n act as interfaces between the user and the VCU 140 associated with endpoints HOa-n.
• the user control devices 112a-n may be a dialing keyboard (e.g.
• the user control devices 112a-n may be keypads, for example, and/or have arrows keys for left, right, up, and down to navigate a cursor on a conference image (i.e., an image associated with a conference) or to shift a position of the conference image.
• the user control devices 112a-n may be software presenting virtual keypads on the displays 118a-n.
• the user control devices 112a-n may include a DTMF generator such as common keypad of a touch-tone telephone and/or a remote control such as that used for a TV set.
• the microphones 114a-n allow users at the endpoints HOa-n to speak within the conference or contribute to the sounds and noises heard by other users.
• the speakers llla-n allow the endpoints HOa-n to hear the conference.
• the cameras 116a-n allow the endpoints HOa-n to input live video data (such as images of the users associated with endpoints HOa-n or images of exhibits) to the conference.
• the displays 118a-n allow the conference to be viewed at the endpoints HOa-n. Those endpoints HOa-n that are missing one of the components may be limited in the ways in which they can participate in the conference.
• the plurality of endpoints llOa-n are connected via the network 130 to the Video Control Unit (VCU) 140.
• the VCU 140 may be a MCU or a Multimedia Gateway (MGW), for example.
• MGW Multimedia Gateway
• the MCU is a conference controlling entity.
• the MCU may be a piece of equipment located in a node of the network 130 or in a temiinal that receives several channels from access ports and, according to certain criteria, processes audiovisual signals and distributes them to connected channels.
• An example of a MCU is MGC-100, which is a product of Polycom, Inc.
• Network 130 may represent a single network or a combination of two or more networks such as Integrated Services Digital Network (ISDN), Public Switched Telephone Network (PSTN), Asynchronous Transfer Mode (ATM), the Internet, and/or an intranet, for example.
• ISDN Integrated Services Digital Network
• PSTN Public Switched Telephone Network
• ATM Asynchronous Transfer Mode
• the Internet and/or an intranet
• MGW Media Gateway
• embodiments having a MCU may also use a MGW, which in turn may also be used for point-to-point communications between individual endpoints.
• the control options target the operation of the MGW.
• the user may be offered a list of subscribers to the MGW, and the user may be allowed to select which subscribers will be contacted via network 130 for participation in the conference, for example.
• the NI 142 may be a logic unit.
• a logical module or logic unit is a modde or unit that performs a certain functionality.
• the terms logical module, logic unit, and variations of these terms are used interchangeably.
• a logical module may comprise one or more elements, which may be hardware and/or software.
• the logical module may be located in one entity or spread over different entities, such as but not limited to a Digital Signal Processing Printed Circuit Board (DSP PCB), a processor, or different programs, for example.
• DSP PCB Digital Signal Processing Printed Circuit Board
• the NT 142 receives the multimedia communication from a plurality of the endpoints llOa-n via the network 130, and processes the multimedia communication according to communication standards such as, but not limited to, H.323, H.321, H.324, Session Initiation Protocol (SIP), and/or H.320.
• the NI 142 delivers audio, video, data, and control streams to the appropriate components of the VCU 140.
• Some communication standards require that the process of the NI 142 include multiplexing the incoming multimedia communication into audio, video, data, and control streams.
• the NI 142 receives separate streams from the various units of VCU 140, and processes the streams according to one of the communication standards.
• the NI 142 then transmits the streams via the multimedia line 120q to the network 130, which in turn transmits the streams via the multimedia lines 120a-n to the endpoints l lOa-n.
• More information about communications between endpoints and/or MCUs over different networks, and information describing signaling, control, compression, and how to set a video call, for example, can be found in the International Telecommunication Union ("ITU") standards H.320, H.321, H.323, H.261 and H.263.
• ITU International Telecommunication Union
• the audio module 160 receives, via the audio line 190, the compressed audio streams of the plurality of endpoints llOa-n from the NI 142. Subsequently, the audio module 160 processes the compressed audio streams, may mix the relevant audio streams (originating from the endpoints llOa-n), and sends the compressed mixed signal via the audio line 190 back to the NI 142, and then back to the endpoints llOa-n. Mixing the audio streams may not be needed if the MGW is used for the VCU 140. In one embodiment, the audio streams that are sent to each of the endpoints llOa-n may be different.
• the audio streams sent to different ones of the endpoints llOa-n may be formatted according to different communications standards, according to the needs of the individual endpoints llOa-n.
• the audio stream may not include the voice of a user associated with the one of the endpoints llOa-n that the audio stream is sent to, but the voice may be included in all other audio streams.
• the audio module 160 includes at least one DTMF module 165 that detects and grabs the DTMF signals from the audio stream.
• the DTMF module 165 then converts the DTMF signals into DTMF control data, and transfers the DTMF control data via the control line 172 to the control module 170.
• the DTMF control data may then be used to control the conference.
• the audio module 160 may also mix the audio streams of the plurality of endpomts llOa-n to form the audio portion of the conference as in the above embodiment in addition to processing the DTMF signals.
• FIG. la may use a speech recognition module (not shown in FIG. la) in addition to, or instead of, the DTMF module 165.
• the speech recognition module may use the user's vocal commands for controlling parameters of the videoconference.
• FIG. la Further embodiments of the present invention may use or have an Interactive Voice Recognition (IVR) module that instructs the user in addition to the visual menu.
• the audio instructions may be an enhancement of the video menu.
• audio module 160 may generate an audio menu for instructing the user regarding how to participate in the conference and/or how to manipulate the parameters of the conference.
• the IVR module is not shown in FIG. la.
• the control module 170 may be a logic unit that controls the operation of the VCU 140. In addition to common operations of the typical MCU, VCU 140 is capable of additional operations as result of having control module 170. Specifically, the control module 170 may include logic modules that process the instructions from the DTMF module 165 sent via the control line 172 and/or processes other dedicated control signals sent via the control line 174. The control module 170 may also control the menu generator 192, and instruct the video module 194 in regard to the output of the menu generator 192. Some unique operations of the control module 170 are described in detail below with respect to FIGs. 2 and 3.
• the Menu Generator (MG) 192 may be a logic unit that generates the menus.
• the MG 192 receives commands via the control line 180 from the control module 170, and generates menu frames with relevant options for selection by the user.
• These menu frames are graphical images that are in a size and format that the video module 194 is capable of handling, and the menu frames are displayed on the displays 118a-n of the endpoints l lOa-n, respectively.
• the menu frames may include text, graphic, and transparent information (i.e., information related to the extent to which the conference video image can be seen as background through a partially transparent foreground image as discussed further below regarding Bank Of Menus (BOM) 410 of FIG. 4).
• the menu frames may be displayed in addition to, or instead of, part of the common video image of the conference.
• the MG 192 may be part of the video module 194. The operation of the MG 192 is described in detail below with respect to FIGs. 2 and 4.
• the video module 194 may be a logic module that receives and sends compressed video streams.
• the video module 194 may include (1) the input modules 195 that handle compressed input video streams from a participating endpoint of the plurality of endpoints llOa-n; and (2) the output modules 196 that generate composed compressed output video streams (that are composed from several input streams to form a video stream representing the conference) for designated endpoints of the plurality of endpomts llOa-n.
• the input modules 195 include at least one video input module, but may include any number of video input modules. For example, there may be one video input module for each of the endpoints llOa-n.
• the video output modules 196 include at least one video output module, but may include any number of video output modules. For example, there may be one video output module for each of the endpoints llOa-n, each video output module producing a screen layout that is individualized to a particular endpoint of the plurality of endpoints 1 lOa-n.
• the uncompressed video data is shared by the input modules 195 and the output modules 196 on the common interface 198 such as, but not limited to, a Time Division Multiplexing (TDM) interface, an Asynchronous Transfer Mode (ATM) interface, and/or shared memory.
• TDM Time Division Multiplexing
• ATM Asynchronous Transfer Mode
• the data on the common interface 198 may be fully uncompressed or partially compressed.
• An exemplary operation of such a video module 194 is described in U.S. Patent Number 6,300,973, cited above.
• Another exemplary embodiment of the video module 194 is described in U.S. Provisional Patent Application Number 60/225,491 and PCT Application Number PCT/IL01/00757, also cited above.
• Different types of output modules may be included in the plurality of output modules 196.
• each one of the plurality of output modules 196 may include an editor 199.
• the video data from MG 192 is grabbed by the appropriate output modules 196 from the common interface 198, and each of the appropriate output modules transfers the video data to the editor 199.
• the editor 199 builds the output video frame from the different video sources, and also composes the menu frame into the next frame of memory.
• the editor 199 may treat this output video frame as an additional video source and add it to the layout as one of the rectangles or windows of the screen layout.
• Each rectangle or window of the screen layout may contain an image from a different endpoint, such as the picture of the user associated with that endpoint.
• video data from the MG 192 may be placed above, below, or on one of the sides of the other images of the screen layout.
• Other exemplary editors may treat the video data from the MG 192 as a special video source and display the menu as partially transparent and in front of the screen layout of the conference using the functionality of the transparent information so that the screen layout behind the menu can still be seen.
• the editor 199 may be the same as the editor of U.S. Patent Number 6,300,973.
• different output modules of the plurality of output modules 196 may have different editors 199.
• the MG 192 may be a separate unit that generates the required menu to more than one of the output modules 196, or the MG 192 may be a module in each of the output modules 196 for generating individualized menus that are requested by the endpoints llOa-n that are receiving the video information from the corresponding output modules 196.
• the menus may be individualized in their entirety.
• the menus may be individualized in their setup, look, and appearance according to the requests of the individual endpoints llOa-n.
• the appearance of the menus may be essentially uniform, but the menus are individualized in terms of when the menus appear and which features are indicated as being selected.
• the presentation of visual control to the endpoints llOa-n is an option that may be selected by a moderator of a conference while he or she reserves and defines the profile of the conference.
• the moderator may be associated with one of the endpo nts 1 lOa-n, and may use the user control devices 112a-n to make the selections and define the profile of the conference.
• the moderator may determine whether the conference participants will have the ability to control the settings (parameters) of the conference (using their respective user control devices 112a-n) during the conference. For example, when allowing the participants to have the ability to control the settings of the conference, the moderator selects a corresponding option "ON" in the conference profile.
• the control module 170 Upon initiating the conference, the control module 170 checks the conference profile and determines whether the option to give the users control of the settings is on. If the option is on, the control module 170 instructs the output modules 196 in the video module 194 to add the video data output from the MG 192 as one of the video sources of the conference.
• FIG. lb is a flowchart of a method 1000 of the operation of the conferencing system 100 of FIG. la.
• FIG. lb describes a single loop including endpoints 1 lOa-n sending signals to the VCU 140, which processes the incoming signals to compose outgoing signals that are sent back to the endpomts llOa-n.
• the incoming signals may include images and sounds (e.g., voices) of individuals at selected ones of the endpoints llOa-n, for example. Some of the images may be associated with sounds, while some images may have no sound associated with them, and likewise some sounds may have no images associated with them.
• the outgoing signals are composites of the incoming signals that represent the conference.
• the loop of FIG. lb is repeated continuously throughout the duration of the conference.
• multiple loops of the method 1000 may overlap and/or occur simultaneously.
• step 1002 the endpoints llOa-n send multimedia streams, via the multimedia lines 120a-n to the network 130.
• the multimedia streams may include sounds, images, and/or control information associated with various end users at the endpoints 1 lOa-n.
• step 1004 the network 130 transfers the multimedia streams, via the multimedia line 120q, to the VCU 140 where the multimedia streams are received by the NI 142.
• sending messages from the endpoints llOa-n to the VCU 140 is accomplished using steps 1002 and 1004.
• the NI 142 separates the multimedia stream into a control stream on the control line 174 for use by the control module 170, an audio stream on the audio line 190 for use by the audio module 160, and a video stream on the video line 184 for use by the video module 194.
• the DTMF module 165 extracts DTMF signals from the audio stream.
• Alternative embodiments may use other control signals instead of or in addition to DTMF signals that are mixed with the audio stream. Those alternative embodiments may use frequencies other than those used for the DTMF or may use other types of modulation.
• the audio module 160 may comprise other types of detectors adapted to these non-DTMF control signals in addition to, or instead of, a detector that includes the DTMF module 165.
• the other module is adapted to the control signaling.
• part of or all the control information may also extracted by NI 142 and sent over control line 174 to control module 160.
• the audio stream is capable of carrying control information in addition to the control stream.
• the DTMF signals are converted into control data that is sent on control line 172 to the control module 170.
• control module 170 may receive control information extracted from both the audio stream and the control stream.
• Control signals for the audio module 160, the video module 194, and the MG 192 are formed based on the control stream, the DTMF information/data, and the control information in step 1012.
• step 1014 the control module 170 sends the audio control signal (formed in step 1012) on the control line 176 to the audio module 160, a video control signal on the control line 178 to the video module 194, and a menu control signal on the control line 180 to the MG 192.
• the MG 192 generates a menu that is sent on video line 182 to video module 194.
• the menu may be in response to a request made in step 1002 by one of the plurality of endpoints llOa-n, and can be used for controlling the conference.
• the MG 192 may generate an audio menu and/or audio enhancements to the visual menu that is sent to the audio module 160 as part of step 1014.
• the audio menu may replace the video menu for those endpoints 1 lOa-n that do not have displays 118a-n.
• a video signal is composed from the video stream carried by video line 184 and the menu carried by video line 182.
• the composed video signal contains information for forming an image of the conference having the menu image superimposed thereon.
• the purpose of the composition of step 1016 is so that several different images (one of which may be that of the menu) may be arranged on one screen in different locations according to a screen layout. Each of the different images may come from a different one of the endpoints llOa-n, except that the image of the menu (if present) comes from the MG 192.
• step 1016 the image associated with the composed video signal (of step 1016) has not been formed on any of the displays 118a-n, yet.
• step 1018 the composed video signals are sent to the endpoints llOa-n allowing the endpoints llOa-n to view the image associated with the video signal composed in step 1016.
• one or more endpoints 1 lOa-n may manipulate the conference parameters via the menu generated.
• Steps 1014, 1016, and 1018 describe forming the video image of the conference on the displays 118a-n, while the discussion of steps 1020 and 1022 that follows describes forming the audio portion of the conference so that it can be heard on the speakers llla-n.
• the audio module 160 mixes the audio streams from the endpoints llOa-n to produce an audio stream of the conference.
• the mixed audio stream is sent to the endpoints 1 lOa-n in step 1022.
• the steps of method 1000 relating to the forming of the composed video image may be independent from the forming of the mixed audio stream, and may therefore be performed simultaneously or in any order with respect to one another.
• FIG. 2a and FIG. 2b are timing diagrams that illustrate screens of an endpoint that operates according to an exemplary method of the present invention.
• Each of FIGs. 2a and 2b are divided into two sections.
• the left section illustrates exemplary images displayed on the screens of the displays llOa-n.
• the right section shows two time axes and arrows illustrating how the events on the two axes are sequenced with respect to one another.
• the left time axis, labeled "ENDPOINT” illustrates the operations of the user of one of the endpoints llOa-n. In other words, the operations of the user occur during time intervals in which the events FIGs.
• the right axis labeled VCU, illustrates the sequence of events associated with the VCU 140. Specifically, during time intervals T2 to T3, T6 to T7, T10 to Til, and T14 to T15, the VCU 140 processes the signals sent by the endpoint and produces a response.
• each sequence of time representing sequences of events starting on the left time axis, continuing on the right time axis, and ending on the left time axis (i.e., T1-T4, T5-T8, T9- T12, and T13-T16) occur during one loop of FIG. lb.
• ENDPOINT and "VCU” are drawn to scale; the relative sizes of the times intervals have no meaning.
• a screen 210 illustrates an ongoing videoconference with a 2X2 layout (i.e., a 2X2 layout is four images arranged in a 2X2 rectangle).
• a menu tab 212 may appear at the bottom left corner of the screen 210 (which is enlarged for illustrative purposes below as menu tab 214).
• the purpose of the menu tab 212 is to instruct the user to push "#" key or any other predefined signaling, when the user wishes to change the current settings (which in turn changes the parameters) of the conference.
• those endpoints llOa-n having other control signaling and having more extensive keyboards or keypads may be capable of selecting the menu tab 212 by pushing the "enter” or “back” key or by clicking on a pointing device (such as a mouse, trackball, touch sensitive pad, touch sensitive screen, or other such device for pointing to, selecting with, and/or moving the cursor).
• a pointing device such as a mouse, trackball, touch sensitive pad, touch sensitive screen, or other such device for pointing to, selecting with, and/or moving the cursor.
• the control module 170 instructs the MG 192 to generate the main menu, and to submit the main menu to an appropriate one of the output modules 196 via the common interface 198 of the video module 194.
• the control module 170 sets the output modules 196 of the video module 194 to grab and/or receive the information from the MG 192, to add the information to the composed video signal, and then to compress the composed video.
• the composed and compressed video signal is transferred to the appropriate one of the endpoints llOa-n.
• the main menu 223 may include eight rectangles.
• the four upper rectangles are function keys 224a-d, which change the settings (thereby changing the parameters of the videoconference) of the communication.
• the function key 224a entitled "ADD PARTI”
• the function key 224b entitled “SPEAKER” chooses one of the endpoints llOa-n functions as the speaker.
• the function key 224c entitled "REMOVE,” removes a participant from the conference
• the function key 224d entitled “LAYOUT”
• Additional functionality may be provided by including other similar additional function keys within the menu 223.
• the additional functionality may control other parameters of the videoconference, such as but not liir ⁇ ted to, visual effects on the screen.
• the parameters may control the appearance of borders between the participants, and background colors.
• other function keys may be utilized instead of the examples provided.
• the cursor is pointing at the function key 224d.
• rectangles 228a-d in the exemplary embodiment indicate information to the user about the functionality of the present user control devices 112a-n of the endpoints llOa-n.
• the rectangles 228a-d merely convey information to the user on how to use the menu.
• Rectangles 228a-d can be used by, and may be convenient for, any of endpoint llOa-n.
• rectangles 228a-d may be particularly useful for those of endpoints llOa-n that use DTMF signals (e.g., those endpoints using telephone keypads) and therefore may not have ready access to an arrow key, for example.
• pressing the "*" key may return the user to menu tab 212 on the ongoing display, thereby restoring the screen image to that of the screen 210 in which the main menu 223 in not displayed.
• the rectangles 228a-d may be replaced with a number of function keys having the navigation functions indicated in rectangles 228a-d or other function keys.
• screen selections can be made by pressing an "enter” or a "back" key of a keyboard, while the cursor can be navigated using arrow keys of the keyboard and or using a pointing device.
• some endpoints llOa-n may be presented with the rectangles 228a-d, while others may have the rectangles 228a-d replaced with function keys of other shapes or forms, for example, according to the selections of the user associated with that endpoint, the conference manager, and/or the system used for the endpoint.
• the control module 170 instructs the MG 192 to generate the speaker submenu (i.e., screen 240) illustrated in FIG. 2b, and to submit the speaker submenu to the appropriate one of the output modules 196 via the common interface 198 of the video module 194. Subsequently, at time t Tll, a screen image including the speaker submenu (i.e., screen 240) is sent to the appropriate one of the endpoints llOa-n.
• selection button 242 is on participant 243
• selection button 244 is on participant 245
• selection button 246 is on participant 247
• selection button 248 is on participant 249
• pressing "7" on the keypad of the appropriate one of the endpoints llOa-n will select the participant 243 of the top left corner (as indicated by selection button 242) as the speaker; pressing “9” will select the participant 245 of the top right corner as the speaker (as indicated by selection button 244); pressing "1” will select the participant 247 of the bottom left corner as the speaker (as indicated by selection button 246); and pressing "3” will select the participant 249 of the bottom right corner as the speaker (as indicated by selection button 248).
• a text menu may be provided as illustrated in FIG. 2c.
• Other embodiments may use a graphical menu with shapes that include the key icon on which the user must press to select the key, as illustrated in FIG. 2d, for example.
• Another type of menu may be an interactive menu.
• An example of an interactive menu is illustrated in FIG. 2e.
• An interactive menu may be used by an end user to mark an area of interest, for example. In one embodiment, the area of interest may be expanded by a zoom function.
• an interactive menu may be used to define a region in which an encoder should concentrate its time and resources in order to provide better video quality in this region.
• the MG 192 generates four cursor lines - two vertical lines 260 and 280, and two horizontal lines 270 and 290.
• Each cursor line can be moved in two directions by using control arrows/numbers 262, 272, 282, and 292. For example, pressing the number "4" will move the vertical line 260 to the left as indicated by the arrow 262. Pressing the "#" may indicate that the area between the four cursor lines is the area that the user is interested in. Pressing the "*" may instruct the system to return to the menu tab.
• vertical lines 260 and 280 and horizontal lines 270 and 290 can be moved with other control signals by moving the cursor onto a desired location, which can be done using a pointing device or using the arrows on a keyboard, for example. Then, the line having the cursor is selected by pressing the enter button on the keyboard or clicking with the pointing device, for example. Next, the selected line is moved by dragging with the pointing device or using the arrow buttons of the keyboard, for example.
• FIG. 2f illustrates another type of menu that is used by an alternative embodiment of the present invention, having menu items 294, 296, and 298.
• This embodiment may use a static menu that is displayed during the whole conference instructing the user that pressing certain keys in his or her user control device (e.g., user control devices 112a-n) will cause the VCU 140 to change some parameters of the conference.
• the menu item 294 instructs the user that pressing "1" will add ten minutes to the reserved duration of the conference, and pressing "3" will toggle the layout on the screen of the relevant endpoint between two states (i.e., menu item 296).
• the first state displays the speaker on a full screen and the second state displays the reserved layout, which is 2x2 for this example.
• FIG. la Other embodiments may implement some functions as instructions written in a user guide or may be written on the user control devices 112a-n (FIG. la) instead of displaying them on the screen. For example, in this embodiment, instructions that pressing "1" will add ten minutes to the reserved duration of the conference and pressing "3" will toggle the screen of the relevant endpoint may appear on the user control devices 112a-n.
• the VCU 140 may not have a menu generator.
• the control module 170 of this embodiment of VCU 140 controls the video module 194 or the audio module 170 according to the selected control option of the user.
• FIG. 3 is a flowchart illustrating an exemplary method 300 that is carried out by the control module 170 according to an embodiment of the present invention.
• the method 300 starts at step 301 where upon initiating the conference, the control module 170 checks the conference profile and determines whether the option of using the visual control is on. If the visual control is on, the control module 170 instructs the output modules 196 that are assigned to the appropriate endpoints to add the video data output from the MG 192 as one of the video sources for the output modules 196. Other embodiments may permanently use the visual control rather than provide them as options.
• Step 310 is a part of a process of periodically checking for the "#" signal indicating that the user wishes to view the main menu.
• step 310 the control module 170 waits a period of time and then checks if the "#" signal from the DTMF module 165 was received. If the "#" signal was not received, the method goes to the beginning of step 310, and again waits for a period of time before checking for the "#" signal. The period of time is short enough so that the user perceives the response of the VCU 140 as essentially instantaneous. If the "#" signal was received, the method continues to step 315 where the control module 170 instructs the MG 192 to submit the main menu with the cursor in an initial position to one of the output modules 196 that corresponds to the one of the endpoints llOa-n that made the request.
• the control module 170 waits for a period of time, and checks for a response from the user of the one of the endpoints 11 Oa-n in step 320. If there is no response, the method returns to the beginning of step 320, and waits for a period of time before checking again. Similar to step 310, the period of time that step 320 waits is short enough that the user perceives the response of the VCU 140 as essentially instantaneous.
• the control module 170 determines how to proceed. If the signal is that of step 322, which is "4" in the current example, the control module 170 performs step 324 (i.e., moving the cursor left if applicable). If the control module 170 cannot move the cursor left, the control module 170 instructs the MG 192 to display the same menu with a different background color around the "4.” The different color indicates to the user that the user made a mistake, and the user has to choose another selection from the menu. However, if the cursor can be moved to the left, the control module 170 instructs the MG 192 and the video module 194 to display the main menu but with the cursor on the rectangle to the left of where it was previously (before the "4" was pressed).
• step 324 i.e., moving the cursor left if applicable.
• the control module 170 instructs the MG 192 to display the same menu with a different background color around the "4.” The different color indicates to the user that the user made a mistake, and the user has to
• step 328 is similar to step 324 except the cursor moves to the right and not to the left. If the signal is that of step 336, which is "2,” the control module 170 performs step 338, which moves the cursor downward. Finally, if the signal is that of step 332, which is "8,” the control unit performs step 334, which moves the cursor upward. In each case if the cursor cannot be moved in the direction requested, the background of the improper selection is changed. In an embodiment, the selections that are improper are indicated as such by the coloring of their respective backgrounds before the user attempts to make an improper selection. After any of steps 324, 328, 338, or 334, method 300 returns to step 315 to wait for the next signal.
• step 320 If while waiting in step 320 the signal is that of step 340, which is "#," the control module 170 selects the appropriate subroutine from a bank of submenu routines 350a-m.
• the "#" signal is the result of the user selecting the function key that is currently being pointed to by the cursor.
• FIG. 2a and 2b which illustrates a submenu routine for choosing the speaker of the conference.
• the number of submenu routines 350a-m is unrelated to the number of endpoints 1 lOa-n.
• step 320 if the signal is that of step 360, which is "*," the control module 170 instructs the MG 192 to deliver the menu tab, and subsequently returns to step 310 and waits for the next "#" signal.
• the purpose of the "*" signal is to return to the menu tab.
• Other exemplary embodiments may use other keys for the same functionality. Alternatively, other functions may be provided for other keys.
• step 1014 of method 1000 (FIG. lb) is closely related to, but not identical to, the method 300. Specifically, step 1014 may correspond to one inner loop of the method 300 (starting with step 320 and ending with a return to step 315, or with one of steps 350a-m, or 360).
• FIG. 4 is a block diagram illustrating an example of the Menu Generator (MG) 192 according to an embodiment of the present invention.
• the MG 192 may include a Bank of Menus (BOM) 410, a Cursor Generator (CG) 413, a text generator 416, a Bank Of Graphic Objects (BOGO) 418, a builder 420, a Bank Of Frames Memory (BOFM) 430, a Common Interface Adaptor (CIA) 440, and a controller 450.
• Alternative embodiments of MG 192 may include more, less, or alternative components.
• the BOM 410 may be a memory storing a plurality of ready menu images or characteristics that are superimposed on the conference image.
• Each menu image is organized by pixel coloring and screen resolution.
• the menus having the resolution of Common Intermediate Format may be stored together in one group, while menus having other resolutions may be stored in other groups.
• CIF is a video conferencing format that specifies a video frame containing 288 lines and 352 pixels per line. Each pixel has the appropriate color values according to some color space representation and may have transparency values also. The transparency value may be selected from a plurality of values from fully transparent to fully opaque.
• the BOM 410 may be a temporary bank of menus that is regenerated for each conference, for example. [0071]
• the menu frame is superimposed on the conference video frame as an upper layer. The superimposing operation is done as a function of the transparency value for each pixel.
• Full transparency means that the video pixel of the layer below the menu (i.e., the layer of the conference) is visible through the menu layer.
• An opaque value means that the video pixel of the layer below the menu (i.e., the layer of the conference) is not visible through the menu layer.
• a partially transparent value means that the value of the pixel is a combination of the value of the menu and the value of the pixel below.
• the text generator 416 may convert a string of text into graphic items, which are added to the menu.
• the text generator 416 may be adapted to different fonts and languages.
• the text generator 416 may be used in the creation of a new BOM 410.
• the text generator 416 may not be present in embodiments having a permanent BOM 410.
• the BOGO 418 includes the different graphic shapes and icons that may be used in the menu.
• the icons may be placed in the menu to give graphical representations of an item.
• text from the text generator 416 may be placed within the icon or near a corresponding icon.
• selecting the icons may cause one of the routines 350a-m (FIG. 3) to be selected.
• the BOGO 418 may be used for the creation of a new BOM 410 if the BOM 410 is temporary. Alternatively, the BOGO 418 may not be present in embodiments having a permanent BOM 410.
• the builder 420 grabs and/or receives the appropriate data from the various data generators (e.g., BOM 410, GC 413, text generator 416, and BOGO 418).
• the builder 420 composes the various elements into a menu frame. Part of the pixels of a menu frame may be transparent or partially transparent. Then, the builder 420 submits the complete menu frame to a section of the BOFM 430 so that it is assigned to the relevant one of the video output modules 196.
• the BOFM 430 may be a memory having a plurality of sections. Each section includes video data of a menu frame. Each section is temporarily assigned to one of the output modules 196 (FIG. la) that is assigned to the endpoints llOa-n (FIG. la) for which the menu is being prepared.
• the Common Interface Adaptor (CIA) 440 acts as the interface between the common interface 198 (FIG. la) of the video module 194 (FIG. la) and the MG 192.
• the implementation of the CIA 440 is based the common interface 198 type. For example, if the common interface 198 is a TDM bus, then the CIA 440 divides the frame memory (which it grabbed and/or received from the BOFM 430) into time slots and submits them to the common interface in the appropriate time slots that have been allocated to the MG 192. If the common interface is a shared memory, then the CIA 440 may be a buffer or collection of buffers.
• the controller 450 controls the operation of the exemplary MG 192.
• the controller 450 receives, from the control module 170, the parameters of the conference.
• the parameters may include information such as, but not limited to, which of the output modules 196 (FIG. la) are associated with the endpomts llOa-n (FIG. la) that are involved in the conference, the frame resolution of each of the endpoints 110 a-n participating in the conference, the type of menus for each of the endpoints llOa-n that are participating, and the data to generate these menus.
• the controller 450 submits this data to the appropriate logic modules of the MG 192.
• the controller allocates the BOFM 430 between the appropriate video output modules 196, and instructs the CIA 440 and the builder 420 about this allocation.
• the controller 450 instructs the various modules to generate the menu tab and to transfer the menu tab to the appropriate one of the output modules 196. Then, the controller 450 waits for the next change in the menu.
• the MG 192 may have other logic modules. For example, some embodiments may not have the BOFM 410. In such a case, the builder 420 builds the appropriate menu by collecting the data from the CG 413, the text generator 416, and the BOGO 418.
• the MG 192 may include the BOFM 410, the CG 413, and the text generator 416, but not the BOGO 418. Furthermore, if in an embodiment the MG 192 is distributed between each of the output modules 196 (FIG. la), then the MG 192 may not use the BOFM 410 or the CIA 440.
• the MG 192 may be located in the output modules 196 and may include a bank of menus.
• the menus in this case, may have been prepared during the conference setup to match the profiles of the endpoints llOa-n (e.g., to have the same resolution), which are assigned to the output modules 196.
• This bank of menus includes all the possible menus and submenus with all the possible locations of the cursor that may be used during the conference.
• the control module 170 instructs the video output modules 196 to select the appropriate menu from this pre prepared bank of menus 410. The selected menu is then composed with the next frame memory.
• FIG. 5 is an exemplary flowchart for a method 500 of operation of the MG 192.
• Method 500 is a general method for generating a plurality of menus, which may also be used to implement step 315 (FIG. 3).
• the controller 450 receives control signals via the control line 180 from the control module 170.
• the controller 450 generates control signals for the bank of menus 410, the cursor generator 413, the text generator 416, the Bank Of Graphic Objects (BOGO) 418, the builder 420, the bank of frame memories 430, and/or the CIA 440 (FIG. 4).
• aspects of the menu or menu characteristics are selected from the bank of menus 410 in step 504. Further, a cursor and its position are generated using cursor generator 413 in step 506. Subsequently, text is converted into graphical images of the corresponding letters (e.g., a bit map of the letter may be generated) using text generator 416 in step 508. In step 510, graphical objects (within which text may be placed) are selected, and a frame is selected from the bank of frames in step 512. In step 516, the builder 420 uses the menu characteristics that were selected, the cursor that was generated, and the graphical images that were selected to build a menu and the menu is added to the selected frame.
• the menu and the selected frame are then sent to the video module 194 in step 518.
• the CIA 440 may place the frame having the menu in a format and location that is expected by video module 194.
• Steps 504, 506, 508, 510, and 512 may be independent processes that may be performed simultaneously or in any order with respect to one another.
• each of the words, “comprise,” “include,” “have,” “with,” and conjugations of these words are used to indicate that the (object or) objects following these words are not necessarily a complete list of the members, components, elements, actions, and/or parts of the subject (or subjects) of these words.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Networks & Wireless Communication (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
• Telephonic Communication Services (AREA)

Applications Claiming Priority (5)

Publications (2)

Family

ID=27616745

Family Applications (1)

Country Status (2)

Cited By (3)

Families Citing this family (9)

Citations (3)

Family Cites Families (4)

• 2003
  • 2003-01-16 EP EP03710697A patent/EP1468559A4/de not_active Ceased
  • 2003-01-16 WO PCT/US2003/001689 patent/WO2003063484A1/en not_active Application Discontinuation

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events