CN102215217B - Establishing a video conference during a phone call - Google Patents

Establishing a video conference during a phone call Download PDF

Info

Publication number
CN102215217B
CN102215217B CN 201010600623 CN201010600623A CN102215217B CN 102215217 B CN102215217 B CN 102215217B CN 201010600623 CN201010600623 CN 201010600623 CN 201010600623 A CN201010600623 A CN 201010600623A CN 102215217 B CN102215217 B CN 102215217B
Authority
CN
Grant status
Grant
Patent type
Prior art keywords
establishing
video
conference
during
phone
Prior art date
Application number
CN 201010600623
Other languages
Chinese (zh)
Other versions
CN102215217A (en )
Inventor
E·C·克兰菲尔
S·O·勒梅
J·S·阿布安
吴锡荣
周小松
小R·加西亚
Original Assignee
苹果公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Abstract

本发明涉及在通话期间建立视频会议。 The present invention relates to the establishment of a video conference during a call. 一些实施例提供一种利用第一移动设备发起视频会议的方法。 Some embodiments provide a method for using a first mobile device to initiate a video conference. 该方法在通过无线通信网络与第二设备的音频通话期间,在第一移动设备上呈现用于从音频通话切换到视频会议的可选用户界面(UI)项目。 The method during a call by the audio network and the second wireless communication device, for switching from presenting the audio communication to video conference selectable user interface (UI) on the first item in the mobile device. 该方法接收对所述可选UI项目的选择。 The method receives a selection of the selectable UI item. 该方法在不终止音频通话的情况下,发起视频会议。 This method without terminating the audio call, initiate a video conference. 该方法在允许第一和第二设备呈现通过视频会议交换的音频和视频数据之前,终止音频通话。 Before the method allows the first and second devices presenting audio and video data through the video conference exchange audio call termination.

Description

在通话期间建立视频会议 Established during a video conference call

[0001] 当Apple的iPhone 4的样机在2010年3月25日从Apple的一个工程师处被窃取时,本申请中要揭示和主张的发明被提前且未经Apple授权地向公众公开了。 [0001] When Apple's iPhone 4 prototype in March 25, 2010 was stolen from one of the engineers at Apple, this application for the invention is disclosed and claimed in advance and is not authorized by Apple to disclose to the public. 在该明显盗窃之前,尚未提交本申请所基于的美国优先权申请。 Before the apparent theft, has not been submitted to the US priority application of the present application is based.

背景技术 Background technique

[0002] 许多目前的便携式设备,比如智能电话机具备视频拍摄功能。 [0002] Many current portable devices such as smart phones have video recording capabilities. 通过电话机上的照相机,便携式设备的用户能够拍摄静止图像和视频。 By the camera on the phone, the user of the portable device capable of capturing still images and video. 不过,在完成视频拍摄之后,为了把拍摄的视频传送给另一方,用户通常必须直接把视频发送给所述另一方,或者把视频上传到另一位置(例如,因特网视频托管网站(hosting site))。 However, after completion of video recording to the transfer of recorded video to another party, a user must typically sent directly to the other video, or video uploaded to another location (e.g., an Internet video hosting website (hosting site) ). 不幸的是,这不允许所述另一方在便携式设备拍摄视频的时候,查看实时视频流。 Unfortunately, this does not allow the other party when shooting video in a portable device to view live video streams.

[0003] 另外,标准的便携式设备只配备一部照相机,并且处理来自该照相机的信息相当困难。 [0003] In addition, standard devices with only a portable camera, and processing information from the camera is very difficult. 理想的设备理应具有多部照相机,并且能够发送实时视频,所述实时视频是来自至少两部照相机的视频的合成。 Apparatus should preferably have multiple cameras, and can transmit real-time video, the composite video is a video in real time from at least two cameras. 就处理多个拍摄的视频流的设备,和与所述设备连接、负责处理实时视频流的传输的网络来说,鉴于可供便携式设备利用的资源有限,这是一个尤其困难的问题。 Processing equipment on multiple captured video stream, and connected to the device, responsible for handling the transmission of real-time video stream networks, in view of the limited use of portable devices available resources, this is a particularly difficult problem.

发明内容 SUMMARY

[0004] 本发明的一些实施例提供具有两部能够拍摄照片和视频的照相机的移动设备。 [0004] Some embodiments of the present invention to provide a mobile device having two cameras capable of taking photos and videos of. 一些实施例的移动设备具有显示拍摄的照片图像和视频图像的显示屏。 Some embodiments of the mobile device having a photo image and a captured video image display screen. 它还包括保存拍摄的图像以便稍后传送给另一个设备的存储器。 It also includes images taken in order to save memory later transferred to another device. 所述设备还具有网络接口,所述网络接口允许该设备在多个设备的用户之间的实时通信会话期间,把拍摄的图像传送给一个或多个设备。 The apparatus also has a network interface, the network interface device that allows real-time during a communication session between a plurality of user devices, the captured image transmitted to the one or more devices. 所述设备还包括编码器,它可使用所述编码器对拍摄的图像编码,以便本地存储或者传送给另一个设备。 The apparatus further includes an encoder, it can use the encoder for encoding an image capturing order stored locally or transmitted to another device. 所述移动设备还包括解码器,所述解码器允许该设备对在实时通信会话期间由另一个设备拍摄的图解解码,或者对本地保存的图像解码。 The mobile device further includes a decoder, allowing the decoder decodes the devices illustrated during a realtime communication session captured by another device, or the local decoded image stored.

[0005] 涉及拍摄的视频图像的传输的实时通信会话的一个例子是视频会议。 [0005] An example of real-time communication session involving transmission of a video image is captured video conference. 在一些实施例中,移动设备在视频会议期间的任意特定时间,只能够传送一部照相机拍摄的视频图像。 In some embodiments, a mobile device at any particular time during the video conference, only capable of transmitting a video camera. 不过,在其它实施例中,移动设备能够在视频会议或者其它实时通信会话期间,同时传送来自其两部照相机拍摄的视频图像。 However, in other embodiments, the mobile device can be a video conference or other realtime communication session during which simultaneous transmission of video images from two cameras.

[0006] 在与另一个设备的视频会议期间,一些实施例的移动设备通过连同由其一部或两部照相机拍摄的视频一起,传送其它类型的内容。 [0006] During a video conference with another device, some of the embodiments of the mobile device with the video captured by a camera with two or transmitting other types of content. 这种其它内容的一个例子包括在该设备的一部照相机拍摄用于视频会议的视频时,由设备的另一部照相机拍摄的低分辨率或高分辨率照片图像。 An example of such other content comprises capturing a video for video conferencing in the device when a camera, captured by another camera apparatus of a low resolution or high resolution photographic images. 这种其它内容的其它例子包括(1)保存在设备上的文件和其它内容,(2)设备的屏幕显示(即,显示在设备的屏幕上的内容),(3)在视频会议或者其它实时通信会话期间,从另一个设备接收的内容,等等。 Other examples of such other content include (1) Save the file and other content on the device, the screen (2) of the display device (i.e., display contents on the screen of the device), (3) or other real-time video conferences during the communication session, the content received from another apparatus, and the like.

[0007] -些实施例的移动设备采用新颖的会议中(in-conference)调整技术,在视频会议期间进行调整。 [0007] - Some mobile devices employ a novel embodiment of the session (in-conference) adjustment techniques, be adjusted during a video conference. 例如,在视频会议期间只传送一部照相机拍摄的视频时,一些实施例的移动设备能够动态切换成传送由其另一部照相机拍摄的视频。 For example, during a video conference only transmit a video captured by the camera, some mobile devices can be dynamically switched embodiments to transmit a video captured by the other camera. 在这种情况下,一些实施例的移动设备把这种切换通知参与视频会议的任何其它设备,使得所述其它设备能够在它那一端提供由这两部照相机拍摄的视频之间的平滑转换。 In this case, some embodiments of the mobile device to any other such switching devices participating in the video conference notice, such that the other device capable of providing a smooth transition between the two videos captured by the camera in which that end of it.

[0008] 在一些实施例中,切换照相机的请求不仅能够起源于在视频会议期间,在其照相机之间进行切换的"本地"设备,而且还能够起源于正接收本地设备拍摄的视频的另一个"远程"设备。 Another [0008] In some embodiments, the camera can not only handover request originated during the video conference, to switch between the camera "local" device, but also the origin of the local device is receiving video photographed "remote" device. 此外,允许一个设备指令另一个设备切换照相机只是一些实施例的设备的远程控制能力的一个例子。 Further, a device to allow the other device to switch instruction is only one example of the camera remote control device to some embodiments. 在一些实施例中,能够远程向设备指令的其它操作的例子包括曝光调整操作(例如,自动曝光),焦距调整操作(例如,自动聚焦)等等。 In some embodiments, remotely including exposure adjustment operation (e.g., automatic exposure), a focal length adjustment operation (e.g., auto focus) and the like to the operation of the device Examples of other instructions. 能够本地或远程指定的新颖的会议中调整的另一个例子是识别拍摄视频中的感兴趣区域(R0I),和使用所述R0I识别修改拍摄照相机的行为,修改带有拍摄照相机的设备的图像处理操作,或者修改带有拍摄照相机的设备的编码操作。 Another example of a novel can be local or remote meeting the specified adjustment is to identify the region of interest captured video (R0I), and using said identification to modify the behavior R0I imaging camera, the modified image processing apparatus with a camera photographing operations, or modify the operation of the coding apparatus with the imaging camera.

[0009] -些实施例的新颖的会议中调整的又一个例子涉及设备产生的合成视频显示的实时修改。 [0009] - a further example of new meeting some embodiments relate to real-time changes to adjust the video synthesizing apparatus generates display. 具体地说,在一些实施例中,移动设备产生同时显不由一个或多个设备的多部照相机拍摄的多个视频的合成显示。 Specifically, in some embodiments, the mobile device generates a plurality of video while significantly help synthesis of multiple camera imaging one or more display devices. 在一些情况下,合成显示把视频放在相邻的显示区中(例如,放在相邻的窗口中)。 In some cases, the synthesized and displayed on the video display area adjacent (e.g., located in an adjacent window). 在其它情况下,合成显示是画中画(PIP)显示,所述PIP显示包括显示两个不同视频的至少两个显示区,其中一个显示区是背景主显示区,另一个显示区是重叠在背景主显示区上的前景插入显示区。 In other cases, synthetic PIP is displayed (PIP) display, the PIP display comprises displaying the at least two display areas of two different video in which a background display area is a main display area, the display area is superimposed on the other background of the foreground on the main display region into the display region.

[0010] 一些实施例中的合成视频显示的实时修改涉及响应于用户对显示区的选择和移动,在合成显示内移动一个或多个显示区。 [0010] In some real-time modification of the embodiment relates to a video display composite response to user selection and movement of the display area, moving one or more display areas in the composite display. 当提供合成显示的设备的屏幕旋转时,一些实施例还在视频会议期间旋转该合成显示。 When the composite display device provides a screen, the rotation of the synthesis of some embodiments also indicated during the video conference. 另外,一些实施例的移动设备允许设备的用户交换(swap)PIP显示中的视频(即,使前景插入显示中的视频出现在背景主显示中,而使背景主显示中的视频出现在前景插入显示中)。 Further, some mobile devices embodiment allows users to exchange video apparatus (swap) PIP display (i.e. the video display is inserted into the foreground appear in the background of the main display, the background appears in the main display video inserted in the foreground display).

[0011] 前面的发明内容意图简单介绍本发明的一些实施例。 [0011] The foregoing summary is intended to introduce some simple embodiments of the invention of the present invention. 并不意味是在本文献中公开的所有发明主题的介绍或综述。 Introduction or synthesis is not meant that all of the subject matter disclosed in this document. 下面的具体实施方式和在具体实施方式中涉及的附图将进一步说明在所述发明内容中描述的实施例,以及其它实施例。 The following detailed description and accompanying drawings relate to the specific embodiment will be further described embodiments, and other embodiments of the embodiments described in the context of the invention. 因此,要理解本文献描述的所有实施例,需要完整地考察发明内容、具体实施方式和附图。 Thus, the present embodiment is to be understood that all embodiments described in the literature, require complete inspection summary, detailed description and accompanying drawings.

附图说明 BRIEF DESCRIPTION

[0012] 附加权利要求中陈述了本发明的新颖特征。 [0012] The appended claims set forth the novel features of the invention. 不过,为了说明,在下述附图中表示了本发明的几个实施例。 However, for purposes of illustration, in the following drawings represent several embodiments of the present invention.

[0013] 图1图解说明一些实施例的合成显示。 [0013] Figure 1 illustrates some embodiments of the composite display.

[0014] 图2图解说明一些实施例的另一种合成显示。 [0014] Figure 2 illustrates other embodiments of a synthetic display.

[0015] 图3概念地图解说明一些实施例的双照相机移动设备的视频处理和编码模块的软件体系结构。 [0015] Figure 3 conceptually illustrates a software architecture for video processing and encoding module dual camera mobile device embodiment.

[0016] 图4概念地图解说明一些实施例的拍摄图像处理单元。 [0016] Figure 4 conceptually illustrates some of the captured image processing unit of the embodiment.

[0017] 图5概念地图解说明基于不同的垂直消隐间隔(VBI)的不同帧速率的例子。 [0017] Figure 5 conceptually illustrates an example of the different frame rates of different vertical blanking interval (the VBI) based.

[0018] 图6概念地图解说明基于不同VBI的不同隔行扫描帧速率的例子。 [0018] Figure 6 conceptually illustrates an example of an interlaced frame rates based on various different VBI's.

[0019] 图7概念地图解说明一些实施例的另一个拍摄图像处理单元。 [0019] FIG 7 conceptually illustrates some of the captured image processing unit to another embodiment.

[0020] 图8概念地图解说明一些实施例的另一个拍摄图像处理单元。 [0020] Figure 8 conceptually illustrates some of the captured image processing unit to another embodiment.

[0021] 图9概念地图解说明一些实施例的双照相机移动设备的视频会议和处理模块的软件体系结构。 [0021] Figure 9 conceptually illustrates a software architecture for video conferencing and processing module dual camera mobile device embodiment.

[0022] 图10概念地图解说明一些实施例的示例视频会议请求消息接发序列。 [0022] Figure 10 conceptually illustrates some exemplary embodiments of the video conference request messaging sequence.

[0023] 图11图解说明一些实施例的视频会议设置操作的用户界面。 [0023] FIG. 11 illustrates a number of embodiments of a video conference setting user interface operation.

[0024] 图12图解说明一些实施例的接受视频会议邀请的用户界面。 [0024] Figure 12 illustrates some of the video conference to accept the invitation user interface embodiments.

[0025] 图13图解说明一些实施例的接受视频会议邀请的另一用户界面。 [0025] Figure 13 illustrates another user interface to accept the number of the video conference invitation embodiment.

[0026] 图14图解说明一些实施例的视频会议设置操作的另一用户界面。 [0026] FIG. 14 illustrates a video conference a number of embodiments of the user interface to provide another operation.

[0027] 图15概念地图解说明一些实施例的设定视频会议的比特率的处理。 [0027] FIG. 15 illustrates conceptually the processing of the bit rate of some of the settings of the video conference embodiment.

[0028] 图16概念地图解说明一些实施例的双照相机移动设备的视频会议和处理模块的另一软件体系结构。 [0028] Figure 16 conceptually illustrates another videoconferencing software architecture dual camera mobile device embodiment and the processing module.

[0029] 图17概念地图解说明一些实施例的双照相机移动设备的另一软件体系结构。 [0029] Figure 17 conceptually illustrates the software architecture of some other mobile device dual camera embodiment.

[0030] 图18概念地图解说明由如图16中图解说明的一些实施例的视频会议管理器执行的处理。 [0030] FIG. 18 illustrates conceptually illustrated in FIG. 16 by the process described in some of the embodiments of the video conference manager executed.

[0031] 图19概念地图解说明一些实施例的时间噪声降低模块的软件体系结构。 [0031] FIG 19 conceptually illustrate some embodiments time noise reduction modules of the software architecture.

[0032] 图20概念地图解说明一些实施例的降低视频图像的时间噪声的处理。 [0032] FIG. 20 illustrates conceptually the processing time of the video image to reduce a number of embodiments of the noise.

[0033] 图21概念地图解说明由如图9中图解说明的一些实施例的图像处理管理器执行的处理。 [0033] FIG. 21 illustrates conceptually the processing illustrated in FIG. 9 manager some image processing performed by the embodiment.

[0034] 图22图解说明一些实施例的曝光调整操作的用户界面。 [0034] FIG. 22 illustrates an embodiment of some embodiments of the exposure adjustment operation of the user interface.

[0035] 图23图解说明一些实施例的焦距调整操作的用户界面。 [0035] FIG. 23 illustrates an embodiment of adjusting the focal length of some embodiments of the user interface operation.

[0036] 图24概念地图解说明由如图16中图解说明的一些实施例的图像处理管理器执行的透视校正处理。 [0036] Figure 24 conceptually illustrates a perspective view of some image correction process manager process performed by the embodiment illustrated in FIG 16 is described.

[0037] 图25概念地图解说明一些实施例的示例透视校正操作。 [0037] Figure 25 conceptually illustrates a perspective view of some exemplary embodiments of the correcting operation.

[0038] 图26概念地图解说明如图16中图解说明的一些实施例的编码器驱动器的软件体系结构。 [0038] Figure 26 conceptually illustrates the software architecture 16 of some embodiments illustrated in the encoder driver in the embodiment shown in FIG.

[0039] 图27概念地图解说明由如图26中图解说明的一些实施例的编码器驱动器执行的图像缩放处理。 [0039] FIG. 27 illustrates conceptually the image encoder of some embodiments illustrated drive 26 by performing scaling processing instructions.

[0040] 图28概念地图解说明由如图16中图解说明的一些实施例的解码器驱动器的软件体系结构。 [0040] Figure 28 conceptually illustrates some of the software architecture of the decoder driver 16 in the embodiment illustrated by the description of FIG.

[0041] 图29概念地图解说明由如图28中图解说明的一些实施例的解码器驱动器执行的图像提取处理。 [0041] Figure 29 conceptually illustrates an image extraction processing decoder driver of some embodiments illustrated in Figure 28 by the execution of the instructions.

[0042] 图30图解说明一些实施例的包括两个速率控制器的编码器驱动器。 [0042] FIG. 30 illustrates two rate controller comprises a number of drive encoder embodiment.

[0043] 图31概念地图解说明如图16中图解说明的一些实施例的连网管理器的软件体系结构。 [0043] Figure 31 conceptually illustrates the software architecture of some of the networking manager 16 embodiment illustrated in FIG.

[0044] 图32图解说明一些实施例的锁定到角落(snap to corner)操作的用户界面。 [0044] FIG. 32 illustrates a number of locking the corner (snap to corner) a user interface of the operation of the embodiment.

[0045] 图33图解说明一些实施例的锁定到角落操作的另一用户界面。 [0045] FIG 33 illustrate some embodiments of the lock to the other corner of the user interface operation.

[0046] 图34图解说明一些实施例的PIP显示旋转操作的用户界面。 [0046] FIG. 34 illustrates a display of some embodiments of the PIP rotational operation of the user interface.

[0047] 图35图解说明一些实施例的PIP显示旋转操作的另一用户界面。 [0047] FIG 35 illustrate some embodiments of the PIP display rotation operation of another user interface.

[0048] 图36图解说明一些实施例的PIP显示旋转操作的另一用户界面。 [0048] FIG 36 illustrate some embodiments of the PIP display rotation operation of another user interface.

[0049] 图37图解说明一些实施例的PIP显示旋转操作的另一用户界面。 [0049] FIG 37 illustrate some embodiments of the PIP display rotation operation of another user interface.

[0050] 图38图解说明一些实施例的缩放PIP显示中的前景插入显示区的用户界面。 [0050] FIG. 38 illustrates a prospect of some embodiments of the PIP display scaling insert a user interface display area.

[0051] 图39图解说明一些实施例的缩放PIP显示中的插入显示区的另一用户界面。 [0051] FIG. 39 illustrates another user interface display area of ​​the insertion of some embodiments of the scaled PIP display.

[0052] 图40图解说明一些实施例的缩放PIP显示中的插入显示区的另一用户界面。 [0052] FIG. 40 illustrates another user interface display area of ​​the insertion of some embodiments of the scaled PIP display.

[0053] 图41图解说明一些实施例的缩放PIP显示中的插入显示区的另一用户界面。 [0053] FIG. 41 illustrates another user interface display area of ​​the insertion of some embodiments of the scaled PIP display.

[0054] 图42图解说明一些实施例的识别显示画面中的感兴趣区域的用户界面。 [0054] FIG. 42 illustrates the identification of some embodiments of the display screen user interface region of interest.

[0055] 图43图解说明一些实施例的识别显示画面中的感兴趣区域的另一用户界面。 [0055] FIG. 43 illustrates the identification of some embodiments shows another embodiment of a user interface screen in the region of interest.

[0056] 图44图解说明一些实施例的识别显示画面中的感兴趣区域的另一用户界面。 [0056] FIG. 44 illustrates the identification of some embodiments shows another embodiment of a user interface screen in the region of interest.

[0057] 图45图解说明一些实施例的在双照相机移动设备上进行本地切换照相机操作的处理。 [0057] FIG. 45 illustrates a process of some embodiments is a local switch camera operation on the dual camera mobile device.

[0058] 图46图解说明一些实施例的切换照相机操作的用户界面。 [0058] FIG. 46 illustrates a camera operation switch of some embodiments of the user interface.

[0059] 图47图解说明一些实施例的切换照相机操作的另一用户界面。 [0059] FIG. 47 illustrates another user interface of some of the switching operation of the embodiment of the camera.

[0060] 图48图解说明一些实施例的切换照相机操作的另一用户界面。 [0060] FIG. 48 illustrates another user interface of some of the switching operation of the embodiment of the camera.

[0061] 图49图解说明一些实施例的切换照相机操作的另一用户界面。 [0061] Figure 49 illustrates another user interface of some of the switching operation of the embodiment of the camera.

[0062] 图50图解说明一些实施例的在双照相机移动设备上进行远程切换照相机操作的处理。 [0062] FIG. 50 illustrates a process of some remote camera operation switch on the dual camera mobile apparatus of the embodiment.

[0063] 图51图解说明一些实施例的远程控制切换照相机操作的用户界面。 [0063] FIG. 51 illustrates a remote control switch to some embodiments of the user interface of the camera operation.

[0064] 图52图解说明一些实施例的远程控制切换照相机操作的另一用户界面。 [0064] FIG. 52 illustrates an embodiment of a remote control of some embodiments of the user interface is switched to another camera operation.

[0065] 图53图解说明一些实施例的远程控制切换照相机操作的另一用户界面。 [0065] FIG. 53 illustrates an embodiment of a remote control of some embodiments of the user interface is switched to another camera operation.

[0066] 图54图解说明一些实施例的远程控制切换照相机操作的另一用户界面。 [0066] FIG. 54 illustrates an embodiment of a remote control of some embodiments of the user interface is switched to another camera operation.

[0067] 图55概念地图解说明一些实施例的进行曝光调整操作的处理。 [0067] Figure 55 conceptually illustrates a process of some exposure adjustment operation of the embodiment.

[0068] 图56图解说明一些实施例的进行曝光调整操作的用户界面。 [0068] FIG 56 illustrates a user interface of some exposure adjustment operation of the embodiment.

[0069] 图57图解说明一些实施例的进行曝光调整操作的另一用户界面。 [0069] Figure 57 illustrates another user interface of some exposure adjustment operation of the embodiment.

[0070] 图58图解说明一些实施例的进行曝光调整操作的另一用户界面。 [0070] FIG. 58 illustrates some embodiments of the exposure adjustment operation of another user interface.

[0071] 图59概念地图解说明由如图16中图解说明的一些实施例的图像处理管理器执行的曝光调整处理。 [0071] FIG. 59 illustrates conceptually illustrated in FIG. 16 is exposed by some of the described embodiments of the image processing manager performing adjustment processing.

[0072] 图60概念地图解说明一些实施例的曝光调整操作。 [0072] FIG 60 conceptually illustrate some embodiments of the exposure adjustment operation.

[0073] 图61概念地图解说明一些实施例的执行焦距调整操作的处理。 [0073] Figure 61 conceptually illustrates a process of some embodiments perform a focus adjustment operation.

[0074] 图62图解说明一些实施例的焦距调整操作的用户界面。 [0074] FIG. 62 illustrates a focus adjustment operation of some embodiments of a user interface.

[0075] 图63图解说明一些实施例的焦距调整操作的另一用户界面。 [0075] FIG 63 illustrate some embodiments of the focal length adjusting operation of another user interface.

[0076] 图64图解说明一些实施例的焦距调整操作的另一用户界面。 [0076] FIG 64 illustrate some embodiments of the focal length adjusting operation of another user interface.

[0077] 图65图解说明一些实施例的关于从一个或多个双照相机移动设备拍摄的视频的不同显示安排。 [0077] FIG. 65 illustrates a number of different arrangements for displaying video captured from a camera mobile devices or more double embodiment.

[0078] 图66图解说明一些实施例的把插入视频的前景叠加在PIP显示中的背景视频上的用户界面。 [0078] FIG. 66 illustrates some of the user interface on the inserted video display foreground superimposed on the background video PIP embodiment.

[0079] 图67图解说明一些实施例的确定视频图像的前景的技术。 [0079] FIG. 67 illustrates a foreground video image determined to some embodiments of the technology.

[0080] 图68图解说明一些实施例的在视频会议期间,交换PIP显示中的插入显示和背景显示的用户界面。 [0080] FIG 68 illustrates a user interface of some embodiments during a video conference, and the exchange inset display background display PIP display.

[0081] 图69图解说明一些实施例的锁定到角落操作的用户界面。 [0081] FIG 69 illustrate some embodiments of the locking corner of the user interface to operate.

[0082] 图70图解说明一些实施例的锁定到角落和推送操作的用户界面。 [0082] FIG 70 illustrate some embodiments of the lock and pushed into the corner of the operation of the user interface.

[0083] 图71图解说明一些实施例的PIP显示旋转操作的用户界面。 [0083] FIG. 71 illustrates a display of some embodiments of the PIP rotational operation of the user interface.

[0084] 图72图解说明一些实施例的PIP显示旋转操作的另一用户界面。 [0084] FIG 72 illustrate some embodiments of the PIP display rotation operation of another user interface.

[0085] 图73图解说明一些实施例的在视频会议期间,从两个远程视频中选择一个视频的用尸界面。 [0085] FIG. 73 illustrates during a video conference, a video selection interface corpse from two remote video in some embodiments.

[0086] 图74图解说明一些实施例的在视频会议期间,从两个本地视频中选择一个视频的用户界面。 [0086] FIG. 74 illustrates some embodiments during a video conference, a video from the selected two local video user interface.

[0087] 图75图解说明一些实施例的会前选择用于视频会议的视频的用户界面。 [0087] FIG 75 illustrates a user interface for selecting a video conference video may Some prior embodiments.

[0088] 图76图解说明一些实施例的由双照相机移动设备拍摄的两个视频之间的带宽分配的例子。 [0088] FIG. 76 illustrates an example of some of the bandwidth allocation between the two mobile video captured by the camera device bis embodiment.

[0089] 图77概念地图解说明一些实施例的管理双照相机移动设备的速率控制器的判优器模块。 [0089] FIG. 77 illustrates conceptually the arbiter a number of management modules dual camera mobile device of an embodiment of the rate controller.

[0090] 图78概念地图解说明一些实施例的对由双照相机移动设备的照相机拍摄的图像进行编码的方法。 [0090] Figure 78 conceptually illustrates some of the camera, the images captured by the dual camera mobile device for encoding embodiment.

[0091] 图79概念地图解说明一些实施例的对由双照相机移动设备的照相机拍摄的图像进行编码的另一方法。 [0091] Another method of some embodiments of the image photographed by the camera of the dual camera mobile device for encoding 79 conceptually illustrates FIG.

[0092] 图80图解说明在图79中图解说明的方法的示例图像合成。 [0092] FIG. 80 illustrates an example of the image synthesizing method described illustrated in FIG. 79.

[0093] 图81概念地图解说明一些实施例的对由双照相机移动设备的照相机拍摄的图像进行编码的另一方法。 [0093] Another method of some embodiments of the image photographed by the camera of the dual camera mobile device for encoding 81 conceptually illustrates FIG.

[0094] 图82概念地图解说明一些实施例的对由双照相机移动设备的照相机拍摄的图像进行解码的方法。 [0094] Figure 82 conceptually illustrates some of the camera, the images captured by the camera mobile device dual decoding embodiment.

[0095] 图83概念地图解说明一些实施例的对由双照相机移动设备的照相机拍摄的图像进行解码的另一方法。 [0095] Another method of some embodiments of the image photographed by the camera of the dual camera mobile device 83 of FIG decoding conceptually illustrated.

[0096] 图84概念地图解说明一些实施例的双照相机移动设备的视频会议和处理模块的另一软件体系结构。 [0096] Figure 84 conceptually illustrates another videoconferencing software architecture dual camera mobile device embodiment and the processing module.

[0097] 图85图解说明一些实施例的多方视频会议的用户界面。 [0097] FIG 85 illustrates a user interface of some embodiments of the multiparty video conferencing embodiment.

[0098] 图86图解说明一些实施例的多方视频会议的另一用户界面。 [0098] FIG. 86 illustrates another user interface of some embodiments of the multiparty video conferencing embodiment.

[0099] 图87图解说明一些实施例的多方视频会议的另一用户界面。 [0099] FIG. 87 illustrates another user interface of some multiparty video conferencing embodiment.

[0100] 图88概念地图解说明一些实施例的应用编程接口(API)体系结构。 [0100] Figure 88 conceptually illustrates several embodiments of the application programming interface (API) architecture.

[0101] 图89图解说明一些实施例的双照相机移动计算设备的体系结构。 [0101] FIG 89 illustrates the architecture of some dual camera mobile computing device embodiment.

[0102] 图90概念地图解说明一些实施例的触摸输入/输出(I/O)设备。 [0102] Figure 90 conceptually illustrates an embodiment of some embodiments of the touch input / output (I / O) devices.

[0103] 图91概念地图解说明一些实施例的示例通信系统。 [0103] Figure 91 conceptually illustrates a communication system according to some example embodiments.

[0104] 图92概念地图解说明一些实施例的另一示例通信系统。 [0104] FIG 92 conceptually illustrates some of the communication system according to another exemplary embodiment.

具体实施方式 detailed description

[0105] 在下面的描述中,出于解释说明的目的,陈述了众多的细节。 [0105] In the following description, for purposes of explanation, numerous details are set forth. 不过,本领域的普通技术人员会认识到可在不利用这些具体细节的情况下实践本发明。 However, those of ordinary skill in the art will recognize that the present invention may be practiced without these specific details. 在其它情况下,为了因不必要的细节而使本发明的描述模糊不清,以方框图的形式表示了公知的结构和设备。 In other instances, the unnecessary detail in order to obscure the description of the present invention, showing a well-known structures and devices are shown in block diagram form.

[0106] 本发明的一些实施例提供一种具有两部能够拍摄照片和视频的照相机的移动设备。 Some embodiments [0106] of the present invention to provide a camera capable of taking two pictures and videos of the mobile device. 移动设备的例子包括移动电话机、智能电话机、个人数字助理(PDA)、膝上型计算机、平板个人计算机或者任何其它类型的移动计算设备。 Examples of mobile devices include mobile telephone, a smart phone, a personal digital assistant (PDA), a laptop computer, a tablet personal computer or any other type of mobile computing device. 本文中使用的照片指的是按单张拍摄模式每次一张地,或者按快速拍摄模式每次几张地由照相机拍摄的静止照片图像。 Photo used herein, refers to the single shooting mode according to one at a time, or press the fast shooting mode every time a few places still photo images taken by the camera. 另一方面, 视频指的是照相机以特定速率(通常称为帧速率)拍摄的一系列视频图像。 On the other hand, it refers to a series of video images of video cameras at a particular rate (typically referred to as frame rate) taken. 拍摄视频的典型帧速率是25帧/秒(fps)、30fps和60fps。 Typical shooting video frame rate is 25 frames / second (fps), 30fps, and 60fps. 一些实施例的移动设备的照相机能够以这些和其它帧速率拍摄视频图像(即,视频帧)。 Some camera mobile device embodiment can capture video images (i.e. video frames) to these and other frame rate.

[0107] 一些实施例的移动设备(1)能够显示拍摄的照片图像和视频图像,(2)能够保存拍摄的图像,以便稍后传送给另一个设备,(3)能够在多个设备的多位用户之间的实时通信会话期间,把拍摄的图像传送给一个或多个设备,和(4)能够对拍摄的图像编码,以便本地存储或者传送给另一个设备。 [0107] The mobile device (1) some embodiments can display photos and video images captured, (2) capable of storing the captured image for transmission to another device, (3) a plurality of devices capable of multiple later during real-time communication session between users, the captured image transmitted to the one or more devices, and (4) capable of encoding the image captured, transferred to local storage or to another device.

[0108] 涉及拍摄的视频图像的传输的实时通信会话的一个例子是视频会议。 [0108] An example of real-time communication session involving transmission of a video image is captured video conference. 在一些实施例中,在视频会议期间的任意特定时间,移动设备只能够传送一部照相机拍摄的视频图像。 In some embodiments, at any particular time during the video conference, the mobile device only capable of transmitting a video camera. 不过,在其它实施例中,在视频会议或者其它实时通信会话期间,移动设备能够同时传送其两部照相机拍摄的视频图像。 However, in other embodiments, during a video conference session, or other real-time communication, the mobile device can simultaneously transmit a video image which two cameras.

[0109] 一些实施例的移动设备产生合成显示,所述合成显示包含由一个或多个设备的多部照相机拍摄的多个视频的同时显示。 [0109] Some embodiments of the mobile device to generate a composite display, the display comprising synthesizing a plurality of video captured by multiple cameras or a plurality of devices are simultaneously displayed. 在一些情况下,合成显示把视频放在相邻的显示区中(例如,放在相邻的窗口中)。 In some cases, the synthesized and displayed on the video display area adjacent (e.g., located in an adjacent window). 图1图解说明合成显示100的一个这种例子,合成显示100 包括两个相邻的显不区105和110,这两个显不区105和110同时显不由一个设备的两部照相机拍摄的,或者由参加视频会议的两个不同设备的两部照相机拍摄的两个视频。 Figure 1 illustrates such a display 100 Synthesis example composite display 100 comprises two adjacent areas 105 and 110 is not significant, the two regions 105 and 110 is not significant while significantly help a device of the two taken by the camera, two or video captured by two cameras two different devices to participate in video conferences.

[0110] 在其它情况下,合成显示是包括显示两个不同视频的至少两个显示区的PIP显示,其中一个显示区是背景主显示区,另一个显示区是重叠在背景主显示区之上的前景插入显示区。 [0110] In other cases, a composite display comprises displaying the at least two display areas PIP two different video display, wherein the display area is a background of the main display area, the other display area is superimposed on the background of the main display area the foreground inset display area. 图2图解说明合成PIP显示200的一个这种例子。 FIG 2 illustrates an example of such a synthetic PIP display 200. 合成PIP显示200包括背景主显示区205和重叠在背景主显示区之上的前景插入显示区210。 Synthesis PIP display 200 includes a background 205 and a main display area overlaps the foreground over the background main display region into display area 210. 这两个显示区205和210 同时显示由一个设备的两部照相机拍摄的,或者由参加视频会议的两个不同设备的两部照相机拍摄的两个视频。 These two display areas 205 and 210 are captured by two cameras simultaneously displaying a device, or both video captured by two cameras of two different devices to participate in a video conference. 尽管在本文中举例说明和讨论的示例合成PIP显示类似于在背景主显示区205内显示整个前景插入显示区210的合成PIP显示200,不过具有重叠在背景主显示区205之上,但是并不完全在背景主显示区205内的前景插入显示区210的其它合成PIP显示也是可能的。 Although illustrated and discussed herein are exemplary synthetic PIP display similar to the display in the background main display area 205 is inserted into the entire foreground synthetic PIP display area 210 display 200, but has superimposed on the background main display area 205, but does not entirely in the background in the foreground of the main display area 205 is inserted into the display region to display other synthetic PIP 210 is also possible.

[0111] 除了在与另一个设备的视频会议期间传送视频内容之外,一些实施例的移动设备能够随同会议的视频内容一起传送其它类型的内容。 [0111] In addition to transmitting the video content during a video conference with another device, the mobile device of some embodiments other types of content can be transmitted along with the video content of the meeting. 这种其它内容的一个例子包括在设备的照相机之一正在拍摄用于视频会议的视频时,由该设备的另一部照相机拍摄的低分辨率或高分辨率照片图像。 An example of such other content included in the one of the camera shooting video of the device is used for a video conference captured by the other camera device or a high-resolution low-resolution photograph image. 这种其它内容的其它例子包括(1)保存在设备上的文件和其它内容,(2)设备的屏幕显示(即,显示在设备的屏幕上的内容),(3)在视频会议或者其它实时通信会话期间,从另一个设备接收的内容,等等。 Other examples of such other content include (1) Save the file and other content on the device, the screen (2) of the display device (i.e., display contents on the screen of the device), (3) or other real-time video conferences during the communication session, the content received from another apparatus, and the like.

[0112] 一些实施例的移动设备采用新颖的会议中调整技术,在视频会议期间进行调整。 [0112] Some embodiments of the mobile device using the novel technology conference adjustment, adjustment during a video conference. 例如,当在视频会议期间只传送一部照相机拍摄的视频时,一些实施例的移动设备能够动态切换成传送由其另一部照相机拍摄的视频。 For example, when the video camera through a transmission only during a video conference, some mobile devices can be dynamically switched embodiments to transmit a video captured by the other camera. 在这种情况下,一些实施例的移动设备把这种切换通知参加视频会议的任何另一个设备,使得所述另一个设备能够在它那一端提供这两部照相机拍摄的视频之间的平滑转换。 A smooth transition between any other device in this case, some embodiments of the mobile device such switching notification of the video conference, so that the other device capable of providing both cameras capturing video at that end of its .

[0113] 在一些实施例中,切换照相机的请求不仅能够起源于在视频会议期间,在其照相机之间进行切换的"本地"设备,而且还能够起源于正接收本地设备拍摄的视频的另一个"远程"设备。 Another [0113] In some embodiments, the camera can not only handover request originated during the video conference, to switch between the camera "local" device, but also the origin of the local device is receiving video photographed "remote" device. 此外,允许一个设备指令另一个设备切换照相机只是一些实施例的设备的远程控制能力的一个例子。 Further, a device to allow the other device to switch instruction is only one example of the camera remote control device to some embodiments. 在一些实施例中,能够远程向设备指令的其它操作的例子包括曝光调整操作(例如,自动曝光),焦距调整操作(例如,自动聚焦)等等。 In some embodiments, remotely including exposure adjustment operation (e.g., automatic exposure), a focal length adjustment operation (e.g., auto focus) and the like to the operation of the device Examples of other instructions. 能够本地或远程指定的新颖的会议中调整的另一个例子是识别拍摄视频中的感兴趣区域(ROI),和使用所述ROI识别修改拍摄照相机的行为,修改带有拍摄照相机的设备的图像处理操作,或者修改带有拍摄照相机的设备的编码操作。 Another example of a novel can be local or remote meeting the specified adjustment is to identify the region of interest captured video (ROI), the ROI identification and use to modify the behavior of the imaging camera, the modified image processing apparatus with a camera photographing operations, or modify the operation of the coding apparatus with the imaging camera.

[0114] 一些实施例的新颖的会议中调整的又一个例子涉及设备产生的合成视频显示的实时修改。 [0114] Some examples of still another embodiment of the novel session embodiment relates to real-time changes in the adjusted composite video display device generates. 具体地说,在一些实施例中,合成视频显示的实时修改涉及响应于用户对显示区的选择和移动,在合成显示内移动一个或多个显示区。 Specifically, in some embodiments, real-time video display relates to modified synthetic response to user selection and movement of the display area, moving one or more display areas in the composite display. 当提供合成显示的设备的屏幕旋转时,一些实施例还在视频会议期间旋转该合成显示。 When the composite display device provides a screen, the rotation of the synthesis of some embodiments also indicated during the video conference. 另外,一些实施例的移动设备允许设备的用户翻转(flip)PIP显示中的视频的顺序(即,使前景插入显示中的视频出现在背景主显示中,而使背景主显示中的视频出现在前景插入显示中)。 Further, some mobile devices embodiment allows the device user turns the video sequence (flip) PIP display (i.e. the video display is inserted into the foreground appear in the background of the main display, the background displayed in the main video appears foreground inset display).

[0115] 下面说明几个更详细的实施例。 [0115] will be described in more detail several embodiments. 第I节提供一些实施例的视频处理体系结构的说明。 Section I provides some indication of the video processing architecture embodiment. 随后,第II节描述一些实施例的拍摄图像处理单元。 Subsequently, Section II describes some of the captured image processing unit of the embodiment. 在一些实施例中,拍摄图像处理单元是设备的负责处理由设备的照相机拍摄的原始图像的组件。 In some embodiments, the captured image processing unit is a component of the original image captured by the camera apparatus is responsible for the processing apparatus.

[0116] 接下来,第III节描述一些实施例的视频会议体系结构。 [0116] Next, Section III describes some of the video conference system structure of the embodiment. 第III节还描述一些实施例的视频会议模块,以及设置单照相机视频会议的几种方式。 Section III describes some video conference module further embodiment, a single camera is provided and several video conference. 之后,第IV节描述一些实施例的会议中调整和控制操作。 Thereafter, Section IV describe some embodiments to adjust the conference and control operations. 第V节随后描述一些实施例的在视频会议期间,传送和显示来自各个设备的多个视频的视频会议特征。 Section V of the subsequent description of some embodiments during a video conference, transfer and display the plurality of video from each video conference device characteristics. 接下来,第VI节描述在视频会议期间,随同非实时内容一起的实时视频的传输。 Next, Section VI described during the video conference, real-time video transmission with the accompanying non-real-time content. 最后,第VII节描述一些实施例的双照相机设备的硬件体系结构。 Finally, Section VII describes some of the hardware architecture of the dual camera apparatus of the embodiment.

[0117] I.视频拍摄和处理 [0117] I. video capture and processing

[0118] 图3概念地图解说明一些实施例的双照相机移动设备的视频处理和编码模块300。 [0118] Figure 3 conceptually illustrates a dual camera mobile apparatus of the embodiment of the video processing and encoding module 300. 在一些实施例中,模块300对由双照相机移动设备的照相机拍摄的图像进行处理,和对视频进行编码。 In some embodiments, module 300 of the image photographed by the camera of the dual camera mobile device is processed, and the video encoding. 如图3中所示,模块300包括拍摄图像处理单元(CIPU)驱动器305、媒体交换模块310、编码器驱动器320和视频处理模块325。 Shown in Figure 3, module 300 includes a captured image processing unit (the CIPU) driver 305, a media exchange module 310, the encoder driver 320 and video processing module 325.

[0119] 在一些实施例中,媒体交换模块310允许设备上的程序(媒体内容的使用者和制造者)交换媒体内容和关于媒体内容的处理的指令。 [0119] In some embodiments, the media exchange module 310 allows programs (manufacturer and user to the media content) on the switching device and instructions regarding media content of the media content processing. 在视频处理和编码模块300中,一些实施例的媒体交换模块310在视频处理模块325和CIPU驱动器305之间,以及在视频处理模块325和编码器驱动器320之间路由这些指令和媒体内容。 And encoding the video processing module 300, some media exchange module 310 in the embodiment of the video processing module 325 and between CIPU driver 305, and the routing between the video processing module 320 and the encoder 325 drives these instructions and media content. 为了使所述指令和媒体内容的路由更容易,一些实施例的媒体交换模块310提供供媒体内容的使用者和制造者使用的一组应用编程接口(API)。 And routing instructions to the media content more easily, some media exchange module 310 of the embodiment provides a set of application programming interface (API) for media content users and manufacturers use. 在一些这样的实施例中,媒体交换模块310是作为运行于双照相机移动设备上的操作系统的一部分的一组一个或多个架构。 In some such embodiments, the media exchange module 310 is run in as a set of one or more of a portion of the architecture of the operating system on a dual camera mobile device. 这种媒体交换模块310的一个例子是苹果公司提供的Core Media架构。 An example of such media exchange module 310 is provided by Apple Core Media framework.

[0120] 视频处理模块325对由设备的照相机拍摄的图像和/或视频进行图像处理。 [0120] The video image processing module 325 and / or video captured by the camera of the image processing apparatus. 这种操作的例子包括曝光调整操作、焦距调整操作、透视校正、动态范围调整、图像缩放、图像合成等等。 Examples of such operations include exposure adjustment operation, the focus adjustment operation, perspective correction, dynamic range adjustment, image scaling, image synthesis and so on. 在一些实施例中,一些图像处理操作也可由媒体交换模块310执行。 In some embodiments, some image processing operations are performed by the media exchange module 310. 例如,如图3 中所示,一些实施例的媒体交换模块310执行降低由设备的照相机拍摄的视频图像中的噪声的时间噪声降低(TNR)操作(例如,用TNR 315)。 For example, as shown in FIG. 3, performing a media exchange module 310 of some embodiments of the temporal noise reduction of a video image captured by the camera apparatus of noise reduction (TNR) operation (e.g., a TNR 315). 下面提供视频处理模块325和媒体交换模块310的这种图像处理操作的另外的例子。 The following examples provide a further video processing module 325 and a media switching module such image processing operations 310.

[0121] 通过媒体交换模块310,视频处理模块325与CIPU驱动器305和编码器驱动器320 接口,如上所述。 [0121] 310, and the video processing module 325 CIPU driver 305 and driver 320 interfaces the encoder switching module through the media, as described above. CIPU驱动器305充当拍摄图像处理单元(CIPU) 330和媒体交换模块310 之间的通信接口。 CIPU driver 305 serving as a captured image processing unit (CIPU) 330 and a communication interface between the media exchange module 310. 如以下进一步所述,CIPU 330是双照相机设备的组件,负责处理在设备的照相机的图像拍摄或者视频拍摄操作期间拍摄的图像。 As described further below, CIPU 330 is a component of a double camera apparatus is responsible for processing the image pickup apparatus or a camera image captured during a video capture operation. CIPU驱动器305通过媒体交换模块310,从视频处理模块325接收对来自设备的一部或两部照相机的图像和/或视频的请求。 CIPU driver 305 through the media exchange module 310, a video processing module 325 receives the request for images and / or one or two video cameras from the device. CIPU驱动器305把所述请求传递给CIPU 330,作为响应,从CIPU 330接收请求的图像和/或视频,CIPU驱动器305随后通过媒体交换模块310,把接收的图像和/或视频发送给视频处理模块325。 CIPU driver 305 the request to CIPU 330., as a response, and then 310, the received image transmitted from the image and / or video, CIPU driver CIPU 330. the request is received 305 through the media exchange module and / or video to the video processing module 325. 通过CIPU驱动器305和媒体交换模块310, 一些实施例的视频处理模块325还向CIPU 330发送指令,以便修改其一些操作(例如,修改照相机的帧速率、曝光调整操作、焦距调整操作等等)。 By CIPU driver 305 and media exchange module 310, some of the video processing module embodiment 325 further to CIPU 330. sends an instruction to modify some of its operations (e.g., a frame rate to change the camera, the exposure adjustment operation, the focus adjustment operation, etc.).

[0122] 编码器驱动器320充当媒体交换模块310和编码器硬件335 (例如,编码器芯片、 芯片系统上的编码组件等等)之间的通信接口。 The communication interface [0122] The encoder driver 320 acts as a media exchange module 310 and an encoder 335 hardware (e.g., encoder chip, the chip component coding system, etc.) between. 在一些实施例中,编码器驱动器320通过媒体交换模块310,从视频处理模块325接收图像和对图像编码的请求。 In some embodiments, the encoder from the video driver 320 receives the image processing module 325 and a request for the encoded image exchange module 310 through the media. 编码器驱动器320 把要编码的图像发送给编码器335,编码器335随后对图像进行照片编码或视频编码。 Encoder driver 320 transmits the image to be coded to the encoder 335, the encoder 335 then photographs the image encoding or video encoding. 当编码器驱动器320从编码器335接收到编码图像时,编码器驱动器320通过媒体交换模块310,把编码图像回送给视频处理模块325。 When the encoder driver 320 from the encoder 335 receives the encoded image, the encoder driver 320 through the media exchange module 310, the encoded image back to the video processing module 325.

[0123] 在一些实施例中,视频处理模块325能够对它从编码器接收的编码图像进行不同的操作。 [0123] In some embodiments, the video processing module 325 can perform different operations it coded image received from the encoder. 所述操作的例子包括把编码图像保存在设备的存储器中,通过设备的网络接口在视频会议中传送编码图像,等等。 Examples of the operations include the encoded image stored in the memory device, transmitting the encoded image in a video conference through a network interface device, and the like.

[0124] 在一些实施例中,视频处理和编码模块300的一些或全部模块被实现成操作系统的一部分。 [0124] In some embodiments, the video processing module, and some or all of the coding module 300 is implemented as part of the operating system. 例如,一些实施例把视频处理和编码模块300的所有四个组件305、310、320和325实现成设备的操作系统的一部分。 For example, some embodiments of the video processing module 300 and encoding all four components 305,310,320 and 325 implemented as part of the operating system of the device. 其它实施例把媒体交换模块310、CIPU驱动器305和编码器驱动器320实现成设备的操作系统的一部分,而把视频处理模块325作为在操作系统上运行的应用。 Other embodiments of the media exchange module 310, CIPU driver 305 and the encoder driver 320 implemented as part of the operating system of the device, and the video processing module 325 as an application running on an operating system. 另外,模块300的其它实现也是可能的。 Further, module 300 other implementations are also possible.

[0125] 现在说明在视频拍摄会话期间,视频处理和编码模块300的操作。 [0125] will now be described during a video capture session, and the encoding operation of the video processing module 300. 为了开始视频拍摄会话,视频处理模块325初始化为视频拍摄会话所需的几个组件。 To begin video capture session, a video processing module 325 is required to initialize the video recording sessions several components. 在一些实施例中,这些组件包括(1) CIPU 330,(2)视频处理模块325的比例缩放和合成模块(未示出),(3)视频处理模块325的图像处理模块(未示出),和(4)编码器335。 In some embodiments, these components include (1) CIPU 330, the ratio of (2) video processing module 325 of the scaling and synthesis module (not shown), (3) a video processing module 325 of the image processing module (not shown) and (4) encoder 335. 另外,当参与视频会议时, 一些实施例的视频处理模块325初始化网络管理器(未示出)。 Further, when the video conference participants, some of the embodiments of the video processing module 325 to initialize the network manager (not shown).

[0126] 通过媒体交换模块310和CIPU驱动器305,视频处理模块把其初始化请求发送给CIPU 330,以使设备的一部或两部照相机开始视频拍摄。 [0126] 310 and through the media exchange module CIPU driver 305, video processing module to send the request to initialize 330. CIPU, one or two so that the video camera starts shooting device. 在一些实施例中,所述请求指定需要拍摄视频的每部照相机的特定帧速率、曝光度和缩放比例大小。 In some embodiments, the request specifies a particular need to shoot a video frame rate of each camera, the exposure size and scaling. 响应所述请求,CIPU 330 开始以指定的帧速率、曝光度和缩放比例大小,返回来自所请求的照相机的视频图像。 Response to the request, CIPU 330 starts with the specified frame rate, exposure and size scaling, video images from the camera to return requested. 这些视频图像通过CIPU驱动器305和媒体交换模块310被返回给视频处理模块325,如上所述, 在把视频图像提供给视频处理模块325之前,媒体交换模块310对视频图像进行TNR操作。 The video driver 310 is returned by CIPU media exchange module 305 and to the video processing module 325, described above, in the video image to the video processing module 325 before, media exchange module 310 video images TNR operation. 在视频处理模块325,视频图像被保存在缓冲器(未示出)中,以便进行另外的图像处理。 In the video processing module 325, the video image is stored in a buffer (not shown), for further image processing.

[0127] 视频处理模块325的图像处理模块取回保存在缓冲器中的视频图像,以进行另外的视频处理。 [0127] The video processing module 325 of the image processing module retrieves a video image stored in the buffer, for additional video processing. 比例缩放和合成模块随后取回处理后的视频图像,以便比例缩放视频图像(如果是在设备的显示屏上的实时显示所必需的话)。 Scaling and synthesis module then retrieves the video image processing, so that scaling the video image (if it is real-time display on the display device if necessary). 在一些实施例中,该模块用由设备的两部照相机拍摄的图像,或者用在视频会议期间由设备的照相机以及另一个设备的照相机拍摄的图像创建合成图像,以便在设备上提供拍摄的视频图像的实时显示,或者创建供编码的合成视频图像。 In some embodiments, the module image captured by the camera device two, or create a composite image in the image captured by a camera during a video conference camera device and the other device in order to provide the video capture device real-time display of the image, or create for encoding composite video image.

[0128] 处理和/或合成后的视频图像通过编码器驱动器320和媒体交换模块310被提供给编码器335。 [0128] The processing of video images and / or post-synthesis module 310 are supplied to the encoder 335 through the encoder driver 320 and media exchange. 编码器335随后对视频图像编码。 The encoder 335 then encodes the video image. 编码图像随后被返回给视频处理模块325 (再次通过编码器驱动器320和媒体交换模块310),以便保存在设备上,或者在视频会议期间传输。 Encoded image is then returned to the video processing module 325 (again through the encoder driver 320 and media exchange module 310), to be stored on the device, or transmission during a video conference. 当设备参加视频会议时,网络管理器(由视频处理模块325初始化)随后取回这些编码图像,对编码图像分包,并通过设备的网络接口(未示出)把编码图像传送给一个或多个其它设备。 When the video conference equipment, a network manager (initialized by the video processing module 325) the encoded image is then retrieved, encoded sub-picture, and a network interface device (not shown) transmits the encoded image to one or more miscellaneous devices.

[0129] II.拍摄图像处理 [0129] II. The captured image processing

[0130] A.单管线 [0130] A. Single line

[0131] 由一些实施例的双照相机移动设备的照相机拍摄的图像是原始的未处理图像。 [0131] an image by a camera dual camera embodiment of the mobile device is captured raw unprocessed images. 在这些图像能够被用于其它操作,比如把图像传送给另一个设备(例如,在视频会议期间)、 保存图像、或者显示图像之前,这些图像需要变换到特定的色空间。 In these images it can be used for other operations, such as transferring the image to another device (e.g., during a video conference), save the image, or before image display, the images need to change to a particular color space. 另外,照相机拍摄的图像需要被处理,以校正错误和/或失真,以及调整图像的颜色、大小等等。 Further, the camera captured the image needs to be processed to correct errors and / or distortion, and color adjustment of the image size and the like. 因此,一些实施例在保存、传送和显示这种图像之前,对图像进行几种处理操作。 Thus, in some embodiments, before saving, transfer and display of such images, the image processing operation several. 所述图像的部分处理由CIPU 330执行。 Part of the image processing performed by CIPU 330.

[0132] 图4中图解说明了所述CIPU的一个例子。 [0132] FIG. 4 illustrates one example of the CIPU. 具体地说,图4概述地图解说明一些实施例的拍摄图像处理单元(CIPU)400。 More specifically, FIG 4 illustrates an outlined some of the captured image processing unit of the embodiment (CIPU) 400. CIPU 400包括或者每次只处理来自设备的照相机之一的图像,或者按照时分多路复用方式(即,按照时间交织方式)同时处理来自设备的两部照相机的图像的单处理管线(single processing pipeline)485。 CIPU 400 comprises one or handle only image from a camera device, or in accordance with a time division multiplexing manner (i.e., in a time interleaved manner) simultaneously process two single pipeline processing equipment from the camera image (single processing pipeline) 485. 可以不同地配置CIPU 400的处理管线485,以处理不同照相机的不同特性和/或操作设置。 CIPU 400 may be configured differently processing pipeline 485, in order to deal with the different characteristics of different cameras and / or operational settings. 一些实施例中的不同照相机特性的例子包括不同的分辨率、噪声传感器、透镜类型(固定透镜或变焦透镜),等等。 Some examples of characteristics of different cameras in the embodiment includes different resolutions, noise sensor, the type of lens (zoom lens or a fixed lens), and the like. 另外,在一些实施例中,设备能够按照其操作照相机的不同操作设置的例子包括图像分辨率大小、帧速率、变焦级别、曝光度等。 Further, in some embodiments, the device can include an image resolution size, frame rate, zoom level, exposure, etc. Examples of different operation according to the operation of the camera settings.

[0133] 如图4中所示,CIPU 400包括传感器模块415,行/帧缓冲器417,坏像素校正(BPC)模块420,透镜阴影(LS)模块425,去马赛克模块430,白平衡(WB)模块435, γ模块440,色空间变换(CSC)模块445,色调、饱和度和对比度(HSC)模块450,定标器模块455, 滤波器模块460,统计引擎465,两组寄存器470,和控制器模块475。 [0133] As shown in FIG, CIPU 400 4 includes a sensor module 415, the line / frame buffer 417, a bad pixel correction (BPC) module 420, a lens shading (LS) module 425, a demosaic module 430, a white balance (WB ) module 435, γ module 440, color space conversion (CSC) block 445, the hue, saturation and contrast (HSC) module 450, a scaling module 455, a filter module 460, statistics engine 465, two sets of registers 470, and The controller module 475. 在一些实施例中,CIPU 400的全部模块都是用硬件(例如,ASIC、FPGA、带有微控制器的S0C,等等)实现的,而在其它实施例中,CIPU 400的一些或全部模块是用软件实现的。 In some embodiments, the CIPU module 400 are all hardware (e.g., ASIC, FPGA, microcontrollers with S0C, and so on), while in other embodiments, some or all of the modules 400 of the CIPU It is implemented in software.

[0134] 如图4中所示,传感器模块415与设备的两部照相机的两个像素阵列410a和410b,及两组传感器405a和405b通信耦接。 [0134] FIG two pixel arrays in the two cameras, the sensor module 415 and the device 4 410a and 410b, 405a and 405b, and two sets of sensors coupled communication. 在一些实施例中,通过每个照相机传感器的移动行业处理器接口(MIPI),使所述通信耦接更容易。 In some embodiments, each of the camera sensor by the Mobile Industry Processor Interface (the MIPI), coupled to said communication easier.

[0135] 通过所述通信耦接,传感器模块415能够向照相机转发指令,以控制每部照相机的操作的各个方面,比如其功率级、变焦级别、焦距、曝光度等。 [0135] coupled via the communication, the sensor module 415 can forward instructions to the camera to control various aspects of the operation of each portion of the camera, such as its power level, the level of zoom, focus, exposure and the like. 在一些实施例中,每部照相机具有四种工作功率模式。 In some embodiments, each camera has four operational power mode. 在第一种工作功率模式下,照相机被断电。 In the first operational power mode, the camera is powered off. 就第二种工作功率模式来说,照相机被通电,不过照相机还未被配置。 Work on the second power mode, the camera is powered on, but the camera has not been configured. 在第三种工作功率模式下,照相机被通电,照相机的传感器被配置,并且照相机传感器的像素收集光子,并把收集的光子转换成数字值。 In a third operational power mode, the camera is powered on, the camera sensor is arranged, and the pixels of the camera sensor to collect photons, and converts the photons collected into a digital value. 不过,照相机传感器还未向传感器模块415发送图像。 However, the camera sensor has not been sent to the image sensor module 415. 最后,在第四种工作功率模式下,照相机处于和第三种功率模式相同的工作功率模式之下,除了照相机现在向传感器模块415发送图像之外。 Finally, at the fourth operational power mode, the camera is under the same power mode and the third operational power mode, the camera transmits the image outside the sensor module 415 in addition to now.

[0136] 在设备的操作期间,照相机可多次从一种工作功率模式切换到另一种工作功率模式。 [0136] During operation of the device, the camera may be repeatedly switched from one operating mode to another power operational power mode. 当切换工作功率模式时,一些实施例要求照相机按照上述顺序切换工作功率模式。 When the power mode switching operation, requires some embodiments camera switches operational power mode in this order. 于是,在这些实施例中,第一种工作功率模式下的照相机只能切换到第二种工作功率模式。 Thus, in these embodiments, the camera under a first operational power mode can only be switched to a second operational power mode. 当照相机处于第二种工作功率模式时,它能够切换到第一种工作功率模式或者第三种工作功率模式。 When the camera is in the second operational power mode, it can be switched to the first operational power mode or the third operational power mode. 类似地,照相机能够从第三种工作功率模式切换到第二种工作功率模式或者第四种工作功率模式。 Similarly, the camera can be switched from the power mode to the third operational power mode or a second operating mode of the fourth operational power. 当照相机处于第四种工作功率模式时,它只能够切换回第三种工作功率模式。 When the camera is the fourth operational power mode, it is only possible to switch back to the third operational power mode.

[0137] 此外,从一种工作功率模式切换到下一种或前一种工作功率模式需要一定量的时间。 [0137] In addition, switching from one operating mode to the power of one or operational power mode of the former requires a certain amount of time. 从而,切换两种或三种工作功率模式慢于切换一种工作功率模式。 Thus, the switching of two or three operational power mode to a slower mode switching operational power. 不同的工作功率模式还消耗不同量的功率。 Different operating power modes also consume different amounts of power. 例如,第四种工作功率模式消耗最大量的功率,第三种工作功率模式消耗的功率高于第一种和第二种工作功率模式,以及第二种工作功率模式消耗的功率高于第一种工作功率模式。 For example, the fourth operational power mode consumes the most amount of power, the power consumption of the third operational power mode higher than the first and second operational power mode and a second operational power mode higher than the first power consumption modes of operation power mode. 在一些实施例中,第一种工作功率模式不消耗任何功率。 In some embodiments, a first operational power mode does not consume any power.

[0138] 当照相机未处于拍摄图像的第四种工作功率模式时,可使照相机保持在其它工作功率模式之一下。 [0138] When the camera is not in the fourth operational power mode of the captured image, the camera can remain in other operating modes at the power. 使不用的照相机保持在哪种工作模式之下的确定取决于允许照相机消耗多少功率和照相机需要多快地响应开始拍摄图像的请求。 The camera determines the need to keep under which operating mode allows the camera depends on how much power the camera and consumption need to respond to how fast the request to start shooting the image. 例如,配置成按照第三种工作功率模式(例如,待机模式)工作的照相机比配置成处于第一种工作功率模式(即,断电) 的照相机消耗更多的功率。 For example, arranged according to a third operational power mode (e.g., standby mode) of the camera is configured to work than in the first operational power mode (i.e., power) cameras consume more power. 不过,当指令照相机拍摄图像时,按照第三种工作功率模式工作的照相机能够比按照第一种工作功率模式工作的照相机更快地切换到第四种工作功率模式。 However, when the instruction camera through image, a camera operating according to a third operational power mode can be switched over to the camera work according to a first operational power mode to a faster fourth operational power mode. 因而,根据不同的要求(例如,对拍摄图像的请求的响应时间,功率消耗),当不拍摄图像时,照相机可被配置成按照不同的工作功率模式工作。 Thus, according to different requirements (e.g., response time to the request of the captured image, power consumption), when no photographic image, the camera may be configured to operate in different operating power modes.

[0139] 如以下进一步所述,当视频处理模块325请求一部或两部照相机开始拍摄图像, 并且传感器模块415通过控制器模块475接收到该请求时,通过它与每部照相机的通信耦接,传感器模块415能够指令一组或两组照相机传感器开始拍摄图像。 [0139] Such as further described below, when the processing module 325 requests a video or two camera shoots an image, and the sensor module 415 receives the request through the controller module 475, through its contact with the communication of each coupling portion of the camera The sensor module 415 can instruct one or both of the camera image sensor to start recording. 拜尔滤镜被叠加在每个照相机传感器之上,从而每个照相机传感器输出拜尔格式图像,所述拜尔格式图像被保存在与每个照相机传感器相关联的像素阵列中。 Bayer filters are superimposed on each camera sensor so that each camera output Bayer pattern image sensor, the Bayer pattern image stored in the pixel array is associated with each joint of the camera sensor. 拜尔格式图像是其中每个像素只保存一种颜色值:红、蓝或绿的图像。 Bayer pattern image in which each pixel value saves only one color: red, green, or blue image.

[0140] 通过它与像素阵列410a和410b的耦接,传感器模块415取回保存在照相机像素阵列410a和410b中的原始拜尔格式图像。 [0140] coupled through its contact with the pixel array 410a and 410b, the sensor module 415 retrieves saved in the camera 410a and 410b in the pixel array of Bayer pattern raw image. 通过控制传感器模块415从照相机的像素阵列取回图像的速率,传感器模块415能够控制由特定照相机拍摄的视频图像的帧速率。 By controlling the rate of the sensor module 415 retrieves an image from the pixel array of the camera, a sensor module 415 capable of controlling the frame rate of the video images captured by a specific camera. 通过控制其图像取回的速率,传感器模块415还能够交织由不同照相机拍摄的图像的读取,以便交织CIPU处理管线485对来自不同照相机的拍摄图像的图像处理。 Control the rate at which the image is retrieved, the sensor module 415 is also capable of reading an image captured by interleaving different cameras, image processing for interleaving captured image 485 pairs from different cameras CIPU processing pipeline. 以下在II. A. 1和II. A. 2的子节中中进一步说明传感器模块415对其图像取回的控制。 The following description further controls the sensor module 415 is retrieved in its image II. A. 1 and II. A. 2 of subsections.

[0141] 传感器模块415把它从像素阵列410a和410b取回的图像行(即,图像的一排排像素)保存在行/帧缓冲器417中。 [0141] The sensor module 415 is retrieved it from the pixel array image lines 410a and 410b (i.e., rows of pixels in the image) stored in a row / frame buffer 417. 通过CIPU处理管线485处理行/帧缓冲器417中的每个图像行。 CIPU processing by the processing pipeline 485 lines / frame buffer 417 in each image row. 如图4中所示,CIPU处理管线485由BPC模块420、LS模块425、去马赛克模块430、WB模块43、γ模块440、CSC模块445、HSC模块450、定标器模块455和滤波器模块460构成。 As shown in FIG. 4, CIPU processing pipeline module 485 by the BPC 420, LS module 425, a demosaic module 430, WB module 43, γ module 440, CSC module 445, HSC module 450, module 455 and a scaler filter module 460 constitution. 在一些实施例中,CIPU处理管线485逐行(即,逐排)地处理来自行/帧缓冲器417的图像,而在其它实施例中,CIPU处理管线485逐帧地处理来自行/帧缓冲器417的整个图像。 In some embodiments, the CIPU processing pipeline 485 progressive (i.e., row by row) from the processed line / frame image buffer 417, while in other embodiments, the CIPU processing line from the processing line 485 from frame to frame / frame buffer an entire image 417.

[0142] 在图4中图解说明的示例管线中,BPC模块420是从行/帧缓冲器417取回图像的模块。 [0142] In the example illustrated in line 4 in FIG, BPC module 420 from the line / frame buffer module 417 retrieves an image. BPC模块420进行坏像素消除操作,该操作设法校正取回的图像中可能由一个或多个有缺陷的照相机传感器引起的坏像素(例如,有缺陷的光子传感器根本不感测光,错误地感测光等等)。 BPC module 420 for elimination of bad pixel operation, which seek to retrieve the image correction may be provided by a bad pixel or more defective due to camera sensor (e.g., a defective photon sensor does not sense light, sensing an error light, etc.). 在一些实施例中,BPC模块420通过比较图像中的特定像素与图像中的一个或多个相邻像素,检测坏像素。 In some embodiments, the BPC block 420 adjacent pixels through a specific pixel in the image in the comparison image or a plurality of detecting bad pixels. 如果所述特定像素的值与所述相邻像素的值之间的差异大于阈值量,那么用颜色(即,红、绿和蓝)与所述特定像素相同的几个相邻像素的值的平均值替换所述特定像素的值。 Several adjacent pixel value If the value of the particular pixel value difference between adjacent pixels is greater than a threshold amount, then the color (i.e., red, green, and blue) with the particular pixel same Alternatively the average value of the specific pixel.

[0143] BPC模块420的操作部分由为该模块保存在CIPU 400的两组寄存器470中的值控制。 [0143] BPC value control module operating portion 420 is stored in two sets of registers 470 CIPU 400 in the module for. 具体地说,为了处理由设备的两部不同照相机拍摄的图像,一些实施例关于每部照相机不同地配置CIPU处理管线485,如上所述。 Specifically, in order to process an image captured by the camera device two different, some embodiments of each camera on CIPU arranged differently processing pipeline 485, as described above. 通过在CIPU 400的两组不同的寄存器470a (Ra) 和470b (Rb)中保存两组不同的值,为两部不同的照相机配置CIPU处理管线485。 By two different sets of values ​​stored in two different registers 400 CIPU 470a (Ra) and 470b (Rb), the processing pipeline 485 is disposed CIPU two different cameras. 每组寄存器470包括用于CIPU处理管线485内的每个模块420-460的一个寄存器(Ra或Rb)。 Each set of registers 470 includes a register for each of the modules 420-460 of (Ra or Rb) in the processing pipeline 485 CIPU. 每个寄存器组中的每个寄存器保存一组定义一个处理管线模块的操作的值。 Each register value set for each register holds a set of line module defines a processing operation. 因此,如图4中所示,寄存器组470a用于指示双照相机移动设备的一部照相机(照相机A)的每个处理管线模块的工作模式,而寄存器组470b用于指示双照相机移动设备的另一部照相机(照相机B)的每个处理管线模块的工作模式。 Thus, as shown in Figure 4, the register group 470a for indicating a dual camera mobile camera device (camera A) of each operating mode of the processing pipeline module 470b for indicating the register bank dual camera mobile device to another a camera (camera B) of the operating mode of each processing pipeline module.

[0144] 对于每部照相机不同地配置CIPU处理管线485的一个例子是把CIPU处理管线485的模块配置成处理大小不同的图像。 One example of [0144] the processing pipeline 485 is configured CIPU differently for each camera is to CIPU processing pipeline module 485 configured to process images of different sizes. 例如,如果照相机传感器405a为640X480像素, 照相机传感器405b为2048 X 1536像素,那么寄存器组470a被配置成保存指令CIPU处理管线485的各个模块处理640X480像素图像的值,以及寄存器组470b被配置成保存指令CIPU处理管线485的各个模块处理2048 X 1536像素图像的值。 For example, if the camera sensor 405a is 640X480 pixels, the camera sensor 405b is 2048 X 1536 pixels, then the register group 470a is configured to store instructions CIPU processing pipeline modules 485 process 640X480 pixel values ​​of the image, and a register group 470b is configured to store CIPU instruction processing pipeline processing modules 485 2048 X 1536 pixel values ​​of the image.

[0145] 在一些实施例中,不同的处理管线配置(即,寄存器值)被保存在不同的简档设置中。 [0145] In some embodiments, different processing line configuration (i.e., register value) is stored in a different profile settings. 在一些这样的实施例中,允许移动设备的用户选择简档设置之一(例如,通过显示在移动设备上的用户界面),以设置一部或多部照相机的操作。 In some such embodiments, the mobile device allows the user to select one of the profile setting (e.g., by displaying a user interface on a mobile device), or to set an operating portion of the camera. 例如,用户可以选择把照相机配置成拍摄高分辨率视频的简档设置,把相同的照相机配置成拍摄低分辨率视频的简档设置,或者把两部照相机都配置成拍摄高分辨率静止图像的简档设置。 For example, the user may select a camera configured to capture a high resolution video profile setting, the camera is configured to capture the same profile setting low resolution video, or the two cameras are configured to capture a high-resolution still image profile setting. 各种不同的配置都是可能的,可被保存在许多不同的简档设置中。 Various configurations are possible and can be stored in many different profile settings. 在其它这样的实施例中,代替允许用户选择简档设置,根据用户选择的应用或活动,自动选择简档设置。 In other such embodiments, instead of allowing the user to select profile settings, in accordance with the application or event selected by the user, automatically selected profile setting. 例如,如果用户选择视频会议应用,那么自动选择把两部照相机配置成拍摄视频的简档,如果用户选择照相应用,那么自动选择把照相机之一配置成拍摄静止图像的简档,等等。 For example, if the user selects a video conferencing application, then automatically selects the two cameras configured to capture video profile, if the user selected photographic applications, the camera automatically selects one of the profiles configured to capture still images, and so on.

[0146] 在BPS模块420之后,LS模块425接收坏像素校正后的图像。 [0146] After the BPS module 420, LS module 425 receives a bad pixel correction image. LS模块425执行透镜阴影校正操作,以校正由产生光衰减效应(即,朝着照相机传感器的边缘,光逐渐减少) 的照相机透镜引起的图像缺陷。 LS lens module 425 performs shading correction operation, to correct the attenuation effect by the generation of light (i.e., towards the edges of the camera sensor, the light is gradually reduced) image defects caused by the camera lens. 这种效应导致图像照明度不均匀(例如,在角落和/或边缘更暗)。 This effect leads to image the illumination unevenness (e.g., the darker corners and / or edges). 为了校正这些图像缺陷,一些实施例的LS模块425估计透镜的照明度衰减的数学模型。 In order to correct the image defects, some of the embodiments of the LS module 425 estimates a mathematical model of the attenuation of the illumination lens. 估计的模型随后被用于补偿图像的透镜衰减,以均匀地照明图像的未被均匀照明的部分。 Model estimation is then used to compensate for the lens attenuation image, not uniform illumination to evenly illuminate the portion of the image. 例如,如果图像的角落的亮度是图像中心的亮度的一半,那么一些实施例的LS模块425把角落像素值乘以2,以便产生均匀的图像。 For example, if the corner of image brightness is half of the luminance of the image center, then some embodiments of the LS 425 module corner pixel values ​​are multiplied 2, in order to produce a uniform image.

[0147] 去马赛克模块430执行去马赛克操作,以便由采样颜色的图像产生全色图像。 [0147] demosaic module 430 performs demosaicing operation, in order to produce full-color image by a color image sample. 如上所述,照相机传感器输出拜尔格式图像,由于拜尔格式图像的每个像素只保存一种色值, 因此拜尔格式图像是不完整的。 As described above, the camera Bayer pattern image sensor outputs, since each pixel of the Bayer pattern image just holding one color value, the Bayer pattern image is incomplete. 去马赛克模块430通过在拜尔格式图像中插入每组颜色的色值,由拜尔格式图像重构红、绿、蓝(RGB)图像。 Demosaic module 430 by inserting each color value in the Bayer pattern color image, the Bayer pattern image reconstruction red, green, blue (RGB) image.

[0148] WB模块435对从去马赛克模块430接收的RGB图像执行白平衡操作,使得图像内容的颜色与现实生活中人眼感觉到的所述内容的颜色相似。 [0148] WB module 435 receives the RGB image white balance module 430 from the demosaicing operation, such that the color image content in real life the similar color content perceived by human eyes. WB模块435通过调整图像的颜色来调整白平衡,以正确地呈现中性颜色(例如,灰、白等)。 WB module 435 adjusts the white balance by adjusting the colors of the image, in order to correctly render neutral color (e.g., gray, and white). 例如,在白炽灯下的一张白纸的图像可能显现为黄色,而人眼感觉所述一张纸为白色。 For example, in an image under a white incandescent may appear yellow, and the human eye as a white piece of paper. 为了解决传感器拍摄的图像的颜色与人眼感觉的图像的颜色之间的差异,WB模块435调整图像的色值,使得拍摄的图像正确地反映人眼感觉的颜色。 In order to solve the difference between the color of the color image sensor with the human eye captured image, the WB module 435 to adjust the color values ​​of the image so that the image captured accurately reflect human eye color.

[0149] 统计引擎465收集在CIPU处理管线485的各个阶段的图像数据。 [0149] Engine 465 collects statistical data of the respective stages of the image processing pipeline 485 CIPU. 例如,图4表示统计引擎465收集在LS模块425、去马赛克模块430和WB模块435之后的图像数据。 For example, Figure 4 shows statistics engine 465 collected in the LS module 425, to the image data after the mosaic module 435 module 430 and a WB. 不同的实施例从CIPU处理管线485的任意数目的不同阶段收集数据。 Different embodiments of the data collected from any number of different stages of the processing pipeline 485 CIPU. 统计引擎465处理收集的数据,并根据处理后的数据,通过控制器模块475和传感器模块415调整照相机传感器405a和405b的操作。 Processing statistical data collection engine 465, and in accordance with the processed data, adjustment operation the camera sensor 405a and 405b by the controller module 475 and sensor module 415. 这种操作的例子包括曝光和聚焦。 Examples of such operations include exposure and focus. 尽管图4显示通过控制器模块475 控制照相机传感器405a和405b的统计引擎465,不过,统计引擎465的其它实施例通过传感器模块415控制照相机传感器。 Although FIG. 4 shows the module by the controller 475 controls the camera sensors 405a and 405b of the statistics engine 465, however, the statistics engine 465 in other embodiments the sensor module 415 by the control of the camera sensor.

[0150] 处理后的数据还可用于调整CIPU 400的各个模块的操作。 [0150] The processed data may also be used to adjust the operation of various modules of CIPU 400. 例如,一些实施例的统计引擎465根据在WB模块435之后收集的数据,调整WB模块435的操作。 For example, some embodiments of the statistics engine 465 based on data collected in the module 435 after WB, WB adjustment operation of the module 435. 在一些这样的实施例中,统计引擎465通过利用处理后的数据提供自动白平衡(AWB)功能,以调整WB模块435的白平衡操作。 In some such embodiments, the statistics engine 465 provides an automatic white balance (AWB) function by using the data processing operation to adjust the white balance WB module 435. 其它实施例可以使用从CIPU处理管线485的任意数目的阶段收集的处理后数据来调整CIPU处理管线485内的任意数目模块的操作。 Other embodiments may use any number of modules to adjust the operation of pipeline processing in 485 CIPU CIPU processed data processing pipeline of any number of stages 485 collected from. 此外,统计引擎465还能够从控制器模块475接收指令,以调整CIPU处理管线485的一个或多个模块的操作。 Further, the statistics engine 465 can also receive instructions from the controller module 475 to adjust the operation of one or more modules 485 CIPU processing pipeline.

[0151] 在从WB模块435接收到图像之后,γ模块440对图像进行γ校正操作,以编码和解码照相机系统的亮度或者三色值。 [0151] After receiving the image from the WB module 435, the image module 440 gamma] gamma] correction operation, encoding and decoding luminance camera system or tristimulus values. 一些实施例的Y模块440通过把10-12比特的线性信号转换成8比特的非线性编码,以校正图像的γ值,进行γ值校正。 Some embodiments of Y modules 440 10-12 bits by converting the linear signal into 8-bit non-linear coding, to the value of γ corrected image performs γ correction value. 一些实施例利用查找表校正Y值。 Some embodiments Y value correction using a lookup table.

[0152] CSC模块445把从γ模块440接收的图像从一个色空间变换到另一个色空间。 [0152] CSC module 445 from module 440 receives the image γ converting from one color space to another color space. 具体地说,CSC模块445把图像从RGB色空间变换到亮度和色度(YUV)色空间。 Specifically, CSC image conversion module 445 from the RGB color space to the luminance and chrominance (the YUV) color space. 不过,CSC模块445的其它实施例可以往来于任意数目的色空间变换图像。 However, other embodiments CSC module 445 may be any number of contacts in the color space conversion image.

[0153] HSC模块450可调整从CSC模块445接收的图像的色调、饱和度、对比度或者它们的任意组合。 [0153] HSC module 450 can adjust the color tone of the image module 445 receives the CSC, saturation, contrast, or any combination thereof. 例如,HSC模块450可以调整这些性质,以降低噪声或者增强图像。 For example, the HSC module 450 may adjust these properties, to reduce noise or enhance the image. 例如,可以增大由低噪声照相机传感器拍摄的图像的饱和度,以使图像显得更鲜艳。 For example, the saturation may be increased by the low noise image captured by the camera sensor in order to make the image look bright. 相反,可以降低由高噪声照相机传感器拍摄的图像的饱和度,以降低这种图像的色噪声。 Conversely, the saturation may be reduced noise picked up by the camera sensor of the image high in order to reduce such an image color noise.

[0154] 在HSC模块450之后,定标器模块455可缩放图像,以调整图像的像素分辨率,或者调整图像的数据大小。 [0154] After the HSC module 450, scaling module 455 may scale the image, to adjust the pixel resolution of the image, or adjust the size of the image data. 例如,定标器模块455还可减小图像的大小,以便适合较小的显示器。 For example, scaling module 455 may also reduce the size of the image to fit smaller display. 例如,定标器模块455能够以许多不同的方式比例缩放图像。 For example, scaling module 455 can be in many different ways scaled image. 例如,定标器模块455能够按比例增大(即,放大)和按比例减小(即,缩小)图像。 For example, scaling module 455 can be scaled up (i.e., amplification) and decrease (i.e., reduced) to scale the image. 定标器模块455还能够按比例缩放图像,或者变形地缩放图像。 Scaler scaling module 455 can also scale the image, the scaled image or deformed.

[0155] 滤波器模块460对从定标器模块455接收的图像应用一种或多种滤波操作,以改变图像的一些或全部像素的一种或多种属性。 [0155] from the image filter module 460 applies one or more filtering operations module 455 receives the scaler to change some or all of the pixels of one or more properties of the image. 滤波器的例子包括低通滤波器、高通滤波器、 带通滤波器、双边滤波器、高斯滤波器,等等。 Examples of the filter comprises a low pass filter, a high pass filter, band pass filter, a bilateral filter, Gaussian filter, and the like. 因而,滤波器模块460能够对图像应用任意多种不同的滤波。 Thus, the filter module 460 can be any number of different filters applied to the image.

[0156] 一些实施例的控制器模块475是控制CIPU 400的操作的微控制器。 [0156] Some embodiments of the controller module 475 to control the operation of the microcontroller of CIPU 400. 在一些实施例中,控制器模块475(1)通过传感器模块41,控制照相机传感器的操作(例如,曝光度), ⑵控制CIPU处理管线485的操作,⑶控制CIPU处理管线485的定时(例如,何时切换照相机传感器,何时切换寄存器,等等),和(4)控制闪光灯/频闪闪光灯(未示出),所述闪光灯/频闪闪光灯是一些实施例的双照相机移动设备的一部分。 In some embodiments, the controller module 475 (1) through the sensor module 41, a control operation (e.g., exposure) of the camera sensor, the control operation CIPU ⑵ processing pipeline 485, the timing control ⑶ CIPU processing pipeline 485 (e.g., when to switch camera sensor, when the switching register, etc.), and (4) to control the flashlight / strobe light (not shown), the flash / strobe light is part of the dual camera mobile device embodiment.

[0157] 控制器模块475的一些实施例处理从统计引擎465和CIPU驱动器480接收的指令。 Some embodiments [0157] The controller module 475 processes instructions received from the statistics engine 465 and the drive 480 CIPU. 在一些实施例中,从CIPU驱动器480接收的指令是来自双照相机移动设备(即,接收自本地设备)的指令,而在其它实施例中,从CIPU驱动器480接收的指令是来自另一个设备的指令(例如,视频会议期间的远程控制)。 In some embodiments, the instruction CIPU driver 480 receives the (i.e., received from the local device) a command from a dual camera mobile devices, while in other embodiments, from the instruction CIPU driver 480 receives from another device command (e.g., a remote control during a video conference). 根据处理后的指令,控制器模块475能够通过规划寄存器470的值,调整CIPU 400的操作。 The instruction processing controller module 470 can be the value of register 475 through the planning, the adjustment operation CIPU 400. 此外,控制器模块475能够在CIPU 400的操作期间,动态重新规划寄存器470的值。 In addition, the controller module 475 during operation CIPU 400 can be dynamically re-programming the value of the register 470.

[0158] 如图4中所示,CIPU 400包括CIPU处理管线485中的许多模块。 [0158] As shown in FIG. 4, CIPU 400 CIPU processing pipeline 485 includes many modules. 不过,本领域的普通技术人员会认识到可以只用一些图解说明的模块,或者用另外的不同模块实现CIPU 400。 However, those of ordinary skill in the art will recognize that only some of the illustrated modules, implemented CIPU 400 with additional or different modules. 另外,由不同模块执行的处理可按照与图4中图解说明的顺序不同的顺序应用于图像。 Further, the processing performed by the different modules can be applied to images in a different order than illustrated in FIG. 4 described order.

[0159] 现在参考图4,说明CIPU 400的一个示例操作。 [0159] Referring now to Figure 4, illustrating an example of the operation of CIPU 400. 为了说明起见,一组寄存器Ra被用于处理由双照相机移动设备的照相机传感器405a拍摄的图像,以及一组寄存器Rb被用于处理由双照相机移动设备的照相机传感器405b拍摄的图像。 For purposes of illustration, a set of register Ra is used for processing an image captured by the camera of the dual camera sensor mobile device 405a, and a set of registers and an image Rb are captured by the camera for processing the dual camera sensor mobile device and 405b. 控制器模块475从CIPU驱动器480接收产生由双照相机移动设备的照相机之一拍摄的图像的指令。 The controller 480 receives generation instruction module 475 captured by one camera of the dual camera mobile device from an image CIPU driver.

[0160] 控制器模块475随后初始化CIPU处理管线485的各个模块,以处理由双照相机移动设备的照相机之一拍摄的图像。 [0160] The controller module 475 then initializes CIPU pipeline processing modules 485 for processing the image captured by one camera of the dual camera mobile device. 在一些实施例中,这包括控制器模块475检查是否使用了寄存器470中的一组正确寄存器。 In some embodiments, it comprises a controller module 475 checks if a set of registers 470 in the correct registers. 例如如果CIPU驱动器480指令控制器模块475产生由照相机传感器405a拍摄的图像,那么控制器模块475检查一组寄存器Ra是否是CIPU 400 的模块读取的一组寄存器。 For example, if CIPU driver 480 instructions of the controller module 475 generates an image captured by the camera sensor 405a, the controller module 475 checks whether a group is a set of registers of register Ra CIPU read module 400. 如果不是,那么控制器模块475在两组寄存器之间切换,使得该组寄存器Ra是由CIPU 400的模块读取的一组寄存器。 If not, the controller module 475 switches between the two sets of registers, register Ra is set such that a set of registers read by the module CIPU 400.

[0161] 对CIPU处理管线485中的每个模块来说,工作模式由保存在所述一组寄存器Ra 中的值指示。 [0161] The processing pipeline 485 CIPU in each module, the operation mode indicated by the value stored in the register Ra are set. 如前所述,一组寄存器470中的值能够在CIPU 400的操作期间被动态重新规贝1J。 As described above, the value set in the register 470 can be dynamically reconfigured during operation of the regulated 1J shell of CIPU 400. 从而,一个图像的处理不同于下一个图像的处理。 Thus, processing in an image processing different from the one image. 尽管CIPU 400的本示例操作的讨论描述CIPU 400中的每个模块读取保存在寄存器中的指示各个模块的工作模式的值,不过在一些软件实现的实施例中,改为把参数传送给CIPU400的各个模块。 Although the present exemplary CIPU 400 discussed in the description of the operation of each module CIPU 400 reads the value stored in operation mode register indicates the various modules, but in some embodiments of a software implementation, the transmission parameters to be changed CIPU400 each module.

[0162] 在一些实施例中,控制器模块475通过指令传感器模块415在从像素阵列410a取回图像之后延迟特定时间量,来初始化传感器模块415。 [0162] In some embodiments, the controller 415 the delay module 475 after a certain amount of time to retrieve the image from the pixel array 410a by instructing the sensor module, the sensor module 415 is initialized. 换句话说,控制器模块475指令传感器模块415以特定速率从像素阵列410a取回图像。 In other words, the controller module 475 instructs the sensor module 415 at a particular rate retrieves an image from the pixel array 410a.

[0163] 随后,控制器模块475通过传感器模块415指令照相机传感器405a拍摄图像。 [0163] Subsequently, the controller module commands the camera sensor 405a 475 415 an image captured by the sensor module. 在一些实施例中,控制器模块475还向照相机传感器405a提供曝光参数和其它照相机操作参数。 In some embodiments, the controller module 475 also provides the camera exposure parameters and other operating parameters of the sensor to the camera 405a. 在其它实施例中,照相机传感器405a使用照相机传感器操作参数的默认值。 In other embodiments, the camera sensor camera sensor 405a using the default values ​​of the operating parameters. 根据所述参数,照相机传感器405a拍摄原始图像,所述原始图像被保存在像素阵列410a中。 Photographing an original image according to the parameters, the camera sensor 405a, the original image is stored in the pixel array 410a. 传感器模块415从像素阵列410a取回所述原始图像,并把图像发送给行/帧缓冲器417以便保存, 之后CIPU处理管线485处理该图像。 Sensor module 415 retrieves from the pixel array 410a of the original image, and the image is transmitted to the line / frame buffer 417 in order to preserve, after CIPU 485 processes the image processing pipeline.

[0164] 在某些情况下,图像可能被行/帧缓冲器417丢弃。 [0164] In some cases, the image may be the line / frame buffer 417 is discarded. 当照相机传感器405a和/或405b以高速率拍摄图像时,传感器模块415可比BPC模块420能够从行/帧缓冲器417取回图像更快地接收图像,并把图像保存在行/帧缓冲器417中(例如,拍摄高帧速率视频时),行/帧缓冲器417会变得完全充满。 When the camera sensor 405a and / or 405b when the captured image frame buffer at a high rate than a sensor module 415 is capable of receiving BPC module 420 lines / frame buffer 417 retrieves an image from the image faster, and the image storing row / 417 (e.g., high frame rate when shooting the video), the line / frame buffer 417 will become completely filled. 当发生这种情况时,一些实施例的行/帧缓冲器417根据先进先出原则丢弃图像(即,帧)。 When this happens, some of the embodiments of the line / frame buffer 417 discards the image (i.e., frame) in accordance with the FIFO principle. 即,当行/帧缓冲器417丢弃一帧图像时,行/ 帧缓冲器417丢弃在行/帧缓冲器417中的所有其它图像之前接收的那帧图像。 That is, when the line / frame buffer 417 to discard a frame of image, the line / frame buffer 417 discards the received frame image row and that all other image / frame buffer 417 before.

[0165] CIPU处理管线485的图像处理从BPC模块420取回来自行/帧缓冲器417的图像,以校正图像中的任何坏像素开始。 [0165] CIPU the image processing pipeline 485 from the module 420 to take back their own BPC / image frame buffer 417 to correct any bad pixels in the image begins. BPC模块420随后把图像发送给LS模块425,以校正图像中的任何不均匀照明度。 BPC module 420 then transmits the image to the LS module 425, illumination unevenness correcting any image. 在校正图像的照明度之后,LS模块425把图像发送给去马赛克模块430,去马赛克模块430处理原始图像,由原始图像产生RGB图像。 After the illumination of the corrected image, the LS module 425 transmits the image to the demosaicing module 430, a demosaic module 430 processes the raw image, the RGB image generated by the original image. 随后,WB模块435 从去马赛克模块430接收RGB图像,并调整RGB图像的白平衡。 Then, the WB module 435 receives the RGB image from the demosaicing module 430, and adjusts the white balance of the RGB image.

[0166] 如上所述,统计引擎465可能已在CIPU处理管线485的各个点收集了一些数据。 [0166] As described above, the statistics engine 465 may have some of the data collected at various points CIPU processing pipeline 485. 例如,如图4中图解所示,统计引擎465在LS模块425、去马赛克模块430和WB模块435之后收集数据。 For example, illustrated in FIG. 4, the statistics engine 465 LS module 425, 435 to collect data after the mosaic module 430, and a WB module. 根据收集的数据,统计引擎465可调整照相机传感器405a的操作和/或CIPU 处理管线485中的一个或多个模块的操作,以便调整从照相机传感器405a的后续图像的拍摄。 The data collected statistics engine 465 to adjust the operation of the camera sensor 405a and / or operation of the processing pipeline 485 CIPU one or more modules in order to adjust subsequent images captured from the camera sensor 405a. 例如,根据收集的数据,统计引擎465可确定当前图像的曝光度过低,从而通过传感器模块415指令照相机传感器405a增大后续所拍图像的曝光度。 For example, based on data collected, the statistics engine 465 may determine the current through the low-exposure image, thereby increasing 405a subsequent exposure images shot by the sensor camera sensor module 415 command. 从而,一些实施例的统计引擎465起一些处理操作的反馈回路的作用。 Thus, some of the effects of the feedback loop processing operation statistics engine 465 up some embodiments.

[0167] 在WB模块435调整图像的白平衡之后,它把图像发送给γ模块440,以进行γ校正(例如,调整图像的Y曲线)。 [0167] After the WB module 435 adjusts the white balance of the image, it sends the image to the γ module 440 to perform the γ correction (e.g., adjustment of the image curve Y). CSC模块445从γ模块440接收γ校正后的图像,并进行色空间变换。 CSC module 445 receives the image from the γ γ correction module 440, and color space conversion. 在本例中,CSC模块445把RGB图像变换成YUV图像。 In the present embodiment, CSC RGB image module 445 into a YUV image. 换句话说,CSC模块445把用RGB色空间表现的图像变换成用YUV色空间表现的图像。 In other words, CSC module 445 with the performance of the RGB color space into an image converted image with YUV color space representation. HSC模块450从CSC模块445接收YUV图像,并调整图像中的各个像素的色调、饱和度和对比度属性。 HSC module 450 receives the YUV image, and adjust the tone of each pixel in the image, saturation and contrast attributes from the CSC block 445. 在HSC模块450之后,定标器模块455缩放图像(例如,放大或缩小图像)。 HSC module 450. After scaling module 455 scales the image (e.g., image enlargement or reduction). 在从定标器模块455接收图像之后,滤波器模块460对图像应用一种或多种滤波。 After the module 455 receives the image from the scaler, the image filter module 460 applies one or more filter pair. 最后,滤波器模块460把处理后的图像发送给CIPU驱动器480。 Finally, the image is sent to the driver 480 CIPU filter module 460 after processing.

[0168] 在上面说明的CIPU 400的这个操作例子中,CIPU处理管线485中的每个模块以某种方式处理图像。 [0168] In this operation example CIPU 400 is explained above, CIPU processing pipeline module 485 each process the image in some way. 不过,CIPU 400处理的其它图像可能不需要CIPU处理管线485的所有模块的处理。 However, other image processing CIPU 400 may not require all the processing modules 485 CIPU processing pipeline. 例如,图像可能不需要白平衡调整、Y校正、比例缩放或滤波。 For example, the image may not need to adjust the white balance, Y correction, scaling or filtering. 因而,CIPU 400能够根据接收的各种输入,比如来自CIPU驱动器480的指令,或者由统计引擎465收集的数据,以任意各种方式处理图像。 Thus, CIPU 400 can be received in accordance with various inputs, such as instructions from the CIPU driver 480, or by the statistics collection engine 465, any image processing in various ways.

[0169] 不同的实施例不同地控制处理图像的速率(即,帧速率)。 [0169] Different embodiments variously control the image processing rates (i.e., frame rate). 控制帧速率的一种方式是通过垂直消隐间隔(VBI)的操纵。 One way is by controlling the frame rate of the vertical blanking interval (the VBI) manipulation. 对于取回图像行以便逐行处理图像的一些实施例来说,VBI是从像素阵列取回由双照相机移动设备的照相机拍摄的视频的一个图像的最后一行,和从像素阵列取回视频的下一个图像的第一行之间的时间差。 For some embodiments in order to retrieve an image line of the image is processed line by line, the last line of the VBI is captured by a camera retrieves the dual camera mobile device of a video image from the pixel array, and the retrieved video from pixel array a time between the first row of the image difference. 在其它实施例中,VBI是从像素阵列取回由双照相机移动设备的照相机拍摄的视频的一个图像,和从像素阵列取回视频的下一个图像之间的时间差。 In other embodiments, the VBI is retrieved dual camera video captured by a camera of the mobile device from the pixel array an image, and the time between retrieving a next video image from pixel array difference.

[0170] 其中能够使用VBI的一个例子在传感器模块415与像素阵列410a和410b之间。 [0170] One example of the VBI can be used between the sensor module 415 in the pixel array 410a and 410b. 例如,传感器模块415的一些实施例逐行地从像素阵列410a和410b取回图像,以及传感器模块415的其它实施例逐个图像地从像素阵列410a和410b取回图像。 For example, some embodiments of the sensor module 415 retrieves an image line by line, and sensor modules 410a and 410b from the pixel array another embodiment of image-wise embodiment 415 retrieves an image from the pixel array 410a and 410b. 从而,通过调整传感器模块415的VBI,能够控制帧速率:增大VBI会降低帧速率,而减小VBI会提高帧速率。 Thus, by adjusting the sensor module 415 of the VBI, the frame rate can be controlled: increasing the frame rate reduces the VBI, but will increase the frame rate to reduce the VBI.

[0171] 1. VBI对单照相机的应用:帧速率控制 Application [0171] 1. VBI single camera: frame rate control

[0172] 图5概念地图解说明基于不同VBI的不同帧速率505、510和515的例子。 [0172] Figure 5 conceptually illustrates an example of the different frame rates of 505, 510 and 515 different VBI based. 每个序列显示在沿着时间线520的各个时刻525-555,拿着吉它的人物的图像,所述图像由双照相机移动设备的照相机之一拍摄。 Each sequence is shown 525-555, holding a guitar image of a person, the image captured by one camera of the dual camera mobile device 520 at various times along the time line. 另外,每个时刻525-555之间的时间都相同,称为一个时间单元。 Further, the time between each time point 525-555 are the same, referred to as a time unit. 为了说明起见,现在将参考图4的传感器模块415和像素阵列410a,说明图5。 To illustrate, with reference now to FIG sensor module 415 and the pixel array 410a 4 and 5 described in FIG. 因而, 每个图像代表沿着时间线520的传感器模块415从像素阵列410a取回图像的时刻。 Thus, each image representing 520 along the time line sensor module 415 retrieves an image from the pixel array 410a of time.

[0173] 在示例的帧速率505中,传感器模块415关于像素阵列410a的VBI被设定为3个时间单元(例如,由控制器模块475设定)。 [0173] In the example of the frame rate 505, the sensor module 415 on the pixel array 410a of the VBI is set to be 3 time units (e.g., set by the controller module 475). 即,传感器模块415沿着时间线520,每隔两个时刻从像素阵列410a取回一巾贞图像。 That is, the sensor module 415 along the timeline 520, two time intervals of a retrieves the image from the pixel array towel Zhen 410a. 如不例的巾贞速率505中所不,传感器模块415在时刻525、540和555取回图像。 The towel 505 does not rate Zhen embodiment are not in, the sensor module 415 retrieves an image at time 555 and 525,540. 从而,示例的帧速率505具有每三个时间单元一帧图像的帧速率。 Thus, an example of the frame rate has a frame rate of 505 units per three times of one frame of image.

[0174] 除了VBI被设定为2个时间单元之外,示例的帧速率510类似于示例的帧速率505。 [0174] In addition to VBI is set to 2 time units outside the frame rate frame rate similar to example 510 Example 505. 从而,传感器模块415沿着时间线520,每隔一个时刻从像素阵列410a取回一巾贞图像。 Thus, the sensor module 415 along the timeline 520, every time a retrieval image from the pixel array towel Zhen 410a. 示例的帧速率510表示传感器模块415在时刻525、535、545和555,从像素阵列410a取回图像。 510 represents an example of the frame rate of the sensor module 415 at time 525,535,545 and 555, retrieves an image from the pixel array 410a. 由于示例的帧速率510的VBI小于示例的帧速率505的VBI,因此示例的帧速率510 的帧速率高于示例的帧速率505的帧速率。 Since the frame rate is the frame rate of the VBI 510 is an example of an example of the frame rate of the VBI less than 505, so an example of a frame rate higher than a frame rate of 510 505 an example of the frame rate.

[0175] 除了传感器模块415关于像素阵列410a的VBI被设定为1个时间单元之外,示例的帧速率515也类似于示例的帧速率505。 [0175] In addition to the sensor module 415 on the pixel array VBI 410a is set to a time outside the unit, the frame rate is also similar to example 515 example 505 the frame rate. 于是,传感器模块415被指令沿着时间线520, 在每个时刻从像素阵列410a取回一巾贞图像。 Thus, the sensor module 415 along the timeline 520 is instructed, a towel at each time retrieving the image from the pixel array Zhen 410a. 如图所不,传感器模块415在时刻525-555,从像素阵列410a取回图像。 FIG do not, at the time the sensor module 415 525-555, retrieves an image from the pixel array 410a. 示例的帧速率515的VBI小于示例的帧速率505和510的VBI。 An example of a frame rate of less than 515 is an example of a VBI VBI frame rate of 505 and 510. 于是,示例的帧速率515的帧速率大于示例的帧速率505和510。 Thus, an example of a frame rate greater than a frame rate of 515 505 and an example of a frame rate of 510.

[0176] 2. VBI对双照相机的应用 Application [0176] 2. VBI double camera

[0177] 一些实施例可能希望同时操作双照相机移动设备的两部照相机(例如,在视频会议期间,从两部照相机传送视频)。 [0177] Some embodiments may be desirable to simultaneously operate two cameras of the dual camera mobile device (e.g., during a video conference, the video transmitted from two cameras). 包括单处理管线的双照相机移动设备的不同实施例提供同时操作双照相机移动设备的两部照相机的不同机制。 Comprise different embodiment of the dual camera mobile device provides a single processing line different mechanisms operate simultaneously two cameras dual camera mobile device.

[0178] 一种这样的机制是通过控制每部照相机的VBI,交织由两部照相机拍摄的图像的处理。 [0178] One such mechanism is the control unit of each camera VBI, interleaving processing of an image captured by the two cameras. 即,在一部照相机的VBI期间,拍摄和处理由另一部照相机拍摄的一个或多个图像, 反之亦然。 That is, during the VBI of a camera, photographing and processing one or more images captured by another camera, and vice versa. 由于上面说明的CIPU 400具有单处理管线485,因此这种机制能够在一些实施例的CIPU 400中实现。 Since CIPU 400 described above having a single processing pipeline 485, and therefore this mechanism can be implemented in some embodiments CIPU 400. 在这样的实施例中,传感器模块415从像素阵列410a和410b之一取回图像,并且取回的图像由CIPU 400在传感器模块415关于另一个像素阵列的VBI期间处理。 In such an embodiment, the sensor module 415 retrieves an image from one of the pixel array 410a and 410b, and retrieved by the image processing with respect to another during the VBI 415 pixel array CIPU 400 in the sensor module.

[0179] 传感器模块415关于每个像素阵列的VBI可被设定为特定值。 [0179] Sensor module 415 pixel array with respect to each VBI may be set to a specific value. 不过,在一些实施例中,VBI不被设定成比CIPU 400取回和处理一帧图像所用时间小的值。 However, in some embodiments, the VBI is set to be not more than CIPU 400 to retrieve and process an image with a small time value. 一些实施例把传感器模块415关于每个像素阵列的VBI设定成相同值。 Some embodiments of the sensor module 415 VBI for each pixel array is set to the same value. 例如,当传感器模块415关于每个像素阵列的VBI被设定成相同值时,传感器模块415交替地从像素阵列410a和410b取回图像。 For example, when the sensor module 415 is set to the same value with respect to each VBI pixel array, the sensor module 415 retrieves an image alternately from the pixel array 410a and 410b. 其它实施例把传感器模块415关于每个像素阵列的VBI设定成不同的值。 Other embodiments of the sensor module 415 VBI for each pixel array is set to different values. 在一些这样的实施例中,传感器模块415关于一个像素阵列的VBI被设定成传感器模块415关于另一个像素阵列的VBI的倍数。 In some such embodiments, the sensor module 415 on a pixel array VBI is set to be a multiple sensor module 415 on another VBI pixel array. 例如,传感器模块415关于一个像素阵列的VBI被设定成2 个时间单元,传感器模块415关于另一个像素阵列的VBI被设定成4个时间单元。 For example, the sensor module 415 on a pixel array VBI is set to two time units, the sensor module 415 on another VBI pixel array is set to four time units. 在这个例子中,相对于传感器模块415从所述另一个像素阵列取回的每一帧图像,传感器模块415 从所述一个像素阵列取回两帧图像。 In this example, with respect to each frame of image sensor module 415 retrieved from the other pixel array, the sensor module 415 retrieves an image from the two pixel arrays.

[0180] 图6概念地图解说明基于不同VBI的不同交织帧速率605、610和615的例子。 Examples of different frame rates based on different interleaving the VBI 605, 610 and 615 [0180] Figure 6 illustrates conceptually. 除了图6包括沿着时间线620的13个时刻625-685之外,图6类似于图5。 In addition to FIG. 6 comprises 620 along the line 13 in time period 625-685, FIG. 6 is similar to FIG. 另外,拿着吉它的人物的图像代表沿着时间线620从一个像素阵列取回该图像的时刻,而戴着学位帽的人物的图像代表沿着时间线620从另一个像素阵列取回图像的时刻。 Further, the image representative of the guitar took 620 retrieves character image from a time of the pixel array along the time line, and an image representative of a degree of wearing the cap 620 retrieves an image from the other pixel array along the time line time.

[0181] 为了说明起见,拿着吉它的人物的图像假定已由双照相机移动设备的照相机传感器405a拍摄,以及戴着学位帽的人物的图像假定已由双照相机移动设备的照相机传感器405b拍摄。 [0181] For purposes of illustration, is holding the image of a person guitar has been assumed that the camera sensor mobile device 405a dual camera photographing, and an image of a person wearing a cap degree assumed by the dual camera sensor mobile device 405b imaging camera. 此外,现在将参考图4的传感器模块415及像素阵列410a和401b说明图6。 Further, with reference now to FIG 4, the sensor module 415 and the pixel array 410a and 401b described in FIG. 6.

[0182] 在示例的交织帧速率605中,传感器模块415关于像素阵列410a和像素阵列401b 的VBI被设定成2个时间单元。 [0182] In the example of interleaving frame rate 605, a sensor module 415 on the pixel array 410a and the VBI pixel array 401b is set to two time units. 如示例的交织帧速率605中所示,传感器模块415在沿着时间线620的时刻625、635、645、655、665、675和685,从像素阵列410 &取回图像,以及传感器模块415在沿着时间线620的时刻630、640、650、660、670和680,从像素阵列410b取回图像。 As an example of interleaving frame rate, the sensor module 415 along the timeline 620 in time 625,635,645,655,665,675 and 685, from the pixel array 410 & retrieves an image, and a sensor module 415 in 605 630,640,650,660,670 and 680, 410b retrieves an image from the pixel array along the time line 620 of the time. 即,在每个时间单元,传感器模块415交替地从像素阵列取回图像。 That is, at every time unit, the sensor module 415 retrieves an image from the alternate pixel array.

[0183] 除了传感器模块415关于像素阵列410a和像素阵列401b的VBI被设定成4个时间单元之外,示例的交织巾贞速率610类似于示例的交织巾贞速率605。 [0183] In addition to the sensor module 415 on the pixel array 410a and the VBI pixel array 401b is set to four time units other, interleaved towels interleaved towels Chen Chen rate rate similar to the example 610 example 605. 示例的交织巾贞速率610 显示传感器模块415在沿着时间线620的时刻625、645、665和685,从像素阵列410a取回图像,以及传感器模块415在沿着时间线620的时刻635、655和675,从像素阵列410b取回图像。 Zhen interleaved towel rate example of the display 610 of the sensor module 415 along the timeline 620 in time 625,645,665 and 685, image retrieval, and a sensor module 415 in time along the timeline 620 from the pixel array 410a 635,655 and 675, retrieves an image from the pixel array 410b. 由于示例的交织帧速率610的VBI大于示例的交织帧速率605的VBI,因此示例的交织帧速率610的帧速率小于示例的交织帧速率605的帧速率。 Since the sample interleaving interleaved frame rate greater than 610 VBI VBI example of a frame rate of 605, so the sample interleaving the frame rate is less than 610 interleaves the frame rate of 605 samples.

[0184] 除了传感器模块415关于像素阵列410a和像素阵列401b的VBI被设定成6个时间单元之外,示例的交织巾贞速率615也类似于示例的交织巾贞速率605。 [0184] In addition to the sensor module 415 on the pixel array 410a and the VBI pixel array 401b is set to six time units other, interleaved towels Zhen rate 615 is also similar to the example of interleaved towels Zhen exemplary rate of 605. 如图6中所示,传感器模块415在沿着时间线620的时刻625、655和685,从像素阵列410a取回图像,以及传感器模块415在沿着时间线620的时刻640和670,从像素阵列410b取回图像。 As shown, the sensor module 415 in time along the timeline 620 and 625,655 685, image retrieval, and a sensor module 415 along the timeline 620 at the time 640 and 670 from the pixel array 410a 6, the pixels from retrieves an image array 410b. 示例的交织帧速率615的VBI大于示例的交织帧速率605和610的VBI。 Interleaving frame rate 615 is an example of interleaving a frame rate greater than VBI VBI example 605 and 610. 从而,示例的交织帧速率615 的帧速率小于示例的交织帧速率605和610的帧速率。 Thus, an example of interleaving the frame rate is less than 615 interleaves the frame rate of 605 and 610 in the example.

[0185] B.多管线 [0185] B. multi-line

[0186] 图7概念地图解说明一些实施例的另一个拍摄图像处理单元(CIPU)700。 [0186] FIG 7 conceptually illustrates some of the captured image processing unit to another embodiment (CIPU) 700. 除了CIPU 700由两个前端处理管线,一个存储器,和一个后端处理管线实现,而不是由单处理管线实现之外,CIPU 700实现和上面说明的CIPU 400相同的功能。 In addition CIPU 700, a memory, and a rear end of the front end processing pipeline implemented by the two processing lines, instead, CIPU 700 realize the same functions as described above by CIPU 400 to single processing pipeline implemented. 因此,将参考CIPU 400的模块,说明CIPU 700的功能的描述。 Thus, with reference to CIPU module 400, the function description describes CIPU 700.

[0187] 如图所示,CIPU 700包括用于照相机传感器405a和像素阵列410a的前端处理管线715、用于照相机传感器405b和像素阵列410b的前端处理管线720、存储器725、控制器模块730、和后端处理管线735。 [0187] As shown, CIPU 700 includes a camera sensor pixel array 405a and the distal end of the processing pipeline 715 410a, 405b for the camera sensor pixel array and the distal end 410b of the processing pipeline 720, a memory 725, a controller module 730, and backend processing pipeline 735. 一些实施例的照相机传感器405a和405b是双照相机移动设备的照相机的传感器。 Some embodiments of the camera sensor is a camera 405a and 405b of the dual camera sensor mobile device.

[0188] 一些实施例的前端处理管线715和720执行CIPU 400的部分图像处理。 [0188] Some embodiments of the front end of the processing pipeline 715 and the image processing section 720 performs the CIPU 400. 因而,不同的实施例可以包括CIPU 400的不同数目的模块。 Thus, various embodiments may include a different number of modules CIPU 400. 例如,一些实施例的每个前端处理管线715和720包括CIPU 400的传感器模块415、BPC模块420、LS模块425、去马赛克模块430、 WB模块435和统计引擎465。 For example, some embodiments of each of the frontend processing pipelines 715 and 720 includes a sensor module 400 CIPU 415, BPC module 420, LS module 425, a demosaic module 430, WB module 435 and statistics engine 465.

[0189] 尽管由于具有相同的模块,前端处理管线715和720执行相同类型的图像处理,不过通过如上关于CIPU 400说明的不同的寄存器值,能够不同地配置每个前端处理管线715 和720中的每个模块。 [0189] Although the same module, the frontend processing pipelines 715 and 720 perform the same type of image processing, as described above but by different register values ​​described on CIPU 400 can be configured differently for each frontend processing pipelines 715 and 720 each module. 此外,由于每个照相机传感器405a和405b都具有它自己的前端处理管线,因此前端处理管线715和720能够相互独立地处理图像。 Further, since each camera sensor 405a and 405b has its own front-end processing pipeline, processing pipeline 715, and thus the front end 720 can be independently processed image. 例如,前端处理管线715 和720能够并行地(即,在相同的时间)、在不同的时间、和以不同的速率处理图像。 For example, front-end processing pipelines 715 and 720 can be parallel (i.e., at the same time), at different times, and process images at different rates.

[0190] 在一些实施例中,每个前端处理管线715和720能够从其对应的照相机传感器和像素阵列取回图像。 [0190] In some embodiments, each of the frontend processing pipelines 715 and 720 to retrieve the image from its corresponding pixel and the camera sensor array. 例如,前端处理管线715从像素阵列410a取回由照相机传感器405a 拍摄的图像,以及前端处理管线720从像素阵列410b取回由照相机传感器405b拍摄的图像。 For example, the front end of the processing pipeline 715 retrieves an image captured by the camera sensor pixel array 405a from 410a, and the front end of the processing pipeline 720 retrieves an image captured by the camera sensor 405b from the pixel array 410b. 当前端处理管线715和720之一从其对应的照相机传感器和像素阵列取回图像时,该前端处理管线处理该图像,并把处理后的图像发送给存储器725。 Front end processing pipeline 715 and the camera sensor pixel array and its corresponding one of 720 retrieves an image, the front end processing of the image processing pipeline, and transmits the processed image to the memory 725. 另外,每个前端处理管线715和720如上所述与控制器模块730通信(例如,通过每个前端处理管线的统计引擎)。 Further, each of the frontend processing pipelines 715 and 720 as described above and a communication controller module 730 (e.g., engine statistics by each front-end processing pipeline).

[0191] 一些实施例的存储器725保存部分处理后的图像,供后端处理管线735完成处理。 [0191] Some embodiments of the memory 725 of the partially processed image is saved, the backend processing pipeline 735 for completion processing. 在这些实施例中,存储器725从前端处理管线715和720接收部分处理后的图像,并把部分处理后的图像发送给后端处理管线735。 In these embodiments, the memory 725 from the image receiving part 715 and the front end of the processing pipeline 720 processing, and sends the image processing section 735 to the rear end of the processing pipeline. 一些实施例把存储器725实现成易失性存储器(例如,随机存取存储器(RAM)),而其它实施例把存储器725实现成非易失性存储器(例如,闪速存储器、硬盘、光盘等)。 Some embodiments of the memory 725 implemented as a volatile memory (e.g., random access memory (the RAM)), while other embodiments the memory 725 implemented as a nonvolatile memory (e.g., flash memory, a hard disk, optical disk, etc.) . 此外,一些实施例的存储器725是内部存储器(例如,RAM),而其它实施例的存储器725是外部存储器(例如,紧凑式闪存(CF)卡,安全数字(SD)卡,等等)。 Further, some embodiments of the memory 725 is an internal memory (e.g., the RAM), and the memory 725 of other embodiments of an external memory (e.g., a compact flash (CF) card, a secure digital (SD) card, etc.).

[0192] 后端处理管线735的一些实施例执行CIPU 700的部分图像处理。 [0192] Some embodiments of the backend processing pipeline 735 performs the processing CIPU 700 of the partial image. 在一些实施例中,后端处理管线735包括前端处理管线715和720不包括的CIPU 400的模块。 In some embodiments, the backend processing pipeline 735 includes a front end 715 and the processing pipeline 720 does not include the module CIPU 400. 例如,参见上面的例子,后端处理管线735应包括CIPU 400的CSC模块445、γ模块440、HSC模块450、定标器模块455和滤波器模块460。 For example, see the above example, should the rear end of the processing pipeline 735 includes CIPU 400 of CSC module 445, γ module 440, HSC module 450, the scaling module 455 and the filter module 460. 因而,这种实施例的后端处理管线735执行前端处理管线715和720不执行的CIPU 400的剩余图像处理。 Thus, the rear end of such an embodiment the processing pipeline 735 performs frontend processing pipelines 715 and 720 remaining image processing is not performed CIPU 400. 因此,后端处理管线735从存储器725取回部分处理后的图像,并对部分处理后的图像执行剩余的图像处理。 Thus, backend processing pipeline processing portion 735 retrieves the image from the memory 725, and image processing performed on the image of the remaining portion after the treatment. 在处理图像之后,后端处理管线735把处理后的图像发送给CIPU驱动器480。 After processing the image, the image is sent to CIPU driver 480 after the rear end of the processing pipeline 735 processing.

[0193] 控制器模块730执行上面参考图4说明的相同功能。 [0193] The controller module 730 performs the same functions described above with reference to FIG 4. 如图7中所示,控制器模块730与前端处理管线715和720及后端处理管线735交互。 As shown in FIG. 7, the controller module 730 frontend processing pipelines 715 and 720 and 735 interacting with backend processing pipeline. 在一些实施例中,控制器模块730包括在后端处理管线735中,而在其它实施例中,控制器模块730包括在前端处理管线715和720之一中。 In some embodiments, the controller module 730 comprises a processing pipeline 735 at the rear end, while in other embodiments, the controller 730 comprises a module 715 in one of the front end 720 and the processing pipeline.

[0194] 现在参考在图7中图解说明的照相机传感器405a和405b、像素阵列401a和410b、 前端处理管线715和720、存储器725和后端处理管线735,说明CIPU 700的操作。 [0194] Referring now to the camera sensor illustrated in FIG. 7 described 405a and 405b, 401a and 410b of the pixel array, the processing pipeline 715 and the front end 720, a memory 725 and backend processing pipeline 735, the instructions CIPU 700. 当前端处理管线715和720之一从其对应的照相机传感器和像素阵列取回图像时,前端处理管线处理该图像,并把部分处理后的图像发送给存储器725。 Front end processing pipeline 715 and the camera sensor pixel array and its corresponding one of the image retrieved 720, the front end of the image processing pipeline processing, and sends the processed image to the memory section 725. 例如,前端处理管线715可从像素阵列410a取回由照相机传感器405a拍摄的图像,或者前端处理管线720可从像素阵列410b取回由照相机传感器405b拍摄的图像。 For example, the front end of the processing pipeline 715 may retrieve an image captured by the camera sensor pixel array 405a from 410a, a front end or processing pipeline 720 may retrieve images captured by the camera sensor 405b from the pixel array 410b. 如上所述,前端处理管线715和720都能够并行处理图像。 As described above, the front end of the processing pipelines 715 and 720 are capable of parallel image processing.

[0195] 后端处理管线735从存储器725取回部分处理后的图像,并处理部分处理后的图像以完成图像的图像处理。 [0195] back-end image processing pipeline 735 portion of process 725 retrieves from the memory, and the image processing section to perform image processing of an image. 在一些实施例中,后端处理管线735基于先进先出的原则取回和处理保存在存储器725中的图像。 In some embodiments, the backend processing pipeline 735 on the FIFO principle to retrieve and process the image stored in the memory 725. 换句话说,存储器725中的特定图像将在先于该特定图像的接收和保存在存储器725中的所有图像之后进行处理,不过,该特定图像将在晚于该特定图像的接收和保存在存储器725中的图像之前进行处理。 In other words, the specific image memory 725 prior to receiving the specific image and stored in the memory 725 after processing all of the images, however, the particular image in the receiver later than the specific image and stored in the memory 725 before processing the image. 在后端处理管线735处理图像之后,它把处理后的图像发送给CIPU驱动器480。 After the back-end processing of the image processing pipeline 735, which transmits the image to the driver 480 CIPU processed.

[0196] 图8概念地图解说明一些实施例的另一个拍摄图像处理单元(CIPU)800。 [0196] Figure 8 conceptually illustrates some of the captured image processing unit to another embodiment (CIPU) 800. 除了CIPU 800由两个单独的处理管线实现,每个照相机传感器具有它自己的单独的处理管线之外,CIPU 800执行和上面说明的CIPU 400相同的功能。 In addition CIPU 800 implemented by two separate processing pipelines, each camera has a sensor other than its own separate processing pipelines, the same functionality and perform CIPU 400 CIPU 800 described above. 因而,将参考CIPU 400的模块,说明CIPU 800的功能的描述。 Thus, with reference to CIPU module 400, described in the description of the function CIPU 800.

[0197] 如图所示,CIPU 800包括用于照相机传感器405a和像素阵列410a的处理管线815,和用于照相机传感器405b和像素阵列410b的处理管线820。 [0197] As shown, CIPU 800 includes a camera sensor pixel array 405a and the processing pipeline 815 410a, 405b, and a camera sensor array and the pixel processing pipeline 820 410b of. 一些实施例的每个处理管线815和820都包括包含在CIPU 400中的所有模块。 Some processing pipelines 815 and 820 of each embodiment includes all of the modules included in the CIPU 400. 于是,这些实施例的每个处理管线815和820的操作均与CIPU 400的操作相同。 Thus, each of these embodiments of the processing pipeline 815 and operation 820 are the same as the operation CIPU 400.

[0198] 由于每个照相机传感器405a和405b都具有它自己的处理管线,因此处理管线815 和820能够相互独立地处理图像。 [0198] Since each camera sensors 405a and 405b has its own processing line, the processing lines 815 and 820 each capable of independently processing an image. 例如,处理管线815和820能够并行地(即,在相同的时间)、在不同的时间、和以不同的速率处理图像。 For example, lines 815 and 820 can be processed in parallel (i.e., at the same time), at different times, and process images at different rates. 另外,通过上面参考CIPU 400说明的不同的寄存器值,能够不同地配置一些实施例的每个处理管线815和820。 Further, reference to different values ​​of the register 400 of the CIPU description, can be arranged in different number of each of the processing pipelines 815 and 820 by the above embodiments.

[0199] 在一些实施例中,CIPU 400的许多模块包括用于执行该模块的一些或全部操作的一个或多个行/帧缓冲器。 [0199] In some embodiments, CIPU number of modules 400 of the module comprises means for performing some or all of the operations of one or more rows / frame buffer. 例如,一些实施例的滤波模块460被实现成执行3 X 3低通滤波。 For example, some embodiments of the filtering module 460 is implemented to perform a 3 X 3 low-pass filtering. 在这样的实施例中,3X3低通滤波器处理图像中的连续三行,以便对所述连续三行的中间行应用3X3低通滤波。 In such an embodiment, the low-pass filter processing 3X3 successive three lines of the image, so that the application 3X3 low-pass filtering said intermediate rows of continuous three lines. 从而,这种实施例的滤波模块460至少需要三个行/帧缓冲器,以便实现3X3低通滤波。 Thus, this embodiment of the filter module 460 requires at least three rows / frame buffer in order to achieve 3X3 low-pass filter. CIPU 400中的其它模块也包括一个或多个行/帧缓冲器,比如BPC 模块420和LS模块425。 CIPU other modules 400 also include one or more line / frame buffer, such as modules 420 and LS BPC module 425.

[0200] CIPU 800的处理管线均可具有不同的行/帧缓冲器大小,以便按照其对应照相机的特征定制图像处理。 Processing pipeline [0200] CIPU 800 may have a different line / frame buffer size, the image processing in order to customize the camera according to the corresponding feature. 例如,如果双照相机移动设备的一部照相机具有2048X 1500像素传感器,那么该2048 X 1500像素传感器的处理管线能够包括2048像素宽的行/帧缓冲器。 For example, if a camera having a dual camera mobile device 2048X 1500 pixel sensor, the processing of the line sensor of 2048 X 1500 pixels 2048 can include a pixel-wide line / frame buffer. 类似地,如果双照相机移动设备的另一部照相机具有640 X 480像素传感器,那么640 X 480像素传感器的处理管线可包括为640像素宽的行/帧缓冲器。 Similarly, if another camera of the dual camera mobile device having a 640 X 480 pixel sensor, the processing pipeline 640 X 480 pixel sensor 640 may include a pixel-wide line / frame buffer. 即,包括在一个处理管线的各个模块中的行/帧缓冲器的大小可不同于包括在另一个处理管线的各个模块中的行/帧缓冲器的大小。 That is, each module comprising a row in a line in the process / different from the frame buffer size can include the size of line / frame buffer in each of the other modules in the processing pipeline.

[0201] III.视频会议 [0201] III. Videoconferencing

[0202] A.视频会议体系结构 [0202] A. Video Conferencing Architecture

[0203] 图9概念地图解说明一些实施例的双照相机移动设备的视频会议和处理模块900 的软件体系结构。 [0203] Figure 9 conceptually illustrates a dual camera apparatus of the embodiment of the mobile video conferencing and processing module 900 of the software architecture. 视频会议和处理模块900包括与上面参考图3说明的对应模块和驱动器305、301和320类似的CIPU驱动器905、媒体交换模块910和编码器驱动器920。 Video conferencing and processing module 900 includes described above with reference to FIG. 3 and drives a corresponding module 305,301 and 320 similar CIPU driver 905, switching module 910 the media drive 920 and the encoder. 视频会议和处理模块900还包括用于实现各种视频会议功能的视频会议模块925、视频会议客户端945和网络接口950。 Video conferencing and processing module further includes a video conference module 900 for implementing various functions of the video conference 925, video conference client 945 and network interface 950. 类似于视频处理和编码模块300,视频会议和处理模块900处理和编码从双照相机移动设备的照相机拍摄的图像。 300, video conferencing and image processing and encoding processing module 900 taken from a camera of the dual camera mobile devices similar to video processing and encoding module.

[0204] 如上参考图3所述,媒体交换模块910允许设备中的媒体内容的使用者和制造者交换媒体内容,和与媒体内容的处理有关的指令。 The [0204] above with reference to FIG 3, media exchange module 910 allows the device media content users and manufacturers to exchange media content and processing instructions related to media content. CIPU驱动器905充当与拍摄图像处理单元(CIPU) 955的通信接口,编码器驱动器920充当与编码器硬件960 (例如,编码器芯片、片上系统上的编码组件,等等)的通信接口。 CIPU driver 905 acts as a captured image processing unit (CIPU) a communication interface, the encoder driver 955 920 acting as the encoder hardware 960 (e.g., encoder chip, encoding component on-chip system, etc.) communications interface.

[0205] -些实施例的视频会议模块925负责各种视频会议功能,比如图像处理、视频会议管理和连网。 [0205] - Some embodiments of the video conference module 925 is responsible for various video conferencing, such as image processing, video conference management and networking. 如图所示,视频会议模块925与媒体交换模块910、视频会议客户端945和网络接口950交互。 As shown, video conferencing module 925 and media switching module 910, a video conference client 945 and the interaction network 950 interfaces. 在一些实施例中,视频会议模块925从视频会议客户端945接收指令, 和向视频会议客户端945发送指令。 In some embodiments, a video conference module 925 from the video conference client 945 receives the instruction, and sends an instruction 945 to the video conference client. 一些实施例的视频会议模块925还通过网络接口950, 向网络发送数据和从网络接收数据,例如,所述网络是局域网(LAN)、无线局域网(WLAN)、 广域网(WAN)、网络的网络(a network of networks)、码分多址接入(CDMA)网络、GSM网络7等等。 Some video conferencing module embodiment 925 through a network interface 950, and receive data from the network to send data to the network, e.g., the network is a local area network (LAN), wireless local area network (WLAN), wide area network (WAN), a network of networks ( a network of networks), Code Division Multiple access (CDMA) network, GSM network 7 and the like.

[0206] 视频会议模块925包括图像处理层930、管理层935和网络层940。 [0206] videoconferencing module 925 includes an image processing layer 930, network management layer 940 and 935. 在一些实施例中,图像处理层930对图像进行图像处理操作,以便用于视频会议。 In some embodiments, the image processing layer 930 performs image processing operations on the image, for use in a video conference. 例如,一些实施例的图像处理层930进行曝光调整、图像缩放、透视校正和动态范围调整,如下进一步详细所述。 For example, some embodiments of the image processing layer 930 of an exposure adjustment, image scaling, and perspective correction dynamic range adjustment, in further detail below. 一些实施例的图像处理层930通过媒体交换模块910发送对来自CIPU955的图像的请求。 Some embodiments of the image processing layer 930 of module 910 sends a request for the image from the CIPU955 by media exchange.

[0207] -些实施例的管理层935控制视频会议模块925的操作。 [0207] - Management some embodiments 935 controls the video conference module 925 operation. 例如,在一些实施例中, 管理层935初始化双照相机移动设备的一部/两部照相机,处理图像和音频,以便传送给远程设备,和处理从远程设备接收的图像和音频。 For example, in some embodiments, one pair management 935 initializes camera mobile device / two cameras, audio and image processing, for transmission to a remote device, and processing of image and audio received from a remote device. 在一些实施例中,管理层935产生用于该设备的合成(例如PIP)显示。 In some embodiments, the management apparatus 935 generates a synthetic (e.g., PIP) display. 此外,管理层935可根据从网络层940接收的连网报告,改变视频会议模块925的操作。 Further, according to the networking management 935 reports received from the network layer 940, changing the operation of a video conference module 925.

[0208] 在一些实施例中,网络层940实现用于视频会议的一些或全部网络功能。 [0208] In some embodiments, the network layer 940 implement some or all functions of a network for video conferencing. 例如,如下所述,除了别的功能之外,一些实施例的网络层940建立视频会议的双照相机移动设备和远程设备之间的网络连接(未示出),把图像传送给远程设备,和从远程设备接收图像。 For example, the following, among other functions, the network layer 940 to establish some embodiments of the network connection (not shown) between the dual camera mobile device and the remote video conference apparatus, the image transmitted to the remote device, and receiving an image from the remote device. 另外,网络层940接收诸如分组丢失、单向等待时间和往返延迟时间之类的连网数据,以及其它各种数据,处理这样的数据,和把数据报告给管理层935。 Further, the network layer 940 receives such packet loss, unidirectional data networking waiting time and the round-trip delay time based, and other various data processing such data, and report to the management data 935.

[0209] -些实施例的视频会议客户端945是利用视频会议模块925的视频会议功能的应用,比如视频会议应用、IP话音(V0IP)应用(例如,Skype)或者即时通信应用。 [0209] - Some video conferencing client terminal 945 is an embodiment of a video conference module using video conferencing application 925, such as a video conferencing application, the IP voice (V0IP) applications (e.g., Skype) or instant messaging application. 在一些实施例中,视频会议客户端945是独立的应用,而在其它实施例中,视频会议客户端945被集成到另一个应用中。 In some embodiments, a video conference client 945 is a standalone application, while in other embodiments, a video conference client 945 is integrated into another application.

[0210] 在一些实施例中,网络接口950是允许视频会议模块925和视频会议客户端945 通过网络(例如,蜂窝网络、局域网、无线网络、网络的网络、因特网等)发送数据和接收数据的通信接口。 [0210] In some embodiments, the network interface 950 is to allow video conference module 925 and a video conference client 945 transmit and receive data over a network (e.g., a cellular network, a local area network, a wireless network, a network of networks, the Internet, etc.) Communication Interface. 例如,如果视频会议模块925想要向因特网上的另一个设备发送数据(例如,由双照相机移动设备的照相机拍摄的图像),那么视频会议模块925通过网络接口950 把图像发送给所述另一个设备。 For example, if a video conference module 925 wants to send data (e.g., captured by the camera of the dual image camera mobile device) to another device on the Internet, the video conference module 925 is sent to the other network 950 via an interface image equipment.

[0211] B.视频会议设置 [0211] B. video conference set

[0212] 图10概念地图解说明一些实施例的示例的视频会议请求消息接发序列1000。 [0212] Figure 10 conceptually illustrates some example embodiments of the video conference request messaging sequence 1000. 图10表示运行于设备1005上的视频会议客户端1010,视频会议服务器1015,和运行于设备1020上的视频会议客户端1025之间的视频会议请求消息接发序列1000。 10 shows the operation of video conferencing client device 1005 on 1010, a video conference between the video conference server 1025 1015, 1020, and run on the device video conference client request messaging sequence 1000. 在一些实施例中, 视频会议客户端1010和1025与图9中所示的视频会议客户端945相同。 In some embodiments, a video conference video conference client 1010 and the client 1025 shown in FIG. 9 and 945 in the same end. 如图10中所示, 一个设备(即,设备1005)请求视频会议,以及另一个设备(S卩,设备1020)应答该请求。 As shown, a device (i.e., device 1005) requests a video conference, and the other devices (S Jie, device 1020) in response to the request 10. 在本申请中描述的双照相机移动设备能够执行这两个操作(即,发出请求和响应请求)。 Dual camera mobile apparatus described in the present application is capable of performing two operations (i.e., request and response request).

[0213] 一些实施例的视频会议服务器1015在视频会议客户端之间路由消息。 [0213] Video conference server 1015 of some embodiments of the video conference between the client route messages. 尽管一些实施例在一个计算设备上实现视频会议服务器1015,不过其它实施例在多个计算设备上实现视频会议服务器1015。 Although some embodiments of video conferencing server 1015 on a computing device, although other embodiments of video conferencing server 1015 on a plurality of computing devices. 在一些实施例中,视频会议服务器是可公开访问的服务器,它能够同时处理和路由众多会议的消息。 In some embodiments, the video conferencing server is publicly accessible server that can simultaneously handle many messages and routing session. 一些实施例的每个视频会议客户端1010和1025经网络接口,比如上面说明的网络接口950,通过网络(例如,蜂窝网络、局域网、无线网络、网络的网络、因特网等)与视频会议服务器1015通信。 Some embodiments of each of the video conference client terminal 1010 via the network interface 950 and a network interface 1025, such as described above, through the network (e.g., a cellular network, a local area network, a wireless network, a network of networks, the Internet, etc.) and the video conference server 1015 communication.

[0214] 当视频会议客户端1010从设备1005的用户接收到(在操作1)开始和设备1020 的视频会议的请求时,一些实施例的视频会议请求消息接发序列1000开始。 [0214] When a video conference client 1010 receives from the device 1005 to a user (operation 1) and requesting the start of the video conference apparatus 1020, a number of embodiments of the video conference request messaging sequence 1000 begins. 当设备1005 的用户选择显示在设备1005上的用户界面的某一用户界面(UI)项目时,一些实施例的视频会议客户端1010接收开始视频会议的请求。 When the user selection device 1005 displays a user interface on the device 1005 a user interface (UI) program, some of the video conference terminal requesting client Example 1010 receives video conference begins. 在下面说明的图11和图14中图解说明了这种用户界面的例子。 FIG 11 described below and illustrated in Figure 14 illustrates an example of such a user interface.

[0215] 在视频会议客户端1010接收到请求之后,视频会议客户端1010向视频会议服务器1015发送(在操作2)视频会议请求,该请求根据用户的输入,把设备1020指示为收件人。 [0215] After the video conference client 1010 receives the request, the client 1010 transmits the video conference video conference server 1015 (operation 2) video conference request, the request based on user input, the device 1020 is indicated as recipient. 视频会议服务器1015把视频会议请求转发给(在操作3)设备1020的视频会议客户端1025。 Video conference server 1015 forwards the request to a video conference (operation 3) video conference client terminal apparatus 1020 1025. 在一些实施例中,视频会议服务器1015利用推送技术,把视频会议请求转发给视频会议客户端1025。 In some embodiments, the video conferencing server 1015 using push technology, forwarding the request to a video conference video conference client 1025. 即,当从视频会议客户端1010接收到请求时,视频会议服务器1015开始向视频会议客户端1025传送视频会议请求,而不是等待客户端1025发送对任何消息的请求。 That is, when receiving from the video session to the requesting client 1010, server 1015 starts a video conference video conference client 1025 to transmit the video conference request, rather than waiting for the client sends a request 1025 for any message.

[0216] 当一些实施例的视频会议客户端1025接收到视频会议请求时,在设备1020上显示用户界面,以向设备1020的用户指出,设备1005的用户发送了开始视频会议的请求,并提示设备1020的用户接受或拒绝该视频会议请求。 [0216] When some of the video conference client embodiment 1025 receives a video conference request, the user interface displayed on the device 1020 to indicate to the device 1020 of the user device 1005 a user sends a request to start a video conference, and prompts user device 1020 to accept or reject the request for the video conference. 在下面说明的图12中图解说明了这种用户界面的例子。 In the following description of FIG. 12 illustrates an example of such a user interface. 在一些实施例中,当视频会议客户端1025接收到(在操作4)接受来自设备1005的用户的视频会议请求的请求时,视频会议客户端1025向视频会议服务器1015 发送(在操作5)视频会议接受。 In some embodiments, when a video conference client 1025 is received (operation 4) accepts requests from the user device 1005 is a video conference request, the video conference client 1025 sends (at operation 5) video to a video conference server 1015 meeting accepted. 当设备1020的用户选择如图12中图解说明的用户界面的某一用户界面项目时,一些实施例的视频会议客户端1025接收到接受视频请求的请求。 When the user selects the device 1020 a user interface items in FIG. 12 illustrates the user interface description, a number of video conference clients embodiment receives a request receiving terminal 1025 requests the video.

[0217] 在视频会议服务器1015从视频会议客户端1025接收到视频会议接受之后,视频会议服务器1015把视频会议接受转发给(在操作6)视频会议客户端1010。 [0217] After the video conference server 1015 received from the video conference client 1025 to accept the video conference, the video conference to accept the conference server 1015 forwards video (operation 6) video conference client 1010. 视频会议服务器1015的一些实施例利用上述推送技术,把视频会议接受转发给视频会议客尸端1010。 Video conference server 1015 of some embodiments of the above-described push technology, forwarding video conference to accept the conference video guest dead end 1010.

[0218] 当接收到视频会议接受时,一些实施例在设备1005和设备1020之间建立(在操作7)视频会议。 [0218] Upon receiving the video conference accepted, some embodiments of the established (operation 7) in the video conference between the device 1005 and the device 1020. 不同的实施例不同地建立视频会议。 Different embodiments establish a video conference differently. 例如,一些实施例的视频会议建立包括:协商设备1005和设备1020之间的连接、确定对视频编码的比特率、和在设备1005和设备1020交换视频。 For example, some embodiments establish a videoconference comprising: negotiate a connection between the device 1005 and the device 1020, determines the bit rate of video encoding, and video switching apparatus 1020 and the device 1005.

[0219] 在上面的例子中,设备1020的用户接受视频会议请求。 [0219] In the above example, the device 1020 requests the user to accept a video conference. 在一些实施例中,设备1020可被配置成(例如,通过设备的优先选择设置)自动接受到来的视频会议请求,而不显示Π。 In some embodiments, the device 1020 may be configured (e.g., by preference settings of the device) to automatically accept the incoming video conference request, without displaying Π. 此外,设备1020的用户也能够拒绝(在操作4)视频会议请求(例如,通过选择在设备1020上显示的用户界面的某个用户界面项目)。 Further, the device 1020 can reject the user (operation 4) video conference request (e.g., by selecting a user interface items in the user interface displayed on the device 1020). 代替发送视频会议接受,视频会议客户端1025向视频会议服务器1015发送视频会议拒绝,视频会议服务器1015把视频会议拒绝转发给视频会议客户端1010。 Instead of receiving transmitted video conference, a video conference client 1025 transmits a video conference video conference server 1015 rejection, video conference server 1015 to forward the video conference rejected video conference client 1010. 从而根本不建立视频会议。 Thus does not establish a video conference.

[0220] 1.视频会议设置用户界面 [0220] 1. Video conference setting user interface

[0221] 在一些实施例中,根据正在进行的通话,发起视频会议。 [0221] In some embodiments, according to the ongoing call, to initiate a video conference. 即,在移动设备的用户与第二位用户通话的时候,经另一方的许可,该用户能够把通话变成视频会议。 That is, when the user of the mobile device and the second user's call, permission of the other, the user is able to call into a video conference. 对本发明的一些实施例来说,图11图解说明由双照相机手持移动设备1100进行的这种视频会议的启动。 Some embodiments of the present invention, Figure 11 illustrates the start of this video conference dual camera of the handheld mobile device 1100. 图11利用设备1100的用户界面("π")1105的五个操作阶段1110、1115、1120、1125和1130,图解说明视频会议的启动。 FIG 11 using a user interface device 1100 ( "π") 1110,1115,1120,1125 five stages of operation 1105 and 1130, illustrates a video conference starts.

[0222] 如图11中所示,UI 1105包括名字栏1135、选择菜单1140和可选择的Π项目1145。 As shown in FIG. 11 [0222], UI 1105 includes a name field 1135, 1140 and select menu items selectable Π 1145. 名字栏1135显示在通话另一端的用户意欲请求与之进行视频会议的人物的名字。 1135 in the user name field to display the other end of the call is intended to request the name of the person with whom videoconferencing. 在本例中,可选择的Π项目1145(它可被实现成可选择的按钮)向用户提供可选择的结束通话(End Call)选项,以结束通话。 In the present embodiment, the selectable items Π 1145 (which may be implemented as selectable buttons) provides selectable call end (End Call) option to the user to end the call. 选择菜单1140显示可选择的Π项目的菜单,比如喇叭扩音器项目1142、静音项目1144、数字小键盘项目1146、电话簿项目1148、保持项目1152、 视频会议项目1154等等。 Select menu display selectable menu Π 1140 items, such as speakerphone Project 1142, Project 1144 mute, numeric keypad Project 1146, Project 1148 phone book, keep the project 1152, project 1154 video conferencing and so on. 不同的实施例不同地显示选择菜单。 Different embodiments differently displayed selection menu. 对图11图解说明的实施例来说,选择菜单1140包括几个同样大小的图标,每个图标代表一个不同的操作。 Example 11 illustrates, the choice of the menu 1140 includes several of the same size icons, each representing a different operation. 其它实施例提供可滚动的菜单,或者给予特定项目以优先级(例如,通过使这些项目变得更大)。 Other embodiments provide a scrollable menu, or give priority to a particular item (e.g., by making these items becomes larger).

[0223] 现在将参考Π 1105在图11中图解说明的五个阶段1110、1115、1120、1125和1130中的状态,说明Π 1105的操作。 And five stages state 1110,1115,1120,1125 1130 [0223] Reference will now be Π 1105 illustrated in Figure 11 described, the instructions Π 1105. 在第一阶段1110中,在手持移动设备用户和Nancy Jones之间已经建立了通话。 In the first phase in 1110, between the handheld mobile device users and Nancy Jones has established a call. 第二阶段1115显示在用户选择可选择的视频会议选项1154(例如,通过手指1150的单指轻点),以激活视频会议工具之后的UI 1105。 The second stage 1115 the user selects the displayed selectable video conferencing options 1154 UI 1105 (e.g., by a finger 1150 of a single finger tap), to activate the tool after the video conference. 在这个例子中,视频会议选项1154(它可被实现成可选择的图标)允许用户在通话期间启动视频会议。 In this example, video conferencing options for 1154 (which can be implemented as selectable icons) allow the user to initiate a video conference during a call. 在第二阶段中,视频会议选项1150被突出显示,以指示视频会议工具已被激活。 In the second stage, video conferencing option 1150 is highlighted to indicate that video conferencing tool has been activated. 不同的实施例可用不同的方式(例如,通过突出显示项目的边框或文本),指示这样的选择。 Various embodiments may in different ways (e.g., by a highlighted border or text items) indicating such a selection.

[0224] 第三阶段1120显示在依据选择视频会议选项1154的选择,设备1100已启动视频会议处理之后的Π 1105。 [0224] The third phase after 1120 show Π 1105 based on selected video conferencing option to select 1154, 1100 has started video conferencing equipment processing. 第三阶段是在设备等待建立视频会议时(例如,在设备等待通话另一端的设备接受或拒绝视频会议时)的过渡保持阶段。 The third stage is to establish a video conference device waits (e.g., when the device is waiting for the other end of the device to accept or reject a video conference call) holding phase transition. 在第三阶段1120中,在建立视频会议连接的时候,设备1100的用户仍然能够与另一设备的用户(即,Nancy Jones)谈话。 In the third stage in 1120, when the establishment of a video conference connection, users will still be able to device 1100 with another user equipment (ie, Nancy Jones) conversation. 另外,一些实施例允许设备1100的用户通过选择显示在Π 1105上的用于取消视频会议请求的可选Π项目(未示出),在第三阶段1120中取消视频会议请求。 Further, some embodiments allow a user device 1100 is displayed by selecting a Cancel [pi selectable items (not shown), a video conference request cancellation 1120 in the third stage in a video conference request in the Π 1105. 在该保持阶段期间, 不同的实施例使用Π 1105中的不同显示来指示该等待状态。 During the holding phase, different embodiments use different display Π 1105 to indicate that the wait state.

[0225] 如图11中所示,在一些实施例中,利用由设备1100拍摄的视频的全屏显示,以及在该视频的底部的"Preview(预览)"符号图解说明第三阶段的等待状态。 [0225] As shown in FIG. 11, in some embodiments, using the video captured by the full-screen display apparatus 1100, and "the Preview (preview)" symbol illustrates a waiting state at the bottom of the third stage of the video. 具体地说,在图11中,第三阶段1120通过在Π 1105的显示区1160中,显示由设备的照相机拍摄的视频的全屏呈现,图解说明视频会议处理的开始。 Specifically, in FIG. 11, the third stage 1120 in the display area 1160 Π 1105, captured by the camera displayed full-screen video presentation device, illustrating the process starts a video conference. 在一些实施例中,正面的照相机是在开始视频会议时,设备选择的默认照相机。 In some embodiments, the front camera is at the beginning of the video conference, the default camera device selection. 通常,在开始视频会议时,所述正面照相机对着设备的用户。 Usually, at the beginning of the video conference, the front facing camera user equipment. 因此,在图11中图解说明的例子中,第三阶段1120把设备1100图解表示成呈现设备1100的用户的全屏视频。 Thus, in the example illustrated in FIG. 11, the third stage 1120 full-screen video device 1100 to present a graphical representation 1100 of the user equipment. 位于在第三阶段1120期间,出现在显示区1160中的视频之下的"Preview"指示进一步突出设备的等待状态。 During the third phase it is located 1120, 1160 appear in the display area of ​​the "Preview" under the instruction waiting state for further video projection device.

[0226] 在一些实施例中,能够不同地表示过渡性的第三保持阶段1120。 [0226] In some embodiments, it can be different represent a transitional stage 1120 holding the third. 例如,一些实施例允许设备1100的用户选择背面的照相机作为启动视频会议的照相机。 For example, some embodiments allow the user to select the device 1100 as a back surface of the camera to start a video conference camera. 为了允许这种选择, 一些实施例允许用户(例如,通过菜单优先选择设置)把背面的照相机指定为启动视频会议的默认照相机,和/或允许用户从在用户选择视频会议选项1154之后,显示背面和正面的照相机的菜单中选择背面的照相机。 To allow for this selection, some embodiments allow a user (e.g., by preference setting menu) on the back of the camera as the startup default video conference camera, and / or allow the user after the user selects from a video conference option 1154, the back surface display and the front of the camera menu, select the camera on the back. 在这些情况的任何一种中,Π 1105(例如,显示区1160)在第三保持阶段1120期间,显示由背面的照相机拍摄的视频。 In any of these cases, Π 1105 (e.g., display area 1160) 1120 during a third holding phase, the display video captured by the camera in the back.

[0227] 另外,其它实施例可通过显示设备1100所摄视频的较小版本,通过显示保存在设备1100上的静止图像,通过提供突出显示设备的等待状态的消息(例如,通过显示"Conference Being Established"(正在建立会议),通过不显示"Preview"指示,等等,可指出视频会议工具的激活。另外,在第三阶段1120中,一些实施例的Π 1105提供如果在该阶段(例如,在用户等待远程用户答复他的请求时),用户决定不进入视频会议,那么允许用户取消进入视频会议,并返回通话状态的结束(End)按钮(未示出)。 [0227] Further, other embodiments may be smaller versions of the video camera by the display device 1100, a still image stored on the device 1100 through the display, projection display apparatus by providing a message waiting state (e.g., by displaying "Conference Being established "(session being established), by not displaying the" Preview "indication, and the like, it may be noted that activate the video conferencing tools. Further, in the third stage 1120, some embodiments provide a Π 1105 If at this stage (e.g., remote user while the user waits for a reply to his request), the user decides not to enter the video conference, the user is allowed to enter the video conferencing cancel and return to end the call state (end) button (not shown).

[0228] 第四阶段1125图解说明在远程用户接受了视频会议请求,并且建立了视频会议连接之后,处于过渡状态的Π 1105。 [0228] In the fourth stage 1125 illustrates the user accepts the remote video conference request, and establishes a video conference Π 1105 after the connection is in a transient state. 在该过渡状态下,显示本地用户的视频(在本例中,由正面的照相机拍摄的视频)的显示区1160的大小逐渐减小(S卩,逐渐缩小),如箭头1175 所示。 In this transitional state, the local user's video display (in this example, captured by the front camera video) size of the display area 1160 is gradually reduced (S Jie, gradually reduced), as shown by arrow 1175. 显示区1160(S卩,本地用户的视频)缩小,使得Π 1105能够在显示区1160之后,显示包含来自远程设备的照相机的视频的显示区1170(例如,显示窗口1170)。 Display area 1160 (S Jie, the user's local video) reduced, so that after possible Π 1105 1160 display area, display area includes a camera video from the remote device 1170 (e.g., a display window 1170). 换句话说,本地用户的视频1160的缩小产生PIP显示1180, PIP显示1180具有本地用户的视频的前景插入显示1160,和远程用户的背景主显示1170。 In other words, reduction of the local user to generate a video PIP display 1160 1180, 1180 foreground video PIP display having a local user insertion background main display 1160, a display 1170 and the remote user. 在本例中,背景主显示1170呈现远程设备的正面照相机正在拍摄其视频的女士(例如,Nancy Jones,远程设备的用户),或者远程设备的背面照相机正在拍摄其视频的女士(例如,Nancy Jones正在拍摄其视频的女士)的视频。 In the present embodiment, the background main display 1170 presents the remote device front camera is shooting Ms their videos (e.g., user Nancy Jones, remote device), or the back camera of the remote device being captured ladies their videos (e.g., Nancy Jones He is shooting its video lady) video. 普通技术人员会认识到图11中所示的过渡性的第四阶段只是一些实施例使用的一种示例方法,其它实施例可以不同地动画演示过渡性的第四阶段。 Of ordinary skill in the art will recognize that the transition of the fourth stage shown in FIG. 11 is only an example of some embodiments of the method used, other embodiments may variously animation transitional phase IV.

[0229] 第四阶段1125还在下面的显示区1155中图解说明可选的Π项目1132。 [0229] The following fourth stage 1125 is also illustrated in the display area 1155 Optional items Π 1132. 可选的Π项目1132(它可被实现成可选择的按钮)在PIP显示1180之下提供可选择的结束会议(End Conference)选项1132。 Optional items Π 1132 (which can be implemented as selectable buttons) under the 1180 end of the meeting provides a selectable (End Conference) 1132 option is displayed in PIP. 用户可以选择结束会议选项1132来结束视频会议(例如, 通过单指轻点)。 The user can select option 1132 to end the meeting video conference (for example, by a single finger tap). 不同的实施例可允许用户用不同的方式结束会议,比如通过转换移动设备上的开关,通过发出话音命令,等等。 Different embodiments may allow the user end of the session in different ways, such as by a switch on the mobile device, by issuing voice commands, and the like. 此外,不同的实施例可允许结束会议选项1132在视频会议期间逐渐消失,从而允许PIP显示1180占据整个显示区1185。 Further, different embodiments may allow the meeting option 1132 gradually disappear during the video conference, 1180 allowing the PIP display area 1185 occupies the entire display. 随后在单指轻点显示区1185的底部时,结束会议选项1132可重新出现,使用户可以利用结束会议选项1132。 Followed by a single finger tap when the display area at the bottom of 1185, the end of the 1132 meeting options may reappear, allowing users to take advantage of the end of the 1132 meeting options. 在一些实施例中,显示区1155的布局和下面进一步详细说明的显示区1155相同。 In some embodiments, region 1155 shows the same layout and below the display area 1155 described in further detail.

[0230] 第五阶段1130图解说明在结束第四过渡状态1125的动画之后的Π 1105。 [0230] The fifth stage 1130 illustrates Π 1105 after the end of the 1125 fourth transient state of animation. 具体地说,第五阶段1130图解说明在视频会议期间,由Π1105呈现的PIP显示1180。 Specifically, the fifth stage 1130 illustrates during a video conference, presented by the Π1105 PIP display 1180. 如上所述,PIP显示1180包括两个视频显示:来自远程照相机的较大的背景显示1170,和来自本地照相机的较小的前景插入显示1160。 As described above, the PIP video display two display 1180 comprises: a camera remote from the larger background display 1170, and smaller foreground from the local display 1160 is inserted into the camera.

[0231] PIP显示1180只是呈现由远程设备和本地设备拍摄的视频的合成视图的一种方式。 [0231] PIP display 1180 is only one manner of presenting a composite view captured by the remote and local devices video. 除了该合成视图之外,一些实施例的设备提供其它合成视图。 In addition to the composite view, some devices provide other embodiments of the synthesized view. 例如,代替具有远程用户的较大的背景显示1170,较大的背景显示1170可以是本地用户,而较小的前景插入显示1160是远程用户。 For example, instead of having large background remote user display 1170, a larger background display 1170 may be a local user, the smaller foreground inset display 1160 is the remote user. 如下进一步所述,一些实施例允许用户在视频会议期间,在作为提供PIP 显示1180的插入视图和主视图的照相机的本地照相机和/或远程照相机之间切换。 As further described below, some embodiments allow a user during a video conference, providing a PIP in the display switching between the local camera inserted into the camera's view and a front view 1180 and / or remote camera.

[0232] 另外,一些实施例允许本地视频和远程视频出现在Π 1105中的两个并排的显示区(例如,左右显示窗口,或者上下显示窗口),或者两个对角排列的显示区中。 [0232] Additionally, some embodiments allow two local and remote videos to appear side by side in Π 1105 in a display area (e.g., around the display window, the display window or down), or the two display areas are aligned diagonally. 在一些实施例中,如下进一步所述,通过设备的优先选择设置,或者通过在视频会议期间,用户能够选择的控件,用户可以规定PIP显示或者默认显示模式的方式。 In some embodiments, as further described below, by setting the priority selection device, or by the control during a video conference, the user can select, the user can specify the PIP display or the default display mode.

[0233] 当图11的设备1100的用户邀请远程用户进行视频会议时,远程用户可接受或拒绝该邀请。 [0233] When the device 1100 of FIG. 11 user video conference invitation remote user, the remote user may accept or reject the invitation. 图12图解说明在六个不同的阶段1210、1215、1220、1225、1230和1235,远程用户的设备1200的Π 1205,所述六个阶段表示在远程用户的设备呈现和接受视频会议邀请的操作序列。 12 illustrates in six different stages Π 1205 1210,1215,1220,1225,1230 and 1235, the remote user device 1200, showing the operation of the six stages and receiving video presentation at a meeting invitation remote user equipment sequence. 下面的Π 1205的描述把设备1200(即,接收到视频会议请求的设备)的用户称为受邀者(invite recipient),把设备1100(即,发送视频会议请求的设备)的用户称为邀请者(invite requestor)。 The following description Π 1205 of the user device 1200 (i.e., the device receiving the video conference request) referred invitees (invite recipient), the device 1100 (i.e., video conference request sending device) is referred to as a user invitation person (invite requestor). 另外,在本例中,假定受邀者的设备1200是双照相机设备,和邀请者的设备一样。 Further, in this embodiment, the apparatus 1200 is assumed invitee dual camera device, and the same equipment inviter. 不过,在其它例子中,这些设备中的一个或者两个都是单照相机设备。 However, in other instances, these two devices are single or a camera device.

[0234] 第一阶段1210图解说明当受邀者从邀请者,John Smith接收到视频会议邀请时的Π 1205。 [0234] The first stage 1210 illustrates the time when Π 1205 invitee received from the invitee, John Smith video conference invitation. 如图12中所示,第一阶段的Π 1205包括名字栏1235、消息栏1240和两个可选的Π项目1245和1250。 As shown in FIG. 12, the first stage Π 1205 comprises a name field 1235, a message bar 1240 and two selectable items 1245 and 1250 [pi. 名字栏1235显示正请求视频会议的人物的名字。 Name field 1235 displays the name of the character is requesting a video conference. 在一些实施例中,名字栏1235显示正请求视频会议的人物的电话号码,而不是人物的名字。 In some embodiments, the name field 1235 displays the name of the character is requesting a video conference phone number, rather than the characters. 消息栏1240显示邀请者给受邀者的邀请。 1240 Message Bar displays the inviter to invite the invitees. 在本例中,消息栏1240中的"Video Conference Invitation (视频会议邀请)"指示邀请者正在请求与受邀者进行视频会议。 In this case, the message bar 1240 "Video Conference Invitation (video conference invitation)" indicates that the invitee is requesting a video conference with the invitees. 可选的UI项目1245和1250(它们可被实现成可选择的按钮)提供可选择的供受邀者用于拒绝或接受邀请的"拒绝请求(Deny Requst)"和"接受请求(Accept Request)"选项1245和1250。 Selectable UI item 1245 and 1250 (which may be implemented as selectable buttons) used to provide a selectable for the invitee to accept or reject the invitation "rejects the request (Deny Requst)" and "accept the request (Accept Request) "options 1245 and 1250. 不同的实施例可不同地显示这些选项和/或显示其它选项。 Different embodiments of these options may be displayed differently and / or other display options.

[0235] 当看到显示在消息栏1240中的"Video Conference Invitation"符号时,通过分别选择Π中的"拒绝请求"选项1245或"接受请求"选项1250,受邀者可拒绝或接受请求。 [0235] When the display see "Video Conference Invitation" symbol in the message bar 1240, respectively, by selecting "request denied" option 1245 or "accept requests" option 1250, the invitee may refuse or accept requests in Π. 第二阶段1215图解说明在图12中所示的例子中,用户选择"接受请求"选项1250。 The second stage 1215 illustrates an example shown in FIG. 12, the user selects "Accept request" option 1250. 在这个例子中,通过用户的手指轻轻点击"接受请求"选项1250,实现所述选择,并通过选项1250 的突出显示,指出所述选择。 In this example, the user's finger just click "accept the request" option 1250, implementing the choice, and the option is highlighted by 1250, pointing out the selection. 在一些实施例中提供其它技术来选择"接受"或"拒绝请求"选项1245和1250(例如,连续两次轻轻点击,等等)以指出所述选择(例如,突出显示Π项目的边框或文本)。 Other techniques provide in some embodiments to select "Accept" or "request denied" option 1245 and 1250 (e.g., two successive click, etc.) to indicate the selection (e.g., item highlighted border or Π text).

[0236] 第三阶段1220显示在受邀者同意加入视频会议之后的Π 1205。 [0236] The third stage 1220 displays agree Π 1205 after the addition of video conferencing in the invitee. 在该阶段,Π 1205进入预览模式,预览模式在显示区1244中显示来自远程设备的正面照相机的视频的全屏呈现。 At this stage, Π 1205 to enter the preview mode, preview mode in the display area 1244 display full-screen video in front of the camera from a remote device is presented. 这种情况下的正面照相机对着远程设备的用户(即,本例中的Nancy Jones)。 In this case the front of the camera facing the user of the remote device (i.e., Nancy Jones in the present embodiment). 因此,按所述预览模式显示她的图像。 Thus, according to the preview mode displaying her image. 这种预览模式使受邀者可以确保她的视频被正确显示,并且在视频会议开始之前(例如,在开始实际传输视频之前),她对其外表感到满意。 This preview mode so that invitees can ensure that her video is displayed correctly, and before the start of a video conference (for example, before starting the actual transmission of the video), she was satisfied with their appearance. 在一些实施例中,可在显示区1244之下显示诸如"Preview"符号之类的符号,以指出受邀者处于预览模式。 In some embodiments, the symbol may be displayed such as "Preview" symbol or the like under the display area 1244, to indicate that the invitee is in preview mode.

[0237] -些实施例允许受邀者选择背面的照相机作为开始视频会议的默认照相机,或者在开始视频会议时选择正面或背面的照相机,如下进一步所述。 [0237] - Some embodiments allow the back of the camera to select invitees as the default start video conference camera, the camera or selecting front or back at the beginning of the video conference, as further described below. 另外,其它实施例不同地显示受邀者的预览显示(例如,在置于显示区1244的角落的较小图像中)。 Further, other embodiments show different invitee preview display (e.g., placed in a corner of the display region 1244 in the image is small). 除此之外的其它实施例不包括这种预览模式,而是在受邀者接受请求之后,立即开始视频会议。 In addition to other embodiments do not include this preview mode, but after they accept the request, immediately start a video conference.

[0238] 在第三阶段,Π 1205显示两个可选的Π项目1275和1246。 [0238] In a third stage, Π 1205 show two selectable items 1275 and 1246 [pi. 其中一个重叠在显示区1244之上,而另一个在显示区1244下面。 Wherein a display area 1244 superimposed on top of the other in the display region 1244 below. 可选的Π项目1275是用户可选择以开始视频会议的"接受"按钮1275。 Project 1275 is an optional Π user may choose to "Accept" button to start a video conference of 1275. 可选的Π项目1246是如果受邀者在此阶段决定不加入视频会议,那么她可选择的"结束"按钮1246。 Optional Π project 1246 if the invitee decides not to join a video conference at this stage, so she can choose the "end" button 1246.

[0239] 第四阶段1225显示在受邀者选择"接受"按钮1275之后的Π1205。 [0239] The fourth stage 1225 is displayed in the invitee select "Accept" Π1205 button after 1275. 在本例中, "接受"按钮1275被突出显示,以指出受邀者随时可开始视频会议。 In this case, the "Accept" button 1275 is highlighted to indicate that invitees can start a video conference at any time. 在其它实施例中可用不同的方式指出这样的选择。 It pointed out in different manners such selection in other embodiments.

[0240] 第五阶段1230图解说明在受邀者接受视频会议请求之后,处于过渡状态的Π 1205。 [0240] 1230 illustrates the fifth stage Π 1205 after they accept a video conference request, in a transient state. 在该过渡阶段,显示受邀者的视频(在本例中,所述视频由正面的照相机拍摄)的显示区1244的大小逐渐减小(即,逐渐缩小),如箭头1260所示。 In this transition period, a video display size invitee (in the present embodiment, the video captured by the front camera) in the display area 1244 is gradually reduced (i.e., gradually reduced), as shown by arrow 1260. 受邀者的视频缩小,使得Π 1205能够在显示区1244之后,显示包含来自邀请者的照相机的视频的显示区1265(例如,显示窗口1265)。 Invitee reduced video, such Π 1205 display area can be after 1244, a display region comprises a video camera from the invitee of 1265 (e.g., a display window 1265). 换句话说,受邀者的视频的缩小产生PIP显示1280,PIP显示1280具有受邀者的视频的前景插入显示区1244,和邀请者的背景主显示1265。 In other words, reduced video of the PIP display 1280 is generated invitee, the invitee having a PIP display 1280 foreground video inset display area 1244, and 1265 background main display inviter.

[0241] 在这个例子中,背景主显示1265呈现本地设备的正面照相机正在拍摄其视频的男士(即,John Smith,本地设备1100的用户)的视频。 [0241] In this example, the background main display 1265 presents the local device front camera is shooting video of their men (ie, John Smith, the local user equipment 1100) video. 在另一个例子中,该视频可以是本地设备的背面照相机拍摄其视频的男士(例如,由John Smith拍摄其视频的男士)的视频。 In another example, the video may be a local device back camera which captured video men (e.g., John Smith which video captured by men) video. 不同地实施例可以不同地动画演示过渡性的第五阶段。 Different embodiments may be variously animation fifth transitional phase.

[0242] 第五阶段1230的Π还显示包括在视频会议期间,使另一用户的音频静音的可选Π项目1285(例如,静音按钮1285),结束视频会议的可选Π项目1287(例如,结束会议按钮1287),和下面进一步说明的切换照相机的可选Π项目1289 (例如,切换照相机按钮1289)的显示区1155(例如,工具条或菜单条)。 [0242] Π fifth stage 1230 further includes a display during the video conference, that the selectable item 1285 Π audio muting another user (e.g., mute button 1285), the end of the video conference selectable items Π 1287 (e.g., end Conference button display area 1287), and described further below Π items selectable switch camera 1289 (e.g., switch camera button 1289) 1155 (e.g., a toolbar or menu bar). 因而,受邀者可以选择任意可选Π项目1285-1289(例如,通过单指轻点),以在视频会议期间执行希望的操作。 Accordingly, any alternative 1285-1289 invitees may be selected (e.g., by a single finger tap), to perform the desired operation during a video conference Π item. 不同的实施例允许受邀者按照不同的方式,例如切换移动设备上的开关,通过给出话音命令等,执行任意操作。 Different embodiments allow invitee in different ways, for example, switch on the mobile device, by giving a voice command or the like, perform any operation.

[0243] 尽管图12表示显示区1155的示例布局,不过,一些实施例提供显示区1155的不同布局,比如图11的显示区1155的布局,该布局只包括用于结束视频会议的可选的结束会议Π选项1132。 [0243] Although FIG. 12 shows an example of the layout of the display area 1155, however, some embodiments provide a different layout of the display area 1155, such as the layout of the display area 1155 of FIG. 11, the arrangement comprising only alternative for ending the videoconference 1132 end of the meeting Π option. 显示区1155的其它布局可以包括用于执行不同功能的任意多种不同的可选Π项目。 Other arrangements of the display region 1155 may include any of a number of different functions for performing different selectable items Π. 此外,第五阶段1230表示显示在Π 1205的底部的显示区1155。 Further, 1230 indicates the fifth stage at the bottom of the display area of ​​Π 1205 1155. 显示区1155 的不同实施例可被显示在Π 1205内的不同位置,和/或定义成不同的形状。 Different embodiments of the display area 1155 may be displayed at different locations within the Π 1205, and / or define different shapes.

[0244] 图12把显示区1155表示成静态显示区(即,显示区1155总是被显示)。 [0244] FIG 12 represents a static display area 1155 the display area (i.e., the display area 1155 is always displayed). 不过, 在一些实施例中,显示区1155是动态显示区。 However, in some embodiments, the display region 1155 is a dynamic display area. 在一些这样的实施例中,显示区1155平常不被显示。 In some such embodiments, the display area 1155 is not displayed normally. 相反,只有当接收到触发事件(例如,比如轻点显示区1280 -次,话音命令之类的用户选择)时,才显示所述显示区1155。 In contrast, only when receiving a trigger event (e.g., such as tap the display region 1280-- views, the user selects a voice command or the like) only when displaying the display region 1155. 在接收到用户选择(例如,选择可选的静音Π项目985),或者规定量的时间(例如,3秒)之后,显示区1155消失,所述规定量的时间可由用户通过移动设备或视频会议应用的优先选择设置指定。 Upon receiving a user selection (e.g., selection of a selectable item of the mute Π 985) after a predetermined time or amount of (e.g., three seconds), the display area 1155 disappears, the predetermined amount of time by the user via the mobile device or video conferencing application preference settings specified. 在一些这样的实施例中,显示区1155在视频会议开始后被自动显示,和按照上面提及的相同方式消失。 In some such embodiments, the display area 1155 is automatically displayed after the start of the video conference, and disappearance in the same manner as mentioned above.

[0245] 第六阶段1235图解说明在结束第五过渡阶段的动画之后的Π1205。 [0245] The sixth stage 1235 illustrates Π1205 after the end of the fifth stage of the transition animation. 具体地说, 第六阶段图解说明在视频会议期间,由Π 1205呈现的PIP显示1280。 Specifically, the sixth stage illustrated during a video conference, presented by the Π 1205 PIP display 1280. 如上所述,PIP显示1280包括两个视频显示:来自本地照相机的较大的背景显示1265,和来自远程照相机的较小的前景插入显示1244。 As described above, the PIP display 1280 includes two video display: the larger background display 1265 from the local camera, and the camera from a remote smaller foreground inset display 1244. PIP显示1280只是呈现由远程设备和本地设备拍摄的视频的合成视图的一种方式。 PIP display 1280 is just a way of presenting a composite view of the video captured by the remote device and the local device. 除了该合成视图之外,一些实施例的设备提供其它的合成视图。 In addition to the composite view, some devices provide other embodiments of the synthesized view. 例如, 代替具有较大的受邀者的背景显示,较大的背景显示可以是邀请者的视频,较小的前景插入显示可以是受邀者的视频。 For example, instead of having large background invitees display, the background display may be a large video inviter, smaller foreground invitee may be inserted into the video display. 如下进一步所述,一些实施例允许用户控制PIP显示中的插入视图和主视图,以可切换地显示本地照相机和远程照相机。 As further described below, some embodiments allow a user to control insert and front views of the PIP display to display the camera switchably local and remote cameras. 另外,一些实施例允许本地视频和远程视频出现在Π 1205中的两个并排显示区(例如,左右显示窗口,或者上下显示窗口),或者两个对角排列的显示区中。 Further, some embodiments allow the local and remote videos Π 1205 appears in the display areas of the two side by side (e.g., left and right display window, the display window or down), or the two display areas are aligned diagonally. 如下进一步所述,通过设备的优先选择设置,或者通过在视频会议期间,用户能够选择的控件,用户可以指定PIP显示的方式或者默认显示模式。 As described further below, by setting the priority selection device, or by the control during a video conference, the user can select, a user can specify how the PIP display or the default display mode.

[0246] 尽管图12按照六个不同的操作阶段表示呈现和接受视频会议邀请的操作序列, 不过,一些实施例可以用较少的阶段实现所述操作。 [0246] Although FIG. 12 shows the operation sequence of video presentation and receive a meeting invitation in accordance with the six different stages of operation, however, some embodiments may be implemented with fewer stages of the operation. 例如,一些这样的实施例可以省略呈现第三阶段1220和第四阶段1225,从而在用户选择"接受请求"选项1250之后,从第二阶段1215进到第五阶段1230。 For example, after some such embodiments may be omitted presented the third stage and fourth stages 1220 1225 so that the user selects "Accept request" option 1250, 1215 into the second stage to fifth stage 1230. 用较少的阶段实现所述操作(即,呈现和接受视频会议邀请)的其它实施例可以省略第一阶段1210和第二阶段1215,从而当受邀者从邀请者接收到视频会议的邀请时,向用户呈现第三阶段1220。 Other embodiments implement the operation (i.e., presentation and video conference invitation acceptance) with fewer stages may be omitted and the second stage of the first stage 1210 1215, so that when received from the invitee to invite the invitee while video conference , presents the third phase of 1220 to the user.

[0247] 图13图解说明通过把第一和第三阶段结合成一个阶段,以及把第二和第四阶段结合成一个阶段,用较少的阶段执行图12中图解说明的操作的例子。 [0247] FIG. 13 illustrates an example by combining the first and third stages to a stage, and the second and fourth stages combined into one stage, illustrated instructions with fewer stages 12 executes. 特别地,图13图解说明在五个不同的阶段1390、1392、1394、1230和1235,远程用户的设备1200的Π 1205。 In particular, FIG. 13 illustrates five different stages 1390,1392,1394,1230 and 1235, the remote user device Π 1205 1200's. 第一阶段1390类似于阶段1110,除了名字栏1295显示名字"John Smith",以指出在通话另一端的人物的名字之外。 The first stage is similar to stage 1390 1110, 1295 in addition to the name column displays the name "John Smith", to indicate the name of the person in the other end of the call outside. 即,已在远程移动设备的用户和本地设备的用户(即,本例中的John Smith)之间建立了通话。 That is, the user has a user device and the remote mobile local device (i.e., John Smith in this example) is established between the call. 第二阶段1392和第三阶段1394类似于图12的第一阶段1210和第二阶段1215,除了第二阶段1392和第三阶段1394还显示远程移动设备的用户(S卩,本例中的Nancy Jones)的预览之外。 The first stage 1392 and second stage 1394 is similar to the third stage 1210 of FIG. 12 and the second stage 1215, 1392 in addition to the second stage and third stage 1394 also displays a user of remote mobile device (S Jie, in the present embodiment, Nancy other than jones) preview. 第四阶段1230和第五阶段1235与图12的第五阶段1230 和第六阶段1235相同。 The same as the fourth stage and fifth stage 1230 of the fifth stage in FIG. 12 1235 1230 and 1235 phase VI.

[0248] 除了在通话期间,通过可选选项激活视频会议工具之外,一些实施例允许双照相机设备的用户直接发起视频会议,而不必首先打电话。 [0248] In addition to during a call, the video conference tools selectable option is activated outside of some embodiments allows users dual camera device to initiate a video conference directly, without having to first call. 图14图解说明发起视频会议的另一种这样的备选方法。 14 illustrates another alternative method of initiating such a video conference. 图14图解说明在七个不同阶段1410、1415、1420、1425、1430、1435和1440的Π 1405,所述七个不同阶段显示启动视频会议的备选操作序列。 14 illustrates in seven different stages 1410,1415,1420,1425,1430,1435 Π 1405 and 1440, of the seven different stages alternate display sequence of operations starts a video conference.

[0249] 在第一阶段1410,用户浏览移动设备上的联系人名单,寻找他希望与之进行视频会议的人物,与他为了通话而查找联系人的方式类似。 [0249] In the first stage 1410, the user through the contact list on the mobile device, with which he hopes to find a video conference people, and he calls and look up contacts in order similar way. 在第二阶段1415,用户选择他愿意与之进行视频会议的人物1455 (例如,通过在该人的名字1455上的单指轻点1460)。 In the second stage 1415, the user selects a video conference with whom he is willing to 1455 figures (for example, by a single person's name on the finger tap 1455 of 1460). 这种选择触发Π 1405显示联系人的信息和各种用户可选选项。 This choice trigger Π 1405 display contact information and various user selectable options. 在这个例子中,Jason的名字1455被突出显示,以指出这是用户意欲与之进行视频会议的人。 In this case, Jason's name 1455 is highlighted to indicate that this is the intended user of a video conference with people. 不同的实施例可以用不同的方式指出这种选择。 Different embodiments may indicate this selection in different ways. 尽管第二阶段1415允许设备1400的用户通过联系人名单,选择用户意欲与之进行视频会议的人,不过一些实施例允许用户通过"Recents (最近)"通话历史选择该人,所述"Recents"通话历史列出设备1400的用户最近与之进行视频会议或者通话的人物的特定号码或名字。 Although the second stage 1415 allows the device 1400 by a user contact list, select the intended user video conference with people, but some embodiments allow the user to "Recents (recently)" call history of the person selected by the "Recents" 1400 call history lists the device user with a specific number or the name of a recent video conference call or characters.

[0250] 在第三阶段1420,在选择了人物的名字1455之后,Π 1405显示所选人物的信息1462,和各种可选Π项目1468U472和1470。 After [0250] In the third stage 1420, the names of the characters in the choice of 1455, Π 1405 1462 displays the selected character information, and a variety of optional Π project 1468U472 and 1470. 在这个例子中,各个可选Π项目1472(可被实现成可选的图标或按钮)之一提供视频会议工具。 In this example, each selectable item Π 1472 (can be implemented as an optional icon or button) one video conferencing tool. 视频会议选项1472允许用户邀请依据联系人1466识别的人物加入视频会议。 1472 video conferencing options allow users to invite contacts to join a video conference based on 1466 figures identified. 不同的实施例不同地(例如,用不同的排列)显示信息1462和可选Π项目1468、1472和1470。 Different embodiments differently (e.g., with a different arrangement) 1462 and the display information items 1468,1472, and 1470 optionally Π.

[0251] 第四阶段1425表示用户选择"视频会议"选项1472(例如,通过单指轻点)。 [0251] The fourth stage 1425 represents a user selects the 'Video Conference' option 1472 (e.g., a single finger tap). 在这个例子中,"视频会议"选项1472被突出显示,以指出视频会议工具1472已被激活。 In this example, "video conference" option 1472 is highlighted to indicate that the video conference tool 1472 has been activated. 在不同的实施例中可不同地指出这种选择(例如,通过突出显示所选图标的文本或边框)。 This selection may be indicated differently (e.g., by highlighting the selected icon or text border) in different embodiments.

[0252] 第五、第六和第七阶段1430U435和1440类似于图11中图解说明的第三、第四和第五阶段1120、1125和1130,可通过参考这些阶段的讨论可理解第五、第六和第七阶段1430U435和1440。 [0252] The fifth, sixth and seventh stages 1430U435 and 1440 illustrated in Figure 11 illustrates a similar third, fourth and fifth stages 1120, 1125 and 1130, may be understood by reference to the discussion of the fifth stage, The sixth and seventh stages 1430U435 and 1440. 简单地说,第五阶段1430图解说明等待远程用户响应视频会议邀请的过渡性的保持阶段。 Simply put, the fifth stage 1430 illustrates the response wait for the remote user to keep a transitional stage video conference invitation. 第六阶段1435图解说明在远程用户接受视频会议请求之后,(显示本地用户的视频的)显示区1480的大小逐渐减小,使得Π 1405能够在显示区1480之后,显示包含来自远程用户的照相机的视频的显示区1492。 1435 illustrates the sixth stage after the video conference request receiving remote user, (user's local video display) size of the display area 1480 is gradually reduced, so that after possible Π 1405 1480 display area, the display includes a camera from a remote user's video display area 1492. 在第七阶段1440,在视频会议期间, Π 1405呈现PIP显示1447。 1440, during a video conference, Π 1405 presented in the seventh stage of the PIP display 1447. 在一些实施例中,第六阶段1435和第七阶段1440中的显示区1155的布局与上面说明的图12的显示区1155的布局相似。 In some embodiments, the display region layout display region sixth stage and seventh stage 1435 1155 1440 in FIG. 12 and described above is similar to the layout 1155.

[0253] 图10、11、12、13和14表示建立视频会议的几种方式。 [0253] FIG. 10, 11 and 14 shows several ways to establish a video conference. 在一些实施例中,在通话期间,通过一个通信通道(通过诸如电路交换通信网络或者分组交换通信网络之类的通信网络)传送音频数据(例如,话音),并在视频会议期间,通过另一个通信通道传送音频数据。 In some embodiments, during a call, through a communication channel (such as a circuit switched communication network via a packet-switched communication network or a communication network or the like) to transmit audio data (e.g., voice), and during a video conference through another transmitting audio data communication channel. 从而,在这样的实施例中,在建立视频会议之前,通过一个通信通道传送音频数据(例如, 话音),以及一旦建立了视频会议,就通过不同的通信通道(而不是在通话期间使用的通信通道)传送音频。 Thus, in such an embodiment, prior to the establishment of a video conference through a communication channel to transmit audio data (e.g., voice), and upon the establishment of a video conference, on through different communication channels (not used for communication during a call channel) transmit audio.

[0254] 为了提供音频数据从通话到视频会议的无缝过渡(例如,切换),一些实施例在建立视频会议之前,并不终止通话。 [0254] In order to provide a seamless transition of audio data from the video conference call (e.g., switching), some embodiments before establishing the video conference, the call does not terminate. 例如,一些实施例在终止通话并开始通过对等通信会话传送音频/视频数据之前,建立对等视频会议连接(例如,在完成图10中图解说明的消息序列之后)。 For example, some embodiments terminate the call and prior to the start peer to peer communications session through transmit audio / video data, video conferencing, etc. establish the connection (e.g., after the completion of the illustrated message sequence described in FIG. 10). 另一方面,其它实施例在终止通话和开始呈现接收的音频/视频数据之前,建立对等视频会议连接(例如在完成图10中图解说明的消息序列之后),并开始通过该对等通信会话传送音频/视频数据。 On the other hand, in other embodiments prior to terminating the call and begin rendering the received audio / video data, the establishment (e.g., after the message sequence described in FIG. 10 illustrates the completion) of the other video conference connection, and starts a communication session via the peer transmitting audio / video data.

[0255] -些实施例的对等视频会议连接允许视频会议中的移动设备直接相互通信(而不是通过例如中央服务器通信)。 [0255] - some peer to peer embodiment of a video conference connection allows direct communication with each other in the video conference mobile device (e.g., rather than by the central server communication). 对等视频会议的一些实施例允许视频会议中的移动设备相互共享资源。 Some embodiments of the video conference, etc. allow the mobile device in a video conference with each other shared resources. 例如,经由视频会议的控制通信通道,通过从一个移动设备向另一个移动设备发送指令,以命令所述另一个移动设备不同地处理图像(即,共享其图像处理资源),比如下面进一步详细描述的曝光调整操作、焦距调整操作和/或切换照相机操作,所述一个移动设备能够远程控制视频会议中的所述另一个移动设备的操作。 For example, a video conference via a control communication channel by another mobile device to send instructions to command the mobile device further treated differently from a mobile device an image (i.e., image processing share its resources), such as described in further detail below exposure adjustment operation, the focus adjustment operation and / or a camera switching operation, the mobile device can be a remote control of the video conference in another operation of the mobile device.

[0256] 2.动态比特率设置 [0256] 2. Dynamic bit rate setting

[0257] -般来说,视频会议中的移动设备通过不同类型的通信网络,比如不同的专用和公共无线通信网络(例如,诸如GSM、UMTS之类的蜂窝网络)的通信通道,相互传递数据(例如,音频和视频图像)。 [0257] - In general, the video conference apparatus through different types of mobile communication networks, such as different public and private wireless communication network (e.g., such as GSM, UMTS cellular network like) of the communication channel, each transmitted data (e.g., audio and video). 下面将参考图91和92说明这种无线通信网络的例子。 Examples 91 and 92 will be described with reference to such a wireless communication network of FIG.

[0258] 由于在特定时间接入通信网络的移动设备的数目不断变化,因此通信网络可用于进行视频会议的带宽在不同时间是不同的。 Bandwidth [0258] Since the number of access to the communication network at a particular time of changing the mobile device, the communication network can be used for videoconferencing at different times are different. 即使在视频会议期间,可用带宽也会改变。 Even during the video conference, the available bandwidth will change. 另夕卜,在视频会议期间用高比特率淹没通信网络,或者企图算出最佳视频会议比特率的大量信令是不可取的。 Another Bu Xi, during a video conference communication network flooded with a high bit rate, or a large number of attempts is calculated optimal video conference signaling bit rate is not desirable.

[0259] 由于这些原因,一些实施例采用指定视频会议的初始最佳比特率的新方法。 [0259] For these reasons, some embodiments of the new method for the specified video conference optimal initial bit rate employed. 为了识别视频会议的初始最佳比特率,所述方法以特定的比特率开始视频会议,如果这些实施例没有检测到会恶化视频会议质量的网络条件,那么所述方法以特定时间间隔递增地增大比特率。 To identify the initial optimal bit rate of the video conference, video conference, the method begins with a specific bit rate, if the embodiment does not detect network conditions may deteriorate the quality of the video conference, the method incrementally spaced by a specific time bitrate.

[0260] 图15中图解说明这种实施例的一个例子。 [0260] FIG. 15 illustrates one example of such an embodiment. 图15概念地图解说明本发明的一些实施例的设定视频会议的比特率的处理1500。 FIG 15 conceptually illustrates some of the bit rate of the video conference is set to an embodiment of the present invention a process 1500. 作为视频会议设置的一部分(例如,作为图10 中图解说明的视频会议建立的一部分)执行处理1500,以根据各种网络条件,动态确定传送数据(例如,音频和视频图像)的比特率。 As part of a video conference set (e.g., set up a video conference as illustrated in FIG. 10 illustrates a part of) the execution process 1500, according to various network conditions, data transfer is determined dynamically (e.g., audio and video) bit rate. 在一些实施例中,处理1500由上面参考图9 说明的视频会议模块925的管理层935执行。 In some embodiments, process 1500 from the video conference module described above with reference to FIG. 9 925 935 management performed. 下面参考图16说明该视频会议模块的更详细形式。 Referring to FIG. 16 illustrates in more detail the form of a video conference module.

[0261] 如图15中所示,通过把比特率设为(在1505)初始比特率,开始处理1500。 [0261] As shown in FIG. 15, the bit rate is set by the (1505) an initial bit rate, process 1500 starts. 在一些实施例中,初始比特率是设备的默认基准速率。 In some embodiments, the initial bit rate is the reference rate default device. 不过,一些实施例允许用户指定初始比特率。 However, some embodiments allow the user to specify the initial bit rate. 在1505,处理1500还通过按初始比特率,通过一个或多个通信通道,向远程设备传送数据(例如,音频和视频图像),来启动视频会议。 1505, 1500 through the processing at the initial bit rate, via one or more communication channels transmitted data (e.g., audio and video) to the remote device to initiate a video conference.

[0262] 随后,处理1500识别(在1510)从视频会议中的远程设备接收的一组网络条件参数。 [0262] Subsequently, process 1500 identifies (at 1510) a set of network condition parameters received from the remote video conference equipment. 在一些实施例中,本地设备通过在开始视频会议时建立的实时传送协议(RTP)通信会话,从远程设备接收所述一组网络条件参数。 In some embodiments, the local device through real time transport protocol established at the beginning of the video conference (RTP) communication session from a remote device to receive the set of network condition parameters. 例如,一些实施例通过RTP的扩展特征,提供网络条件参数。 For example, some embodiments feature by extending the RTP, there is provided a network condition parameters. 此外,通过指出扩展报头在RTP分组报头中的存在,并定义关于附加信息的扩展报头,一些实施例的RTP扩展特征能够被用于传送任意类型的信息(例如,所述一组网络条件参数)。 Further, by indicating an extension header is present in the RTP packet header, and define the extension header information about the number of RTP extension features of the embodiments can be used to transmit any type of information (e.g., the set network condition parameters) .

[0263] 在不同的实施例中,视频会议中的各个设备传送多组不同的网络条件/拥塞参数。 [0263] In various embodiments, the video conference apparatus transmits respective different sets of network condition / congestion parameter. 在下面说明的实施例中,所述一组网络条件参数包括单向等待时间和带宽估计比特率。 In an embodiment described below, the set condition parameters comprising a unidirectional network latency and bandwidth estimation bit rate. 在其它实施例中,所述一组网络条件参数包括分组丢失数据和往返时间(RTT)延迟数据。 In other embodiments, the set of parameters comprises a delay network conditions data and data packet loss round-trip time (RTT). 因而,不同的实施例可在所述一组网络条件参数中包括任意多个不同的网络条件参数。 Thus, various embodiments may include any number of different network conditions in the parameter set of network condition parameters.

[0264] 在一些实施例中,从视频会议的远程设备接收的一组网络条件参数以按在操作1505设定的初始比特率,在视频会议期间,从本地移动设备(即,执行处理1500的移动设备)传送给远程设备的数据(例如,音频和视频)为基础。 [0264] In some embodiments, a set of network condition parameters from the remote video conference equipment according to the received operation 1505 to set initial bit rate, during a video conference, from the local mobile device (i.e., a process 1500 the mobile device) of data transmitted to a remote device (e.g., audio and video) is based. 例如,在一些实施例中,通过利用音频分组的时间戳,计算音频分组通过网络连接,从本地移动设备传播到远程设备所用的时间,远程设备能够确定单向等待时间。 For example, in some embodiments, by utilizing the time stamp of the audio packet, audio packet connection is calculated through the network to the propagation time of the local mobile device used by the remote device from the remote device to determine the one-way latency. 具体地说,在一些实施例中,每个音频分组被打上时间戳。 Specifically, in some embodiments, each audio packet is time stamped. 在没有分组延迟的情况下,远程设备应与时间戳之差相等的设定时间间隔,接收音频分组。 In the absence of delayed packets, the remote device should be set equal to the difference between the time stamp of the spacer, to receive audio packets. 不过,当存在单向等待延迟时,远程设备以比时间戳之差大的时间每隔接收音频分组。 However, when there is a one-way latency delay, to the remote device is larger than the difference between the time stamp of the audio packet receiving intervals.

[0265] 另外,在一些实施例中,远程设备通过检查接收到视频分组的时间,接收到相邻视频分组的时间,和所述相邻视频分组的大小,确定带宽估计比特率。 [0265] Further, in some embodiments, the remote device receives the time by checking the video packet, video packet temporally adjacent received, the video packets and the size of the adjacent bit rate determines the bandwidth estimation. 即,接收两个连续视频分组的时间差和第二个视频分组的大小被用于估计网络连接的可用带宽。 That is, receiving two consecutive video packets and the size of the second time difference between the video packets are used to estimate the available bandwidth of the network connection. 一些实施例通过检查多对连续视频分组,确定带宽估计比特率。 Some embodiments by examining a plurality of pairs of consecutive video packets, determines the bandwidth of the estimated bit rate. 上面的例子利用特定类型的数据(即,用于确定单向等待时间的音频数据,和用于确定带宽估计比特率的视频数据)。 The above example using particular type of data (i.e., audio data for determining the one-way latency time and bandwidth estimation for determining the video data bit rate). 不过,在一些实施例中,也可以使用通过本地移动设备和远程设备之间的网络连接传送的其它类型的数据。 However, in some embodiments, it may also be used other types of data transmission through a network connection between a local mobile device and a remote device.

[0266] 在识别(在1510)所述一组网络条件之后,处理1500随后确定(在1515)单向等待时间是否已恶化到超过定义的阈值量。 [0266] In recognition (in 1510) after the set of network conditions, then the process 1500 determines (at 1515) whether a one-way latency time is deteriorated to a threshold amount exceeds a defined. 在一些实施例中,所述阈值量被定义为特定的等待时间量,如果当前的单向等待时间和先前的单向等待时间之间的差值超过特定等待时间量,那么确定单向等待时间恶化到超过阈值量。 In some embodiments, the threshold amount is defined as a specific amount of waiting time, if the current and the previous way latency difference between the one-way latency time exceeds a certain amount of time to wait, it is determined that the one-way latency deteriorated beyond a threshold amount. 在其它实施例中,阈值量被定义为单向等待时间的特定变化率。 In other embodiments, the threshold amount is defined as the rate of change of a particular one-way latency. 因而,当一组单向等待时间(例如,当前的单向等待时间和先前的单向等待时间)的变化率超过特定的变化率,那么确定单向等待时间恶化到超过阈值量。 Thus, when a group of one-way latency (e.g., the current and the previous latency unidirectional way latency) rate of change exceeds a particular rate of change, it is determined that one-way latency value exceeding a threshold amount of time to deteriorate.

[0267] 当确定单向等待时间恶化到超过阈值量时,处理1500结束。 [0267] When it is determined to deteriorate way latency exceeds a threshold amount, the process 1500 ends. 否则,处理1500确定(在1520)当前比特率是否达到了带宽估计比特率。 Otherwise, the process 1500 determines (at 1520) whether the current bit rate of the bandwidth estimation bit rate. 在一些实施例中,带宽估计比特率指示网络连接的可用带宽量(例如,15kb/s (kbps))。 In some embodiments, the bandwidth estimation bit rate indicating an amount of available bandwidth (e.g., 15kb / s (kbps)) of the network connection. 当处理1500确定当前比特率超过带宽估计比特率时,处理1500结束。 When the process 1500 determines that the current bit rate exceeds the bit rate bandwidth estimation, process 1500 ends. 当处理1500确定当前比特率未超过带宽估计比特率时,处理1500进入操作1525。 When the process 1500 determines that the current bit rate does not exceed the bandwidth estimation bit rate, process 1500 proceeds to operation 1525.

[0268] 在1525,处理1500确定当前比特率是否达到定义的最大比特率。 [0268] In 1525, process 1500 determines whether the current bit rate reach the maximum bit rate defined. 当处理1500确定当前比特率超过定义的最大比特率时,处理1500结束。 When the process 1500 determines that the current bit rate exceeds a defined maximum bit rate, process 1500 ends. 否则,处理1500进入操作1530, 把当前比特率增大规定量。 Otherwise, the process 1500 proceeds to operation 1530, the current bit rate is increased by a predetermined amount. 不同的实施例不同地定义增大比特率的量。 Different embodiments defined differently increase the amount of bit rate. 增大当前比特率的规定量的例子包括32kbps、64kpbs、以及任意多种其它增大比特率的量值。 Increasing the predetermined amount of current examples include bit rate 32kbps, 64kpbs, and any of a variety of other values ​​of the bit rate is increased.

[0269] 随后,处理确定(在1535)规定的时间量是否已过去。 [0269] Subsequently, the process determines (at 1535) whether a specified amount of time has elapsed. 规定的时间量可以是1秒、2 秒、5秒或者任何其它可能的时间量,因为不同的实施例不同地定义所述时间量。 A predetermined amount of time may be 1 second, 2 seconds, 5 seconds or any other amount of time possible, because different embodiments of the variously defined amount of time. 处理1500 等待规定的时间量过去,以便让远程设备能够接收以新增大的比特率(在操作1530),从本地移动设备传送的数据(例如,音频和视频图像),并根据新增大的比特率确定网络条件参数。 Waiting a predetermined amount of processing time over 1500, to allow the new remote device capable of receiving a large bit rate (in operation 1530), data (e.g., audio and video) transmitted from the home mobile device, and in accordance with the new large network condition determining a bit rate parameter. 如果处理1500确定规定的时间量还未过去,那么处理1500返回操作1535,直到规定的时间量过去为止。 If the processing 1500 determined amount of time has not been specified in the past, the process 1500 returns to operation 1535, the amount so far in the past until a specified time. 当处理1500确定规定的时间量已过去时,处理1500返回操作1510。 When the process 1500 determines a predetermined amount of time has elapsed, the process 1500 returns to operation 1510. 如上所述继续进行处理1500的从1510开始的操作,直到处理1500结束为止。 Process 1500 continues as described above from the beginning of operation 1510 until the process 1500 ends.

[0270] 当处理1500结束时(S卩,在操作1515、1520或1525之后),完成视频会议的比特率的设置,并且确定了最佳比特率。 [0270] When the process at the end of 1500 (S Jie, after the operation of 1515,1520 or 1525), to complete the setting of the bit rate of the video conference, and determines the optimal bit rate. 由于视频会议的可用带宽在视频会议期间可能变化,因此,一些实施例继续根据从远程设备接收的一组网络条件参数(即,单向等待时间和带宽估计比特率),调整比特率。 Since the available bandwidth of the video conference may change during a video conference, therefore, in accordance with some embodiments, a set of network conditions continue parameters received from a remote device (i.e., one-way latency and bandwidth estimated bit rate), adjust the bit rate. 可通过增大比特率,在视频会议期间调整比特率。 By increasing the bit rate, the bit rate is adjusted during the video conference. 例如,如果处理1500因单向等待时间恶化到超过规定的阈值量而结束,并且在视频会议期间,单向等待时间得到改善,那么一些实施例增大比特率。 For example, if the processing 1500 to deteriorate due to the one-way latency exceeds a predetermined threshold amount ended, and during the video conference, the one-way latency is improved, then the bit rate is increased to some embodiments. 类似地,如果处理1500因比特率超过带宽估计比特率而结束,并且在视频会议期间,带宽估计比特率增大,那么一些实施例增大比特率。 Similarly, if the bit rate exceeds the processing 1500 by the bandwidth estimation bit rate end, and during the video conference, the bandwidth estimation bit rate increases, the bit rate is increased to some embodiments.

[0271] 相反,可通过降低比特率,在视频会议期间调整比特率。 [0271] In contrast, by reducing the bit rate, the bit rate is adjusted during the video conference. 例如,如果在视频会议期间,单向等待时间继续恶化到超过规定的阈值量,那么一些实施例降低比特率。 For example, if during a video conference, the continued deterioration of the one-way latency exceeds a predetermined threshold amount, then some embodiments of the bit rate reduction. 另外,如果在视频会议期间,比特率继续超过带宽估计比特率(例如,带宽估计比特率继续降低),那么一些实施例降低比特率。 Further, if during a video conference, the bit rate continues to exceed the bandwidth estimation bit rate (e.g., bit rate continues to decrease the bandwidth estimation), then some embodiments of the bit rate reduction.

[0272] 此外,处理1500使用单向等待时间和带宽估计比特率来确定是否增大比特率。 [0272] In addition, process 1500 using one-way latency and bandwidth to determine whether the estimated bit rate to increase the bit rate. 不过,普通技术人员会认识到在不同的实施例中,可以使用任意多种网络条件参数来确定是否增大比特率。 However, ordinary skill in the art will recognize that various embodiments may use any variety of network condition parameters to determine whether to increase the bit rate. 例如,确定是否增大比特率可以只基于RTT延迟数据或者分组丢失数据。 For example, it is determined whether the bit rate increase may be based only on data RTT delay or packet loss data.

[0273] C.视频会议体系结构 [0273] C. Video Conferencing Architecture

[0274] 如上所述,图16概念地图解说明一些实施例的双照相机移动设备的视频会议和处理模块1600的软件体系结构。 [0274] As described above, FIG 16 conceptually illustrates the software architecture of the mobile video conferencing dual camera apparatus embodiment 1600 and the processing module. 如图所示,视频会议和处理模块1600包括客户端应用1665、视频会议模块1602、媒体交换模块1620、缓冲器1625、拍摄图像处理单元(CIPU)驱动器1630、编码器驱动器1635和解码器驱动器1640。 As shown, video conferencing and processing module 1600 includes a client application 1665, a video conference module 1602, media exchange module 1620, a buffer 1625, a captured image processing unit (the CIPU) drives 1630, the encoder driver 1635 and a decoder driver 1640 . 在一些实施例中,缓冲器1625是保存视频的图像以便在双照相机移动设备的显示器1645上显示的帧缓冲器。 In some embodiments, the frame buffer 1625 is a buffer for holding a video image for display on the display 1645 pairs camera mobile device.

[0275] 在一些实施例中,客户端应用1665与图9的视频会议客户端945相同。 [0275] In some embodiments, the client application 1665 and the video conference client terminal 945 is the same as FIG. 如上所述,客户端应用1665可被集成到另一个应用中,或者实现成独立的应用。 As described above, the client application 1665 may be integrated into another application, or implemented as a separate application. 客户端应用1665 可以是利用视频会议模块1602的视频会议功能的应用,比如视频会议应用、IP话音(V0IP) 应用(例如,Skype)或者即时消息通信应用。 The client application 1665 may be a video-conferencing application module 1602 videoconferencing capabilities, such as video conferencing applications, the IP voice (V0IP) applications (e.g., Skype) or instant messaging applications.

[0276] -些实施例的客户端应用1665向视频会议模块1602发送指令,比如开始会议和结束会议的指令,从视频会议模块1602接收指令,把来自双照相机移动设备的用户的指令路由到视频会议模块1602,和产生显示在双照相机移动设备上并允许用户与应用交互的用户界面。 [0276] - these clients embodiment is applied 1665 to the video conference 1602 sends an instruction module, such as an instruction to start session and end of the meeting, from 1602 receives an instruction video conferencing module, the user's instruction from the dual camera mobile device is routed to the video conference module 1602, and generates a display on the mobile device and the dual camera allows the user to interact with an application user interface.

[0277] D.视频会议管理器 [0277] D. Video Conference Manager

[0278] 如图16中所示,视频会议模块1602包括视频会议管理器1604、图像处理管理器1608、连网管理器1614、和缓冲器1606、1610、1612、1616和1618。 [0278] As shown in FIG. 16, a video conference module 1602 includes a video conference manager 1604, an image processing manager 1608, networking manager 1614, 1606,1610,1612,1616 and 1618, and buffers. 在一些实施例中,视频会议模块1602与图9中图解说明的视频会议模块925相同,从而执行上面关于视频会议模块925所述的一些或全部的相同功能。 In some embodiments, the same video conference module 1602 illustrated in FIG. 9 videoconferencing module 925, thereby performing the above with respect to some or all of the same functions as the video conference module 925.

[0279] 在一些实施例中,视频会议管理器1604负责当视频会议开始时,初始化视频会议模块1602的一些或所有其它模块(例如,图像处理管理器1608和连网管理器1614),在视频会议期间控制视频会议模块1602的操作,和当视频会议结束时,停止视频会议模块1602 的一些或所有其它模块的操作。 [0279] In some embodiments, the video conference manager 1604 is responsible when starting the video conference, the video conference module 1602 to initialize some or all of the other modules (e.g., image processing and networking management manager 1608 1614), the video during the video conference session control module 1602 operation, and when the end of the video conference, the video conference module stops operation of some or all of the other modules 1602.

[0280] 一些实施例的视频会议管理器1604还处理从视频会议中的一个或多个设备接收的图像,和由双照相机移动设备的一部或两部照相机拍摄的图像,以便显示在双照相机移动设备上。 [0280] Some video conference manager 1604 further embodiment one or more devices processed image received from the video conference, and an image captured by a camera or two double camera mobile device in order to display the camera-bis on mobile devices. 例如,一些实施例的视频会议管理器1604从缓冲器1618取回从参与视频会议的另一个设备接收的解码图像,和从缓冲器1606取回由CIPU 1650处理的图像(即,由双照相机移动设备拍摄的图像)。 For example, some embodiments of the video conference manager 1604 retrieves from the decoded image buffer 1618 received from another device is participating in a video conference, and retrieved by the image processing CIPU 1650 from the buffer 1606 (i.e., the camera is moved by bis the device captured image). 在一些实施例中,在把图像显示在双照相机移动设备上之前,视频会议管理器1604还比例缩放和合成图像。 In some embodiments, before displaying an image on the dual camera mobile device 1604 further scaling ratio of the video conference manager composite image. 即,在一些实施例中,视频会议管理器1604产生PIP或其它合成视图,以显示在移动设备上。 That is, in some embodiments, generates a video conference manager 1604 or other synthetic PIP view for display on the mobile device. 一些实施例比例缩放从缓冲器1606 和1618取回的图像,而其它实施例只比例缩放从缓冲器1606和1618之一取回的图像。 Some embodiments of the scaling buffer 1606 and 1618 from the retrieved image, while other embodiments only scaling retrieved from the buffer 1606 and one of 1618 images.

[0281] 尽管图16把视频会议管理器1604图解表示成视频会议模块1602的一部分,不过视频会议管理器1604的一些实施例被实现成与视频会议模块1602分离的组件。 [0281] Although FIG. 16 illustrates a video conference manager 1604 shown as part of a video conference module 1602, but the video conference manager 1604 of some embodiments is implemented as a separate component of the video conference module 1602. 因而,单个视频会议管理器1604可被用于管理和控制几个视频会议模块1602。 Thus, a single video conference manager 1604 may be used to manage and control several video conference module 1602. 例如,一些实施例将在本地设备上运行独立的视频会议模块以与多方会议中的每一方交互,本地设备上的这些视频会议模块均由一个视频会议管理器管理和控制。 For example, some embodiments will be running on the local device independent video conference and multi-party conference modules each party interaction, the video conferencing module on the local device by the manager to manage and control a video conference.

[0282] 在图像被编码器1655编码之前,一些实施例的图像处理管理器1608处理由双照相机移动设备的照相机拍摄的图像。 [0282] In the image encoder 1655 is encoded before, some of the embodiments of the image processing manager 1608 processes the image captured by the camera of the dual camera mobile device. 例如,图像处理管理器1608的一些实施例对CIPU 1650处理的图像执行曝光调整、焦距调整、透视校正、动态范围调整和图像缩放中的一个或多个操作。 For example, image processing for image processing CIPU 1650 manager 1608 of some embodiments perform exposure adjustment, focus adjustment, perspective correction, image scaling and dynamic range adjustment of one or more operations. 在一些实施例中,图像处理管理器1608控制传送给视频会议中的另一个设备的编码图像的巾贞速率。 In some embodiments, the image processing control manager 1608 transmits the image to the encoding rate Zhen towel another video conference apparatus.

[0283] 连网管理器1614的一些实施例管理双照相机移动设备和参加视频会议的另一个设备之间的一个或多个连接。 [0283] Some networking manager 1614 between the dual camera mobile device and another video conference apparatus embodiment of one or more connection management. 例如,一些实施例的连网管理器1614在视频会议开始时,建立双照相机移动设备和视频会议的另一个设备之间的连接,以及在视频会议结束时,中断这些连接。 For example, some embodiments of the networking manager 1614 at the start of the video conference, establish a connection between the dual camera mobile device to another device and video conferencing, as well as at the end of the video conference, interrupt the connection.

[0284] 在视频会议期间,连网管理器1614把由编码器1655编码的图像传送给视频会议的另一个设备,以及把从视频会议的另一个设备接收的图像路由给解码器1660以便解码。 [0284] During a video conference, another networking device manager 1614 video conference, and the image received from another routing device to the video conference to the decoder 1660 for decoding by the image encoder 1655 transmits encoded. 在一些实施例中,连网管理器1614,而不是图像处理管理器1608,控制传送给视频会议的另一个设备的图像的帧速率。 In some embodiments, the networking manager 1614, instead of the image processing manager 1608, a control frame rate of the image transfer apparatus to another video conference. 例如,连网管理器1614的一些这样的实施例通过丢弃(即, 不传送)应该被传送给视频会议的另一个设备的编码帧中的一些编码帧,控制帧速率。 For example, some such networking manager 1614 by dropping embodiments (i.e., not transmitted) should be transmitted to some of the other coded frame coded frames of video conferencing equipment, the frame rate control.

[0285] 如图所示,一些实施例的媒体交换模块1620包括照相机源模块1622、视频压缩模块1624和视频解压缩模块1626。 [0285] As shown, some of the media exchange module 1620 of the embodiment includes a camera source module 1622, the video decompression module 1624 and module 1626 to video compression. 媒体交换模块1620与图3中所示的媒体交换模块310相同,同时提供更多的细节。 The same switching module 1620 media exchange with the media module 310 shown in FIG. 3, while providing more details. 照相机源模块1622通过CIPU驱动器1630,在视频会议模块1602 和CIPU1650之间路由消息和媒体内容,视频压缩模块1624通过编码器驱动器1635,在视频会议模块1602和编码器1655之间路由消息和媒体内容,以及视频解压缩模块1626通过解码器驱动器1640,在视频会议模块1602和解码器1660之间路由消息和媒体内容。 The camera source module 1622 through CIPU driver 1630, between a video conference module 1602 and CIPU1650 routing messages and media content, video compression module 1624 by the encoder driver 1635, in a video conference module 1602 and the encoder between 1655 routing messages and media content , and a video decompression module 1626 by the decoder driver 1640, in a video conference module 1602 and a decoder routing messages between 1660 and media content. 一些实施例把包括在媒体交换模块310中的TNR模块315 (图16中未示出)实现成照相机源模块1622的一部分,而其它实施例把TNR模块315实现成视频压缩模块1624的一部分。 Some embodiments include the TNR module 315 (not shown in FIG. 16) implemented as part of the camera module 1622 in a source media exchange module 310, while other embodiments the TNR module 315 implemented as part of video compression module 1624.

[0286] 在一些实施例中,CIPU驱动器1630和编码器驱动器1635与图3中图解说明的CITO驱动器305和编码器驱动器320相同。 [0286] In some embodiments, the same CIPU driver 1630 and the encoder 1635 and the drive illustrated in FIG. 3 CITO driver 305 driver 320 and the encoder. 一些实施例的解码器驱动器1640充当视频解压缩模块1626和解码器1660之间的通信接口。 Some embodiments of the decoder driver 1640 serving as a video decompression module 1626 and the communication interface between the decoder 1660. 在这样的实施例中,解码器1660解码通过连网管理器1614从视频会议的另一个设备接收的并通过视频解压缩模块1626路由的图像。 In such an embodiment, the module 1626 routes decoded by an image decoder 1660 networking manager 1614 received from another device and compressed by a video conference video decompression. 在图像被解码之后,图像通过解码器驱动器1640和视频压缩模块1626被送回视频会议模块1602。 After an image is decoded by the image decoder driver 1640 and a video compression module 1626 is sent back to the video conference module 1602.

[0287] 除了在视频会议期间进行视频处理之外,一些实施例的双照相机移动设备的视频会议和处理模块1600还在视频会议期间,执行音频处理操作。 [0287] In addition to the video processing during a video conference, dual camera during a video conference in the embodiment of the mobile device and the processing module 1600 still video conferencing, audio processing operations performed. 图17图解说明这样的软件体系结构。 FIG 17 illustrates such a software architecture. 如图所示,视频会议和处理模块1600包括视频会议模块1602 (它包括视频会议管理器1604、图像处理管理器1608和连网管理器1614)、媒体交换模块1620和客户端应用1665。 As shown, video conferencing and processing module 1600 includes a video conference module 1602 (which includes the video conference manager 1604, an image processing and networking management manager 1608 1614), the media exchange module 1620 and a client application 1665. 图17中省略了图16中所示的视频会议和处理模块1600的其它组件和模块,以简化说明。 FIG 17 is omitted in the video conference shown in FIG. 16 and other components of the processing module and modules 1600, to simplify the description. 视频会议和处理模块1600还包括帧缓冲器1705和170、音频处理管理器1715和音频驱动器1720。 Video conferencing and processing module 1600 further includes a frame buffer 170 and 1705, the audio processing manager 1715, and an audio driver 1720. 在一些实施例中,音频处理管理器1715被实现成单独的软件模块,而在其它实施例中,音频处理管理器1715被实现成媒体交换模块1620的一部分。 In some embodiments, the audio processing manager 1715 is implemented as a separate software module, while in other embodiments, the audio processing manager 1715 module 1620 is implemented as part of the media exchange.

[0288] 音频处理管理器1715处理由双照相机移动设备捕获的音频数据,以便传送给视频会议中的另一个设备。 [0288] The audio processing manager 1715 process audio data captured by the dual camera mobile device in order to transmit to another device in the video conference. 例如,音频处理管理器1715通过音频驱动器1720接收由麦克风1725捕获的音频数据,对音频数据编码,之后把编码的音频数据保存在缓冲器1705中, 以便传送给所述另一个设备。 For example, the audio processing manager 17151720 received through the audio data from the audio driver captured by the microphone 1725, the audio data is encoded, then the encoded audio data stored in the buffer 1705 for transmission to said another device. 音频处理管理器1715还处理由视频会议中的另一个设备捕获并从所述另一个设备接收的音频数据。 Audio processing manager 1715 also processes a video conference captured by another device, and the audio data received from the other device. 例如,音频处理管理器1715从缓冲器1710取回音频数据,对音频数据解码,解码后的音频数据随后通过音频驱动器1720被输出给扬声器1730。 For example, the audio processing manager 1715 retrieves the audio data from the buffer 1710, the audio data decoding, the decoded audio data is then output to a speaker 1730 through the audio driver 1720.

[0289] 在一些实施例中,视频会议模块1602连同音频处理管理器1715及其关联的缓冲器一起是更大的会议模块的一部分。 [0289] In some embodiments, a video conference module 1602 along with the audio processing manager 1715 and its associated buffer together are part of a larger conference modules. 当在不交换视频内容的情况下,在几个设备之间进行多方音频会议时,视频会议和处理模块1600只利用连网管理器1614和音频处理管理器1715,以使通过网际协议(IP)层的音频交换更容易。 When the video content without switching, the multi-party audio conference between several devices, video conferencing and processing module 1600 using the networking manager 1614 and the audio processing manager 1715, so that an Internet Protocol (IP) audio Interchange layer easier.

[0290] 现在参考图18,说明一些实施例的视频会议管理器1604的操作。 [0290] Referring now to Figure 18, illustrate operation of some embodiments of the video conference manager 1604. 图18概念地图解说明由一些实施例的视频会议管理器,比如图16中图解说明的视频会议管理器1604执行的处理1800。 Processing 1800 in FIG. 18 conceptually illustrates a video conference manager of some embodiments, such as illustrated in FIG. 16 video conference manager 1604 is performed. 这可等同于由图9的管理层935执行。 This may be performed by management equivalent to 935 in FIG. 9. 在一些实施例中,当双照相机移动设备的用户接受(例如,通过显示在双照相机移动设备上的用户界面)视频会议请求,或者当另一个设备的用户接受双照相机移动设备的用户发送的请求时,视频会议管理器1604 执行处理1800。 In some embodiments, when the dual camera mobile device accepts user (e.g., by displaying the user interface on the dual camera mobile device) video conference request, or when another user accepts a user equipment dual camera mobile device transmits a request when the video conference manager 1604 performs processing 1800.

[0291] 通过接收(在1805)开始视频会议的指令,开始处理1800。 [0291] By receiving the instruction to start a video conference (1805), process 1800 begins. 在一些实施例中,指令是从客户端应用1665接收的,或者是通过显示在双照相机移动设备上的用户界面从用户接收的,并由客户端应用1665转发给视频会议管理器1604。 In some embodiments, the application instruction is received from the client 1665, or is received from a user through the user interface displayed on the dual camera mobile device 1665 forwards by the client application 1604 to the video session manager. 例如,在一些实施例中,当双照相机移动设备的用户接受视频会议请求时,指令是通过用户界面接收的并由客户端应用转发。 For example, in some embodiments, when the user double video camera mobile device accepts the meeting request, the instruction is received through the user interface by forwarding the client application. 另一方面,当另一个设备的用户接受从本地设备发出的请求时,一些实施例从客户端应用接收指令,而没有用户界面交互(不过,可存在先前的发送初始请求的用户界面交互)。 On the other hand, when the user of the other device accepts requests from the local device, some embodiments receive instructions from the application client, but no user interface interaction (however, there may be user interface interactions previous sending the initial request).

[0292] 随后,处理1800初始化(在1810)与视频会议管理器1604交互的第一模块。 [0292] Subsequently, the process 1800 initializes (1810) the first module interacts with the video conference manager 1604. 一些实施例的与视频会议管理器1604交互的模块包括CIPU 1650、图像处理管理器1608、音频处理管理器1715和连网管理器1614。 Some interaction with the video conference manager module 1604 embodiments include CIPU 1650, image processing manager 1608, an audio processing manager 1715 and the networking manager 1614.

[0293] 在一些实施例中,初始化CIPU 1650包括指令CIPU 1650开始处理由双照相机移动设备的一部或两部照相机拍摄的图像。 [0293] In some embodiments, including the initialization instruction CIPU 1650 CIPU 1650 starts processing the image captured by a camera or two double camera mobile device. 一些实施例通过指令图像处理管理器1608开始从缓冲器1610取回图像,并处理和编码取回的图像,初始化图像处理管理器1608。 Some embodiments 1608 starts by retrieving an instruction from the manager of the image processing the image buffer 1610, and the retrieved image processing and coding, initialization of the image processing manager 1608. 为了初始化音频处理管理器1715, 一些实施例指令音频处理管理器1715开始编码由麦克风1725 捕获的音频数据,和解码保存在缓冲器1710中的音频数据(从另一个设备接收的音频数据),以便输出给扬声器1730。 To initialize the audio processing manager 1715, some embodiments of instruction audio processing manager 1715 starts to save encoded audio data (audio data received from another device) in the buffer 1710 by the microphone 1725 captures audio data, and decoded to output to the speaker 1730. 一些实施例的连网管理器1614的初始化包括指令连网管理器1614建立和视频会议中的另一个设备的网络连接。 Some embodiments of the networking manager 1614 includes an instruction to initialize the networking manager 1614 to establish a network connection to another device and a video conference.

[0294] 处理1800随后确定(在1815)是否还剩有要初始化的任何模块。 [0294] process 1800 then determines (at 1815) whether there is left any module to be initialized. 当剩有要初始化的模块时,处理1800返回操作1810,以初始化另一个模块。 When the remaining modules have to be initialized, the process 1800 returns to operation 1810 to initialize another module. 当所有需要的模块都已被初始化时,处理1800产生(在1820)供显示在双照相机移动设备(即,本地显示器)上的合成图像。 When all the required modules have been initialized, the process 1800 is generated (1820) in the composite image for display on a dual camera mobile device (i.e., local display). 这些合成图像可包括示于下面说明的图65中的那些合成图像(即,PIP或其它合成显示),并且可包括来自本地的双照相机移动设备的照相机的图像和来自参与视频会议的另一个设备的照相机的图像的各种组合。 The composite image may include those shown in the composite image will be described in FIG. 65 (i.e., the PIP display or other synthetic), and may include an image of another camera dual camera apparatus from the local mobile device from participating in a video conference the various combinations of a camera image.

[0295] 接下来,处理1800确定(在1825)是否对视频会议做出了改变。 [0295] Next, the process 1800 determines (at 1825) whether the video conference made a change. 一些实施例通过用户与显示在双照相机移动设备上的用户界面的交互,接收对视频会议的改变,而其它实施例通过连网管理器1614,从另一个设备接收对视频会议的改变(即,远程控制)。 Some interactive user interface on the dual camera mobile device user with the embodiments of the display, receiving a change to the video conference, while other embodiments by networking manager 1614, receiving a change to the video conference from another device (i.e., remote control). 在一些实施例中,也可从客户端应用1665,或者视频会议模块1602中的其它模块接收对视频会议设置的改变。 In some embodiments, may also be applied from a client 1665, a video conference module 1602 or other module receiving a change to the video conference setting. 视频会议设置还会由于网络条件的变化而改变。 Video conference setting also due to changes in network conditions change.

[0296] 当发生改变时,处理1800确定(在1830)对视频会议的改变是否是对网络设置的改变。 [0296] When a change occurs, the process 1800 determines (at 1830) whether to change to the video conference is to change the network settings. 在一些实施例中,所述改变或者是网络设置改变或者是图像拍摄设置改变。 In some embodiments, the network setting is changed or changing or setting changed image capturing. 当对视频会议的改变是对网络设置的改变时,处理修改(在1840)网络设置,随后进入操作1845。 When the changes to the video conference is to change the network settings, the process modifications (at 1840) the network settings, and then proceeds to operation 1845. 一些实施例的网络设置改变包括改变对图像编码的比特率,或者向另一个设备传送图像的帧速率。 Some embodiments of the network setting change comprises changing the bit rate for encoding an image, or the image transfer frame rate to another device.

[0297] 当对视频会议的改变不是对网络设置的改变时,处理1800确定所述改变是对图像拍摄设置的改变,从而进入操作1835。 [0297] When the change is not change to the video conference network setting, the process 1800 determines that the change is a change of the image capturing setting so proceeds to operation 1835. 处理1800随后执行(在1835)对图像拍摄设置的改变。 Process 1800 is then performed (at 1835) to change the settings of the image capturing. 在一些实施例中,对图像拍摄设置的改变可包括切换照相机(即,切换双照相机移动设备上的哪部照相机拍摄视频),焦距调整,曝光调整,显示或不显示来自双照相机移动设备的一部或两部照相机的图像,和放大或缩小显示在双照相机移动设备上的图像,以及其它设置改变。 In some embodiments, changes to the image capturing settings may include a switch camera (i.e., the switching of which portion of the camera on the dual camera mobile device shooting video), focus adjustment, exposure adjustment, displaying or not displaying a from the dual camera mobile device two cameras or image portion, is enlarged or reduced and displayed on the dual image camera mobile device, and other settings changed.

[0298] 在操作1845,处理1800确定是否结束视频会议。 [0298] In operation 1845, process 1800 determines whether to end video conferencing. 当处理1800确定不结束视频会议时,处理1800返回操作1820。 When the process 1800 is determined not to end the video conference, the process 1800 returns to operation 1820. 当处理1800确定视频会议将结束时,处理1800结束。 When the process 1800 determines video conference ends, the processing 1800 ends. 当处理1800从客户端应用1665接收到结束视频会议的指令时(即,归因于通过本地双照相机移动设备的用户界面接收的,或者从参与视频会议的另一个设备接收的指令),处理1800 的一些实施例确定结束视频会议。 When the process 1800 from the client application when receiving an instruction to end video conference (i.e., due to the locally received through the user interface of the dual camera mobile device, or received from another device participating in video conferences instructions) 1665, process 1800 Some embodiments determine the end of the video conference.

[0299] 在一些实施例中,当视频会议结束时,视频会议管理器1604执行未示于处理1800 中的各种操作。 [0299] In some embodiments, when the end of the video conference, the video conference manager 1604 performs various operations is not shown in the processing 1800. 一些实施例指令CIPU 1650停止产生图像,指令连网管理器1614中断与视频会议中的另一个设备的网络连接,和指令图像处理管理器1608停止处理和编码图像。 Some embodiments of the instruction to stop generating the image CIPU 1650, the interrupt instruction networking manager 1614 is connected to another network device in the video conference, and instructs the image processing manager 1608 stops processing and encoding the image.

[0300] E.时间噪声降低 [0300] E. temporal noise reduction

[0301] 一些实施例包括处理视频图像,以降低视频中的噪声的特殊的时间噪声降低模块。 [0301] Some embodiments include processing video images, the video special time to reduce the noise in the noise reduction module. 一些实施例的时间噪声降低模块比较视频序列中的后续图像,以识别和从视频中消除有害的噪声。 Some embodiments of the time noise reduction module compares subsequent images of the video sequence to identify and eliminate unwanted noise from the video.

[0302] 图19概念地图解说明一些实施例的这种时间噪声降低(TNR)模块190的软件体系结构。 [0302] Figure 19 conceptually illustrates some such embodiments time software architecture noise reduction (TNR) of the module 190. 一些实施例把TNR模块1900实现成应用的一部分(例如,实现成如图3中所示的媒体交换模块的一部分),而其它实施例把TNR模块1900实现成由其它应用使用的独立应用。 Some embodiments of the TNR module 1900 implemented as part of an application (e.g., implemented as part of the media shown in FIG. 3 to FIG switching module), while other embodiments the module 1900 TNR implemented as stand-alone application used by other applications. 除此以外的其它实施例把TNR模块1900实现成在双照相机移动设备上运行的操作系统的一部分。 Other embodiments other than the TNR module 1900 is implemented as part of running on the dual camera mobile device operating system. 在一些实施例中,TNR模块1900由一组API实现,所述一组API向其它应用提供TNR模块1900的一些或全部功能。 In some embodiments, TNR module 1900 is implemented by a set of API, the API provides a set of some or all functions TNR module 1900 to other applications.

[0303] 如图19中所示,TNR模块1900包括TNR管理器1905、差分模块1910、像素平均模块1915和运动历史模块1920。 [0303] As shown in FIG, 19 includes a module 1900 TNR TNR manager 1905, a differential module 1910, module 1915 pixel averaging module 1920 and a motion history. 尽管图19把三个模块1910、1915和1920表示成独立的模块,不过一些实施例在单个模块中实现下面说明的这些模块的功能。 Although FIG. 19 1910, 1915 and 1920 represent three modules as separate modules, but Some embodiments implement functions of these modules will be described in a single module. 一些实施例的TNR模块1900接收输入图像、基准图像和运动历史作为输入。 Some embodiments of the TNR module 1900 receiving an input image, as the reference image and the input motion history. 在一些实施例中,输入图像是目前正在处理的图像,而基准图像是视频序列中输入图像与之比较的在先图像。 In some embodiments, the input image is an image currently being processed, the input image is a reference image in a video sequence to compare with the previous image. TNR模块1900 输出输出图像(一种形式的噪声降低的输入图像)和输出运动历史。 TNR module 1900 outputs the output image (a form of noise reduction of an input image) and the output motion history.

[0304] 一些实施例的TNR管理器1905指引数据在TNR模块1900内的流动。 [0304] Some TNR manager 1905 guidance data flows in the embodiment of module 1900 TNR. 如图所示, TNR管理器1905接收输入图像、基准图像和运动历史。 As shown, the TNR manager 1905 receives the input image, the reference image and the motion history. TNR管理器1905还输出输出图像和输出运动历史。 TNR manager 1905 also outputs the output video and motion history. TNR管理器1905把输入图像和基准图像发送给差分模块1910,从差分模块1910接收差分图像。 TNR manager 1905 of the input image and the reference image sent to the differential module 1910, module 1910 receives the difference from the difference image.

[0305] 在一些实施例中,差分模块1910处理从TNR管理器1905接收的数据,并把处理后的数据发送给TNR管理器1905。 [0305] In some embodiments, the differential module 1910 process the data received from the 1905 TNR manager, and sends the processed data to the manager TNR 1905. 如图所示,差分模块1910从TNR管理器1905接收输入图像和基准图像。 As shown, the difference module 1910 TNR manager 1905 receives from the input image and the reference image. 一些实施例的差分模块1910通过从另一个图像的像素值中减去一个图像的像素值,产生差分图像。 Some embodiments of the differential module 1910 by subtracting the pixel values ​​of one image from another image pixel value, generating a differential image. 差分图像被发送给TNR管理器1905。 Difference image is sent to the manager TNR 1905. 一些实施例的差分图像指示两个图像之间的差异,以便识别与前一图像相比,输入图像的已发生变化的部分,和输入图像的保持不变的部分。 A differential image indicating a difference between the two images according to some embodiments, in order to identify compared with the previous image, some of these changes has occurred, the input image and the input image remains constant portion.

[0306] TNR管理器1905还把输入图像和基准图像发送给像素平均模块1915。 [0306] TNR manager 1905 also input image and the reference image to the pixel averaging module 1915. 如图所示, 一些实施例还把运动历史发送给像素平均模块1915。 As shown, some embodiments also the motion history to a pixel averaging module 1915. 不过,其它实施例可以只发送输入图像和基准图像,而不发送运动历史。 However, other embodiments may transmit only the input image and the reference image, without transmitting the motion history. 在任何一个实施例中,TNR管理器1905都从像素平均模块1915接收处理后的图像。 In any embodiment, TNR manager 1905 are image pixel averaging module 1915 from the reception processing.

[0307] -些实施例的像素平均模块1915利用运动历史来确定是否关于图像中的特定位置,从输入图像和基准图像中获得像素的平均值。 [0307] - some of the pixel averaging module 1915 of the embodiment using the motion history to determine whether a specific position on an image, the pixel mean value obtained from the input image and the reference image. 在一些实施例中,运动历史包括输入图像中的每个像素的概率值。 In some embodiments, the motion history image comprising probability values ​​for each input pixel. 特定的概率值代表相对于基准图像中的对应像素,输入图像中的对应像素已发生变化(即,动态像素)的概率。 Particular probability value representative of the reference image with respect to the corresponding pixels, the corresponding pixel in the input image has changed (i.e., dynamic pixel) occurrence probability. 例如,如果输入图像中的特定像素的概率值为20,那么指示相对于基准图像中的对应像素,输入图像中的特定像素已变化的概率为20%。 For example, if the probability of a particular input pixel in the image is 20, it indicates a probability with respect to a particular pixel corresponding to a pixel in the reference image, the input image has changed to 20%. 作为另一个例子,如果输入图像中的特定像素的概率值为0,那么指示相对于基准图像中的对应像素,输入图像中的特定像素未变化(即,静态像素)。 As another example, if the probability of a particular pixel in the image input is 0, that indicates that a particular pixel corresponding to the pixel with respect to the reference image, the input image is not changed (i.e., static pixels).

[0308] 不同的实施例不同地保存输入图像的概率值。 [0308] Different embodiments variously stored probability values ​​of the input image. 一些实施例可把输入图像的每个像素的概率值保存在一个数据阵列中。 Some embodiments may the probability of each pixel of the input image value is stored in a data array. 其它实施例可把概率值保存在维数与视频的图像的分辨率相同的矩阵中(例如,一组阵列中)。 Other embodiments may be the probability values ​​stored in the same image dimension and resolution video matrix (e.g., a group of the array). 例如,如果视频的图像的分辨率为320X240,那么矩阵也为320X240。 For example, if the video resolution of the image is 320X240, then the matrix is ​​also 320X240.

[0309] 当除了输入图像和基准图像之外,像素平均模块1915还从TNR管理器1905接收到运动历史时,像素平均模块1915读取输入图像中的每个像素的概率值。 [0309] In addition, when the input image and the reference image, pixel averaging module 1915 also receives from the TNR manager 1905 to the motion history, the pixel averaging module 1915 reads the input probability value of each pixel in the image. 如果输入图像中的特定像素的概率值低于规定的阈值(例如,5%,20%),那么像素平均模块1915根据在该特定像素,大概不会存在运动,从而在该像素的图像之间的差异可能归因于噪声的假定,求该特定像素值和基准图像中的对应像素值的平均值。 If the probability value of a specific pixel of the input image is below a predetermined threshold value (e.g., 5%, 20%), the pixel averaging module 1915 in accordance with the particular pixel, probably not motion is present between the image so that the pixel the difference may be attributed to the assumed noise, averaging the pixel values ​​of the specific pixel value and the corresponding reference image.

[0310] 如果输入图像中的特定像素的概率值不低于规定的阈值,那么像素平均模块1915 不修改输入图像的特定像素(即,在该像素的像素值保持和输入图像中的一样)。 [0310] If the probability of a particular pixel in the input image value is not less than a predetermined threshold value, the pixel averaging module 1915 does not modify the specific pixel of the input image (i.e., holding, and as the input image pixel value of the pixel). 这是因为运动很可能位于该特定像素,从而图像之间的差异很可能不是噪声的结果。 This is probably because the movement of the particular pixel is located, so that the result is the difference between the images may not noise. 在一些实施例中,当运动历史不被发送给像素平均模块1915时,像素平均模块1915求输入图像中的每个像素与基准图像中的对应像素的平均值。 In some embodiments, when the motion history is not transmitted to the pixel averaging module is 1915, the pixel averaging module 1915 averaging corresponding pixels of each pixel of the reference image in the input image. 由像素平均模块1915输出并被发送给TNR管理器1905的处理后的图像包括未被平均的任何像素的输入图像像素值,和被像素平均模块1915平均的任何像素的平均像素值。 And the image transmitted by the pixel averaging module 1915 outputs to the TNR manager process 1905 includes any non-averaged pixel values ​​of pixels of the input image, and the average of the average pixel value of any pixel averaging module 1915.

[0311] 在一些实施例中,运动历史模块1920处理从TNR管理器1905接收的数据,并把结果数据回送给TNR管理器1905。 [0311] In some embodiments, the motion history module 1920 process the data received from the 1905 TNR manager, and the results back to the data manager TNR 1905. 一些实施例的运动历史模块1920从TNR管理器1905接收输入图像和运动历史。 Some embodiments of the motion history module 1920 TNR manager 1905 receives an input image and a motion history. 一些实施例把该数据输入Bayes估计器,以便产生可在关于下一个输入图像的像素平均中使用的新的运动历史(即,一组概率值)。 Some embodiments of the data input to the Bayes estimator, to produce a new motion history (i.e., a set of probability values) may be used in about the next pixel averaging input image. 其它实施例使用其它估计器来产生新的运动历史。 Other embodiments use other estimator to generate a new motion history.

[0312] 现在参考图20,说明TNR模块1900的操作。 [0312] Referring now to Figure 20, an operation module 1900 TNR. 图20概念地图解说明一些实施例的降低视频的图像的时间噪声的处理2000。 Figure 20 conceptually illustrates a time down to some embodiments of the video image 2000 processed noise. 从TNR管理器1905接收(在2005)输入图像、基准图像和运动历史开始处理2000。 1905 received from the TNR manager (in 2005) of the input image, the reference image and motion history begins processing 2000. 输入图像是目前为了降噪而处理的图像。 Current input image is an image for noise reduction and treatment. 在一些实施例中,基准图像是从CIPU接收的视频的图像序列中的前一个图像。 In some embodiments, the reference image is a preceding image from the received sequence of video CIPU. 不过,在其它实施例中,基准图像是由前一个输入图像的处理产生的输出图像(即,TNR模块1900的输出)。 However, in other embodiments, the reference image is the output image generated by the processing of the previous input image (i.e., an output of module 1900 TNR). 运动历史是由前一个输入图像的处理产生的输出运动历史。 Motion history output is generated by the motion history a pre-processed input image.

[0313] 当输入图像是视频的第一个图像时,一些实施例的TNR模块1900并不处理第一个图像(即,不对第一个图像应用TNR)。 [0313] When the input image is the first image of the video, the TNR module 1900 of some embodiments is not the first image processing (i.e., not the first image application TNR). 换句话说,TNR管理器1905接收第一个图像,并且只是输出第一个图像。 In other words, the TNR manager 1905 receives the first image, and only the output of the first image. 在其它实施例中,当输入图像是视频的第一个图像时,第一个图像被用作输入图像和基准图像,TNR模块1900如下所述处理该图像。 In other embodiments, when the input image is the first image of the video when the first image is used as the input image and the reference image, the TNR module 1900 processes the image as described below. 此外,当输入图像是视频的第一图像时,运动历史为空(即,空值,全为零,等等),TNR管理器1905只是输出空的运动历史,作为输出运动历史。 Further, when the input image is the first image of the video, the motion history is empty (i.e., the null value, all zero, etc.), the TNR manager 1905 only outputs an empty motion history, as an output motion history.

[0314] TNR管理器1905随后确定(在2010)输入图像是否是静态的。 [0314] TNR manager 1905 then determines (at 2010) whether the input image is static. 为了做出该确定, 一些实施例把输入图像和基准图像发送给差分模块1910,从差分模块1910接收差分图像。 To make this determination, some embodiments sends the input image and the reference image to the difference module 1910, 1910 receive the differential image from the differential module. 当两个图像之间的差异低于规定的阈值(例如,5%差异,10%差异等)时,一些实施例把输入图像分类为静态图像。 When the difference is below a predetermined threshold value (e.g., 5% difference 10% differences) between the two images, some embodiments of the input image is classified as a static image.

[0315] 当输入图像是静态图像时,TNR管理器1905把输入图像和基准图像发送给像素平均模块1915,以计算(在2015)输入图像的像素与基准图像的像素的平均值,以便从静态图像中减少任何噪声。 [0315] When the input image is a still image, the TNR manager 1905 of the input image and the reference image to the pixel averaging module 1915 to calculate the average pixel of the reference image (2015) of the input image, from the static to reduce any noise in the image. 处理随后进入下面说明的操作2040。 The process then proceeds to operation 2040 described below.

[0316] 当输入图像不是静态图像时,TNR管理器把输入图像、基准图像和运动历史发送给像素平均模块1915以便处理。 [0316] When the input image is not a still image, the TNR manager input image, the reference image and a moving history to a pixel averaging module 1915 for processing. 像素平均模块1915选择(在2020)输入图像中的一个像素。 Pixel averaging module 1915 selects one pixel (in 2020) the input image. 通过利用运动历史,像素平均模块1915确定(在2025)像素的运动概率是否低于特定阈值,如上所述。 By using the motion history, the pixel averaging module 1915 determines (2025) whether the probability of the pixel motion below a certain threshold, as described above.

[0317] 如果所选像素的概率低于特定阈值,那么像素平均模块1915计算(在2030)输入图像的所述像素与基准图像中的对应像素的平均值。 [0317] If the probability of the selected pixel is below a certain threshold, then the pixel averaging module 1915 calculates an average value of the corresponding pixel (at 2030) the reference pixel in the input image. 否则,该像素不被求平均值,并且输出图像将与在所述特定像素的输入图像相同。 Otherwise, the pixels are not averaged, and outputs the same as the input image in the specific image pixel. 像素平均模块1915随后确定(在2035)在输入图像中是否剩有任何未选择的像素。 Pixel averaging module 1915 then determines (at 2035) in the input image if there are any remaining unselected pixels. 如果存在还未被处理的任何像素,那么处理返回操作2020以选择下一个像素。 If any pixel not yet been processed exists, the process returns to operation 2020 to select the next pixel. 像素平均模块1915执行操作2020-2030,直到评估了所有像素为止。 Pixel averaging module 1915 to perform operations 2020-2030 until evaluated all of the pixels so far.

[0318] 处理随后更新(在2040)运动历史。 [0318] The process then updated (in 2040) sports history. 如图19中所示和如上所述,运动历史模块1920根据输入图像更新运动历史。 As shown in FIG. 19 and described above, the motion history of the input image update module 1920 according to the motion history. 新的运动历史连同来自像素平均模块的处理后图像一起由TNR管理器输出。 New motion history output by the TNR manager together with the post-processing image from the pixel averaging module.

[0319] F.图像处理管理器&编码器 [0319] F. The image processing manager & Encoder

[0320] 除了由CIPU和/或CIPU驱动器执行的时间噪声降低和图像处理操作之外,一些实施例在视频会议模块925的图像处理层930执行各种图像处理操作。 [0320] and the noise reduction image processing operation performed by the addition CIPU time and / or a drive other than CIPU some embodiments videoconferencing module 930 performs various image processing operations of the image processing layer 925 in the embodiment. 这些图像处理操作可包括曝光调整、焦距调整、透视校正、动态范围的调整和图像缩放等等。 These image processing operations may include exposure adjustment, focus adjustment, perspective correction, and dynamic range adjustment image scaling like.

[0321] 图21概念地图解说明执行这样的图像处理操作的处理2100。 [0321] FIG. 21 illustrates conceptually the execution of such image processing operations of process 2100. 在一些实施例中,处理2100的一些或全部操作由图16的图像处理管理器1608和编码器驱动器1635的组合执行。 In some embodiments, some or all of process 2100 performed by a combination of image processing operations manager 1608 and the encoder driver 1635 of FIG. 16. 在一些这样的实施例中,图像处理管理器1608执行基于像素的处理(例如,缩放、动态范围调整、透视校正等)。 In some such embodiments, the image processing manager 1608 performs pixel processing (e.g., scaling, dynamic range adjustment, perspective correction, etc.) based. 一些实施例在视频会议期间,对将被传送给参与视频会议的另一个设备的图像执行处理2100。 In some embodiments, during a video conference, performed on the image to be transmitted to another device involved in the video conference process 2100.

[0322] 现在参考图16,说明处理2100。 [0322] Referring now to Figure 16, the processing 2100. 通过从缓冲器1606取回(在2105)图像,开始该处理。 By retrieving from the buffer 1606 (2105) image, the process starts. 在一些实施例中,取回的图像是视频的图像(即,图像序列中的一个图像)。 In some embodiments, the retrieved image is a video image (i.e., in a sequence of images). 该视频可由在其上执行处理2100的设备的照相机拍摄。 The video captured by the camera in which the execution of the processing device 2100.

[0323] 随后,处理2100对取回的图像执行(在2110)曝光调整。 [0323] Subsequently, the processing performed on the image 2100 pairs retrieved (2110) to adjust the exposure. 一些实施例通过显示在双照相机移动设备上的用户界面,进行曝光调整。 Some embodiments of the user interface by displaying on the dual camera mobile device, exposure adjustment. 图22图解说明这种实施例的示例曝光调整操作。 FIG 22 illustrates an exemplary embodiment of such an exposure adjustment operation.

[0324] 图22参考设备2200的Π 2205的三个阶段2210、2215和2220,说明曝光调整操作。 [0324] FIG. 22 with reference to three stages apparatus Π 2205 2200 2210,2215 and 2220, the exposure adjustment operation described. 第一阶段2210图解说明包括显示区2225和显示区1155的Π 2205。 The first stage comprises a Π 2205 2210 illustrates a display area 2225, and a display area 1155. 如图所示,显示区2225显示太阳和脸及身体都暗黑的男士的图像2230。 As shown, the display area 2225 displays an image 2230 and the sun face and body are dark men. 暗黑的脸和身体显示该男士未得到恰当曝光。 Dark face and body show the man has not been properly exposed. 图像2230可以是由设备2200的照相机拍摄的视频图像。 2230 may be a video image captured by the camera device 2200. 如图所示,显示区1155包括用于结束视频会议的可选Π项目2250。 As shown, the end of the display area 1155 includes a selectable item Π video conference 2250. 在一些实施例中,显示区1155的布局与上面说明的图12的显不区1155的布局相同。 In some embodiments, region 1155 shows the same layout and the layout described above is not significant region 1155 in FIG. 12.

[0325] 第二阶段2215图解说明设备2200的用户通过选择显示区2225的某一区域,发起曝光调整操作。 [0325] The second stage 2215 illustrates the device 2200 by a user selecting a region of the display area 2225, and initiate an exposure adjustment operation. 在这个例子中,通过把手指2235放在显示区2225内的任何地方实现选择。 In this example, choose achieved by 2235 on a finger anywhere within the display area 2225. 在一些实施例中,用户从可能的图像设置调整的菜单中选择曝光调整。 In some embodiments, the user may select from the exposure adjustment to adjust the image settings menu.

[0326] 第三阶段2220显示在完成曝光调整操作之后,所述男士的图像2240。 [0326] The third stage 2220 is displayed after completion of the exposure adjustment operation, an image of the 2240 men. 如图所示, 图像2240类似于图像2230,不过图像2240中的男士被恰当曝光。 As shown, an image 2240 is similar to 2230, but in 2240 men image is properly exposed. 在一些实施例中,恰当曝光的图像是在不恰当曝光的图像之后拍摄的图像。 In some embodiments, the image is properly exposed image after exposure is not appropriate in the captured image. 在第二阶段2215中发起的曝光调整操作调整由设备2200的照相机拍摄的后续图像的曝光。 Initiated in the second stage 2215 in the subsequent exposure adjustment operation to adjust the exposure of the image captured by the camera device 2200.

[0327] 返回图21,处理2100随后对图像执行(在2115)焦距调整。 [0327] Returning to Figure 21, and then focus adjustment processing 2100 performed on the image (at 2115). 一些实施例通过显示在双照相机移动设备上的用户界面,进行焦距调整。 Some embodiments of the user interface by displaying on the dual camera mobile device performs focus adjustment. 图23概念地图解说明这种焦距调整操作的例子。 Figure 23 conceptually illustrates an example of such a focus adjustment operation.

[0328] 图23参考设备2300的Π 2305的三个不同阶段2310、2315和2320,图解说明焦距调整操作。 [0328] 2300 in FIG. 23 with reference to three different stages of the apparatus of Π 2305 2310,2315 and 2320, illustrates the focus adjustment operation. 第一阶段2310图解说明包括显示区2325和显示区1155的Π 2305。 The first stage comprises a Π 2305 2310 illustrates a display area 2325, and a display area 1155. 显示区2325呈现由设备2300的照相机拍摄的男士的模糊图像2330。 Display area 2325 showing the blurred image 2330 captured by the camera device 2300 men. 这种模糊说明该男士的图像2330未对准焦点。 This description of the man fuzzy image 2330 is not in focus. 即,当照相机拍摄该男士的图像2330时,照相机的透镜未被聚焦于该男士。 That is, when the camera captured image of the 2330 men, the camera lens is not focused on the man. 另外,图像2330可以是由设备2300的照相机拍摄的视频图像。 Further, the image 2330 may be a video image captured by the camera device 2300. 如图所示,显示区1155 包括用于结束视频会议的可选Π项目2350。 As shown, the end of the display area 1155 includes a selectable item Π video conference 2350. 在一些实施例中,显示区1155的布局和上面说明的图12的显不区1155的显不区1155的布局相同。 In some embodiments, region 1155 shows the same layout and the layout described above is not significant not significant region 12 of the region 1155 1155.

[0329] 第二阶段2315图解说明设备2300的用户通过选择显示区2325的一个区域,发起焦距调整操作。 [0329] The second stage 2315 illustrates the device 2300 by a user selecting a region of the display area 2325, initiates a focus adjustment operation. 在这个例子中,通过把手指2335放在显示区2225内的任何地方实现选择。 In this example, choose achieved by 2335 on a finger anywhere within the display area 2225. 在一些实施例中,用户从可能的图像设置调整的菜单中选择焦距调整。 In some embodiments, the user may select an image from a focus adjustment setting adjustment menu.

[0330] 第三阶段2320显示在完成焦距调整操作之后,所述男士的图像2340。 [0330] The third stage 2320 is displayed in operation after completion of the focus adjustment, the image 2340 men. 如图所示, 图像2340和图像2330相同,不过图像2340中的男士显得更清晰。 As shown, the same image 2340 and the image 2330, the image 2340 in men but appear clearer. 这说明照相机的透镜被恰当地聚集于该男士。 This shows that the camera lens is properly gathered to the man. 在一些实施例中,恰当聚焦的图像是在不恰当焦距的图像之后拍摄的图像。 In some embodiments, the appropriate focused image is an image captured after the image focal inappropriate. 在第二阶段2315中发起的焦距调整操作调整由设备2300的照相机拍摄的后续图像的焦距。 The focal length of the image in a second subsequent stage 2315 initiation of a focus adjustment operation of the adjustment device 2300 captured by a camera.

[0331] 返回图21,处理2100对图像进行(在2120)图像缩放。 [0331] Returning to Figure 21, the image process 2100 (2120) image scaling. 一些实施例对图像进行图像缩放,以减少用于对图像编码的比特数(即,降低比特率)。 Some embodiments of the picture image scaling, to reduce the number of bits for encoding an image (i.e., lower bit rate). 在一些实施例中,处理2100 进行图像缩放,如下面参考图26所述。 In some embodiments, image scaling process 2100, as described below with reference to Figure 26.

[0332] 处理2100随后对图像进行(在2125)透视校正。 [0332] Then the image processing 2100 (in 2125) the perspective correction. 在一些实施例中,处理2100进行透视校正,如下在图24中所述。 In some embodiments, the process 2100 perspective correction, as in the FIG. 24. 这样的透视校正涉及利用由识别双照相机移动设备的定向和移动的一个或多个加速计和/或陀螺仪传感器获得的数据。 Such a perspective correction data relates to a mobile device identifying dual camera movement and orientation of one or more accelerometers and / or a gyro sensor is obtained. 所述数据随后被用于修改图像,以校正不正确的透视。 The image data is then used to modify, to correct incorrect perspective.

[0333] 在对图像进行透视校正之后,处理2100调整(在2130)图像的动态范围。 [0333] After the image perspective correction, adjustment processing 2100 (in 2130) the dynamic range of the image. 在一些实施例中,图像的动态范围是图像中的每个像素能够具有可能值的范围。 In some embodiments, dynamic range image is an image in each pixel can have a range of possible values. 例如,具有0-255 的动态范围的图像可被调整到0-128的范围,或者任何其它数值范围。 For example, an image having a dynamic range of 0-255 may be adjusted to a range of 0-128, or any other range of values. 调整图像的动态范围可减少将用于对图像编码的比特的数量(即,降低比特率),从而使图像平滑。 Dynamic range adjustment image may reduce the number of bits used for encoding an image (i.e., lower bit rate), thereby smoothing the image.

[0334] 调整图像的动态范围还可用于各种其它目的。 [0334] adjust the dynamic range of the image it can be used for various other purposes. 一个目的是减少图像噪声(例如, 图像是由有噪声的照相机传感器拍摄的)。 An object is to reduce the image noise (e.g., an image captured by the camera sensor has a noise). 为了减少噪声,可以调整图像的动态范围,使得暗电平被重新定义,以包括浅黑(即,粉碎的黑色(crush black))。 To reduce noise, the dynamic range of the image can be adjusted, so that the black level is redefined to include a light black (i.e., pulverized black (crush black)). 以此方式,图像的噪声被减少。 Noise this way, the image is reduced. 动态范围调整的另一个目的是调整一种或多种颜色或者颜色范围,以便增强图像。 Another object of the dynamic range adjustment is to adjust one or more colors or color range, in order to enhance the image. 例如,一些实施例可假定正面照相机拍摄的图像是人脸的图像。 For example, some embodiments may be assumed that the image captured by the camera is an image of the front face. 因此,可以调整该图像的动态范围,以增强红色和粉红色,使人的面颊显得红润/更红润。 Therefore, you can adjust the dynamic range of the image to enhance red and pink, people seem rosy cheeks / more rosy. 动态范围调整操作还可用于其它目的。 Dynamic range adjustment operation may also be used for other purposes.

[0335] 最后,处理2100确定(在2135)用于对图像编码的一个或多个速率控制器参数。 [0335] Finally, the process 2100 determines (at 2135) used for one or more image encoding rate controller parameters. 在一些实施例中,这样的速率控制器参数可包括量化参数和帧类型(例如,预测帧、双向帧、帧内编码帧)。 In some embodiments, such parameters may include a rate controller quantization parameter and a frame type (e.g., a predictive frame, bidirectional frame, intra-coded frame). 该处理随后结束。 The process then ends.

[0336] 尽管处理2100的各个操作被图解表示成按照特定的顺序执行,不过,普通技术人员会认识到这些操作中的许多操作(曝光调整、焦距调整、透视校正等)可按照任意顺序执行,并且彼此无关。 [0336] Although processing of each operation 2100 is illustrated as a particular order of execution in accordance with, but of ordinary skill in the art will recognize that many of the operations (exposure adjustment, focus adjustment, perspective correction, etc.) of these operations may be performed in any order, and independent of each other. 即,一些实施例的处理可在曝光调整之前进行焦距调整,或者可对图21 中图解说明的处理进行类似的修改。 That is, some embodiments may be treated prior to exposure adjustment to adjust the focus, or the process may be illustrated in FIG. 21 to be described is similar modification.

[0337] 1.透视校正 [0337] 1. Perspective Correction

[0338] 如上所述,一些实施例在显示或传送图像之前,对图像进行透视校正。 [0338] As described above, in some embodiments, before displaying or transmitting the image, the image perspective correction. 在一些情况下,双照相机移动设备上的一部或多部照相机不会完全面向其被摄物体,并且在未校正的图像中,被摄物体会显得失真。 In some cases, a portion of the camera or on the dual camera mobile device which is not completely oriented subject, and uncorrected image, the subject will appear distorted. 可以使用透视校正来处理图像,使得图像将忠实地反映图像中的被摄对象在人看来的样子。 Perspective correction can be used to process the image, so that the image will be faithfully reflected in the eyes of the subject like objects in the image.

[0339] 图24概述地图解说明由一些实施例的图像处理管理器,比如图16中图解说明的图像处理管理器执行的透视校正处理2400。 [0339] FIG 24 illustrates an overview of the embodiment of the image processing by the manager of some embodiments, such as illustrated in perspective view in FIG. 16 described image processing manager 2400 performs a correction process. 一些实施例的处理2400由图9中所示的图像处理层930 (它可包含图像处理管理器1608)执行。 Some embodiments of process 2400 performed by the image processing layer 930 shown in FIG. 9 (which may include image processing manager 1608). 一些实施例在处理2100的操作2125 执行处理2400,以便在显示或传送最近拍摄的视频图像之前,校正所述图像的透视。 In some embodiments of the processing operation 2100 2125 2400 performs processing for displaying a video image prior to transmission or taken recently, perspective correction of the image.

[0340] 处理2400从自加速计传感器接收(在2405)数据开始,在一些实施例中,所述加速计传感器是双照相机移动设备的一部分。 [0340] Processing 2400 receives (at 2405) from the accelerometer sensor data from the start, in some embodiments, the accelerometer sensor is part of a double camera mobile device. 一些实施例的加速计传感器测量沿着一个或多个轴的设备的速度的变化率(即,设备的加速度)。 Some embodiments of the accelerometer sensor for measuring rate of change of speed along one or more axes of the devices (i.e., acceleration of the device). 该处理还从陀螺仪传感器接收(在2410)数据,在一些实施例中,所述陀螺仪传感器也可以是双照相机移动设备的一部分。 The process also receives from the gyro sensor (2410) data, in some embodiments, the gyro sensor may be part of the dual camera mobile device. 一些实施例的陀螺仪和加速计传感器可被单独地或者组合地用于识别双照相机移动设备的定向。 Some gyroscope and accelerometer sensor of the embodiment may be used alone or in combination for identifying dual camera mobile device orientation.

[0341] 随后,处理2400根据从加速计和陀螺仪传感器获得的数据,确定(在2415)要进行的透视校正量。 [0341] Then, the process 2400 in accordance with data obtained from an accelerometer and a gyro sensor, determines the amount of perspective correction to be performed (at 2415). 通常,当所述定向离轴更远时,为了产生最佳的图像,需要更多的透视校正。 Typically, when the further off-axis orientation, in order to produce the best image, the need for more perspective correction. 一些实施例根据设备的定向,计算代表透视校正量的扭曲参数(warp parameter)。 Some embodiments according to the orientation of the device, calculating a representative perspective distortion correction amount parameters (warp parameter).

[0342] 在确定要进行的透视校正量之后,处理2400接收(在2420)由双照相机移动设备的照相机拍摄的图像。 [0342] After determining the amount of perspective correction to be performed, an image captured by the camera of the dual camera mobile processing device 2400 receives (at 2420). 可对由该照相机拍摄的视频序列中的每个图像进行所述处理。 The process may be performed for each video sequence captured by the camera in. 一些实施例可对分别来自双照相机移动设备上的两部照相机的每个图像进行单独的计算。 Some embodiments may be calculated for each individual image respectively from the two cameras on the dual camera mobile device.

[0343] 所述处理随后根据确定的透视校正量,修改(在2425)图像。 [0343] The process then determines a correction amount according to a perspective view, modify (at 2425) an image. 除了扭曲参数或者透视校正量的其它表示之外,一些实施例还使用基线图像或者其它信息(例如,用户输入的应关于其进行校正的点)。 In addition to the parameter or perspective distortion correction amount other than represented, Some embodiments also use a baseline image or other information (e.g., user input corresponding points on its correcting). 在修改图像之后,处理2400结束。 After modifying the image, process 2400 ends.

[0344] 图25概念地图解说明一些实施例的示例图像处理操作。 [0344] Figure 25 conceptually illustrates an example of the image processing operations of some embodiments. 图25图解说明由不使用透视校正的第一图像处理模块2520执行的第一图像处理操作2505,和由使用透视校正的第二图像处理模块2565执行的第二图像处理操作2550。 FIG 25 illustrates a first image processing operation without using a 2505 perspective correction processing module 2520 performs a first image and a second image processing by the second image using the perspective correction operations performed by processing module 2565 2550.

[0345] 如图所示,对由成一定角度向下观看块体的空间透视产生的块体2515的第一图像2510进行第一图像处理操作2505。 [0345] As shown, for viewing by a downwardly angled space to generate a perspective view of the block of the first image block is 25,102,515 2505 a first image processing operation thereof. 与块体2515的底部相比,块体的顶部离所述透视更近。 Compared with the bottom block body 2515, a top perspective view of the block from the closer. 因而,块体2515显得朝着拍摄第一图像2510的照相机倾斜。 Thus, block 2515 appear inclined toward the camera capturing the first image 2510. 图25还表示了在由第一图像处理模块2520处理之后的处理后的第一图像2525。 FIG 25 also shows a first image 2525 processed after processing by the first image processing module 2520. 如图所示,处理后的第一图像2525 中的块体2515出现相同的后处理,因为第一图像处理模块2520未进行任何透视校正。 As shown, a first block image 2525 processed in the same post-treatment appeared 2515, no perspective correction as a first image processing module 2520 is not performed.

[0346] 对块体2560的第二图像2555进行第二图像处理操作2550。 [0346] The second body image 2555 2560 2550 second image processing operation. 块体2560和第一图像2510中的块体2515相同。 The same block 2560 and block 2510 of the first 2515 image. 图25还表示在由第二图像处理模块2565的透视校正器2570 进行第二图像2555的处理之后的处理后的第二图像2575。 FIG 25 also shows a second image 2575 processed after processing the second image 2555 by a second image processing module 2565 of the perspective corrector 2570. 透视校正器2570可使用处理2400,以便校正第二图像2555的透视。 Perspective correction using the processing device 2570 may be 2400, 2555 so as to correct the second image perspective. 根据来自加速计和陀螺仪的指出拍摄第二图像2555 的照相机成向下的角度倾斜的数据(可能还根据其它数据),透视校正器2575能够校正第二图像,使得在处理后的第二图像2575中,块体看来好像是笔直地观看的。 The photographing noted from the accelerometer and gyroscope 2555 a camera image into a second downwardly inclined angle data (possibly from other data), the perspective corrector capable of correcting the second image 2575, such that the second image after processing 2575, the bulk seems like a straight viewing.

[0347] 2.缩放和比特流处理 [0347] 2. The scaling process and a bit stream

[0348] 图像缩放和比特流处理属于在上面参考图21说明的由一些实施例的图像处理层930执行的功能之列。 [0348] Scaling and columns belonging to a bit stream processing functions performed by some of the image processing layer of Example 93 021 explained above with reference to the FIG. 图像缩放(在操作2130进行)涉及在一些实施例中比例放大或比例缩小图像(即,修改用于表示该图像的像素的数目)。 Image scaling (in operation 2130) relates to a scaled up or scaled down in some embodiments, the image (i.e., modify the number of pixels used to represent the image). 在一些实施例中,比特流处理涉及把指示缩放后图像的大小的数据插入比特流中。 In some embodiments, the bit stream processing relates to the size of the image after scaling data indicating the inserted into the bitstream. 在一些实施例中,这种缩放和比特流处理由编码器驱动器(例如,驱动器1635)执行。 In some embodiments, the scaling process executed by a bit-stream, and the encoder driver (e.g., driver 1635).

[0349] 图26概念地图解说明一些实施例的这种编码器驱动器2600的软件体系结构,并且显示由编码器驱动器2600对示例图像2605进行的示例缩放和比特流处理操作。 [0349] Figure 26 conceptually illustrates some of the software architecture of this embodiment of the encoder driver 2600, and display performed by the example encoder 2600 of exemplary drive image scaling 2605 and a bit stream processing operations. 在一些实施例中,图像2605是由双照相机移动设备的照相机拍摄的、用于传送给视频会议中的另一个设备的视频的图像。 In some embodiments, an image 2605 is captured by the camera of the dual camera mobile device, to another image of the video transmitting apparatus for a video conference. 参见图16,在一些实施例中,视频图像已通过CIPU驱动器1630和照相机源模块1622从CIPU 1650传播到缓冲器1606,图像处理管理器1608从缓冲器1606 取回所述视频图像。 Referring to Figure 16, in some embodiments, the video image has been propagated through CIPU driver 1630 and the camera 1650 CIPU source module 1622 to the buffer 1606 from the image processing manager 1608 retrieves the image from the video buffer 1606. 在图像处理管理器1608中经过图像处理(例如,焦距调整、曝光调整、 透视校正)之后,图像通过缓冲器1610和视频压缩模块1624被发送给编码器驱动器1635。 In the image processing manager 1608 after image processing (e.g., focus adjustment, exposure adjustment, perspective correction), image and video compression 1610 through the buffer module 1624 is transmitted to the encoder driver 1635.

[0350] 如图所示,编码器驱动器2600包括处理层2610和速率控制器2645。 [0350] As shown, the encoder 2600 includes a processing driver layer 2610 and a rate controller 2645. 在下面说明的图30中图解说明了一些实施例的速率控制器的例子。 FIG 30 described below are some examples illustrating embodiments of a rate controller. 处理层2610包括图像缩放器2615 和比特流管理器2625。 Treatment layer 2610 includes an image scaler 2615 and the bit stream manager 2625. 在一些实施例中,这些模块在图像被编码之前和之后对图像进行各种操作。 In some embodiments, these modules before and after the image is encoded perform various operations on the image. 尽管在本例中,图像缩放器被表示成编码器驱动器2600的处理层2610的一部分, 不过一些实施例把图像缩放器实现成图像处理管理器1608而不是编码器驱动器2600的一部分(即,在把图像和大小数据发送给编码器驱动器之前进行图像缩放)。 Rather than part of the encoder driver 2600. Although in the present embodiment, the image scaling is represented a part of to encode driver treatment layer 2600 of 2610, although some embodiments to implement image scaler to the image processing manager 1608 (i.e., scaling the image size before sending the image data to the encoder and drives).

[0351] 如图所示,在图像通过速率控制器2645被发送给编码器2650之前,图像缩放器2615缩放图像。 [0351] As shown, before the image is sent by the rate controller 2645 to the encoder 2650, an image scaler 2615 scales the image. 图像2605通过缩放器2615被发送并被比例缩小成图像2630。 26052615 image is transmitted to the image 2630 scaled down and through the sealer. 除了比例缩小图像之外,一些实施例还能够比例放大图像。 In addition to scaling down the image outside, some embodiments can also be scaled image.

[0352] 如图26中所示,一些实施例比例缩小输入的图像(例如,图像2605),随后把比例缩小的图像(例如,2630)重叠在空间冗余图像(例如,图像2635)上,所述空间冗余图像的大小(就像素来说)与输入图像相同(即,图像2605的像素的行数和列数与空间冗余图像2635的像素的行数和列数相同)。 [0352] As shown, some embodiments of the scaled down image (e.g., image 2605) 26 input, followed by the scaled-down image (e.g., 2630) overlap spatially redundant image (e.g., image 2635), the size of the spatial redundancy of the image (in pixels for) the same input image (i.e., the number of rows of pixels of same number of rows and columns of the image 2605 and the spatial redundancy of the image pixels 2635 and the number of columns). 一些实施例把比例缩小的图像2630重叠在空间冗余图像的左上角(如图所示,以产生合成图像2640),而其它实施例把比例缩小的图像重叠到空间冗余图像的不同部分(例如,中央、左上角、正上方、正下方、右下方等等)。 Some embodiments of the scaled image 2630 is superimposed on the image spatial redundancy left corner (as shown, to produce a composite image 2640), while other embodiments the scaled-down image is superimposed to different parts of the spatial redundancy of the image ( For example, the center of the upper left corner, just above, just below the bottom right, and so on).

[0353] 在一些实施例中,空间冗余图像是基本上全部为一种颜色(例如,黑色、蓝色、红色、白色等),或者具有重复图案(例如,方格、条纹等)的图像。 [0353] In some embodiments, the spatial redundancy of the image is substantially all one color (e.g., black, blue, red, white, etc.), or with a repetitive pattern (e.g., checks, stripes, etc.) of the image . 例如,图26中所示的空间冗余图像2635具有重复的十字形图案。 For example, spatial redundancy in the image 2635 shown in FIG 26 has a crisscross pattern repeating. 由于重复性的缘故,合成图像2640的空间冗余部分能够容易地被编码器压缩成少量的数据。 Since repetitive reason, spatial redundancy portion of the composite image 2640 can be easily compression encoder into a small amount of data. 此外,如果一系列的图像都被比例缩小,并且对序列中的每个图像来说,使用的空间冗余图像都相同,那么时间压缩可被用于进一步减少为呈现编码图像所需要的数据的重。 Further, if the series of images are reduced scale, and for each image in the sequence, the spatial redundancies are used in the same image, the time compression may be used to further reduce the data required to render the encoded image weight.

[0354] 图像缩放器2615的一些实施例还产生指示缩放后的图像的大小(例如,比例缩放的图像2630的大小)的大小数据2620,并把产生的大小数据2620发送给比特流管理器2625。 Size of the image after the [0354] image scaler some embodiments 2615 also generates an indication scaling (e.g., scaling of the size of the image 2630) of the size of the data 2620, and the size of the generated data 2620 to a bit stream manager 2625 . 一些实施例的大小数据2620按照缩放后的图像2630的像素的行数和像素的列数(即,高度和宽度),指示缩放后的图像2630的大小。 Some embodiments of the size of the data in accordance with the number 2620 (i.e., height and width) of the scaled image 2630 pixels in rows and columns of pixels, the scaled image 2630 indicating the size. 在一些实施例中,大小数据2620还指示缩放后的图像2630在合成图像2640中的位置。 In some embodiments, the size of the image data 2620 also indicates the position 2630 in the scaled image 2640 is synthesized.

[0355] 在图像被缩放之后,合成图像2640通过速率控制器2645被发送给编码器2650。 [0355] After an image is scaled, the synthesized image 2640 is transmitted by the rate controller 2645 to the encoder 2650. 在一些实施例中,如下进一步详细所述,速率控制器2645控制由编码器2650输出的图像的比特率(即,数据大小)。 In some embodiments, in further detail below, the rate controller 2645 controls a bit rate of the image output by the encoder 2650 (i.e., data size). 一些实施例的编码器2650压缩和编码图像。 Some embodiments of the encoder 2650 compresses and encodes the image. 编码器2650可以使用H. 264编码或者另一种编码方法。 The encoder 2650 H. 264 coding, or another coding method may be used.

[0356] 一些实施例的比特流管理器2625从编码器2650接收一个或多个编码图像的比特流,并把大小数据插入比特流中。 [0356] Some embodiments of the bitstream manager of 26,252,650 or receiving a plurality of encoded image bit stream from the encoder, and the size of the data into the bitstream. 例如,在一些实施例中,比特流管理器2625从图像缩放器2615接收大小数据2620,并把大小数据2620插入从编码器2650接收的编码合成图像2640 的比特流2655中。 For example, in some embodiments, manager 2625 receives a bit stream from the data size of the image scaler 2615 2620, 2620 and the size of the data inserted from the encoded synthetic image encoder 2650 receives the bit stream 2640 2655. 这种情况下,比特流管理器2625的输出是包括大小数据2620的修改后的比特流2660。 In this case, the output bit stream is a bit manager 2625 includes modifying the size of the data stream 2620 2660. 不同的实施例把大小数据2620插入比特流2655中的不同位置。 Different embodiments the size of the data inserted into the bitstream 2620 of 2655 different positions. 例如,t匕特流2660显示大小数据2620被插入在比特流2660的开始。 For example, t dagger display size of the data bit stream 2660 is inserted in the bit stream 2620 start 2660. 不过,其它实施例把大小数据2620插入在比特流2655的结尾、插入在比特流2655的中间、或者插入在比特流2655内的任何其它位置。 However, other embodiments of the size of the data 2620 is inserted into the end of the bitstream 2655, the bitstream is inserted in the middle of 2655, or into any other position within the 2655 bit stream.

[0357] 在一些实施例中,比特流2655是包括合成图像2640的一系列的一个或多个编码图像的比特流。 [0357] In some embodiments, the bitstream 2655 including a series of synthetic images 2640 of a plurality of coded bit streams or images. 在一些这样的实施例中,该系列中的图像都被缩放到相同的大小,并且大小数据2620指示这些缩放后的图像的大小。 In some such embodiments, the series of images are scaled to the same size, and the data size 2620 indicating the size of the image after scaling. 在图像被传送给在视频会议另一端的设备之后, 接收设备能够从比特流中提取大小信息,并利用所述大小信息正确地解码接收的图像。 After transmitted to the other end of video conferencing equipment, the receiving apparatus can extract the size information in the image from the bit stream, and using the image size information received correctly decoded.

[0358] 图27概念地图解说明由双照相机移动设备的编码器驱动器,比如驱动器2600执行的图像缩放处理2700。 [0358] FIG. 27 illustrates conceptually the encoder driver dual camera mobile device, such as an image of the driver 2600 performs scaling processing 2700. 处理2700从接收(在2705)由双照相机移动设备的照相机拍摄的图像(例如,图像2605)开始。 2700 starts receiving (at 2705) an image (e.g., image 2605) captured by a camera of the dual camera mobile device. 当双照相机设备用两部照相机拍摄图像时,一些实施例对来自两部照相机的图像执行处理2700。 When the double image capturing camera device with two cameras, some embodiments of the image processing performed from two cameras 2700.

[0359] 随后,处理2700缩放(在2710)接收的图像。 [0359] Subsequently, the received image scaling process 2700 (2710). 如上所述,不同的实施例不同地缩放图像2605。 As described above, various embodiments of the image 2605 scaled differently. 例如,图26中的图像2605被比例缩小并被重叠在空间冗余图像2635上,从而产生合成图像2640。 For example, the image 2605 in FIG. 26 is superimposed on a reduced scale and spatial redundancy in image 2635, 2640 to produce a composite image.

[0360] 处理2700随后把缩放后的图像(例如,包括缩放后的图像2630的合成图像2640) 发送给(在2715)编码器2650,以便编码。 [0360] 2700 then sends the scaled image (e.g., a composite image including the scaled image 2630 2640) to the (at 2715) the encoder 2650 to encode. 处理2700的一些实施例把缩放后的图像2630(包括在合成图像2640中)通过确定编码器编码图像的比特率的速率控制器发送给编码器2650。 Some embodiments of the process 2700 scaled image 2630 (including the composite image 2640) by determining the transmission bit rate encoder of the encoded image to the encoder rate controller 2650. 一些实施例的编码器2650压缩和编码图像(例如,利用离散余弦变换、量化、熵编码等等),并把带有编码图像的比特流返回给编码器驱动器2600。 Some embodiments of the encoder 2650 compresses and encodes the image (e.g., using a discrete cosine transform, quantization, entropy encoding, etc.), and the bit stream with the encoded image back to the encoder driver 2600.

[0361] 随后,处理2700把指示缩放后的图像的大小的数据(例如,大小数据2620)发送给(在2720)比特流管理器。 [0361] Then, the data size of the image after scaling indication 2700 (e.g., data size 2620) to (2720) the bit stream manager. 如图26中所示,在一些实施例中,该操作是在编码器驱动器2600中进行的(即,编码器驱动器2600中的一个模块把大小数据发送给编码器驱动器2600中的另一个模块)。 , In some embodiments, the operation is performed in FIG. 26 in the encoder driver 2600 (i.e., a module 2600 encoder driver sends the size of the data to the encoder drive in another module 2600) .

[0362] 在缩放后的图像被编码器2650编码之后,处理2700从编码器接收(在2725)比特流。 [0362] After an image is encoded by the encoder 2650 after the scaling process 2700 receives (at 2725) the bit stream from the encoder. 如图所示,一些实施例在另外已接收大小数据的比特流管理器接收所述比特流。 As shown, in some embodiments additional data size of the received bit stream manager receives the bit stream. 接收的比特流包括编码的合成图像,还可包括视频序列中的一个或多个另外的图像。 Receiving a composite image comprising a bitstream encoded video sequence may comprise one or more additional images.

[0363] 处理2700随后把指示缩放后的图像的大小的数据(例如,大小数据2620)插入比特流中,随后结束。 [0363] 2700 followed by data indicating the size of the image after scaling (e.g., data size 2620) is inserted into the bitstream, then it ends. 如图26中所示,在一些实施例中,该操作也是由比特流管理器执行的。 As shown in FIG. 26, in some embodiments, the operation is performed by the bit stream manager. 如上所述,不同的实施例把大小数据插入比特流的不同部分。 As described above, the size of the various embodiments of data into different portions of the bitstream. 在图解说明的例子中,大小数据2620被插入在比特流2655的开始处,如所得到的比特流2660中所示。 In the illustrated example, the size of the data 2620 is inserted in the bit stream at the beginning of 2655, 2660 as shown in the bit stream obtained. 现在,该比特流可被传送给参与视频会议的另一个设备,在所述另一个设备,该比特流能够被解码和观看。 Now, the bitstream may be transmitted to other devices participating in the video conference, said another apparatus, the bit stream can be decoded and viewed.

[0364] 在一些实施例中,解码器驱动器(例如,驱动器1640)执行编码器驱动器的相反功能。 [0364] In some embodiments, the decoder driver (e.g., driver 1640) encoder performs the inverse function of the drive. 即,解码器驱动器从接收的比特流提取大小数据,把比特流传送给解码器,并利用大小数据缩放解码的图像。 That is, the decoder driver is extracted from the bit stream received size data, the bit stream to the decoder, and use the size of the decoded image data scaled. 图28概念地图解说明一些实施例的这种解码器驱动器2800的软件体系结构,并且表示由解码器驱动器2800对示例的比特流2825进行的示例比特流处理和缩放操作。 FIG 28 conceptually illustrates some of the software architecture of this embodiment of the decoder driver 2800, and 2800 represented by a bit sample of an example of the stream decoder driver 2825 for a bit stream processing and scaling operations.

[0365] 在一些实施例中,比特流2825是包括由视频会议中的一个设备的照相机拍摄的视频的编码图像的比特流(例如,来自诸如驱动器2600之类的编码器驱动器的比特流),并被传送给解码器驱动器2800所运行于的设备。 [0365] In some embodiments, the bit stream 2825 is a bit stream coded image taken by the camera a device of the video conference video (e.g., from the encoder driver such as driver 2600 such bit stream), and transmitted to the decoder driver running on the device 2800. 参见图16,在一些实施例中,该比特流已被连网管理器1614接收,并被发送给缓冲器1616,所述比特流从缓冲器1616被视频解压缩模块1626取回,并被发送给解码器驱动器1640。 Referring to Figure 16, in some embodiments, the bit stream has been received networking manager 1614, buffer 1616 and sent to the buffer from the bit stream compressed video decompression module 1616 retrieves 1626, and transmitted 1640 to the decoder driver.

[0366] 如图所示,解码器驱动器2800包括处理层2805。 [0366] As shown, the decoder driver 2800 includes a processing layer 2805. 处理层2805包括图像缩放器2810和比特流管理器2820。 Treatment layer 2805 includes an image scaler 2810 and the bit stream manager 2820. 在一些实施例中,这些模块2810和2820在图像被解码之前和之后,对接收的图像进行各种操作。 In some embodiments, these modules 2810 and 2820 after and before the image is decoded, an image received various operations. 尽管在本例中,图像缩放器2810被表示成解码器驱动器2800的处理层2805的一部分,不过,一些实施例把图像缩放器实现成图像处理管理器1608,而不是解码器驱动器的一部分(S卩,在从解码器驱动器2800发送图像之后,进行图像缩放)。 Although in the present embodiment, the image scaler 2810 is represented by a part as the decoder driver treatment layer 2800 of 2805, however, some embodiments to implement image scaler to the image processing manager 1608, instead of the decoder driver part (S Jie, 2800 after transmission of the image from the decoder driver, image scaling).

[0367] 如图所示,一些实施例的比特流管理器2820接收一个或多个编码图像(S卩,视频序列中的图像)的比特流,并从该比特流中提取大小数据,之后把比特流发送给解码器2835进行解码。 [0367] As shown, some of the embodiments of the bitstream manager 2820 receives one or more encoded images (S Jie, a video sequence of images) of the bit stream, and extracts the size of data from the bit stream, after the the bit stream to a decoder 2835 for decoding. 例如,如图28中所示,比特流管理器2820接收一个编码图像的比特流2825,从比特流2825提取大小数据2815,并把得到的比特流2830 (没有大小数据2815)发送给解码器2835进行解码。 For example, as shown in FIG bitstream manager 2820 receives a bit stream 28 encoded image 2825, 2825 to extract from the bit stream data size 2815, and the resulting bit stream 2830 (2815 no size data) 2835 sent to the decoder decoded. 如图所示,在一些实施例中,比特流管理器2820把提取的大小数据2815发送给图像缩放器2810。 As shown, in some embodiments, the flow manager 2820 transmits the extracted data bit size of the image scaler 2815 to 2810.

[0368] 一些实施例的大小数据2815和由编码器驱动器2600插入比特流中的大小数据2620相同。 [0368] Example embodiments of the same size of the data number 2815 and 2620 by the encoder 2600 is inserted into the drive in the size of the data bit stream. 如上在图26的说明中所述,一些实施例的大小数据2815按照子图像2845的像素的行数和像素的列数,指示子图像2845的大小。 As described in the description of FIG. 26, the size of some embodiments data 2815 in accordance with the number of columns of sub-pixel 2845 and the number of rows of pixels, indicative of the sub-image size of 2845. 大小数据2815还可指示子图像2845在更大的空间冗余图像2840内的位置。 Size of the data sub-image 2815 may also indicate a position 2845 within a larger spatial redundancy of the image 2840. 在这个例子中,比特流2825显示插入在比特流2825 的开始处的大小数据2815。 In this example, display 2825 is inserted into the bit stream size of the data bit stream at the beginning of 2825 to 2815. 不过,如上所述,不同的实施例把大小数据2815插入比特流2825的不同位置。 However, as described above, various embodiments of the size of the data inserted into the bitstream 2815 of 2825 different positions.

[0369] 一些实施例的图像缩放器2810利用从比特流管理器2820接收的大小数据,从图像中提取子图像。 [0369] Some embodiments of the image scaler 2810 use, extracted from the sub-image size of the image data bit stream received from the manager 2820. 例如,图28图解说明图像缩放器2810从解码器2835接收包括子图像2845的图像2840。 For example, FIG. 28 illustrates an image scaler 2810 receives an image 2840 includes sub-image 2845 from the decoder 2835. 如图所示,一些实施例的图像缩放器2810从图像2840提取子图像2845。 As shown, some of the embodiments of the image scaler 28102840 extracting sub-images from the image 2845. 提取的图像随后可被显示在双照相机移动设备上。 Then extracted image may be displayed on a dual camera mobile device.

[0370] 图29概念地图解说明一些实施例的由参与视频会议的设备的解码器驱动器,t匕如驱动器2800执行的图像提取处理2900。 [0370] FIG. 29 illustrates conceptually the decoder driver by the number of devices participating in a video conference embodiment, t dagger driver 2800 performs image extraction processing 2900. 通过接收(在2905)编码图像的比特流(例如, 比特流2825),开始该处理。 By receiving (at 2905) a bit stream of encoded image (e.g., bit stream 2825), the process is started. 所述比特流可以发送自与解码器驱动器所运行于的设备进行视频会议的另一个设备,或者可被保存在所述设备的存储器中。 The bit stream can be transmitted from the decoder driver running on the videoconferencing equipment other device, or may be stored in a memory of the device. 当所述设备从多个来源接收图像时,一些实施例对来自每个来源的图像进行处理2900。 When the device receives an image from a plurality of sources, some embodiments of the images from each source 2900 for processing.

[0371] 随后,处理2900从比特流提取(在2910)大小数据。 [0371] Subsequently, the size of the data extraction process stream 2900 (2910) from the bit. 如上所述,可在比特流中的不同位置找到所述大小数据。 As described above, the size of the data can be found in the bit stream at different locations. 一些实施例知道在何处寻找所述大小数据,而其它实施例寻找特定的标记,所述标记指出所述大小数据位于接收的比特流中的何处。 Some embodiments of the size of the data to know where to look for, while other embodiments find particular marker, the marker indicates that the size of the data is located in the received bit stream where. 在一些实施例中, 大小数据指出子图像的大小(例如,每一行中的像素数和每一列中的像素数)和子图像在编码图像中的位置。 In some embodiments, the sub-image size data indicates the size (e.g., number of pixels in each row and each column of pixels) and sub-picture position in the encoded image.

[0372] 处理2900随后把提取的大小数据发送给(在2915)图像缩放器。 [0372] process 2900 then transmits the extracted data to a size (at 2915) an image scaler. 如图28中所示,在一些实施例中,该操作是在解码器驱动器内进行的(即,解码器驱动器中的一个模块把大小数据发送给解码器驱动器中的另一个模块)。 As shown in FIG. 28, in some embodiments, the operation is performed in the decoder driver (i.e., the decoder driver is a module the size of the transmit data to another module in the decoder driver).

[0373] 处理2900还把比特流发送给(在2920)解码器,以便解码。 [0373] Processing 2900 also transmits the bit stream to a (2920) decoder, for decoding. 在一些实施例中,解码器解压缩和解码比特流(例如,利用反向离散余弦变换、反向量化等),并把重构的图像返回给解码器驱动器。 In some embodiments, the decoder decompresses and decodes the bit stream (e.g., using the inverse discrete cosine transform, inverse quantization, etc.), and the reconstructed image is returned to the decoder driver.

[0374] 在比特流被解码器解码之后,处理2900从解码器接收(在2925)解码图像。 [0374] After the bit stream is decoded by the decoder, the process 2900 receives (at 2925) the decoded picture from the decoder. 如图所示,一些实施例在还已从比特流管理器接收大小数据的图像缩放器接收解码图像。 As shown, some embodiments also receives the decoded image data from the received bit stream size image scaler manager. 处理随后利用接收的大小数据,从解码图像提取(在2930)子图像。 Processing then use the size of the received data, extracting (2930) a sub-image from the decoded image. 如图所示,如大小数据2815 中所示那样,从解码图像2840的左上角提取子图像2845。 As shown, as the size of the data as shown in 2815, the sub-image extraction 2845 from the upper left corner of the decoded image 2840. 现在,提取的子图像可被显示在显示设备(例如,双照相机移动设备的屏幕)上。 Now, the extracted sub-image may be displayed on a display device (e.g., dual camera mobile device's screen).

[0375] 3.速率控制器 [0375] 3. The rate controller

[0376] 在一些实施例中,设备的两部照相机具有不同的多组特性。 [0376] In some embodiments, two different camera apparatus having a plurality of sets of characteristics. 例如,在一些实施例中,正面照相机是为运动视频图像的拍摄而优化的较低分辨率照相机,而背面照相机是为静止图像的拍摄而优化的较高分辨率照相机。 For example, in some embodiments, the front camera for taking moving video images optimized lower resolution camera, while the back of the camera for photographing a still image of the higher resolution of the camera optimized. 由于诸如设备的成本、功能和/或几何形状之类的各种原因,其它实施例可以使用不同特性的照相机的不同组合。 For various reasons such as cost, functionality and / or geometry of the like, various other combinations of different characteristics of the camera may be used according to embodiments.

[0377] 具有不同特性的照相机能够引入不同的伪像。 [0377] The camera can be introduced with different characteristics of different artifacts. 例如,与较低分辨率照相机相比,较高分辨率照相机会显示更多的噪声。 For example, compared with the lower resolution cameras, high-resolution camera will appear more noise. 与由较低分辨率照相机拍摄的图像相比,由较高分辨率照相机拍摄的图像会表现出更高水平的空间或时间复杂性。 Compared with the low-resolution image captured by the camera, captured by a higher resolution camera images will show a spatial or temporal higher level of complexity. 另外,具有不同光学性质的不同照相机会在拍摄的图像中引入不同的Y值。 Further, different cameras having different optical properties may introduce different Y values ​​in the captured image. 由不同的照相机用于拍摄图像的不同感光机构也会引入不同的伪像。 Photosensitive means for taking different images from different cameras will introduce different artifacts.

[0378] 这些照相机特有的伪像中的一些隐藏由其它来源产生的伪像。 [0378] Camera-specific artifacts hide some of these artifacts generated by other sources. 例如,在由具有高水平噪声的高分辨率照相机拍摄的图像中,作为视频编码处理的副产物的伪像变得不太可见。 For example, in an image captured by a high resolution camera with a high level of noise, as a by-product of a video encoding process becomes less visible artifacts. 当编码噪声(比如量化失真)隐藏在照相机特有的伪像之后时,视频编码处理可使用更大的量化步长来获得较小的比特率。 When the coding noise (such as quantization distortion) is hidden behind the camera-specific artifacts, a video encoding process using a larger quantization step size to obtain a smaller bit rate. 另一方面,当照相机引入较少的伪像时(比如就较低分辨率照相机来说),视频编码处理能够利用更细小的量化步长,以便避免由量化引起的不可接受的视觉失真水平。 On the other hand, when the camera is introduced fewer artifacts (such as for lower resolution camera), a video encoding process capable of using finer quantization step size, in order to avoid an unacceptable level of visual distortion due to quantization. 从而,与忽视这些照相机特有的特性的视频编码处理相比,优化成利用或者补偿这些照相机特有的特性的视频编码处理能够获得更好的速率-失真折衷。 Thus, as compared with the camera to ignore these unique characteristics of video coding process is optimized to use the camera or to compensate for these characteristics peculiar to a video encoding process to obtain a better rate - distortion trade-off.

[0379] 为了把这些照相机特有的特性用于实现速率-失真折衷,一些实施例实现分别相对于两部照相机的每个照相机优化的两种视频编码处理。 [0379] In order to put these cameras for achieving specific characteristics of a rate - distortion tradeoff, some embodiments are implemented with respect to each of the two cameras of the camera to optimize both a video encoding process. 图30图解说明具有关于两部照相机3060和3070的两个视频编码处理的系统的例子。 FIG 30 illustrates an example of a system having two video cameras on two 3060 and 3070 of the encoding process of. 如图30中所示,系统3000包括编码器驱动器3010、速率控制器3020和3040、和视频编码器3030。 As shown in Figure 30 includes system 3000 drives an encoder 3010, a rate controller 3020 and 3040, and a video encoder 3030. 编码器3030把从视频照相机3060和3070拍摄的视频图像编码成比特流3080和3090。 The encoder 3030 video captured from video cameras 3060 and 3070 into a bitstream 3080 and 3090.

[0380] 在一些实施例中,视频编码器驱动器3010是一个或多个处理单元上运行的软件模块。 [0380] In some embodiments, the video encoder driver 3010 is a software module running on one or more processing units. 它提供视频编码器303和系统的其它组件,比如视频照相机、图像处理模块、网络管理模块和存储缓冲器之间的接口。 It provides other components of video encoder 303 and systems, such as the interface between a video camera, an image processing module, a network management module and the memory buffer. 编码器驱动器3010控制拍摄的视频图像从照相机和图像处理模块到视频编码器3030的流动,它还提供编码比特流3080和3090到存储缓冲器和网络管理模块的管道。 Encoder driver 3010 controls a video image captured from the camera and an image processing module to the flow of the video encoder 3030, it provides the encoded bit stream 3080 and the pipe 3090 to the buffer memory, and network management module.

[0381] 如图30中所示,编码器驱动器3010包括速率控制器的两个不同实例3020和3040。 [0381] As shown, the encoder 3010 includes a driver 30 in two different instances 3020 and 3040 of rate controller. 这些多个实例可以是用于两部不同照相机的两个不同的速率控制器,或者是关于两部不同的照相机,按照两种不同的方式配置的一个速率控制器。 These examples may be a plurality of two different cameras of two different rate controller, or on two different cameras, a rate controller configured in accordance with two different ways. 具体地说,在一些实施例中,这两个速率控制器3020和3040代表两个单独的速率控制器。 Specifically, in some embodiments, both the rate controllers 3020 and 3040 represent two separate rate controller. 另一方面,在其它实施例中,这两个速率控制器3020和3040是单个速率控制器的两种不同配置。 On the other hand, in other embodiments, both the rate controllers 3020 and 3040 are two different configurations of a single rate controller.

[0382] 图30还显示包括状态缓冲器3015的编码器驱动器3010,状态缓冲器3015保存在视频会议期间使用的速率控制操作的编码状态信息。 [0382] FIG 30 further shows the status of the encoder buffer drives 3015 3010, 3015 saved status buffer rate used during a video conference state information of the operation control code. 具体地说,在一些实施例中,这两个不同的速率控制器,或者相同速率控制器的两种不同配置在视频会议期间,共享保存在状态缓冲器3015中的相同的编码状态信息。 Specifically, in some embodiments, two different rate controller, or two different configurations of the same rate controller during a video conference, held in the shared state buffer 3015 in the same encoding state information. 状态信息的这种共享允许双视频拍摄视频会议中的统一的速率控制器操作。 This shared state information allows dual video shoot uniform rate controller operating a video conference. 这种共享还允许单视频拍摄视频会议中,切换照相机操作期间的最佳视频编码(即,允许关于当前照相机所摄视频的编码的速率控制操作使用由关于前一照相机所摄视频的编码的速率控制操作保持的编码状态信息)。 This also allows a single shared video capture video conference, the optimal switching operation of the camera during the video coding (i.e., allowing the rate of the encoded video camera on the current intake control operation by the coding rate used on a front video camera of the pickup coding state maintaining operation of the control information). 图30把状态缓冲器3015表示成编码器驱动器3010的一部分,不过其它实施例可在编码器驱动器3010之外实现状态缓冲器3015。 FIG 30 represents the state buffer 3015 as part of the encoder driver 3010, although other embodiments may be implemented in addition to the state buffer 3015 drives an encoder 3010.

[0383] 在状态缓冲器3015中,不同的实施例保存代表编码状态信息的不同类型的数据(例如,不同类型的编码参数)。 [0383] In the state buffer 3015 in different embodiments represent different types of data stored coding state information (e.g., different types of coding parameters). 这种编码状态信息的一个例子是视频会议的当前目标比特率。 An example of such information is the current coding state target bit rate of the video conference. 上面在III.B节中说明了识别目标比特率的一种方式。 The above described way of identifying a target bit rate in III.B section. 这种编码状态信息的其它例子包括缓冲器充满度、最大缓冲器充满度、一个或多个最近编码的帧的比特率、以及其它编码状态信息。 Other examples of such a coding state information includes a bit rate buffer fullness, the maximum buffer fullness, or a plurality of the most recently encoded frame, status information and other encoded.

[0384] 速率控制器随后能够使用目标比特率(或者保存在状态缓冲器中的另一个编码状态参数)来计算在其速率控制操作中使用的一个或多个参数。 [0384] The rate controller can then use the target bit rate (or other coding state parameters stored in the status buffer) to compute one or more parameters used in the operation of the rate control. 例如,如下进一步所述,一些实施例的速率控制器使用当前的目标比特率来计算宏块或帧的量化参数QP。 For example, as described further below, some of the embodiments rate controller calculates a quantization parameter QP of the macro block or frame of the current target bit rate. 例如,一些实施例利用当前的目标比特率来计算量化调整参数,这些实施例从量化调整参数得到宏块和/或帧的量化参数QP。 For example, some embodiments utilize the current target bit rate to calculate a quantization parameter adjustment, these embodiments are obtained from the quantization parameter adjustment macroblock quantization parameter QP and / or frame. 因此,在视频会议中的照相机切换操作期间,在(两个速率控制器,或者一个速率控制器的两种不同配置的)两种速率控制操作之间共享目标比特率允许对当前照相机拍摄的视频编码的速率控制操作受益于对前一照相机拍摄的视频编码的在先速率控制操作的编码状态数据。 Thus, the camera in a video conference during a switching operation, the target bitrate share between the current operation allows the video taken by the camera in two rates (the rate of two different configurations of the two controllers, or a rate controller) controls coding control operation rate benefit prior to encoding rate of the video taken by the camera before a coding state of the operation control data.

[0385] 图30图解说明包括两个不同的速率控制器实例3020和3040的编码器驱动器3010。 [0385] FIG. 30 illustrates two different examples of rate controllers 3020 and 3040 drives an encoder 3010. 不过,在其它实施例中,这些速率控制器实例3020和3040被内置到视频编码器3030 中。 However, in other embodiments, these examples the rate controller 3020 and 3040 are built into the video encoder 3030. 视频编码器3030把由照相机3060和3070拍摄的视频图像编码成数字比特流3080和3090。 The video encoder 3030 and 3060 captured by the camera 3070 into a digital video coding bitstream 3080 and 3090. 在一些实施例中,视频编码器产生符合常规的视频编码标准(例如,H. 264MPEG-4)的比特流。 In some embodiments, the video encoder generates consistent with conventional video coding standards (e.g., H. 264MPEG-4) bitstream. 在一些这样的实施例中,视频编码器执行包括运动估计、离散余弦变换("DCT")、 量化和熵编码的编码操作。 In some such embodiments, include a video encoder performs motion estimation, discrete cosine transform ( "DCT"), quantization, and entropy encoding of the encoding operation. 视频编码器还进行与编码操作的功能相反的解码操作。 The video encoder also contrary to the function of the encoding operation of the decoding operation.

[0386] 在一些实施例中,编码器3030包括用于进行量化的量化器模块3032。 [0386] In some embodiments, the encoder 3030 includes a quantizing module 3032. 量化器模块由来自速率控制器3020或3040的量化参数3022或3042控制。 Quantization by the quantization parameter control module from the rate controller 3020 or 3040 3022 or 3042. 在一些实施例中,每个量化参数由对应的速率控制器设定,并且是与该速率控制器相关的照相机的一种或多种属性的函数,如下进一步所述。 In some embodiments, each corresponding to the quantization parameter set by the rate controller and is a function of or related to the rate controller of the various properties of the camera, as described further below. 速率控制器能够通过设定更大的量化步长减少用于编码的比特数,或者通过设定更小的量化步长增大所使用的比特数。 Rate controller reduces the number of bits used for encoding by setting a larger quantization step size, or to increase the number of bits used by setting a smaller quantization step size. 通过控制量化步长,速率控制器还确定在编码的视频图像中会引入多少失真。 By controlling the quantization step size, the encoding rate controller further determines a video image introduces much distortion. 从而,速率控制器能够实现比特率和图像质量之间的折衷。 Thus, the rate controller to achieve a compromise between image quality and bit rate. 在实现速率-失真折衷时,速率控制器监控比特率,以便不会上溢存储缓冲器、下溢存储缓冲器、或者超过传输通道容量。 In the achievable rate - distortion when a tradeoff, the bit rate controller monitors the rate, in order not to overflow the storage buffer, memory buffer underflow, or more than the transmission channel capacity. 速率控制器还必须控制比特率,以便提供最佳的可能图像质量和避免由量化引起的不可接受的图像质量失真。 Rate controller must also control the bit rate, in order to provide the best possible image quality and the image quality due to quantization to avoid unacceptable distortion. 在一些实施例中,每个速率控制器把按照一组状态数据值表示的监控数据保存在状态缓冲器3015中。 In some embodiments, the controller monitor each rate in accordance with a set of data representing the state of data values ​​stored in the state buffer 3015. 在一些实施例中,速率控制器3020和3040使用照相机特有的属性来优化速率-失真折衷。 In some embodiments, the rate controller 3020 and the camera 3040 used to optimize specific properties rate - distortion tradeoffs.

[0387] 在一些实施例中,每个速率控制器通过直接对其量化参数应用修正系数,优化速率-失真折衷。 [0387] In some embodiments, each rate controller through its direct application quantization parameter correction coefficient optimal rate - distortion trade-off. 在一些这样的实施例中,修正系数是预先确定的并和照相机一起被内置在设备中;设备不需要动态计算这些修正系数。 In some such embodiments, the correction coefficient is determined in advance, and the camera together and built in the device; these devices need to dynamically calculate the correction coefficient. 在其它实施例中,系统利用由照相机拍摄的输入图像,动态确定特定于照相机的适当修正系数。 In other embodiments, the system using the input image photographed by a camera, camera to dynamically determine the specific appropriate correction factor. 在一些这样的实施例中,系统在多次编码中,分析由照相机拍摄的一系列的输入视频图像,以收集关于照相机的某些统计数据。 In some such embodiments, the multiple encoding system, the analysis of a series of input video image captured by a camera, to collect statistics on certain camera. 系统随后利用这些统计数据得到关于该照相机优化的量化参数的修正系数。 The system then use these statistics to obtain a correction coefficient on the quantization parameter optimization of the camera.

[0388] 在一些实施例中,这些照相机特有的修正系数经由视频图像的视觉掩蔽属性,被应用于量化参数。 [0388] In some embodiments, these specific correction coefficient camera via a visual masking attributes of the video image, is applied to the quantization parameter. 图像或一部分图像的视觉掩蔽属性是在所述图像或图像部分中能够容忍多少编码伪像的指示。 Visual masking attributes of the image or portion of an image in the image or image portion indicating how many coding artifacts can be tolerated. 一些实施例计算量化所述图像或图像部分的亮度能量的视觉掩蔽属性,而其它实施例计算量化所述图像或图像部分的活化能或复杂性的视觉掩蔽属性。 Calculating the number of quantized energy image luminance or image portion visual masking attributes embodiments, while other embodiments calculate a quantization of the image or image portion or activation energy complexity visual masking attributes. 与如何计算视觉掩蔽属性无关,一些实施例利用视觉掩蔽属性来计算一个视频帧的修正或掩蔽量化参数。 Regardless of how to calculate the visual masking attributes Some embodiments utilize visual masking attributes to calculate a corrected video frame or masking quantization parameter. 一些这样的实施例把掩蔽量化参数计算为帧级视觉掩蔽属性《Pfmme和基准视觉掩蔽属性9R的函数。 Some such embodiments the masking frame level quantization parameter is calculated as a function of visual masking attributes "Pfmme reference and visual masking attributes and 9R. 在一些实施例中,用视频掩蔽属性和<PR修正的量化参数被表示成: In some embodiments, a video-masking properties and the modified quantization parameter PR <is expressed as:

[0389] (1) [0389] (1)

[0390] 其中MQP&ame是巾贞的掩蔽或修正量化参数,QP_是初始或标称量化值,以及β 是适合于本地统计数据的常数。 [0390] wherein MQP & ame correction quantization parameter is masked or towel Zhen, QP_ quantized initial or nominal value and β is a constant adapted to the local statistics. 在一些实施例中,基准视觉掩蔽属性q>R和标称量化参数QP_是根据网络条件的初始或周期性评估预先确定的。 In some embodiments, the reference visual masking attributes q> R and the nominal quantization parameter QP_ initial or periodic assessment is based on a network condition determined in advance.

[0391] 在一些实施例中,方程式(1)中的视觉掩蔽属性(pframe被计算为 [0391] In some embodiments, visual masking attributes equation (1) (PFRAME is calculated as

[0392] [0392]

(2) (2)

[0393] 其中avgFrameLuma是巾贞的平均亮度值,以及avgFrameSAD是巾贞的绝对差的平均值和数。 [0393] where the average luminance value is avgFrameLuma Fok towel, and the average of the absolute differences is avgFrameSAD towel several infidelity. 常数a,0,C,D和E适合于本地统计数据。 Constants a, 0, C, D and E are adapted to the local statistics. 在一些实施例中,这些常数适合于照相机特有的特性。 In some embodiments, the constants in a camera-specific features.

[0394] 一些实施例还计算视频图像的某一部分,比如宏块的掩蔽量化参数。 [0394] Some embodiments also calculates certain portion of the video image, such as masking quantization parameter of a macroblock. 在这些情况下,掩蔽量化参数被计算为宏块视觉掩蔽属性Φμβ的函数: In these cases, the masking property of the quantization parameter is calculated as a function of the macro block Φμβ visual masking:

[0395] (3) [0395] (3)

[0396] 其中,在一些实施例中,β m是适合于本地统计数据的常数,MQPframe是利用方程式(1)和(2)计算的。 [0396] wherein, in some embodiments, β m is suitable to the local statistics constant, MQPframe using Equation (1) and (2) calculated. 在一些实施例中,方程式(3)中的视觉掩蔽属性q>MB被计算为 In some embodiments, the visual Equation (3) in the masking properties q> MB is calculated as

[0397] (4) [0397] (4)

[0398] 其中avgMBLuma是宏块的平均亮度值,以及avgMBSAD是宏块的绝对差的平均值和数。 [0398] wherein avgMBLuma average brightness value of the macroblock, and a macroblock avgMBSAD is the average sum of the absolute differences. 常数α,β,Α,Β和C适合于本地统计数据。 Constants α, β, Α, Β C and suitable to the local statistics. 在一些实施例中,这些常数适合于照相机特有的特性。 In some embodiments, the constants in a camera-specific features.

[0399] 胜于如上所述利用多个照相机特有的常数来计算修正的量化参数,一些实施例通过只利用单个照相机特有的系数来计算量化参数,实现特定于照相机的速率控制。 [0399] As described above rather than using a plurality of cameras to calculate the constants unique to the quantization parameter modification, some embodiments by using only a single camera-specific factor to calculate the quantization parameter, to achieve a particular rate of the camera control. 例如,已知视觉掩蔽属性q>fr ame和φΜΒ,以及量化参数QP&ame,一些实施例利用单个照相机特有的系数μ来计算宏块的量化参数: For example, it is known visual masking attributes q> fr ame and φΜΒ, and the quantization parameter QP & ame, some embodiments use a single camera to calculate the specific coefficient μ macroblock quantization parameter:

[0400] (5) [0400] (5)

[0401] 为了计算方程式(5),一些实施例把帧和宏块的复杂性度量分别用作视觉掩蔽属性 [0401] In order to calculate Equation (5), some embodiments of the frame and macroblock complexity metrics are used as visual masking attributes

,

[0402] -些实施例在QPM的计算中应用不同的照相机特有的系数。 [0402] - some embodiments apply different camera-specific factor in the calculation of QPM. 例如,在一些实施例中,QPmb被计算为 For example, in some embodiments, QPmb is calculated as

[0403] (6) [0403] (6)

[0404] 其中P是按照照相机特有的特性调整的系数。 [0404] wherein P is a coefficient adjusted in accordance with the specific features of the camera.

[0405] 如上所述,状态缓冲器3015保存在视频会议期间,两个不同的速率控制器实例3020和3040能够共享的编码状态信息,以便从它们的速率控制操作获得更好的编码结果。 [0405] As described above, stored in the state buffer 3015 during a video conference, two different rate controller instances 30203040 coding states can share the information and to control the operation of the encoding result better from their rates. 在一些实施例中,目标比特率RT是这种共享的状态信息的一个例子。 In some embodiments, the target bit rate RT is an example of such shared state information. 该比特率是对一系列帧编码的理想比特率。 The bit rate is the bit rate over a series of encoded frames. 一般来说,该比特率是用比特/秒为单位表示的,并且是根据比如上面在III. B节中描述的处理确定的。 In general, the bit rate in bits / sec is represented, and is determined according to the processing described above in the example III. Section B.

[0406] 如上所述,一些实施例的速率控制器利用目标比特率来计算它输出给视频编码器3030的帧和/或宏块量化参数QP。 [0406] As described above, some of the embodiments of the rate controller uses it to calculate the target bit rate to the video encoder outputs frame 3030 and / or the macro-block quantization parameter QP. 例如,一些实施例利用当前的目标比特率来计算量化调整参数,这些实施例从计算的量化调整参数得到宏块和/或帧的量化参数QP。 For example, some embodiments utilize the current target bit rate adjustment parameter to calculate a quantization, macroblock obtained these embodiments and / or frame from the quantization adjustment parameter calculated quantization parameter QP. 在一些实施例中,量化调整参数是利用通过把前一帧的比特率,或者前一帧的比特率的运行期均值除以当前的目标比特率而计算的分数表示的。 In some embodiments, adjusting the quantization parameter is calculated by using the operation of the bit rate of the previous frame, the previous frame or the average bit rate divided by the current target bit rate expressed as a fraction. 在其它实施例中,所述调整参数不是按照这种方式精确计算的,相反一般地说(1)正比于前一帧的比特率或者前一帧的比特率的运行期均值,和(2)反比于当前的目标比特率。 In other embodiments, the adjustment parameters are not precisely calculated in this way, generally opposite to (1) is proportional to the running average of the bit rate of the previous frame or previous frame bit rate, and (2) inversely proportional to the current target bit rate.

[0407] 在计算这样的量化调整参数之后,一些实施例的速率控制器利用该参数调整它所计算的宏块和/或帧量化参数。 [0407] After the calculation of such quantization parameter adjustments, the rate controller of some embodiments it is calculated by using the adjustment parameter macroblock and / or frame quantization parameter. 进行这种调整的一种方式是把计算的宏块和/或帧量化参数乘以量化调整参数。 One way of making this adjustment is the calculated macro block and / or frame quantization parameter is multiplied by the quantization parameter adjustment. 进行这种调整的另一种方式是根据量化调整参数计算偏移量化参数值,随后把该偏移参数应用于计算的宏块和/或帧量化参数(例如减去)。 Another way of making this adjustment is based on and / or frame quantization parameter adjusted quantization parameter calculates an offset value quantization parameter, then the parameter used to calculate the offset macroblock (e.g., subtracted). 这些实施例的速率控制器随后把调整的宏块和/或帧量化参数输出给视频编码器3030。 The rate controller to adjust the subsequent embodiments macroblocks and / or frame quantization parameter is output to the video encoder 3030.

[0408] 在其它实施例中,速率控制器利用目标比特率计算在其速率控制操作中使用的其它参数。 [0408] In other embodiments, the controller calculates the rate of the other parameters used in the operation in which the rate control using the target bit rate. 例如,在一些实施例中,速率控制器利用该目标比特率修正宏块或帧的视觉掩蔽强度。 For example, in some embodiments, the rate controller with which the correction target bit rate or the frame macroblock visual masking strength.

[0409] G.连网管理器 [0409] G. networking manager

[0410] 图31概念地图解说明一些实施例的连网管理器3100,比如图16中图解说明的连网管理器1614的软件体系结构。 [0410] Figure 31 conceptually illustrates an embodiment of some of the networking manager 3100, such as illustrated in FIG. 16 described software architecture of the networking manager 1614. 如上所述,连网管理器3100管理它所运行于的双照相机移动设备和视频会议中的远程设备之间的网络连接(例如,连接建立、连接监控、连接调整、 连接中断等)。 As described above, the network connection between the mobile device and dual camera video conference networking management manager 3100 in which it is running on a remote device (e.g., connection establishment, connection monitoring, adjusting connection, connection interruption, etc). 在视频会议期间,一些实施例的连网管理器3100还处理传送给远程设备的数据,和处理从远程设备接收的数据。 During a video conference, a number of networking manager 3100 further embodiment of the data transmitted to the remote processing device, and processing data received from the remote device.

[0411] 如图31中所示,连网管理器3100包括会话协商管理器3105、发射器模块3115、通用传输缓冲器3120、通用传输缓冲器管理器3122、虚拟传送协议(VTP)管理器3125、接收器模块3130和媒体传送管理器3135。 As shown, the networking manager [0411] FIG. 31 3100 includes a session negotiation manager 3105, a transmitter module 3115, universal transmission buffer 3120, buffer manager 3122 universal transmission, virtual transmission protocol (VTP) manager 3125 , the receiver module 3130 and a media transport manager 3135.

[0412] 会话协商管理器3105包括协议管理器3110。 [0412] session negotiating manager 3105 includes a protocol manager 3110. 协议管理器3110确保在视频会议期间,发射器模块3115使用正确的通信协议向远程设备传送数据,和强制服从使用的通信协议的规则。 Protocol Manager 3110 to ensure that during the video conference, the transmitter module 3115 using the correct communication protocol to transfer data to a remote device, and forced to obey the rules of the communication protocol used. 协议管理器3110的一些实施例支持多种通信协议,比如实时传送协议(RTP)、传送控制协议(TCP)、用户数据报协议(UDP)、和超文本传送协议(HTTP),等等。 Some embodiments of the protocol manager 3110 supports multiple communication protocols such as Real-time Transport Protocol (the RTP), transmission control protocol (TCP), User Datagram Protocol (the UDP), and hypertext transfer protocol (HTTP), and the like.

[0413] 会话协商管理器3105负责建立双照相机移动设备和参与视频会议的一个或多个远程设备之间的连接,以及在会议之后中断这些连接。 [0413] session negotiating manager 3105 is responsible for establishing a connection between the dual camera mobile device and participate in a video conference with one or more remote devices, and after the meeting break these connections. 在一些实施例中,会话协商管理器3105还负责建立双照相机移动设备和视频会议中的远程设备之间的(例如,传送和接收视频和/或音频流的)多媒体通信会话(例如,利用会话发起协议(SIP))。 In some embodiments, the session manager 3105 is also responsible for the negotiation to establish (e.g., transmit and receive video and / or audio stream) multimedia communication session between the dual camera mobile device and a remote video conference equipment (e.g., using a session Initiation protocol (SIP)).

[0414] 会话协商管理器3105还从媒体传送管理器3135接收反馈数据,根据反馈数据,通过通用传输缓冲器管理器3122,确定通用传输缓冲器3120的操作(例如,是传送还是丢弃分组/帧)。 [0414] negotiation session manager 3105 also receives feedback data from the media transport manager 3135, based on feedback data, through a common transmission buffer manager 3122, the operation determines the universal transmission buffer 3120 (e.g., transmit or discard the packet / frame ). 在一些实施例中,这种反馈可包括单向等待时间和带宽估计比特率。 In some embodiments, the feedback may include a one-way latency and bandwidth estimation bit rate. 在其它实施例中,反馈包括分组丢失信息和往返延迟时间(例如,根据发送给视频会议中的远程设备的分组和来自远程设备的确认的接收确定的)。 In other embodiments, the feedback information including packet loss and round-trip delay time (e.g., determined based on the received packet to the remote video conference devices and acknowledgment from the remote device). 根据来自媒体传送管理器3135的信息,会话协商管理器3105能够确定是否正在发送过多的分组,和指令通用传输缓冲器管理器3122,让通用传输缓冲器3120传送较少的分组(即,如图15中所述调整比特率)。 The information transmitted from the media manager 3135, session negotiation manager 3105 can determine whether too many packets are being transmitted, and instructions universal transmission buffer manager 3122, so that less common packet transmission buffer 3120 transmits (i.e., such as Figure 15 said adjusted bit rate).

[0415] 发射器模块3115从视频缓冲器(例如,图16的缓冲器1612)取回编码图像(例如,以比特率的形式),并对图像分包以便通过通用传输缓冲器3120和虚拟传送协议管理器3125,传送给视频会议中的远程设备。 [0415] The transmitter module 3115 from the video buffer (e.g., buffer 1612 of FIG. 16) retrieves an image encoding (e.g., in the form of bit rate), and to transmit the image through the sub-transmission buffer 3120 and virtual universal protocol Manager 3125, transmitted to a remote video conference equipment. 产生编码图像并把编码图像发送给发射器模块3115的方式可以基于从媒体传送管理器3115和/或会话协商管理器3105接收的指令或数据。 Generating an encoded image and transmits the encoded image to a transmitter module 3115 embodiment may be based on an instruction 3105 received from the media transport manager 3115 and / or the manager or session negotiation data. 在一些实施例中,对图像分包涉及把接收的比特流分成一组分组,每个分组均具有特定的大小(即,由会话协商管理器3105按照特定协议指定的大小),和增加任何需要的报头(例如,地址报头、协议指定报头等等)。 In some embodiments, relates to a sub-image into the bit stream received a set of packets, each packet has a particular size (i.e., specified by a particular protocol 3105 by the session manager negotiation size), and adding any desired header (e.g., header address, protocol header specifies the like).

[0416] 通用传输缓冲器管理器3122根据从会话协商管理器3105接收的数据和/或指令,控制通用传输缓冲器3120的操作。 [0416] 3122 universal transmission buffer manager 3105 according to the received data from the session manager negotiation and / or instructions, the control operation of the universal transmission buffer 3120. 例如,可以指令通用传输缓冲器管理器3122命令通用传输缓冲器3120传送数据、停止传送数据、丢弃数据等等。 For example, the universal transmission instruction command buffer manager 3122 universal transmission buffer 3120 data transfer, data transfer is stopped, discard the data and the like. 如上所述,在一些实施例中, 当参与会议的远程设备好像丢失分组时,根据从远程设备接收的确认,能够识别这种情况。 As described above, in some embodiments, the remote device when in the meeting if a packet is lost, according to the received acknowledgment from the remote device, can recognize this situation. 为了减少分组丢失,可以指令通用传输缓冲器管理器3122以较低的速率向远程设备传送分组。 In order to reduce packet loss, it may instruct the universal transmission buffer manager 3122 at a lower rate transmitting packets to the remote device.

[0417] 通用传输缓冲器3120保存从发射器模块3115接收的数据,并且通过VTP管理器3125把该数据传送给远程设备。 [0417] universal transmission buffer 3120 to save the data 3115 received from the transmitter module, and transmitted to a remote device manager 3125 by VTP data. 如上所述,根据从通用传输缓冲器管理器3122接收的指令,通用传输缓冲器3120可丢弃分组(例如,视频的图像)。 As described above, in accordance with instructions received from the 3122 universal transmission buffer manager, universal transmission buffer 3120 may drop the packet (e.g., video images).

[0418] 在一些实施例中,RTP被用于在视频会议期间通过UDP传递数据分组(例如,音频分组和视频分组)。 [0418] In some embodiments, the RTP packets is used to transfer data (e.g., video packets and audio packets) during a video conference over UDP. 其它实施例利用RTP在视频会议期间通过TCP传递数据分组。 Other embodiments utilize RTP data packets transmitted by TCP during the video conference. 在不同的实施例中,也可使用其它传送层协议。 In various embodiments, it is also possible to use other transport layer protocol.

[0419] 一些实施例用一对端口号(即,源端口号和目的地端口号),定义两个移动设备之间的特定通信通道。 [0419] Some embodiments by a port number (i.e., source port number and destination port number), the definition of a particular communication channel between two mobile devices. 例如,移动设备之间的一个通信通道可用一对端口号(例如,源端口50 和目的地端口100)定义,以及移动设备之间的另一个不同的通信通道可用另一对不同的端口号(例如,源端口75和目的地端口150)定义。 For example, a communication channel between a pair of ports available mobile device number (e.g., source port and destination port 50 100) is defined, and the other different communication channels available between a mobile device to another different port numbers ( For example, a source port 150 and destination port 75) is defined. 一些实施例还利用一对网际协议(IP) 地址来定义通信通道。 Some embodiments also using a pair of Internet Protocol (IP) address to define the communication channel. 在一些实施例中,不同的通信通道被用于传送不同类型的数据分组。 In some embodiments, different communication channels are used to transmit different types of data packets. 例如,可在单独的通信通道中传送视频数据分组、音频数据分组和控制信令数据分组。 For example, the video data packets may be transmitted in a separate communication channel, the audio data packets and control signaling data packets. 因而,视频通信通道传送视频数据分组,以及音频通信通道传送音频数据分组。 Accordingly, the video communication channel packets to transmit video data, and an audio communication channel transmitting audio data packets.

[0420] 在一些实施例中,控制通信通道用于视频会议期间本地移动设备和远程设备之间的消息接发。 [0420] In some embodiments, a control message for a communication channel between a local mobile device and a remote device during a video conference hair. 这种消息接发的例子包括发送和接收请求、通知、以及对所述请求和通知的确认。 Such examples include messaging send and receive requests, notifications and confirmation of the request and notification. 消息接发的另一个例子包括从一个设备向另一个设备发送远程控制指令消息。 Another example of messaging includes transmitting from one device to another remote control instruction message. 例如, 通过经本地设备的控制通信通道,从本地设备向远程设备发送远程控制远程设备的操作的指令,能够执行下面说明的远程控制操作(例如,指令设备只发送来自特定照相机的图像, 或者只利用特定照相机拍摄图像)。 For example, by transmitting a remote control instruction operation of a remote device, capable of performing remote control operation will be described from the local device to the remote device via the control communication channel local device (e.g., instruction apparatus transmits only the image from a particular camera, or only with specific camera takes the picture). 不同的实施例利用不同的协议,比如实时传送控制协议(RTCP)、RTP扩展、SIP等实现所述控制通信。 Different embodiments use different protocols, such as Real-time Transport Control Protocol (RTCP), RTP extension, the SIP communication and the like to achieve the control. 例如,一些实施例利用RTP扩展在视频会议中在两个移动设备之间传递一组控制消息,以及使用SIP分组在视频会议期间在所述两个移动设备之间传递另一组控制消息。 For example, some embodiments utilize RTP extension transmitting a set of control messages between two mobile devices in a video conference, and another set of transmission between the two mobile devices using SIP packets during the video conference control message.

[0421] 一些实施例的VTP管理器3125允许通过单一通信通道(例如,利用相同的一对端口号),传送指定通过不同的通信通道(例如,利用不同的多对端口号)传送的不同类型的数据分组。 [0421] Some VTP manager Example 3125 allows a single communication channel (e.g., using the same pair of port number), the transfer is specified by different communication channels (e.g., using different pairs of port numbers) different types of transfers data packets. 用于此目的一种技术涉及识别数据分组类型,通过提取数据分组的一对指定的端口号识别数据分组被规定要通过其传送的通信通道,和通过把数据分组的一对端口号修改成所述单一通信通道的一对端口号来规定数据分组将通过所述单一通信通道传送(即, 所有的数据分组通过相同的一对端口号被传送)。 This object relates to a technique for identifying the type of data packets, the data packets is specified by extracting a port number identifying the specified data packets to be transmitted through the communication channel, and by modifying one pair of data packet into the port number of the said single communication channel to a predetermined port number to the data packet transmitted through the single communication channel (i.e., all data packets are transmitted through a same port number).

[0422] 为了明了每种数据分组的一对初始端口号,一些实施例保存相对于数据分组类型的一对初始端口号的映射。 [0422] In order to understand the data packets for each pair of initial port number, some embodiments, the mapping data stored initial packet type a port number with respect. 一些这样的实施例随后利用协议的分组类型字段来区分被多路复用到一个通信通道中的不同分组。 Some such embodiments to distinguish different packets then are multiplexed to a communication channel using a packet protocol type field. 例如,一些具有VTP管理器的实施例把音频、视频和控制分组多路复用到一个RTP流中,利用RTP分组类型字段来区分在一个RTP通道中被传送给视频会议中的另一个设备的音频、视频和控制分组。 For example, some embodiments VTP manager having audio, video and control packets multiplexed into one RTP stream, using the RTP packet type field to distinguish between the other device is transmitted to the video conference in a RTP channel audio, video and control packets. 在一些这样的实施例中,VTP管理器还把SIP分组中的控制消息路由给所述另一个设备。 In some such embodiments, VTP manager also controls the routing SIP message packet to said another device.

[0423] 一些实施例识别检查数据分组签名(即,分组报头格式),以区分利用不同的协议传递的不同分组(例如,区分利用RTP传送的分组和利用SIP传送的分组)。 [0423] Some embodiments of data packet inspection to identify the signature embodiment (i.e., a packet header format), with different protocols to distinguish different packet transmission (e.g., distinguished using RTP packet and transmitted using a packet transmitted SIP). 在这样的实施例中,在确定不同协议的数据分组之后,如上所述检查使用相同协议的数据分组(例如,利用RTP的音频数据和视频数据)的字段,以识别不同的数据类型。 In such an embodiment, after determining the data packets of different protocols, checking the data packet using the same protocol as described above (e.g., use of RTP audio data and video data) fields to identify the different data types. 按照这种方式,VTP管理器3125通过单一通信通道,传送预定通过不同的通信通道传送的不同数据分组。 In this manner, the VTP manager 3125 through a single communication channel, data packets transmitted through different predetermined different communication channels transmitted.

[0424] 尽管上面说明了通过单一通信通道组合不同类型的数据的一种方式,不过其它实施例利用其它技术把不同的分组类型多路复用到一个通信流中。 [0424] Although the above embodiment described a combination of different types of data through a single communications channel, although other embodiments utilize other techniques of different types of multiplexed into a packet in the traffic flow. 例如,一些实施例的一种技术涉及留意数据分组的一对初始端口号,并把所述一对初始端口号保存在数据分组本身中,以便稍后提取。 For example, some embodiments relate to a technique original pair of attention port number of data packets, and the port number of the pair of initial data stored in the packet itself, so as to extract later. 还存在把两个视频会议参加者之间的不同类型的数据组合到一个端口对通道中的其它技术。 There is also the combination of different types of data between the two video conference participants to a port other technologies channel pair.

[0425] 当VTP管理器3125通过虚拟通信通道,从远程设备接收到数据分组时,VTP管理器3125检查数据分组的签名以识别利用不同的协议发送的不同分组。 [0425] When VTP manager 3125 through the virtual communication channel, received from a remote device to the data packet, checks the signature VTP manager 3125 to identify the packet data using different packets of different transmission protocols. 这种签名可被用于区分SIP分组和RTP分组。 Such signatures may be used to distinguish a packet SIP and RTP packets. 一些实施例的VTP管理器还利用一些或全部分组的分组类型字段,多路分解被多路复用到单一虚拟化通道中的各种不同类型的分组(例如,音频、视频和控制分组)。 VTP manager Some embodiments also use some or all of the packet type field of the packet, the demultiplexer are multiplexed into a variety of different types of packets (e.g., audio, video and control packets) in a single virtual channel. 在识别这些不同类型的分组之后,VTP管理器根据它保存的端口对号和分组类型的映射,把每个不同类型的分组和其对应的端口对号关联起来。 After the identification of these different types of packets, according to the VTP port manager permutation mapping and its packet type saved, the check mark of packets each associated with different types and its corresponding port up. VTP 3125随后用识别的一对端口号修改数据分组的一对端口号,并转发数据分组以便解包。 VTP 3125 then modifies a data packet, the port number identified by a port number and forwards the data packets to unpack. 在利用不同的技术把不同的分组类型多路复用到单一通道中的其它实施例中,VTP管理器使用不同的技术解析分组。 In other embodiments utilize different technologies to the different types of packets multiplexed into a single channel of, VTP manager parse packets using different techniques.

[0426] 通过利用这样的技术多路复用和多路分解不同的分组,VTP管理器3125创建单一的虚拟化通信通道(例如,单对端口号),通过所述单一的虚拟化通信通道传送视频数据、 音频数据和控制信令数据,以及通过所述单一的虚拟化通信通道从远程设备接收音频、视频和控制分组。 [0426] By using different packet decomposition techniques such multiplexing and demultiplexing, the VTP manager 3125 to create a single virtual communication channel (e.g., a single port number), by the communication of a single virtual channel transport video data, audio data and control signaling data, received from a remote device, and audio, video and control packets via said single communication channel virtualization. 从而,从网络的观点来看,数据是通过所述单一的虚拟化通信通道传送的, 而从会话协商管理器3105和协议管理器3110的观点来看,视频数据、音频数据和控制信令数据是通过不同的通信通道传送的。 Thus, from the network point of view, the data is transmitted through said single virtual communication channel, from the viewpoint manager 3105 and the session negotiation protocol manager view 3110, video data, audio data and control signaling data It is transmitted through different communication channels.

[0427] 类似于传送给视频会议中的远程设备的图像,从视频会议中的远程设备传送的图像是按分组格式接收的。 [0427] similar to the image transferred to the remote video conference apparatus, the image transmitted from the remote video conference apparatus is received by the packet format. 接收器模块3130接收分组,并对分组解包以便重构图像,之后把图像保存在视频缓冲器(例如,图16的缓冲器1616)中以便解码。 Receiver module 3130 receives the packet, and a packet unpacking order to reconstruct an image, then the image stored in the video buffer (e.g., buffer 1616 of FIG. 16) in order to decode. 在一些实施例中,对图像解包涉及除去任何报头,和由分组重构只具有图像数据(可能有大小数据)的比特流。 In some embodiments, the image relates to unpack remove any headers, and a packet having only the reconstructed image data (size of the data may have) a bit stream.

[0428] 媒体传送管理器3135处理从网络接收的反馈数据(例如,单向等待时间、带宽估计比特率、分组丢失数据、往返延迟时间数据,等等),以动态和自适应地调整数据传输的速率(即,比特率)。 [0428] Media manager 3135 to handle the transfer of data received from the network (e.g., one-way latency, bandwidth estimation bit rate, data packet loss round-trip delay time data, etc.) to dynamically and adaptively adjusts the data transmission the rate (i.e., bit rate). 在一些其它实施例中,媒体传送管理器3135还根据处理后的反馈数据控制容错,还可把反馈数据发送给视频会议管理器1604,以便调整视频会议模块1602的其它操作,比如比例缩放、大小调整和编码。 In some other embodiments, a media manager 3135 also controls the transmission based on feedback fault tolerant data processing, it may send feedback data to the video session manager 1604 to adjust the video conference module 1602 other operations, such as scaling, size adjustment and coding. 除了当会议中的远程设备不能处理所有分组时,使通用传输缓冲器丢弃分组之外,视频会议模块和编码器能够使用较低比特率对图像编码, 使得对每个图像来说将发送较少的分组。 Except when the remote conference apparatus can not process all packets, so the packet is dropped outside the common transmission buffer, and a video conference module encoder can use a lower bit rate coding, such that each image is transmitted less grouping.

[0429] 在一些实施例中,媒体传送管理器3135还可监控设备的其它变量,比如会影响如何配置照相机的工作功率模式的功耗和热级,如上所述。 [0429] In some embodiments, a media manager 3135 may also transmit other variables monitoring equipment, such as power consumption can affect how to configure the operational power mode and the heat level of the camera, as described above. 该数据还可被用作加入反馈数据中的附加输入(例如,如果设备正变得过热,那么媒体传送管理器3135可设法使处理慢下来)。 The data may also be added as an additional input feedback data (e.g., if the device is overheated, the media transport manager 3135 may seek to slow down the process).

[0430] 现在参考图16,说明连网管理器3100的几个示例操作。 [0430] Referring now to Figure 16, illustrate several examples of operation of the networking manager 3100. 首先说明把由双照相机移动设备的照相机拍摄的图像传送给视频会议中的远程设备,之后说明从远程设备接收图像。 First, the image captured by the camera transmits the dual camera mobile device to the remote video conference equipment, after receiving the image description from the remote device. 发射器模块3115从缓冲器1612取回将被传送给视频会议中的远程设备的编码图像。 Transmitter module 3115 retrieves the encoded images are transmitted to the remote video conference apparatus 1612 from the buffer.

[0431] 协议管理器3110确定要使用的恰当协议(例如,传送音频和视频的RTP),会话协商管理器3105把该协议通知给发射器模块3115。 [0431] The protocol manager 3110 determines the appropriate protocol (e.g., transmitting audio and video RTP) to be used, the session negotiation protocol manager 3105 notifies the transmitter module 3115. 之后,发射器模块3115对图像进行分包, 并把分包的图像发送给通用传输缓冲器3120。 Thereafter, the transmitter module 3115 pairs of sub-images, and transmits the packetized transmission buffer 3120 to a general image. 通用传输缓冲器管理器3122从会话协商管理器3105接收指令,以命令通用传输缓冲器3120传送或丢弃图像。 Universal transmission buffer manager 3122 receives an instruction from the session negotiation manager 3105, generic command transmission buffer 3120 to transmit or discard the image. VTP管理器3125从通用传输缓冲器3120接收分组,并处理分组,以便通过单一通信通道把分组传送给远程设备。 VTP manager 3125 receives the packet from the common transmission buffer 3120, and processing packets for the packet transmitted to the remote devices via a single communication channel.

[0432] 当从远程设备接收图像时,VTP管理器3125通过虚拟化的单一通信通道,从远程设备接收分包的图像,并处理分组以便通过分配用来接收图像的通信通道(例如,视频通信通道),把图像引导到接收器模块3130。 [0432] When, the VTP virtualization manager 3125 through a single communication channel, the receiving apparatus receives an image from the remote device from the remote sub-image, and packets for processing to an image received by the communication channel allocation (e.g., video communication channel), the image is directed to the receiver module 3130.

[0433] 接收器模块3130对分组解包以重构图像,并把图像发送给缓冲器1616,以便由解码器1660解码。 [0433] The receiver module 3130 to reconstruct an image, and transmits the image packet depacketizer to the buffer 1616, a decoder 1660 for decoding. 接收器模块3130还把控制信令消息转发给媒体传送管理器3130 (例如, 来自视频会议中的远程设备的接收分组的确认)。 Receiver module 3130 also forwards control signaling message to the media transport manager 3130 (e.g., acknowledgment packets received from the remote video conference device).

[0434] 上面说明了连网管理器3100的几个示例操作。 [0434] The above description of several exemplary networking manager 3100 to operate. 这些只是说明性的例子,因为各种其它实施例将利用不同的模块,或者在各种功能不同地分布在模块之间的情况下,执行这些操作或不同的操作。 These examples are illustrative only, because various other embodiments will utilize different modules, or different in the various functions distributed among the modules in the case, these actions or perform different operations. 此外,连网管理器3100的模块或者其它模块可执行另外的操作,t匕如动态比特率调整。 In addition, the networking manager module or other modules 3100 may perform additional operations, t dagger The dynamic bit rate adjustment.

[0435] IV.会议中调整和控制操作 [0435] IV. Conference adjustment and control operations

[0436] A.画中画修改 [0436] A. Modify PIP

[0437] 1.锁定到角落(snap-to-corner) [0437] 1. Lock the corner (snap-to-corner)

[0438] 本发明的一些实施例允许双照相机移动设备的用户通过来回移动构成合成显示的一个或多个显示区,修改显示在设备上的合成显示。 [0438] Some embodiments allow a user double camera mobile device according to the present invention comprises one or more display regions displayed by the synthesized moving back and forth, change the display on the display device in the synthesis. 一个这样的例子是来回移动PIP显示的插入显示区。 One such example is inserted into the display area of ​​the back and forth movement of the PIP display. 图32图解说明在视频会议期间执行的这样一个例子。 FIG 32 illustrates an example of such a performed during a video conference. 在视频会议中,由于各种原因,比如当前景插入显示区遮挡了背景显示区的感兴趣区域,用户可能想要移动前景插入显示区。 In a video conference, for various reasons, such as when the foreground inset display area obscured by the background display area of ​​the region of interest, the user may want to move the foreground inset display area.

[0439] 图32参考Π 3205的五个不同阶段3210、3215、3220、3225和3230,图解说明插入显示区3240在设备的Π 3205中的移动。 [0439] FIG. 32 with reference to five different stages Π 3205 3210,3215,3220,3225 and 3230, illustrates the insertion movement of the display area 3240 in Π 3205 apparatus. 第一阶段3210图解说明在该设备的本地用户和远程设备的远程用户之间的视频会议期间的Π 3205。 The first stage 3210 illustrates Π 3205 during a video conference between the local user and remote user equipment of the remote device. 图32中的Π 3205显示与在开始视频会议之后,在图11的第五阶段中显示的PIP显示相同的PIP显示。 Π 3205 in FIG. 32 after the start of displaying a video conference, a PIP display in the fifth stage of FIG. 11 show the same PIP display. 在这个例子中,由本地用户的设备拍摄的视频被显示在插入显示区3240中,由远程用户的设备拍摄的视频被显示在背景显示区3235中。 In this example, the video captured by the local user device is displayed in the inset display area 3240, the video captured by the remote user device 3235 is displayed in the background display area. 如图所示,显示区1155包括用于结束视频会议的可选Π项目3245。 As shown, the end of the display area 1155 includes a selectable item Π video conference 3245. 在一些实施例中,显不区1155的布局与上面说明的图12的显不区1155的布局相同。 In some embodiments, the region is not substantially the same region without significant layout 1155 of FIG. 12 and 1155 of the layout described above.

[0440] 第二阶段3215图解说明用户通过选择插入显示区3240,启动锁定到角落操作。 [0440] The second stage 3215 illustrates the user display area 3240 by selecting the insert, to a corner lock operation starts. 在这个例子中,通过把手指3255放在插入显示区3240内的任意地方,实现选择。 In this example, by inserting a finger in 3255 anywhere within the display area 3240, implementation choices. 如图所示, 用插入显示区3240的粗边框3260,显示这种选择。 As shown, the display area with the insertion of the thick border 32603240, show that this selection. 不同的实施例可用不同的方式指出这样的选择,比如通过突出显示区3240,通过使显示区3240摆动等等。 Various embodiments may be indicated in different ways such selection, such as by highlighting the region 3240, the display section 3240 by swinging the like.

[0441] 第三阶段3220图解说明在用户开始把PIP显示3250的插入显示区3240从PIP显示3250中的一个区域移动到该PIP显示3250中的另一个区域之后的Π 3205。 [0441] In the third stage 3220 illustrates the user begins to insert the PIP display area 3250 of the display 3240 is a region 3250 is moved to the PIP display area Π 3205 3250 after another from the PIP display. 在这个例子中,插入显示区3240已开始从PIP显示3250的左下角移动到PIP显示3250的右下角,如箭头3265所示。 In this example, the display area 3240 are inserted starting from the bottom left corner of the PIP display 3250 moves to the lower right corner of the PIP display 3250, as shown by arrow 3265. 在这个例子中,在第二阶段3215中选择插入显示之后,用户通过朝着PIP 显示3250的右下角拖动其手指3255,移动插入显示区3240。 In this example, after selecting the inset display 3215 in the second stage, the lower right corner of the PIP display toward the user by dragging their finger 3250 3255, 3240 to move the insertion display area. 一些实施例提供在PIP显示3250中来回移动插入显示区3240的其它技术。 Some embodiments provide the PIP display 3250 is inserted moves back and forth in the display area 3240 other techniques.

[0442] 第四阶段3225图解说明在用户从设备3200的屏幕上移开其手指3255之后的状态下的Π 3205。 [0442] The fourth stage 3225 illustrates Π 3205 in the state after the user removes his finger from the screen 3255 of the device 3200. 在这种状态下,插入显示区3240仍然朝着根据在第三阶段3220中的用户手指移动而被识别的PIP显示3250的右下角移动。 In this state, the inset display area 3240 displays the lower right corner is still moving toward the user's finger 3250 in the third stage 3220 movement of identified PIP. 换句话说,在手指3255开始朝着PIP 显示3250的右下角移动插入显示区3240之后,即使移开了手指3255, UI 3205仍然保持这种移动。 Other words, after 3255 began to move the fingers moving lower right corner of the PIP display 3250 of the inset display area 3240, even removed the finger 3255, UI 3205 remains such movement. 为了保持所述移动,在用户移开其手指3255之前,一些实施例的Π 3205要求用户的拖动操作大于特定的阈值量(例如,大于特定的距离,或者长于特定的时间);否则,这些实施例会在稍微移动插入显示区3240之后,使插入显示区3240保持在其初始的左下角位置,或者根本不移动插入显示区3240。 In order to maintain the movement, before the user removes his finger 3255, some of the embodiments of Π 3205 requires the user's drag operation greater than a certain threshold amount (e.g., greater than a certain distance or longer than a certain time); otherwise, the after the embodiments will slightly moved display area 3240 is inserted, the insertion display area 3240 remains in its original position of the lower left corner, or not move the inset display area 3240.

[0443] 不过,尽管即使用户在插入显示区到达其新位置之前,停止了他的拖动操作,一些实施例仍然允许插入显示区继续移动,不过其它实施例要求用户保持拖动操作,直到插入显示区到达其新位置为止。 [0443] However, although even before the user inserted a new display area has reached its position, stopped his drag operation, some embodiments permit insertion still continues to move the display area, although other embodiments require the user to maintain a drag operation, until the insertion display area reaches its new location. 一些实施例提供移动插入显示区的其它技术。 Some embodiments provide mobile insertion display area other techniques. 例如,一些实施例要求用户在插入显示区3240实际开始移动之前,指定要把插入显示区3240引导到何处, 等等。 For example, some embodiments require the user actually starts moving before the 3240 insertion display area, display area 3240 is inserted should specify where to boot, and the like. 一些实施例还通过仅仅以不同角度倾斜移动设备,使显示区可以滑动和锁定到角落。 Some embodiments are also inclined at different angles by simply mobile device, the display area may slide and lock into the corner.

[0444] 第五阶段3230图解说明在插入显示区3240到达其位于PIP显示3250的右下角的新位置之后的Π 3205。 [0444] In the fifth stage 3230 illustrates the insertion region 3240 reaches a display positioned after the PIP display Π 3205 3250 new lower-right corner. 在第五阶段3230中,粗边框3260的消除指示锁定到角落操作已完成。 In the fifth stage in 3230, eliminating the thick border 3260 indicates the lock into the corner is complete.

[0445] 为了使在上述第三、第四和第五阶段3220、3225和3230中图解说明的移动更容易,一些实施例的Π 3205采用一旦用户使插入显示区朝着PIP显示3250的某个角落移动,就允许插入显示区快速锁定到该角落的锁定规则(snapping rule)。 [0445] Once the user in order to make the insertion of a PIP display area 3250 toward the third, fourth and fifth stage moving 3220,3225 and 3230 illustrated in the description easier Π above-described embodiments 3205, corner movement inset display area allows fast locking to the rules of the locking corner (snapping rule). 例如,当用户朝着特定角落拖动插入显示区3240超过阈值量时,一些实施例的Π 3205识别插入显示区3240 的运动方向,确定所述运动已超过阈值量,从而随后自动移动插入显示区3240,而不用用户进一步输入插入显示区3240可被锁定到的Π 3205中的下一个网格点。 For example, when the user drags a particular corner of the insert toward the display area 3240 exceeds a threshold amount, the identification number Π 3205 Example insertion movement direction of the display area 3240, determining the amount of motion exceeds a threshold, for subsequent inserted automatically move the display area 3240, without further user input to the next grid points of the display area 3240 may be inserted to be locked in Π 3205. 在一些实施例中, 为锁定插入显示区3240而提供的唯一网格点是位于PIP显示3250的四个角落的网格点。 In some embodiments, the only grid point locking insert provided display area 3240 is located in the four corners of the PIP display grid points 3250. 其它实施例提供插入显示区3240可被锁定到的Π 3205中(例如,PIP显示3250中)的其它网格点(即,插入显示区3240的侧边或顶点可被置于其上或者与之对齐的其它网格点)。 Other embodiments provide other grid points is inserted into the display area 3240 may be locked to the Π 3205 (e.g., the PIP display 3250) (i.e., the vertices or sides of the inset display area 3240 may be placed on or in connection with other alignment grid points).

[0446] 另外的实施例可以不采用网格点,使得插入显示区可被置于PIP显示3250中的任意点。 [0446] Further embodiments may not employ a grid point, so that the insert can be placed in the display area at any point in the PIP display 3250. 再另外的实施例提供允许用户打开或关闭UI的锁定到网格点特征。 Yet other embodiments provide a UI allowing a user to open or close to the grid point lock features. 此外,除了从设备拍摄的视频之外,不同的实施例可允许用户对各种项目,比如图标等执行锁定到角落操作。 Further, in addition to the video capture device from, different embodiments may allow the user to various items, such as icons, performing the locking operation into the corner.

[0447] 图33图解说明Π 3205中的锁定到角落操作的另外两个例子3330和3335。 [0447] FIG. 33 illustrates a Π 3205 is locked to the corner of the other two examples of operation 3330 and 3335. 这些另外的锁定到角落操作显示根据用户的垂直或对角拖动操作,在PIP显示3250中被垂直或对角移动的插入显示区3240。 These additional lock into the corner of the user operation shows a vertical or diagonal drag operation, the PIP display is vertically inserted in the display area 3250 or 3240 of the angular movement.

[0448] 即使图32和33图解说明了插入显示区在PIP显示内的移动,不过普通技术人员会认识到其它实施例可以利用类似的技术在其它类型的PIP显示或其它类型的合成显示中移动显示区。 [0448] Even FIGS. 32 and 33 illustrate the insertion display area is moved in the PIP display, but of ordinary skill in the art will recognize that other embodiments may utilize similar techniques displayed in other types of PIP composite display or other types of mobile display Area. 例如,如下进一步所述,一些实施例的PIP显示具有两个或者更多的前景插入显示,通过利用与上面参考图32和33说明的技术类似的技术,能够在PIP显示中移动这些插入显示。 For example, as described further below, some of the embodiments of the PIP display having two or more foreground inset display, by using the above with reference to FIGS. 32 and 33 similar to the technique described in the art, can be moved in the PIP display displays these inserts. 另外,一些实施例利用类似的技术在合成显示中来回移动显示区(例如,通过用户拖动移动,把一个显示区从屏幕的左侧移动到屏幕的右侧)。 In addition, some embodiments utilize similar techniques moved back and forth in the display area in composite display (e.g., by dragging a user moves to a display area of ​​the screen moves from left to right of the screen). 此外,合成显示的一个或多个显示区的移动会导致双照相机移动设备的图像处理操作的变化,比如响应用户的输入,使视频会议管理器1604重新合成合成显示中的显示区。 Further, moving the one or more display area lead to variations in the composite display image processing dual camera mobile device operation, such as in response to user input, so that the video conference manager 1604 resynthesized display area in composite display. 如下进一步所述,一些实施例采用当第二显示区被从第三位置移动到第一位置时,把第一显示区从第一位置推开的锁定和推送技术。 As further described below, some embodiments employ when the second display area is moved from the third position to the first position, the first display area from the first open position and the locking push technology.

[0449] 2.旋转 [0449] 2. Turn

[0450] 当用于视频会议的移动设备的用户在会议期间旋转所述移动设备时,一些实施例旋转在视频会议期间呈现的PIP显示。 [0450] When the mobile device user for a video conference session during rotation of the mobile device, some embodiments of rotation during a video conference embodiment presented PIP display. 图34图解说明当使设备3400从垂直位置旋转到水平位置时,设备3400的Π1105的旋转。 FIG. 34 illustrates the time when the device 3400 is rotated from a vertical position to a horizontal position, the rotation of the device 3400 Π1105. 当屏幕的长边是垂直的时,设备3400被坚握,而当屏蔽的长边是水平的时,设备3400被横握。 When the screen is perpendicular to the long side of the device 3400 grip is firm, and when the long side of the shield is horizontal, the device 3400 is horizontal grip. 在图34中图解说明的例子中,UI 1105从为设备的坚握而优化的纵向视图旋转成为设备3400的横握而优化的横向视图。 In the example illustrated in FIG. 34 in, UI 1105 from the firm grip of the longitudinal view of the device be optimized rotary optimized lateral view of the device 3400 held horizontally. 这种旋转功能使得当移动设备3400被坚握或横握时,用户都可以观看以直立位置显示的Π 1105。 This rotation is such that when the mobile device 3400 firm grip or lateral grip, the user can view Π 1105 displayed in an upright position.

[0451] 图34按照六个不同的操作阶段3410、3415、3420、3425、3430和3435,图解说明Π 1105的旋转。 [0451] FIG. 34 according to six different operating phases 3410,3415,3420,3425,3430 and 3435, illustrates the rotation of Π 1105. 第一阶段3410图解说明在该设备的本地用户和远程设备的远程用户之间的视频会议期间的Π1105。 The first stage 3410 illustrates Π1105 during a video conference between a local user of the remote user device and the remote device. 图34中的Π 1105显示与在建立视频会议之后,在图11的第五阶段中显示的PIP显示相同的PIP显示1180。 Π 1105 Figure 34 displays after the establishment of video conferencing, PIP display in the fifth stage of Figure 11 shows the same PIP display 1180. 在这个例子中,本地用户的设备拍摄的视频显示在插入显示区1160中,远程用户的设备拍摄的视频显示在背景显示区1170中。 In this example, the local user of video equipment is displayed in the shooting area inset display 1160, the video captured by the remote user device 1170 displayed in the display area in the background. 在PIP 显示1180之下的显示区1155中包括用户可选择以结束视频会议(例如,通过单指轻点) 的可选Π项目3485 (例如,"结束会议(End Conference) "按钮3485)。 Under the display area 1180 includes a user may select 1155 to end the videoconference (e.g., by a single finger tap) Π optional item 3485 (e.g., the button "end of the session (End Conference)" 3485) in the PIP display.

[0452] 第二阶段3415图解说明在用户开始向一边倾斜设备3400之后的Π 1105。 [0452] 3415 illustrates the second stage begins after the user tilts Π 1105 to the device 3400 side. 在这个例子中,用户开始把设备3400从被坚握倾斜到被横握,如箭头3460所示。 In this example, the user starts the device 3400 to be inclined from the horizontal grip firm grip, as indicated by arrow 3460 in FIG. Π 1105的外观没有变化。 Π 1105 appearance has not changed. 在其它情况下,用户可能改为想把设备3400从横握倾斜到被坚握,在这些情况下,Π1105从水平优化的视图切换成垂直优化的视图。 In other cases, the user may want to grip the device 3400 from Kennedy are inclined to the horizontal grip, in these cases, Π1105 view from the horizontal to optimize handover optimization in a vertical view.

[0453] 第三阶段3420图解说明在设备3400已从被坚握倾斜到被横握之后的状态下的Π 1105。 [0453] In the third stage 3420 illustrates the apparatus 3400 from being held inclined to Π 1105 is in a state after the firm lateral grip. 在这种状态下,Π 1105的外观仍然没有变化。 In this state, Π appearance of 1105 remains unchanged. 在一些实施例中,在使设备3400倾斜超过阈值量并保持超过阈值量一段时间之后,触发旋转操作。 In some embodiments, when the apparatus 3400 is inclined beyond a threshold amount and remains after a period of time exceeding a threshold amount, rotational operation is triggered. 在图34中图解说明的例子中,假定阈值量和旋转速度不会导致Π 1105旋转,直到设备已放置在水平位置之后的较短时间间隔为止。 In the example illustrated in FIG. 34 illustrates, it is assumed threshold amount and the rotation speed does not cause Π 1105 rotated until the device has been placed in a horizontal position after a short time interval so far. 不同的实施例具有用于触发旋转操作的不同阈值量和等待时间。 Different embodiments having different threshold values ​​for triggering the amount of rotational operation and the waiting time. 例如,一些实施例可具有如此低的触发旋转操作的阈值,使得不管设备3400的定向,都使Π 1105 显得好像它总是以直立位置被显示似的。 For example, some embodiments may have a threshold to trigger such a low rotation operation, so that regardless of the orientation of the device 3400, so that all Π 1105 appear as if it is always displayed in an upright position like. 在其它实施例中,设备3400的用户可以规定何时可以触发旋转操作(例如,通过菜单优先选择设置)。 In other embodiments, the device 3400 may be triggered when the user can specify the rotational operation (e.g., setting preferences menu). 另外,一些实施例可在设备被倾斜超过阈值量之后,不延迟所述旋转。 Further, some embodiments may be after the device is tilted more than a threshold amount, the rotation is not delayed. 此外,不同的实施例可以允许用不同的方式触发旋转操作,比如通过切换移动设备上的开关、通过发出话音命令、依据通过菜单进行的选择,等等。 Further, different embodiments may allow rotating operation triggered in different ways, such as by the changeover switch on the mobile device, by issuing a voice command, menu selection by the basis, and the like.

[0454] 第四阶段3425图解说明在开始旋转操作之后的Π 1105。 [0454] The fourth stage 3425 illustrates Π 1105 after the start of the rotating operation. 一些实施例动画演示旋转显示区,以向用户提供关于旋转操作的反馈。 Some embodiments rotate the display area animation embodiment, to provide feedback to the user about the rotational operation. 图34图解说明这样的一种动画的例子。 FIG 34 illustrates an example of such an animation. 具体地说,图34在其第四阶段3425表现显示区1180和1155 -起开始旋转。 Specifically, FIG. 343,425 at the fourth stage performance display area 1180 and 1155-- from start to rotate. 显示区1180和1155围绕经过Π 1105的中心的轴3465(即,z轴)旋转。 Display area 1180 and 1155 about axis 3465 through the center of the Π 1105 (i.e., z-axis). 显示区1180和1155被旋转相同的数量,不过沿与设备3400的旋转(例如,通过设备3400的倾斜)相反的方向旋转。 Display area 1180 and 1155 are rotated by the same amount, but the direction of rotation of the device 3400 (e.g., by tilting the device 3400) rotate in opposite directions. 在这个例子中,由于设备3400沿顺时针方向旋转了90° (通过从坚握变成横握),因此旋转操作会使显示区1180和1155沿逆时针方向旋转90°。 In this example, since the device 3400 is rotated clockwise 90 ° (from the firm grip by grip becomes horizontal), so that the rotation action causes display areas 1180 and 1155 is rotated counterclockwise 90 °. 当显示区1180和1155旋转时,显示区1180和1155按比例地缩小,以适合Π 1105,使得显示区1180和1155仍然可以完全出现在Π 1105上。 When rotating the display area 1180 and 1155, display areas 1180 and 1155 reduced proportionally to fit Π 1105, 1180 and 1155 so that the display area can be completely still appear on Π 1105. 一些实施例可提供指示设备3400的状态的消息(例如,通过显示字词"Rotating(旋转)")。 Some embodiments may provide an indication of a device status message 3400 (e.g., by displaying the word "Rotating (rotation)").

[0455] 第五阶段3430图解说明在显示区1180和1155从纵向视图到横向视图逆时针旋转90°之后的Π 1105。 [0455] The fifth stage 3430 illustrates a display region 1180 and from 1155 to Π 1105 after longitudinal view of a lateral view rotated counterclockwise by 90 °. 在这个阶段,显示区1180和1155已被旋转,不过还未展开到Π 1105的整个宽度。 At this stage, the display area 1180 and 1155 have been rotated, but has not yet commenced the entire width of Π 1105. 箭头3475指出在第五阶段的结尾,显示区1180和1155将开始侧向展开,以适合Π 1105的整个宽度。 3475 arrow pointed at the end of the fifth stage, the display area 1180 and 1155 will begin to spread laterally to fit the entire width of Π 1105. 不同的实施例可不包括该阶段,因为可以与第四阶段3425 中的旋转同时地进行所述展开。 Different embodiments may not include this stage, since the unfolding may be performed simultaneously with the rotation of the fourth stage 3425.

[0456] 第六阶段3435图解说明在显示区1180和1155已被展开,从而占据Π 1105的整个显示之后的Π 1105。 [0456] 3435 illustrates the sixth stage 1180 and 1155 have been expanded in the display area, so as to occupy the entire display Π 1105 after the Π 1105. 如上所述,其它实施例可以不同地实现这种旋转。 As described above, other embodiments may be implemented differently this rotation. 对一些实施例来说,仅仅把设备的屏幕旋转超过阈值量就可触发所述旋转操作,而不管设备3400的定向。 For some embodiments, the only the screen of the device beyond a threshold amount of rotation of the rotating operation can be triggered, regardless of the orientation of the device 3400.

[0457] 另外,其它实施例可提供用于指示旋转操作的不同动画。 [0457] Further, other embodiments may provide different animations for indicating a rotational operation. 在图34中执行的旋转操作涉及显示区1180和1155围绕Π 1105的中心旋转。 Rotating operation performed in FIG. 34 relates to the display region 1180 and 1155 around the center of rotation of Π 1105. 另一方面,可以使显示区围绕它们各自显示区的中心轴单独旋转。 On the other hand, the display area can display about their respective central axes of rotation of the individual regions. 图35中示出了一种这样的方法。 Figure 35 illustrates one such method. 图35表示动画演示Π 1105的PIP显示1180的显示区1170和1160的旋转的备选方法。 35 shows an alternative method of rotating a display region 1180 and 1170 1160 Π PIP 1105 animation display. 图35中图解说明的PIP 显示1180和图11中图解说明的PIP显示1180相同。 PIP 35 illustrated in FIG PIP display 1180 and FIG. 11 illustrates the same display 1180.

[0458] 图35按照六个不同的操作阶段3410、3415、3420、3525、3530和3535,图解说明PIP显示1180的旋转。 [0458] FIG. 35 according to six different operating phases 3410,3415,3420,3525,3530 and 3535, illustrates the rotation of the PIP display 1180. UI 1105的前三个阶段的操作与如在图34中的Π 1105中说明的前三个阶段的操作相同。 The first three stages of operation as UI 1105 as illustrated in FIG. 34 Π 1105 in the first three stages of the same. 在图34和35的第三阶段,设备3500都已从坚握变成横握,并且Π 1105的旋转还未开始。 In the third stage of FIG. 34 and 35, from the device 3500 are held horizontally into firm grip and rotation Π 1105 has not yet started.

[0459] 第四阶段3525图解说明动画演示所述旋转的备选方法。 An alternative method of animation 3525 illustrates the rotation of [0459] the fourth stage. 在第四阶段,旋转操作已开始。 In the fourth stage, the rotating operation has begun. 具体地说,第四阶段3525表现显示区1170和1160的旋转的开始。 Specifically, the fourth stage 3525 starts to rotate the display performance of 1170 and 1160. 显示区1170和1160分别围绕穿过每个显示区的中心的轴3567和3565(即,z轴)旋转。 Display area 1170 and 1160, respectively, around an axis through the center of each of the display areas 3567 and 3565 (i.e., z-axis). 显示区1170和1160被旋转相同的量,不过沿与设备3500的旋转(例如,通过设备3500的倾斜)相反的方向旋转。 Display area 1170 and 1160 are rotated by the same amount, but the direction of rotation of the device 3500 (e.g., by tilting the device 3500) rotation in the opposite direction. 与上面在图34的第四阶段3425中图解说明的类似,由于设备3500沿顺时针方向旋转了90° (通过从坚握变成横握),因此旋转操作会使显示区1170和1160沿逆时针方向旋转90°。 Similar to FIG. 34 described above in the fourth stage illustrated in 3425, since the device 3500 is rotated clockwise 90 ° (from the firm grip by grip becomes horizontal), so that the rotation action causes the display region 1170 and 1160 in the reverse clockwise rotation of 90 °. 当显示区1170和1160旋转时,显示区1170和1160按比例缩小以适合Π 1105,使得显示区1170和1160仍然可以完全出现在Π 1105上。 When rotating the display area 1170 and 1160, display areas 1170 and 1160 scaled down to fit Π 1105, 1170 and 1160 so that the display area can be completely still appear on Π 1105.

[0460] 第五阶段3530图解说明在显示区1170和1160都从纵向视图到横向视图逆时针旋转90°之后的Π 1105。 [0460] In the fifth stage 3530 illustrates a display region 1170 and 1160 are the view from portrait to landscape view Π 1105 after rotated counterclockwise 90 °. 在这个阶段,显示区1170和1160已被旋转,不过还未展开到UI 1105的整个宽度。 At this stage, the display area 1170 and 1160 have been rotated, but has not yet commenced the entire width of the UI 1105. 此外,显示区1160还未被移动到其最终位置。 Further, the display region 1160 has not been moved to its final position. 插入显示区1160在PIP 显示1180中的最终位置由如在第一阶段3410中所示的插入显示区1160在PIP显示1180 中的位置确定(例如,插入显示区1160在PIP显示1180的左下角)。 Display area 1160 is inserted in the PIP display 1180 as the final position of the insert shown in display area 1160 in the first stage 3410 1180 PIP display position determination (e.g., display area 1160 displays inserted in the lower left corner 1180 PIP) . 在这个阶段,插入显示区1160仍然在Π 1105的左上角。 At this stage, the inset display area 1160 is still in the upper left corner Π 1105.

[0461] 箭头3580指出在第五阶段3530的结尾,显示区1170和1160将开始侧向展开,直到主显示区1170适合横握的设备的Π 1105的整个宽度为止。 [0461] 3580 indicated by arrow 3530 at the end of the fifth stage, the display area 1170, and 1160 will begin to expand laterally, until the main display area 1170 for lateral grip device 1105 until the entire width of Π. 此外,箭头3575指出插入显示区1160将滑动到PIP显示1180的左下角。 Further, indicated by arrow 3575 is inserted into the slide display area 1160 displays the lower left corner 1180 of the PIP.

[0462] 不同的实施例可不同地实现这一点。 [0462] The various embodiments may be implemented differently from this. 在一些实施例中,插入显不区1160的移动可以和主显示区1170的展开同时发生,或者可以顺序发生。 In some embodiments, not significantly move the insertion region 1160 may expand and simultaneously the main display area 1170, or may occur sequentially. 此外,一些实施例可以在主显示区1170展开前、展开中或展开后缩放插入显示区1160,以产生新的PIP显示1180。 Further, some embodiments may be deployed before the main display area 1170, expand or expand the scale display area 1160 is inserted, to produce a new PIP display 1180. 在这个例子中,在显示区1160和1170旋转时,显示区1155消失。 In this example, when the rotation display area 1160 and 1170, the display area 1155 disappears. 不过,在一些实施例中,显示区1155可在旋转期间保留在Π 1105上,并和显示区1160和1170-起旋转。 However, in some embodiments, the display area 1155 may be retained during the rotation of the Π 1105, and 1160 and 1170 and display area from rotating.

[0463] 第六阶段3535图解说明在插入显示区1160到达其新位置,并且显示区1160和1170已被恰当展开,以适合Π 1105的整个宽度之后的Π 1105。 [0463] In the sixth stage 3535 illustrates the insertion display area 1160 reaches its new position, and display areas 1160 and 1170 have been properly expanded to fit the entire width Π 1105 after the Π 1105. 在这个例子中,插入显示区1160现在位于PIP显示1180的左下角,重叠在主显示区1170上。 In this example, the inset display area 1160 are in the lower left corner 1180 of the PIP display, superimposed on the main display area 1170. PIP显示1180现有具有和第一阶段3410的PIP显示1180相同的显示排列。 PIP display 1180 and the first stage 3410 having a conventional PIP display the same display array 1180. 第六阶段中在PIP显示1180之下出现的显示区1155指示旋转操作已完成。 In the sixth stage display area 1180 appears below 1155 indicates a rotational operation has been completed in PIP. 如上所述,仅仅把设备的屏幕旋转超过阈值量就可触发所述旋转操作,而不管设备3500的定向。 As described above, only the screen of the device beyond a threshold amount of rotation of the rotating operation can be triggered, regardless of the orientation of the device 3500.

[0464] 在上面参考图34和35说明的例子中,显示区1170的定向也改变(S卩,从纵向变成横向)。 [0464] In the examples 35 and explained above with reference to FIG. 34, the display area 1170 also changes the orientation (S Jie, from the longitudinal into the lateral). 即,在第三阶段3420中旋转显示区1170之后,通过水平展开PIP显示1180,使得它适合整个Π 1105,显示区1170的定向从纵向变成横向。 That is, the third stage 3420 is rotated in the display area after 1170, 1180 by the PIP display spread horizontally, so that it is suitable for the entire Π 1105, a display area 1170 from the longitudinal orientation into the transverse direction. 在一些实施例中,当设备3500被旋转时,由远程设备拍摄的视频旋转,不过显示由远程设备拍摄的视频的显示区的定向保持不变。 In some embodiments, when the device 3500 is rotated, the video captured by the remote rotation, but the display orientation of the display area of ​​video captured by a remote device remains unchanged. 图36中图解说明了一个这样的例子。 FIG 36 illustrates one such example. 图36类似于图35,除了显示在显示区1170 中的视频旋转,不过显示区1170保持沿纵向方向显示。 FIG 36 is similar to FIG. 35, in addition to displaying the video display area 1170 is rotated in, but the longitudinal direction of the holder 1170 display area.

[0465] 图36中还图解说明了其中显示区1155保持在相同位置(而不是如图35中所示那样旋转并且水平展开,以填满PIP显示1180)的旋转操作的例子。 [0465] Figure 36 also illustrates a region 1155 in which the display is maintained at the same position in the rotating operation example (not shown in FIG. 35 as the rotating and horizontally expand to fill the PIP display 1180) a. 此外,图36包括与上面在图12中说明的显示区1155的布局相同的显示区1155的布局。 Further, FIG. 36 described above includes a display region in the layout of FIG. 12 the same layout of the display area 1155 1155. 如图所示,在阶段3640、 3645、3650、3655、3685和3690中,当设备3500旋转时,显示区1155保留在相同位置。 As shown, at stage 3640, the 3645,3650,3655,3685 and 3690 when the device 3500 is rotated, the display area 1155 remains in the same position.

[0466] 一些实施例提供一种旋转操作,其中显示由本地设备拍摄的视频的显示区的定向改变(而不是如图35中所示那样保持相同的定向),以反映在对本地设备进行旋转操作之后本地设备的定向。 [0466] Some embodiments provide a rotating operation, wherein the display changes the orientation of the display area of ​​video captured by the local device (not maintain the same orientation as shown in FIG. 35), is rotated to reflect the local device local orientation of the device after the operation. 图36参考六个不同的阶段3640、3645、3650、3655、3685和3690,图解说明Π1105的这种旋转操作的例子。 36 with reference to FIG six different stages 3640,3645,3650,3655,3685 and 3690, illustrates an example of such a rotation operation Π1105. 在图36中,第一阶段3640沿纵向方向显示插入显示区1160,插入显示区1160显示由设备3500的照相机拍摄的视频。 In FIG 36, the display 3640 is inserted into the first stage of the display area 1160 along the longitudinal direction, is inserted into the display area 1160 displays a video captured by the camera device 3500. 第二和第三阶段3645和3650与图35的第二和第三阶段3415和3420相同,因为它们表示设备3500在旋转操作的各个阶段的倾斜。 The third stage and the same second and third stages 3645 and 3650 of FIG. 35 and second 34153420 because they represent the tilting device 3500 at various stages of the rotational operation. 此时,设备3500的照相机在沿横向方向拍摄图像。 In this case, the device 3500 of the captured image in the camera in the transverse direction. 为了指出这种转变, 一些实施例提供如在第四和第五阶段3655和3685中所示的动画,而其它实施例根本不提供任何动画。 To indicate this transition, some embodiments provide embodiments shown in the animation as the fourth and fifth stages 3655 and 3685 in, while other embodiments provides no animation.

[0467] 在第四阶段3655,显示在插入显示区1160中的图像被旋转,不过插入显示区1160 本身不被旋转,因为第二和第三阶段3445和3650中设备3500的倾斜已把插入显示区1160 旋转到横向方向。 [0467] In a fourth stage 3655, the display image is rotated in the display area 1160 is inserted, but inserted into the display area 1160 itself is not rotated, since the second and third phases inclined 3445 and 3650 in the display apparatus 3500 is inserted Yiba region 1160 is rotated to the lateral direction. 在第五阶段3685中,插入显示区1160中的旋转图像被水平展开以填充插入显示区1160,插入显示区1160开始朝着PIP显示1180的左下侧区域移动,以把插入显不区1160置于和插入显不区1160在第一阶段3640的PIP显不中相同的相对位置。 In a fifth stage 3685, the rotated image in the inset display area 1160 is expanded to fill the horizontal display area 1160 is inserted, is inserted into the lower left display area 1160 starts displaying area 1180 moves toward the side of the PIP, is not significant to the insertion region 1160 disposed the same relative positions and do not significantly insertion region 1160 in the first phase is not noticeable in 3640 PIP.

[0468] 在一些实施例中,显示由远程设备拍摄的视频的显示区的定向也变化,以反映在对远程设备进行旋转操作之后远程设备的定向。 [0468] In some embodiments, the display orientation of the display area of ​​video captured by the remote device is also changed to reflect the orientation after rotational operation of the remote device remote device. 图37图解说明设备3500的Π 1105的四个不同阶段,其中(1)显示由本地设备拍摄的视频的显示区(本例中的显示区1160)的定向变化,以反映在对本地设备进行旋转操作之后本地设备的定向,和(2)显示由远程设备拍摄的视频的显示区(本例中的显示区1170)的定向变化,以反映在对远程设备进行旋转操作之后本地设备的定向。 Figure 37 illustrates four different stages of Π 1105 of the device 3500, wherein (1) shows the change in orientation of the display region (display region 1160 in the present embodiment) of the video captured by the local device to reflect the local device rotates directional changes the orientation of the local device after the operation, and (2) displays a video captured by the remote device display area (display area 1170 in this example) to reflect the orientation of the local device to the remote device after a rotation operation.

[0469] 在第一阶段3705,UI 1105和图36中的Π 1105相同。 [0469] In the first stage 3705, in the same UI Π 1105 and 36 1105. 具体地说,第一阶段3705 显示纵向定向的显示区1160和1170,因为设备3500是按纵向定向显示的,以及远程设备是纵向定向的(未示出)。 Specifically, the first stage 3705 portrait orientation display area 1160 and 1170, because the device 3500 is displayed in portrait orientation, and the remote device is longitudinally oriented (not shown). 从第一阶段3705到第二阶段3710,通过把设备3500从直立位置到横向位置旋转90°,对本地设备进行旋转操作。 3705 from the first stage to the second stage 3710, the device 3500 by the rotation from the upright position to the lateral position of 90 °, rotating operation of the local device. 第二阶段3710显示在完成设备3500的旋转操作之后的Π 1105。 The second stage 3710 displays Π 1105 after completion of the rotation of the operation device 3500. 在第二阶段,显示在显示区1170和1160中的视频已旋转到直立位置。 In the second stage, the video displayed in the display region 1170 and 1160 is rotated to an upright position. 不过,只有本地拍摄视频的显示区1160从纵向定向旋转到横向定向,因为只对本地设备(即,设备3500)进行了旋转操作。 However, only a local shooting video display area 1160 is rotated from a portrait orientation to a landscape orientation, because only a local device (i.e., device 3500) in the rotating operation. 显示区1170保持纵向定向。 Display area 1170 remains vertical orientation.

[0470] 从第二阶段3710到第三阶段3715,通过把远程设备从直立位置旋转到横向位置(未示出),对远程设备进行旋转操作。 [0470] 3715, rotational operation of the remote device by the remote device is rotated from an upright position to a lateral position (not shown) from the second stage to the third stage 3710. 第三阶段3715显示在完成远程设备的旋转操作之后的Π 1105。 The third stage 3715 Π 1105 displays operation is completed after the rotation of the remote device. 在第三阶段,显示在显示区1170中的视频和远程拍摄视频的显示区1170从纵向定向旋转到横向定向,因为只对远程设备进行了旋转操作。 In the third stage, and remote video display in the display region 1170 of the shooting video display area 1170 is rotated from a portrait orientation to a landscape orientation, since only the rotating operation of the remote device. 从而,UI 1105的第三阶段均按横向定向显示本地和远程拍摄视频的显示区1170和1160。 Thus, UI 1105 of the third stage are by lateral directional display local and remote video imaging display areas 1170 and 1160.

[0471] 从第三阶段3715到第四阶段3720,通过把设备3500从横向位置到直立位置旋转90°,对本地设备进行旋转操作。 [0471] 3720, a rotation operation on the local device through the device 3500 is rotated 90 ° from the horizontal position to the upright position from the third stage 3715 to the fourth stage. 第四阶段3720显示在完成该旋转操作之后的Π 1105。 The fourth stage display Π 1105 3720 After completion of the rotating operation. 在第四阶段3720,显示在显示区1160和1170中的视频已旋转到直立位置。 In the fourth stage 3720, the video displayed in the display region 1160 and 1170 is rotated to an upright position. 不过,只有本地拍摄视频的显示区1160从横向定向旋转到纵向定向,因为只对本地设备(即,设备3500) 进行了旋转操作。 However, only a local shooting video display area 1160 is rotated from the portrait orientation to the landscape orientation, because only a local device (i.e., device 3500) in the rotating operation. 显示区1170保持横向定向。 Display area 1170 remains oriented transversely.

[0472] 从第四阶段3720到第一阶段3705,通过把远程设备从横向位置到直立位置旋转90° (未示出),对远程设备进行旋转操作。 [0472], rotational operation of the remote device by the remote device is rotated 90 ° from the horizontal position to an upright position (not shown) from the first stage to the fourth stage 3720 3705. 在这种情况下,第一阶段3705显示在完成该旋转操作之后的显示区1170。 In this case, the first display region 3705 Phase After completion of the rotating operation 1170. 于是,该阶段的Π 1105显示纵向定向的显示区1160和1170。 Thus, at this stage Π 1105 portrait orientation display area 1160 and 1170. 尽管图37图解说明了一系列的不同旋转操作,不过其它实施例可以按照任意许多不同序列,进行任意许多旋转操作。 Although FIG. 37 illustrates a number of different rotational operation, but other embodiments in many different embodiments may be any sequence, arbitrarily many rotating operation.

[0473] 图34、35、36和37说明在视频会议期间对本地和远程设备进行的旋转操作。 [0473] FIGS. 34, 35 and 37 illustrates the rotational operation of the local and remote devices during a video conference. 当对本地移动设备进行旋转操作时,一些实施例把旋转操作通知给远程设备,以便让远程设备能对本地设备的视频进行任何修改(例如旋转显示本地设备的视频的显示区)。 When the rotating operation of the local mobile device, some embodiments of the rotary operation notification to the remote device, so that any remote device capable of modifications (e.g. rotating the display area of ​​the video of the local device) to the local video equipment. 类似地,当对远程设备进行旋转操作时,远程设备把该操作通知给本地设备,使本地设备可以对远程设备的视频进行任何修改。 Similarly, when the rotational operation of the remote device, the remote device to the local device notifies the operator, so that the local device can be any modification to the remote video device. 一些实施例提供在视频会议期间,在本地设备和远程设备之间传送旋转操作的通知的控制通信通道。 Some embodiments provide that during the video conference, the transfer control of the communication channel notification rotational operation between the local and remote devices.

[0474] 尽管图34、35、36和37图解说明能够实现旋转的动画的不同方式,不过普通技术人员会认识到其它实施例可以用不同的方式显示旋转的动画。 [0474] Although FIGS. 34, 35 and 37 illustrate different ways to achieve the rotation animation, but of ordinary skill in the art will recognize that other embodiments may display animated rotation in different ways. 另外,旋转操作的动画能够引起本地移动设备的图像处理操作的变化,比如使视频会议管理器1604在Π 1105中以不同的角度重新合成显示区,和比例缩放显示在显示区中的图像。 Further, the rotational operation of the animation can cause local changes in the image processing operation of the mobile device, such that the video conference manager Π 1105 to 1604 in different angles re-synthesis display area, and displaying an image scaled in the display area.

[0475] 3.窗口大小调整 [0475] 3. The window resizing

[0476] 一些实施例允许移动设备的用户调整在视频会议期间呈现的PIP显示的插入显示区的大小。 [0476] Some embodiments allow the user to adjust the size of the mobile device is inserted into the display region during a video conference presentation PIP display. 不同的实施例提供缩放插入显示区的不同技术。 Various embodiments provide different techniques inserted scaling display area. 图38图解说明缩放插入显示区的一种方法。 Figure 38 illustrates a method for scaling a display area is inserted. 在该方法中,移动设备的用户通过选择插入显示区的一个角落,随后扩大或缩小插入显示区,来调整插入显示区的大小。 In this method, the user of the mobile device by selecting a display area of ​​a corner insert, followed by enlarging or reducing the display area is inserted, is inserted to adjust the size of the display area.

[0477] 在图38中,移动设备3825的Π 3800在与另一个移动设备的远程用户的视频会议期间,呈现PIP显示3865。 [0477] In FIG. 38, the mobile device Π 3800 3825 during the remote user device with another mobile video conferencing, 3865 presents a PIP display. PIP显示3865包括两个视频显示:背景主显示区3830和前景插入显示区3835。 PIP display two video display 3865 includes: a main display area 3830 and the background of the foreground display region 3835 is inserted. 背景主显示区3830占据大部分的PIP显示3865,而前景插入显示区3835 较小并重叠在背景主显示区3830上。 BACKGROUND main display area 3830 occupies most of the PIP display 3865, and the foreground and a small inset display area 3835 superimposed on the background main display area 3830. 在这个例子中,背景主显示区3830呈现拿着吉它的人物的视频,该人物被假定为由远程设备的正面照相机拍摄其视频的人物,或者由远程设备的背面照相机拍摄其视频的人物。 In this example, the background main display area 3830 presents holding a guitar figures of the video, the person is assumed by the remote device's front camera shooting video of their character, or captured by the remote device's back camera video of their character. 前景插入显示区3835呈现戴着帽子的人物的视频,在这个例子中,该人物被假定为由本地设备的正面照相机拍摄其视频的人物,或者由本地设备的背面照相机拍摄其视频的人物。 The foreground inset display area 3835 presents a video of a hat person, in this case, the person is assumed by the local device's front camera shooting video of their character, or captured by the local device's back camera video of their characters. 在PIP显示3865下面是包括标记成"End Conference (结束会议)"的可选Π项目3860(例如,按钮3860)的显示区1155,可选Π项目3860允许用户通过选择该项目来结束视频会议。 The following is shown in the PIP 3865 includes a flag to "End Conference (the meeting)" Π selectable item 3860 (e.g., button 3860) a display area 1155, Π selectable item 3860 allows the user to select the item to the end of the video conference.

[0478] PIP显示3865只是呈现由远程设备和本地设备拍摄的视频的合成视图的一种方式。 [0478] PIP display 3865 is only one manner of presenting the synthesized video view captured by a remote device and a local device. 一些实施例可以提供其它的合成视图。 Some embodiments may provide other composite views. 例如,代替具有用于来自远程设备的视频的较大背景显示,较大的背景显示可以是来自本地设备的视频,以及较小的前景插入显示可以是来自远程设备的视频。 For example, instead of having large background video from a remote device for display, a larger background display may be a video from the local device, and a smaller display can be inserted into the foreground video from the remote device. 另外,一些实施例允许本地视频和远程视频出现在Π3800中的两个并排显示区(例如,左右显示窗口,或者上下显示窗口),或者两个对角排列的显示区中。 Further, some embodiments allow the local and remote videos to appear side by side in the two display areas Π3800 (e.g., around the display window, the display window or down), or the two display areas are aligned diagonally. 在一些实施例中,PIP显示的方式或者默认显示模式可由用户规定。 In some embodiments, the PIP display mode or the default display mode specified by the user. 在其它实施例中,PIP 显示还可包含一个较大的背景显示,和两个较小的前景插入显示。 In other embodiments, the PIP display may also include a larger background display, and two smaller foreground inset display.

[0479] 图38按照Π 3800的四个操作阶段,图解说明缩放操作。 [0479] FIG. 38 in terms of four operating phases Π 3800, illustrating a zooming operation. 在第一阶段3805,前景插入显示区3835显著小于背景主显示区3830。 In the first stage 3805, inserted into the foreground display region 3835 is significantly less than the background main display area 3830. 另外在这个例子中,前景插入显示区3835 位于PIP显示3865的右下角。 Also in this case, the foreground display region 3835 is inserted into the lower right corner 3865 of the PIP display is located. 在其它例子中,前景插入显示区3835可具有不同的大小,或者位于PIP显示3865中的不同区域。 In other examples, insertion foreground display region 3835 may have different sizes, or in the PIP display 3865 in different areas.

[0480] 在第二阶段3810,发起缩放操作。 [0480] In the second stage 3810, initiated zoom. 在这个例子中,通过选择用户想要缩放的插入显示区3835的一个角落(例如,通过把手指3840放在插入显示区3835的左上角上),发起该操作。 In this example, by selecting a corner inset display area 3835 of the user you want to zoom (for example, by the insertion of a finger on the 3840 display area on the upper left corner of 3835), to initiate the operation. Π 3800的第二阶段3810用插入显示区3835的粗边框3845指出这种选择。 The second stage Π 3800 3810 3845 Slot thick border display area 3835 indicated this choice. 在这个阶段,用户能够扩大或缩小插入显示区3835 (例如,通过在PIP显示3865上远离插入显示区3835或者朝着插入显示区3835拖动其手指3840)。 At this stage, the user can enlarge or reduce the display area 3835 is inserted (e.g., 3835, or by dragging their finger away from the insertion display area 3865 on display 3835 in the PIP display area 3840 toward the insertion).

[0481] 第三阶段3815图解说明在如箭头3850所示,通过远离插入显示区3835移动其手指3840(即在本例中,通过朝着Π 3800的左上角对角移动他的手指),用户开始扩大插入显示区3835之后的Π 3800。 [0481] In the third stage 3815 as illustrated by arrow 3850, by the display area 3835 away from the insertion moves his finger 3840 (in this example, i.e., toward the upper left corner of Π 3800 by diagonal moves his finger), the user inset display area began to expand after 3835 Π 3800. 另外如箭头3855所示,手指3840的移动在高度和宽度方向按比例地扩大了插入显示区3835。 Also as shown by arrow 3855, the finger 3840 moves proportionally enlarged in height and width directions of the display area 3835 is inserted. 在其它例子中,用户能够利用相同的技术(S卩,通过朝着插入显示区3835拖动手指),缩小插入显示区3835。 In other examples, the user can use the same technology (S Jie, by dragging a finger 3835 toward the insertion display area), reduce the display area 3835 is inserted.

[0482] 第四阶段3820显示在已经完成插入显示区3835的缩放之后的Π 3800。 [0482] The fourth stage 3820 displays Π 3800 3835 after scaling has completed insertion of the display area. 在这个例子中,一旦插入显示区3835已经达到理想的大小,那么用户就通过停止其手指3840的拖动,并从PIP显示3865移开他的手指,完成插入显示区3835的缩放。 In this case, once the inset display area 3835 has reached the desired size, then the user stops by dragging their finger 3840, and from the PIP display 3865 moved his fingers to complete the inset display area Zoom 3835. 作为该过程的结果, 缩放后的插入显示区3835大于其在第一阶段3805中的初始大小。 As a result of this process, the scaled insert display area 3835 is larger than the initial size of the first stage 3805. 粗边框3845的消除指示现在完成了插入显示区缩放操作。 Thick border indicates the 3845 is now complete elimination of the inset display area zoom operation.

[0483] 一些实施例提供允许用户在视频会议期间,在PIP显示3865中缩放插入显示区3835的其它技术。 [0483] Some embodiments allows a user during a video conference, the display 3865 is inserted in the zoom display area 3835 other techniques in PIP. 图39图解说明一种这样的其它技术。 Figure 39 illustrates one other such techniques. 图39图解说明一种通过选择插入显示区3835的一个边缘(即,在插入显示区3835的侧边之一),随后扩大或缩小插入显示区3835,来缩放插入显示区3835的技术。 Figure 39 illustrates a plug an edge of the display area 3835 by selecting (i.e., one side of the insertion display area 3835), followed by insertion enlarged or reduced display region 3835 to display area 3835 scaling technology is inserted.

[0484] 图39按照图38的Π 3800的四个操作阶段,图解说明该缩放操作。 [0484] FIG. 39 in terms of four operating phases Π 3800 of FIG. 38, illustrating the zooming operation. 图39的第一阶段3805和图38中的第一阶段3805相同。 FIG 39 is the same as the first stage of the first stage 3805 and 38 3805. 具体地说,在这个阶段,设备3925的Π 3800 图解说明具有较大的背景主显示区3830和位于PIP显示3865的右下角的较小的前景插入显示区3835的PIP显示3865。 Specifically, at this stage, Π 3800 3925 illustrated device description has a larger background main display area 3830 positioned smaller PIP display foreground lower right corner of display area 3865 is inserted into the PIP display 3865 and 3835. 尽管图38和39图解说明在相同的Π 3800中缩放插入显示区3835的两种不同技术,不过普通技术人员会认识到一些实施例将不在相同的Π中提供这两种技术。 While FIGS. 38 and 39 illustrate the same scaling Π 3800 inserted in the display region 3835 of two different technologies, but ordinary skill in the art will recognize that some of the same will not be provided in these two techniques [pi embodiment.

[0485] 第二阶段3910图解说明缩放操作的开始。 [0485] The second stage 3910 illustrates the start of the zoom operation. 在这个例子中,用户通过选择该用户想要缩放的插入显示区3835的一个侧边(例如,把手指3840放在插入显示区3835的顶边或侧边上),发起缩放操作。 In this example, the user selects a display region of the insertion side of the user you want to zoom 3835 (e.g., the finger is inserted into the display area 3840 on the top edge or side edge 3835), initiate a zooming operation. 在这个例子中,用户把他的手指3840放在插入显示区3835的顶边上,以便实现所述选择。 In this example, the user to insert his finger on the display area 3840 a top edge 3835, in order to achieve the selection. 第二阶段3910用插入显示区3835的粗边框3845指示这种选择。 The second stage 3910 is inserted with a thick border display area 3845 3835 This selection is indicated.

[0486] 第三阶段3915图解说明在如箭头3950所示,用户通过使其手指3840移离插入显示区3835(即,垂直地朝着PIP显示3865的顶部移动),开始扩大插入显示区3835之后的Π 3800。 [0486] 3915 illustrates the third stage as shown by arrow after the user moves away from the display region 3835 by inserting her finger 38403950 (i.e., vertically towards the top of the display moving the PIP 3865), is inserted to expand the display area 3835 begins the Π 3800. 另外,如箭头3955所示,手指3840的移动在高度和宽度方向都按比例扩大了插入显示区3835。 Further, as shown by arrow 3955, the finger 3840 moves in the height and width directions of the inserted scaled display area 3835. 在其它例子中,用户通过利用相同的技术(即,通过朝着插入显示区3835 拖动手指3840),缩小显示区3835。 In other examples, a user by utilizing the same technology (i.e., 3835 by dragging the finger 3840 toward the insertion display area), reduce the display area 3835.

[0487] 第四阶段3920显示在完成插入显示区3835的缩放之后的Π3800。 [0487] The fourth stage 3920 Π3800 displayed after completion of the insertion of the 3835 display area zoom. 在这个例子中,一旦插入显示区3835达到理想的大小后,那么用户就通过停止其手指3840的拖动,并从设备的显示屏移开他的手指3840,来完成插入显示区3835的缩放。 After this case, once the inset display area 3835 to achieve the desired size, then the user stops by dragging their finger 3840, and moved his finger from the display device 3840, 3835 to complete the scale of the inset display area. 作为该过程的结果, 缩放后的插入显示区3835大于其在第一阶段3805中的初始大小。 As a result of this process, the scaled insert display area 3835 is larger than the initial size of the first stage 3805. 粗边框3845的消除指示现在完成了插入显示区缩放操作。 Thick border indicates the 3845 is now complete elimination of the inset display area zoom operation.

[0488] 响应拖动操作,一些实施例在高度和宽度方向,按比例地调整插入显示区3835的大小,如图38和39中所示。 [0488] in response to a drag operation, in some embodiments the height and width directions, proportionally adjust the size of the inset display area 3835, 38 and 39 in FIG. 其它实施例可允许用户调整插入显示区3835的高度和/或宽度,而不影响另一属性。 Other embodiments may allow the user to adjust the height of the insertion display area and / or width of 3835, without affecting other properties. 图40图解说明一种这样的缩放过程的例子。 Figure 40 illustrates illustrates a example of such a scaling process.

[0489] 具体地说,图40图解说明移动设备4025的与图38的Π 3800类似的Π 3800, 除了当选择插入显示区3835的边缘之一并水平或垂直移动该边缘时,图40的UI 3800允许插入显示区3835沿水平方向和/或垂直方向扩展之外。 [0489] More specifically, FIG 40 illustrates a mobile device with similar Π 3800 Π 3800 to 4025 in FIG. 38, except when the UI display area is inserted into one of the selected edge 3835 and the edge of the horizontal or vertical movement, FIG. 40 3800 display area 3835 allows the insertion of a horizontal direction and / or vertical extension beyond. 图40图解说明Π 3800中的与图38的PIP显示3865类似的PIP显示3865,除了现在插入显示区3835在PIP显示3865 的右上角之外。 FIG 40 illustrates Π 3800 in FIG. 38 and PIP display PIP display 3865 is similar to 3865, except now to displaying inset display area 3835 in the upper right corner 3865 of PIP. PIP显示3865包括两个视频显示:背景主显示区3830和前景插入显示区3835。 PIP display two video display 3865 includes: a main display area 3830 and the background of the foreground display region 3835 is inserted. 在这个例子中,背景主显示区3830呈现由远程设备的正面照相机或背面照相机拍摄的视频。 In this example, the background main display area 3830 presents a video captured by the front camera or the back camera remote device. 前景插入显示区3835呈现由本地设备的正面照相机或背景照相机拍摄的视频。 The foreground inset display area 3835 presents a video captured by the front camera or the local background camera device.

[0490] 类似于图38,图40按照Π 3800的四个操作阶段,图解说明缩放操作。 [0490] Like FIG. 38, FIG. 40 in accordance with a zoom operation of four operating phases Π 3800, illustration. 第一阶段4005与图38的第一阶段3805类似,除了现在插入显示区3835在右上角之外。 Like the first stage of the first stage 4005 of FIG. 38 and 3805, in addition to the display area 3835 is now inserted into the upper right corner. 其它三个阶段4010、4015和4020类似于三个阶段3910、3915和3920,除了插入显示区3835的底边的选择和移动只导致插入显示区3835沿垂直方向扩大,而不影响插入显示区3835的宽度之外。 Other three stages 4010,4015 3910,3915 and 4020 and 3920 similar to the three stages, except that insert bottom display area 3835 of the selection and movement only results in enlarged inset display area 3835 in the vertical direction, without affecting the inset display area 3835 the width of the outside.

[0491] 图38、39和40提供通过选择插入显示区3835的一个角落或侧边,允许用户缩放PIP显示3865的插入显示区3835的示例实施例。 [0491] FIGS. 38, 39 and 40 is provided by selecting inserting a corner or side of the display region 3835, allows the user to zoom inserted PIP display example of the display region 3865 3835 Example. 一些实施例提供缩放插入窗口3835的其它技术。 Some embodiments provide other techniques inserted scaling window 3835. 例如,图41图解说明一些实施例允许通过选择插入显示区3835的内部,缩放插入显示区3835。 For example, FIG. 41 illustrates an embodiment of some embodiments allows selection inserted inside the display region 3835, the zoom display area 3835 is inserted. 在这种方法中,用户通过把两根手指4155和4156放在屏幕上,并且彼此远离或者接近地拖动所述两根手指,调整插入显示区3835的大小。 In this method, the user 4155 and 4156 the two fingers on the screen, and away from each other or close to dragging the two fingers to adjust the size of the display area 3835 is inserted.

[0492] 在图41中,移动设备4140的Π 3800在与另一个移动设备的远程用户的视频会议期间,提供PIP显示3865。 [0492] In FIG. 41, the mobile device Π 3800 4140 during the remote user device with another mobile video conferencing, there is provided a PIP display 3865. 为了简化Π 3800的说明,图41图解说明Π 3800中与图38 的PIP显示3865类似的PIP显示3865。 To simplify the description of Π 3800, FIG. 41 illustrates Π 3800 in FIG. 38 and PIP display PIP display 3865 is similar to 3865.

[0493] 图41按照Π 3800的七个操作阶段,图解说明该缩放操作。 [0493] FIG. 41 illustrates the operation of the zoom operation in accordance with the seven stages of Π 3800, illustration. 前四个阶段3805、 4110、4115和4120表示插入显示区3835的扩大,而后三个阶段表示插入显示区3835的缩小。 The first four stages 3805, 4110,4115 and 4120 represent insertion enlarged display area 3835 of the last three stages to zoom out the display region 3835 of the insert. 图41中的第一阶段3805和图38中的第一阶段3805相同。 The same as the first stage in first stage 41 3805 and 38 3805. 具体地说,在这个阶段,UI 3800图解说明具有较大的背景主显示区3830和较小的前景插入显示区3835的PIP显示3865。 Specifically, at this stage, UI 3800 illustrated has a large main display area 3830 and the background of the smaller PIP display area 3835 is inserted into the foreground of the display 3865. 在这个例子中,背景主显示区3830呈现由远程设备的正面照相机或背面照相机拍摄的视频。 In this example, the background main display area 3830 presents a video captured by the front camera or the back camera remote device. 前景插入显示区3835呈现由本地设备的正面照相机或背面照相机拍摄的视频。 The foreground inset display area 3835 presents a video captured by the front camera or the back camera of the local device.

[0494] 第二阶段4110图解说明在发起缩放操作之后的Π 3800。 [0494] The second stage 4110 illustrates Π 3800 after the initiating zooming operation. 在这个例子中,用户通过选择该用户想要缩放的插入显示区3835 (例如,通过把两根手指4155和4156放在插入显示区3835内),发起缩放操作。 In this example, the user selects the user wants to insert the display area 3835 scaled (e.g., by two fingers 4155 and 4156 inserted in the display area 3835), initiate a zooming operation. Π 3800的第二阶段4110用插入显示区3835的粗边框4190指出这种选择。 The second stage Π 3800 is inserted into the display area 4110 with thick border 41903835 noted that selection.

[0495] 第三阶段4115图解说明在如箭头4160所示,用户通过相互远离地移动其手指4155和4156 ( S卩,朝着PIP显示3865的左上角移动手指4155,朝着PIP显示3865的右下角移动手指4156),开始扩大插入显示区3835之后的Π 3800。 [0495] In the third stage 4115 as illustrated by arrow 4160, the user moves their finger away from the 4155 and 4156 (S Jie by mutual movement toward the upper left corner of the PIP display 3865 of finger 4155 toward the right the PIP display 3865 the angular movement of a finger 4156), begins to expand Π 3800 3835 inserted after display area. 如箭头4165所示,手指4155 和4156的移动在高度和宽度方向按比例地扩大了插入显示区3835。 As indicated by arrows 4165, 4155 and 4156 move the finger in the height and width proportionally enlarged inset display area 3835.

[0496] 第四阶段4120显示在完成插入显示区3835的缩放之后的Π3800。 [0496] In the fourth stage 4120 Π3800 displayed after completion of the insertion of the display area 3835 of scaling. 在这个例子中, 用户通过停止其手指4155和4156的拖动,并且从设备的显示屏移开其手指4155和4156, 来完成插入显示区3835的缩放。 In this example, the user stops dragging his fingers 4155 and 4156, and 4155 and 4156 remove his fingers from the display device to complete the scale display area 3835 is inserted. 作为该过程的结果,缩放后的插入显示区3835大于其在第一阶段3805中的初始大小。 As a result of this process, the scaled insert display area 3835 is larger than the initial size of the first stage 3805. 粗边框4190的消除指示现在已完成插入显示区缩放操作。 Thick border indicates the elimination of 4190 is now complete inset display area zoom operation.

[0497] 在第五阶段4125,通过把两根手指4155和4156放在插入显示区3835上,用户重新选择插入显示区3835。 [0497] In a fifth stage 4125 through 4155 and 4156 the two fingers inserted in the display region 3835, the user re-select display area 3835 is inserted. 第六阶段4130图解说明在如箭头4170所示,用户通过相互接近地移动其手指4155和4156,开始缩小插入显示区3835之后的Π 3800。 Sixth stage as shown in illustration 4130, the user moves his finger close to each other 4155 and 4156, began to narrow arrow display area 4170 is inserted after 3835 in Π 3800. 如箭头4175所示, 手指4155和4156的移动在高度和宽度方向按比例地缩小了插入显示区3835。 As indicated by arrows 4175, 4155 and 4156 of the finger movement proportionally reduced in height and width directions of the display area 3835 is inserted.

[0498] 第七阶段4135类似于图41的第四阶段4120,除了通过所述操作,插入显示区3835的大小已缩小之外。 [0498] FIG seventh stage 4135 is similar to the fourth stage 4120, in addition to by the operation, the display area size of insert 3835 has been reduced to 41. 粗边框4190的消除指示现在已完成插入显示区缩放操作。 Thick border indicates the elimination of 4190 is now complete inset display area zoom operation.

[0499] 图38-41的上述说明举例说明了允许用户缩放PIP显示的插入显示区的几种示例用户界面。 Described above [0499] FIG 38-41 illustrates several exemplary user interface allows users to zoom the displayed PIP inset display area. 在一些实施例中,插入显示区的缩放会引起双照相机移动设备的图像处理操作的变化,比如使视频会议管理器1604响应用户的输入,改变PIP显示中的插入显示区的比例缩放和合成。 In some embodiments, the inset display area zoom will cause changes in the image processing dual camera mobile device operation, such that the video conference manager 1604 in response to user input, altering the ratio inset display area PIP display scaling and synthesis. 另外,在一些实施例中,图38-41中的显不区1155的布局和上面说明的图12的显不区1155的布局相同。 Further, in some embodiments, FIG. 38-41 were not the layout area 1155 described above in FIG region 12 is not substantially the same layout 1155.

[0500] 4.识别感兴趣区域 [0500] 4. Identify region of interest

[0501] 一些实施例允许用户在视频会议期间,识别显示的视频中的感兴趣区域(R0I),以便修改图像处理(例如,图16中的图像处理管理器1608),编码(例如,图16中的编码器1655),移动设备及其照相机在视频会议期间的行为,或者它们的组合。 [0501] Some embodiments allow a user during a video conference, the video display to identify a region of interest (R0I), so as to modify the image processing (e.g., FIG. 16 of an image processing manager 1608), coding (e.g., 16 in FIG. an encoder 1655), and the behavior of the mobile device during the video conference camera, or a combination thereof. 不同的实施例提供不同的技术来识别视频中的这种感兴趣区域。 Different embodiments provide different techniques to identify such a region of interest in the video. 图42图解说明用于识别视频的感兴趣区域, 以便提高视频的图像质量的一些实施例的用户界面。 Figure 42 illustrates a region of interest for identifying the video in order to improve the image quality of some of the video of the user interface of the embodiment.

[0502] 在图42中,移动设备4225的Π 4200在与另一个移动设备的远程用户的视频会议期间,呈现PIP显示4265。 [0502] In FIG. 42, Π 4225 mobile device 4200 and another mobile device during the video conference remote users, presenting the PIP display 4265. 图42中的PIP显示基本上与图41中的PIP显示类似。 FIG 42 PIP display substantially displays the like in FIG. 41 PIP. 具体地说,图42中的PIP显示包括两个视频显示:背景主显示4230和前景插入显示4235。 Specifically, in FIG. 42 PIP video display includes two display: the foreground and background main display 4230 display 4235 is inserted. 在这个例子中,背景主显示4230呈现树和戴着帽子的人物的视频,所述树和人物被假定为由远程设备的正面照相机拍摄其视频的树和人物,或者由远程设备的背面照相机拍摄其视频的树和人物。 In this example, the background main video display 4230 presents a tree and a hat character, the character is assumed to be a tree, and the remote device by the front camera shooting video of trees and its people, or captured by the back camera of the remote device tree and characters of their video. 前景插入显示4235呈现男士的视频,在这个例子中,所述男士被假定为由本地设备的正面照相机拍摄其视频的男士,或者由本地设备的背面照相机拍摄其视频的人物。 Foreground inset display 4235 presents video men, in this example, is assumed by the front camera man local imaging apparatus whose video men, or captured by the back camera whose video equipment local character. PIP显示的下面是包括标记成"End Conference (结束会议)"的可选UI项目4260 (例如, 按钮4260)的显示区1155,可选Π项目4260允许用户通过选择该项目来结束视频会议。 The following is a PIP display flag to "End Conference (the meeting)" selectable UI item 4260 (e.g., button 4260) a display area 1155, Π selectable item 4260 allows the user to select the item to the end of the video session.

[0503] 该PIP显示只是呈现由远程设备和本地设备拍摄的视频的合成视图的一种方式。 [0503] The PIP display is only one manner of presenting the synthesized video view captured by a remote device and a local device. 一些实施例可以提供其它的合成视图。 Some embodiments may provide other composite views. 例如,代替具有用于来自远程设备的视频的较大背景显示,较大的背景显示可以是来自本地设备的视频,较小的前景插入显示可以是来自远程设备的视频。 For example, instead of having large background video from a remote device for display, a larger background display may be a video from the local device, can be smaller foreground inset display video from the remote device. 另外,一些实施例允许本地视频和远程视频出现在Π中的两个并排显示区(例如,左右显示窗口,或者上下显示窗口),或者两个对角排列的显示区中。 Further, some embodiments allow the local and remote videos to appear side by side in Π two display areas (e.g., around the display window, the display window or down), or the two display areas are aligned diagonally. 在其它实施例中,PIP显示还可包含一个较大的背景显示,和两个较小的前景插入显示。 In other embodiments, the PIP display may also include a larger background display, and two smaller foreground inset display. 在一些实施例中, PIP显示的方式或者默认显示模式可由用户规定。 In some embodiments, the PIP display mode or the default display mode specified by the user.

[0504] 图42按照Π 4200的四个操作阶段,图解说明R0I标识操作。 [0504] FIG. 42 in terms of four operating phases Π 4200, illustrating R0I identification operation. 如第一阶段4205 中所示,呈现在背景显示4230中的视频具有很低的质量(S卩,视频图像模糊)。 As shown in the first stage 4205, the display 4230 presents the background video has a very low mass (S Jie, video image blur). 在这个例子中,移动设备4225的用户意欲把背景显示4230中出现人物的脸部4270的区域识别为感兴趣区域。 In this example, the user mobile device 4225 is intended to show the background appears in the character's face region 4230 is identified 4270 for the region of interest.

[0505] 在第二阶段4210中,发起识别感兴趣区域的操作。 [0505] In the second stage 4210, to initiate the operation to identify the region of interest. 在这个例子中,通过选择呈现在背景显示4230中的视频之中用户想要识别为感兴趣区域的区域(例如,通过在设备的屏幕上,在背景显示4230中的显示的人物脸部4270附近的位置轻点手指4250),发起该操作。 In this example, by selecting the background presented in the 4230 video display into the user wants a region identified as a region of interest (for example, through the screen of the device, in the background of the characters displayed near the display face 4230 4270 location finger tap 4250), initiated the operation.

[0506] 如第三阶段4215中所示,用户的区域选择使Π 4200画出环绕用户选择的区域的环绕框4275(例如,虚线方框4275)。 [0506] As shown in the third stage 4215, the user choose to make a region around the region Π 4200 selected by the user to draw the enclosure 4275 (e.g., a broken line block 4275). 第四阶段4220显示在结束感兴趣区域的标识之后的Π 4200。 The fourth stage 4220 show Π 4200 after the end of the identified area of ​​interest. 作为该过程的结果,与第一阶段4205中相比,感兴趣区域内的视频的质量已得到显著改善。 As a result of this process, compared with the first stage 4205, the video quality within the region of interest has been significantly improved. 环绕框4275的消除指示现在已完成R0I选择操作。 Enclosure 4275 now indicates cancellation R0I selection operation has been completed. 在一些实施例中,R0I标识处理还对显示在远程设备上的相同视频造成与它对本地设备4225造成的变化相同的变化。 In some embodiments, R0I identification process also causes the same change caused by the change in its local device 4225 pairs of display on the remote device in the same video. 例如,在这个例子中,显示在远程设备上的相同视频的感兴趣区域内的画面质量也得到显著改善。 For example, in this example, the picture quality of the display within the region of interest of the same video on the remote device is also significantly improved.

[0507] 在一些实施例中,用户可在第三阶段4215放大或缩小环绕框4275(例如,通过把手指4250放在显示画面上,并朝着屏幕的右上角移动手指4250,以放大环绕框4275,或者朝着屏幕的左下角移动手指4250,以缩小环绕框4275)。 [0507] In some embodiments, a user may zoom in or out in the third stage 4215 enclosure 4275 (e.g., by a finger 4250 on the screen of the display, and moving the finger toward the upper right corner of the screen 4250, in order to enlarge the surrounding box 4275, or movement of the finger toward the lower left corner of the screen 4250 to narrow enclosure 4275). 一些实施例还允许用户在第三阶段4215旋转环绕框4275 (例如,通过把手指4250放在显示画面上,并在显示画面上水平或垂直移动手指4250)。 Some embodiments also allow a user in the third stage rotary enclosure 4275 4215 (e.g., by a finger 4250 on the display screen, and displayed on the screen horizontally or vertically moving the finger 4250). 在一些其它实施例中,所述区域的选择根本不会使Π 4200在第三阶段4215中画出环绕框4275。 In some other embodiments, the selected area will not cause Π 4200 enclosure 4275 shown in the third stage 4215.

[0508] 其它实施例提供允许用户识别视频中的感兴趣区域的不同技术。 [0508] Other embodiments provide different techniques allow the user to identify regions of interest in the video. 图43图解说明一种这样的其它技术。 Figure 43 illustrates one other such techniques. 在图43中,用户通过画出包围感兴趣区域的形状,识别感兴趣区域。 In Figure 43, the user draw a shape surrounding the region of interest, to identify the region of interest. 在这个例子中,所述形状是矩形,不过也可以是其它形状(例如,任何其它多边形、圆形、椭圆形等)。 In this example, a rectangular shape, but it may be other shapes (e.g., any other polygonal, circular, oval, etc.). 一些实施例在还提供在图42中图解说明的那种技术的设备Π中,提供图43的备选技术。 In some embodiments of the device also provides a technology that Π 42 illustrated in FIG illustrated, alternative techniques to provide 43 of FIG. 不过,其它实施例不在相同Π中提供这两种技术。 However, other embodiments are not the same as the two technologies are provided in Π.

[0509] 图43按照Π 4200的五个操作阶段,图解说明这种R0I标识操作。 [0509] FIG. 43 according to the five stages of operation Π 4200, illustrates this operation R0I identification. 图43中的第一阶段4205和图42中的第一阶段4205相同。 The same as the first stage of the first stage in FIG. 43 and FIG. 42 4205 4205. 具体地说,在第一阶段4205中,UI 4200图解说明具有较大的背景主显示4230,和位于PIP显示4265的左下角的较小的前景插入显示4235 的PIP 显示4265。 Specifically, in a first stage 4205, UI 4200 illustrated has a larger background main display 4230, and is located in the lower left corner foreground smaller PIP display of the inset display 4265 4235 4265 PIP display.

[0510] 在第二阶段4310,发起识别感兴趣区域的操作。 [0510] In the second stage 4310, to initiate the operation to identify the region of interest. 在这个例子中,通过持续一段时间选择定义呈现在背景主显示4230中呈现的视频中的感兴趣区域的第一位置(例如,通过在设备的屏幕上,持续一段时间把手指4350放在背景显示4230中的所显示人物脸部4270附近的位置上),发起所述操作。 In this example, by selecting a period of time defined in the context of presenting a first position of the main display area 4230 in the video presentation of interest (e.g., through the screen device, a period of time in the background display finger 4350 position near the face of a person in 4270 on the display 4230), initiating the operation. 在第三阶段4315, Π 4200利用临近背景显示区4230上的所选第一位置的圆点4355,指示已选择了所述第一位置4370。 In the third stage 4315, Π 4200 4355 using near background display dot position of a first selected area on the 4230, indicating that said first selected position 4370.

[0511] 第四阶段4320图解说明在用户选择了定义感兴趣区域的第二位置4375之后的Π 4200。 [0511] In the fourth stage 4320 illustrates the user selects the Π 4200 4375 after the second position the defined region of interest. 在这个例子中,通过如箭头4360所示,在圆点4355出现之后,从第一位置开始在设备的屏幕内拖动手指4350,并停止在位于背景显示区430中的显示的帽子和显示的树之间的位置,用户选择第二位置4375。 In this example, by, after the 4355 dots appear, as indicated by arrow 4360 start dragging from a first position within the screen of the device's finger 4350, and stops located in the background display region 430 and display the display hat position between the tree, the user selects a second position 4375. 如第四阶段中所示,这种拖动导致Π4200画出感兴趣区域的矩形边框4365,所述矩形边框4365具有在其相对顶点上的第一和第二位置4370和4375。 As shown in the fourth stage, this leads to drag the region of interest shown Π4200 bounding rectangle 4365, a rectangular frame 4365 having first and second opposite positions 4370 and 4375 on the vertex.

[0512] 第五阶段4325图解说明在感兴趣区域的标识已完成之后的Π4200。 [0512] 4325 illustrates the fifth stage Π4200 After identifying the region of interest has been completed. 在这个例子中,用户通过一旦标识了期望的感兴趣区域,就停止手指4350的拖动并从设备的显示屏移开手指4350,来完成感兴趣区域的标识。 In this example, once the user by identifying a desired area of ​​interest, stop dragging your finger away finger 4350 and 4350 from the display device to complete the identified area of ​​interest. 第五阶段4325图解说明通过该拖拉(drawing)过程,与第一阶段4205中相比,感兴趣区域内的视频的质量已得到显著改善。 4325 illustrates the fifth stage through the dragging (Drawing) process, compared with the first stage 4205, the video quality within the region of interest has been significantly improved. 在一些实施例中,和它对本地设备4225所造成的变化一样,这种拖拉过程也对远程设备上的显示画面造成相同的变化。 In some embodiments, and its variation caused by the local device 4225, as this process also causes drag on the same display screen changes on the remote device. 例如,在这个例子中,显示在远程设备上的相同视频的感兴趣区域内的图像质量会得到显著改善。 For example, in this example, display image quality within the region of interest in the same video on the remote device will be significantly improved.

[0513] 上面的图42和43的描述举例说明了识别视频中的感兴趣区域,以便改善所识别区域的图像质量的不同方式。 Description [0513] The above Figures 42 and 43 illustrate a region of interest identified in the video, different ways to improve the image quality of the identified region. 在一些实施例中,改善识别的感兴趣区域的画面质量会导致双照相机移动设备的编码操作的变化,比如当对视频编码时,对所识别的区域分配更多的比特。 In some embodiments, the image quality improves the identified region of interest causes a change in the encoded dual camera mobile device operation, such as when the video coding, to allocate more bits to the identified region.

[0514] 一些实施例允许用户识别视频中的感兴趣区域,以对移动设备或其照相机做出不同的改变。 [0514] Some embodiments allow a user to identify a region of interest video, in order to make various changes to the mobile device or a camera. 例如,图44图解说明识别视频中的感兴趣区域,以在显示画面上扩大或缩小感兴趣区域的例子。 For example, FIG. 44 illustrates a region of interest in identifying the video in order to expand or shrink example a region of interest on the display screen. 在这种方法中,用户通过把显示画面上的某一区域选为感兴趣区域的中心,随后扩大或缩小所述感兴趣区域,识别视频中的感兴趣区域。 In this method, the user of a selected area on the display screen the center of the region of interest, followed by enlarging or reducing the region of interest, to identify a region of interest in the video.

[0515] 在图44中,移动设备4425的Π 4400在与另一个移动设备的远程用户的视频会议期间,呈现PIP显示4265。 [0515] In FIG. 44, Π 4425 4400 mobile device with another mobile device during the video conference remote users, presenting the PIP display 4265. 图44中的PIP显示4265基本上与图42的PIP显示4265类似,不过图44的前景插入显示4235位于PIP显示4265的左下角。 FIG 44 displays the PIP display 4265 is substantially similar to PIP 4265 of FIG. 42, FIG. 44, but the foreground inset display 4235 is located in the lower left corner 4265 of the PIP display.

[0516] 图44按照Π 4400的四个操作阶段,图解说明R0I选择操作。 [0516] FIG. 44 in terms of four operating phases Π 4400, illustrating R0I selecting operation. 如第一阶段4405 中所示,背景显示4430呈现具有在背景显示4430的左侧的男士,和在背景显示4430的右侧的树4440的视频。 As shown in the first stage 4405, the background display 4430 presents a display having a background of 4430 men left, and right of the video background display 4430 in tree 4440. 此外,树4440相对较小,只占据背景显示区4430的右侧。 Furthermore, the tree 4440 is relatively small, occupying only the right side of the background display region 4430. 在这个例子中,移动设备4425的用户意欲把背景显示区4430上出现树4440的区域识别为感兴趣区域。 In this example, the user mobile device 4425 is intended to appear on the background display region 4430 in the region identified as a region of interest 4440 tree.

[0517] 在第二阶段4410中,发起识别感兴趣区域的操作。 [0517] In the second stage 4410, to initiate the operation to identify the region of interest. 在这个例子中,通过选择呈现在背景显示4430中的视频之中的、用户希望识别为感兴趣区域的区域4440 (例如,通过把两根手指4445和4446放在背景显示区4430上、显示树4440的地方),发起所述操作。 In this example, by selecting from among the presented background display 4430 in the video, the user wishes to identify a region of interest, the region 4440 (e.g., by two fingers 4445 and 4446 on the background display region 4430 displays a tree local 4440), initiating the operation. 在第二阶段4410,通过彼此远离地拖动其手指4445和4446,用户能够使感兴趣区域4440扩大,并占据更大部分的背景显示区4430。 In the second stage 4410, away from one another by dragging their finger 4445 and 4446, the user can expand the area of ​​interest 4440, and occupies a larger portion of the background display region 4430. 通过相互接近地拖动其手指4445和4446,用户还能够使感兴趣区域4440缩小,从而占据较小部分的背景显示区4430。 Close to one another by dragging their finger 4445 and 4446, the user 4440 can also reduce the area of ​​interest, so as to occupy a smaller portion of the background display region 4430.

[0518] 第三阶段4415图解说明在如箭头4450所示,用户通过彼此远离地移动其手指4445和4446(即,手指4445朝着背景显示区4430的左上角移动,手指4446朝着背景显示区4430的右下角移动),开始使感兴趣区域4440扩大以占据更大部分的背景显示区4430 之后的Π4400。 [0518] In the third stage 4415 as illustrated by arrow 4450, the user moves their finger away from the 4445 and 4446 (i.e., the finger 4445 moves the upper left corner of region 4430 toward each other by a background, finger toward the background display region 4446 4430 bottom right move), began to expand the area of ​​interest 4440 to occupy a larger portion of the background display area Π4400 after 4430. 在一些实施例中,手指移动还对远程设备的显示造成和它对本地设备造成的变化一样的变化。 In some embodiments, the finger movement also caused changes in the local device and it caused the same changes to display the remote device. 例如,在这个例子中,相同视频的感兴趣区域将扩大,从而占据远程设备的更大部分的背景显示区4430。 For example, in this example, the region of interest of the same video will expand to occupy a larger portion of the background display region 4430 of the remote device. 在一些实施例中,本地显示和/或远程显示中的感兴趣区域的扩大导致一个或两个移动设备或其照相机修改它们的一个或多个其它操作,如下进一步所述。 In some embodiments, the local display area to expand and / or remote display of interest results in one or both of the camera or a mobile device to modify their one or more other operations, as further described below.

[0519] 第四阶段4420显示在感兴趣区域的标识已完成之后的Π 4400。 [0519] The fourth stage 4420 Π 4400 displayed after identification of the region of interest has been completed. 在这个例子中, 一旦感兴趣区域在背景显示区4430中达到期望的比例后,那么用户通过停止用户的手指4445和4446的拖动,并且从设备的显不屏上移开手指4445和4446,来完成感兴趣区域的标识。 In this example, once the region 4430 to achieve the desired proportion of the region of interest displayed in the background, then the user stops dragging the user's finger 4445 and 4446 and the fingers 4445 and 4446 away from the screen on the display device is not, to complete the identified area of ​​interest. 作为该过程的结果,感兴趣区域占据了大部分的背景显示区4430。 As a result of this process, the region of interest occupies most of the background display area 4430. 现在完成感兴趣区域的标识操作。 Logo is now complete operating region of interest.

[0520] 上面的一些例子举例说明用户如何识别视频中的感兴趣区域,以提高视频中的所选感兴趣区域内的图像质量(例如,通过增大对视频的感兴趣区域编码的比特率)。 [0520] Some examples above illustrate how the user to identify a region of interest video, to improve image quality in the selected region of interest in the video (e.g., by increasing the area of ​​interest of the video encoding bit rate) . 在一些实施例中,识别视频中的感兴趣区域导致移动设备的图像处理操作,比如曝光、比例缩放、 聚焦等的变化。 In some embodiments, identifying the video in the region of interest results in the image processing operation of the mobile device, such as exposure, scaling, and other changes in focus. 例如,识别视频中的感兴趣区域会导致视频会议管理器1604不同地比例缩放和合成视频的图像(例如,识别要变焦的感兴趣区域)。 For example, to identify a region of interest in the video will cause the video conference manager 1604 and differently scaled composite video image (e.g., to identify the zoom area of ​​interest).

[0521] 在其它实施例中,识别视频中的感兴趣区域导致移动设备的照相机的操作的变化(例如,帧速率、变焦、曝光、比例缩放、聚焦等)。 [0521] In other embodiments, the region of interest to identify the video camera causes a change in the operation of the mobile device (e.g., a frame rate, zoom, exposure, zoom ratio, focus, etc.). 在另外的其它实施例中,识别视频中的感兴趣区域导致移动设备的编码操作的变化,比如向识别的区域分配更多的比特、比例缩放等等。 In still other embodiments, the identified region of interest in the video encoding of the mobile device causes a change in the operation, such as assigning more bits to the region identified, scaling and so on. 另外,尽管上面说明的示例R0I标识操作只会对移动设备或其照相机造成上述修改中的一种修改,不过在一些其它实施例中,R0I标识操作会对移动设备或其照相机的操作造成多于一种的修改。 Further, although the above example of identifying is R0I described above will result in a modification of the modification of a mobile device or a camera, but in other embodiments, the mobile device will operate R0I identify or operate a camera caused by more than one modification. 另外,在一些实施例中,图42-44的显不区1155的布局和上面说明的图12的显不区1155的布局相同。 Further, in some embodiments, not significantly FIG layout region 42-44 1155 described above in FIG. 12 is the same region without significant layout 1155.

[0522] B.切换照相机 [0522] B. switch camera

[0523] -些实施例提供在视频会议期间,切换照相机(S卩,改变拍摄图像的照相机)的方法。 [0523] - Some embodiments provide a method during a video conference, the camera switches (S Jie, change camera through image). 不同的实施例提供实现切换照相机操作的不同方法。 Different embodiments provide different methods to achieve the switching operation of the camera. 一些实施例提供由双照相机移动设备执行的切换该设备的照相机(即,本地切换)的方法,而其它实施例为双照相机移动设备提供指令视频会议中的另一个双照相机移动设备切换所述另一个设备的照相机(即, 远程切换)的方法。 Some embodiments provide a camera apparatus by switching the mobile device to perform the dual camera (i.e., a local switch) method, while other embodiments of the mobile device to switch to another dual camera dual camera mobile device provides an instruction in the other video conference the method of a camera apparatus (i.e., remote switching) of. 除此之外的其它实施例同时提供进行本地切换和远程切换的方法。 In addition to other embodiments also provide a method for the local switch and the remote switch. IV. B. 1节将说明在双照相机移动设备上进行本地切换照相机操作的处理。 IV. B. Section 1 will be described for processing a local switch camera operation on the dual camera mobile device. IV. B. 2节将说明在双照相机移动设备上进行远程切换照相机操作的处理。 IV. B. processing section 2 will be described camera operation switch on the remote dual camera mobile device.

[0524] 1.本地切换照相机 [0524] 1. Local switching camera

[0525] 图45图解说明一些实施例在本地双照相机移动设备执行的,在与包括至少一部照相机的远程移动设备的视频会议期间,在本地设备的两部照相机之间进行切换的处理4500。 [0525] FIG. 45 illustrates in some embodiments, the process 4500 during a video conference camera comprises at least one remote mobile device, to switch between the two cameras of the local devices in the local mobile device to perform the dual camera. 在一些实施例中,处理4500由图16中所示的视频会议管理器1604执行。 In some embodiments, the processing performed by a video conference 45001604 manager 16 shown in FIG. 为了说明起见,下面的讨论将把本地双照相机移动设备的一部照相机称为照相机1,把本地双照相机移动设备的另一部照相机称为照相机2。 For illustrative purposes, the following discussion will be a local dual camera mobile camera apparatus 1 is called a camera, the other camera local dual camera mobile device called a camera 2.

[0526] 通过在本地双照相机移动设备和远程移动设备之间开始(在4505)视频会议,开始处理4500。 [0526] By the start (at 4505) a video conference between a local dual camera mobile device and the remote mobile device, process 4500 starts. 随后,处理4500把来自本地双照相机移动设备的当前所选照相机(例如,照相机1)的视频图像发送给(在4510)远程移动设备,以便在远程移动设备上显示。 Subsequently, the processing of the currently selected camera 4500 from the local dual camera mobile device (e.g., camera 1) is transmitted to the video images (at 4510) a remote mobile device, for display on the remote mobile device. 在4510, 处理4500还根据所述视频图像和它从远程移动设备接收的视频图像,产生和显示合成图像。 In 4510, the process 4500 in accordance with the further video image and the video image it receives from the remote mobile device, generates and displays a composite image.

[0527] 处理4500随后确定(在4515)是否接收到结束视频会议的请求。 [0527] process 4500 then determines (at 4515) whether a request to end the video conference. 如上所述,在一些实施例中,可以应本地双照相机移动设备的用户的请求(例如,通过本地双照相机移动设备的用户界面),或者应远程移动设备的用户的请求(例如,通过远程移动设备的用户界面),结束视频会议。 As described above, in some embodiments, the user may be a local dual camera mobile device requests (e.g., through a user interface local dual camera mobile device), the remote or mobile device should request the user (e.g., via remote mobile user interface device), the end of the video conference. 当处理4500接收到结束视频会议的请求时,处理4500结束。 When the process 4500 receives a request to end the video conference, the process 4500 ends.

[0528] 当处理4500未接收到结束视频会议的请求时,处理4500随后确定(在4520)本地双照相机移动设备的用户是否已指令该本地设备切换用于视频会议的照相机。 [0528] When the process end request is not received 4500 the video conference, the process 4500 then determines whether a local user has dual camera mobile device instructing the local switching device is a camera for a video conference (4520). 当处理4500确定(在4520)所述本地设备未被指令切换照相机时,处理4500返回操作4510。 When the process 4500 determines (at 4520) the local apparatus is not the camera switching instruction, the process returns to operation 45,104,500. 不过,当处理4500确定(在4520)所述本地设备已被指令切换照相机时,处理4500进入操作4525。 However, when the process 4500 determines (at 4520) when the apparatus has been switched local camera command, the process 4500 proceeds to operation 4525.

[0529] 在4525,处理4500向远程移动设备发送指出本地双照相机移动设备要切换照相机的通知。 [0529] In 4525, the process informs the local dual camera mobile device to switch to the camera remote mobile device 4500 transmits noted. 在一些实施例中,处理4500通过如上所述,与音频通道和视频通道一起由VTP 管理器3125多路复用的视频会议控制通道发送所述通知。 In some embodiments, sending the notification process 4500 as described above, with the video channel and the audio channel with the control channel by the video session manager 3125 VTP multiplexed.

[0530] 在发送其通知之后,处理4500进行(在4530)切换照相机操作。 [0530] After transmitting its notification processing 4500 (in 4530) the switching operation of the camera. 在一些实施例中, 进行(在4530)切换照相机操作包括指令CIPU停止用照相机1拍摄视频图像,开始用照相机2拍摄视频图像。 In some embodiments, a (at 4530) the camera operation switch includes an instruction to stop capturing a video image with CIPU camera 1 starts the image captured by the camera 2 video. 这些指令可以仅仅指令CIPU切换捕获来自与照相机2关联的像素阵列的图像,并开始处理这些图像。 These instructions may merely command CIPU 2 switching array of image pixels associated with the camera from capturing and begins processing the images. 另一方面,在一些实施例中,给CIPU的指令可以伴随一组初始化参数,所述一组初始化参数指令CIPU : (1)根据特定的一组设置操作照相机2,(2)以特定的帧速率捕获由照相机2产生的视频,和/或(3)根据特定的一组设置(例如,分辨率等)处理来自照相机2的视频图像。 On the other hand, in some embodiments, it may be accompanied by instructions for CIPU a set of initialization parameters, said set of initialization parameters instructions CIPU: (1) 2, (2) in a particular frame in accordance with a particular set of settings to operate the camera capturing video rate generated by the camera 2, and / or (3) for processing a video image from the camera 2 according to a particular set of settings (e.g., resolution, etc.).

[0531] 在一些实施例中,切换照相机指令(在4530)还包括把未使用的照相机切换成如上所述的第四种工作功率模式的指令。 [0531] In some embodiments, the camera switching instruction (in 4530) further includes instructions to the camera is switched to the unused fourth operational power mode as described above. 在这个例子中,切换照相机指令包括给照相机2的、 切换到其第四种工作功率模式的指令。 In this example, comprises a switch camera command to the camera 2 is switched to its fourth operational command power mode. 另外,切换照相机指令还包括给照相机1的、从其第四种工作功率模式切换到另一种工作功率模式,比如第一种工作功率模式,以节省电力,或者切换到第三种工作功率模式,使得当被要求拍摄图像时,它能够快速切换到第四种工作功率模式,并开始拍摄图像的指令。 Moreover, further comprising a switch camera command to the camera, the fourth switch 1 from its operational power mode to another operational power mode, such as a first operational power mode to conserve power, or switch to a third operational power mode such that when the captured image is required, it is possible to quickly switch to the fourth operational power mode and starts shooting instruction image. 切换照相机操作4530还涉及合成由本地双照相机移动设备的照相机2拍摄的图像(而不是由照相机1拍摄的图像)和从远程移动设备接收的图像,以便在双照相机移动设备上显示。 The camera operation switch 4530 further relates to an image (instead of the image captured by the camera 1) and the mobile device received from the remote image to be displayed on the dual camera mobile device photographed by the camera synthesized local dual camera mobile device 2.

[0532] 在4530指令切换照相机之后,处理4500在本地双照相机移动设备上进行(在4535)切换照相机动画,以显示在来自照相机1的图像的显示和来自照相机2的图像的显示之间的转变。 [0532] After the instruction to switch the camera 4530, the process 4500 (at 4535) on the local switch camera animation dual camera mobile device to display the image from the transition between the display and the display of the camera 1 from the image of the camera 2 . 在本地双照相机移动设备上的切换照相机动画之后,处理4500循环经过操作4510-4520,直到接收到结束视频会议请求或者新的切换照相机请求为止。 Switch camera on the local dual camera mobile device after the animation process 4500 cycles through operations 4510-4520, the end of the video conference until receiving a new request or a handover request until the camera.

[0533] 图46图解说明一些实施例如何允许通过双照相机移动设备的Π1105请求切换照相机操作,和这些实施例如何动画演示切换照相机操作的一个例子。 [0533] FIG. 46 illustrates how some embodiments, the camera allows a double Π1105 requesting mobile device camera operation switch, and how to animate these embodiments an example of the switching operation of the camera. 图46按照该设备的Π 1105的八个不同的操作阶段4610、4615、4620、4625、4630、4635、4640和4645,图解说明切换照相机操作。 FIG camera operation switch 46 of the apparatus 1105 according Π eight different phases of operation 4610,4615,4620,4625,4630,4635,4640 and 4645, is illustrated. UI 1105的前四个阶段图解说明接收用户的切换照相机请求的例子。 The first four stages illustrated UI 1105 described an example of receiving a user request to switch cameras. 在本发明的一些实施例中,该设备的用户具有产生这种请求的其它机制。 In some embodiments of the present invention, the user of the device having other mechanisms generating such a request.

[0534] 第一阶段4610和图11的Π 1105的第一阶段1130相同,它表示在建立视频会议之后的Π 1105。 [0534] FIG same Π 4610 and 1105 of the first stage 11 of the first stage 1130, which represents a [pi 1105 after establishing the video conference. 在这个阶段,UI 1105显示包括两个视频显示的PIP显示:来自远程照相机的较大的背景显示,和来自本地照相机的较小的前景插入显示。 At this stage, UI 1105 PIP display includes two video displays show: larger context from the remote camera display, and smaller foreground inset display from the local camera. 在这个例子中,背景主显示区1170呈现一位女士的视频,在这个例子中,所述女士被假定为由远程设备拍摄其视频的女士,而前景插入显示区1160呈现一位男士的视频,在这个例子中,所述男士被假定为由本地设备的正面照相机拍摄其视频的男士。 In this example, the background main display area 1170 video showing a woman, in this case, Ms. Ms. by the remote device is assumed to shoot their video, while the foreground inset display area 1160 presents a video of a man, in this example, the men to be assumed by the front camera imaging device local men their videos.

[0535] 第二阶段4615随后显示通过Π 1105的PIP显示区1180的选择,开始切换照相机操作。 [0535] The second stage 4615 is then displayed on the display area 1180 selected by the PIP Π 1105, switch camera operation starts. 如图所示,通过把用户的手指4670放在PIP显示1180上,实现所述选择。 As shown, the 1180, said selection by the user's finger on the PIP display 4670. 第三阶段4620显示包括用于在视频会议期间,请求在本地设备4600的照相机间进行切换的可选Π 项目4675(例如,切换照相机按钮4675)的Π 1105。 The third stage 4620 comprises a display during the video conference, request selectable items [pi switching between the local device 4600 of the camera 4675 (e.g., switch camera button 4675) is Π 1105. 第四阶段4625图解说明在本地设备4600的用户选择(例如,通过单指轻点)可选Π 项目4675之后,和在通过可选Π 项目4675的突出显示指出这种选择之后的Π 1105。 In the fourth stage 4625 illustrates the user selecting the local device 4600 (e.g., a single finger tap) [pi selectable items after 4675, and pointed out that this choice Π 1105 after an optional [pi by the highlighted item 4675. 通过选择该可选Π 项目4675,用户指令设备4600在视频会议期间,从设备4600的正面照相机切换到设备4600的背面照相机。 4675, the user instructs the device 4600 during a video conference, by selecting the selectable switching Π project from the front to the back of the camera apparatus 4600 4600 camera device. 在其中设备4600的背面照相机拍摄视频的其它例子中,用户对可选Π 项目4675的选择会指令设备4600从设备4600的背面照相机切换到设备4600的正面照相机。 Other examples of devices in which the back camera shooting video of 4600, a user selection of the selectable item Π apparatus 4675 instructs switching from the rear surface 4600 of the camera 4600 to the device 4600 of the front camera device. 在第四阶段之后,视频会议管理器向CIPU发送指令,以及远程设备开始切换照相机操作。 After the fourth stage, the video conference manager sends an instruction to the CIPU, and a remote switching device starts camera operation.

[0536] UI 1105的后四个阶段4630、4635、4640和4645图解说明本地设备上的切换照相机动画的例子。 [0536] UI 1105 after four stages 4630,4635,4640 and 4645 illustrates an example of a switch camera animation on the local device. 该动画意图造成从本地设备的正面照相机和背面照相机拍摄的视频被并发地显示在观察窗格的正反两面上的印象,在任意时间,所述观察窗格只能够让用户看到其正反两面之一。 The animation is displayed with intent to cause impression on both sides of the viewing pane from a video camera on the back and front camera captured the local device concurrently, at any time, the viewing pane only allows users to see the pros and cons one of both sides. 当在视频会议中间请求切换照相机时,使该观察窗格仿佛围绕垂直轴旋转, 使得在观察窗格的先前向用户显示一部照相机的视频的一个侧面上呈现的一部照相机的视频转动离开用户,直到它被观察窗格的另一个侧面代替为止,所述另一个侧面显示另一部照相机的视频。 When requesting a handover in the middle of the video conference camera, so that if the viewing pane about a vertical axis of rotation, so that the video on one side of a previously displayed video camera to a user's viewing pane presentation of a camera is rotated away from the user until it is in place until the other side of the viewing pane, the other side of the display portion of another video camera. 这种感受到的观察窗格的旋转动画和现象是通过(1)在用于一部照相机的显示区中,逐渐缩小来自该部照相机的视频图像,并对所述视频图像应用透视校正操作, 随后(2)在所述显示区中,逐渐扩大来自另一部照相机的视频图像,并减小对所述视频图像的透视校正操作实现的。 This felt and observed phenomena rotation animation pane by (1) in the display area for a camera, a video image is gradually reduced from the camera unit, said video image and apply perspective correction operation, followed by (2) in the display area, the video image gradually enlarged from another camera, and to reduce the perspective correction operation on the video image to achieve.

[0537] 因此,第五阶段4630图解说明围绕垂直轴4682的"观察窗格的旋转"的开始。 [0537] Thus, the fifth stage began around 4630 illustrates the "observation pane rotation" of the vertical axis 4682. 为了给出观察窗格的旋转现象,UI 1105减小了视频显示区1160中的正面照相机的视频图像的大小,并应用了透视操作,以显得与视频图像的左侧相比,视频图像的右侧更远离用户。 In order to give the rotation phenomena observed pane, UI 1105 reduces the size of the video image of the front camera video display area 1160, and the application operating perspective, appear to the right compared to the left side of the video image, the video image side farther from the user.

[0538] 第六阶段4635图解说明观察窗格已旋转90°,使得用户只能看到窗格的边缘,如显示在显示区1160的中央的细线4686所示。 [0538] sixth stage 4635 illustrates the viewing pane has been rotated 90 °, so that the user can see the edge of the pane, as shown in thin lines in the central display region 1160 of the display 4686. 第七阶段4640图解说明观察窗格继续旋转, 使得观察窗格4688的背面现在逐渐出现在用户面前,以显示从用户的背面照相机拍摄的视频。 Seventh stage 4640 illustrates the viewing pane continues to rotate, so that the viewing pane now back surface 4688 gradually presented to the user, to display video captured from a user's back camera. 同样,在一些实施例中,旋转动画的这种表现是通过减小背面照相机的视频图像在视频显示区4688中的大小,并应用透视操作,以使得与视频图像的右侧相比,视频图像的左侧更远离用户来实现的。 Also, in some embodiments, this performance is the rotation animation video image by reducing the size of the back surface of the camera in the video display area 4688, and operation application perspective, compared to the right so that the video image, the video image the left side farther from the user to achieve.

[0539] 第八阶段4645图解说明表示切换照相机操作的动画的完成。 [0539] 4645 illustrates an eighth stage of the animation indicating the completion of the switching operation of the camera. 具体地说,这个阶段在显示区1160中显示由设备4600的背面照相机拍摄的汽车的视频图像。 Specifically, at this stage in the display area 1160 displays a video image captured by the back camera apparatus 4600 of an automobile.

[0540] 上面参考图46说明的例子通过切换照相机用户界面调用切换照相机操作。 46 illustrate an example of the [0540] above with reference to FIG call switching operation by switching the camera the camera user interface. 其它实施例不同地调用切换照相机操作。 Other embodiments different call switching operation of the camera. 例如,一些实施例通过使切换照相机可选Π项目永久地显示在视频会议期间的UI,例如图47的Π 1105上,调用切换照相机操作。 For example, some embodiments of the camera by switching selectable items [pi permanently display the UI during a video conference, for example, a Π 1105 of FIG. 47, camera operation switch calls. 在图47中, 切换照相机按钮1289连同静音按钮1285和结束会议按钮1287 -起被显示在显示区1155 中。 In Figure 47, the button switch camera mute button 1285 and 1289, together with the conference button 1287-- from being displayed in the display area 1155. 显不区1155的布局和上面参考图12说明的显不区1155的布局相同。 Region 1155 is not substantially the same region without significant layout described above with reference to a layout 1155 of FIG. 12.

[0541] 图47 按照六个阶段:4610、4790、4630、4635、4640 和4645,图解说明Π 1105 的切换照相机操作。 [0541] FIG. 47 according to six stages: 4610,4790,4630,4635,4640 and 4645, illustrate Π camera operation switch 1105. 图47的第一阶段4610类似于图46的第一阶段4610,除了显示区1155 的布局显示静音按钮1285,结束会议按钮1287和切换照相机按钮1289,而不是单一的结束会议按钮之外。 The first stage 4610 of FIG. 47 is similar to FIG. 46 of the first stage 4610, in addition to the layout of the display area 1155 to display a mute button 1285, the conference button 1287 and a switch button 1289 a camera, rather than a single end of the session other than the button. 第二阶段4790图解说明在本地设备4600的用户选择(例如,通过利用手指4670的单指轻点)切换照相机可选Π项目1289之后的Π 1105。 The second stage 4790 illustrates a user selection (e.g., by a single finger with a finger tap 4670) optional switch camera [pi Π 1105 after local devices 4600 1289 projects. 在这个例子中,通过选择可选Π项目1289,用户指令设备4600在视频会议期间,从设备4600的正面照相机切换到设备4600的背面照相机。 In this example, 1289, 4600 instructing device user during the video conference, switch selectable by selecting items from a front Π camera apparatus 4600 to the apparatus 4600 back camera. 图47的后四个阶段与图46的后四个阶段类似,除了显示区1155的布局和上面在第一阶段4610中说明的布局相同之外,于是不再进一步说明,以避免用不必要的详情模糊本发明的描述。 After four stages in FIG. 47 is similar to Figure 46 of four phases, in addition to displaying the same layout and the layout area 1155 described above in addition to the first stage 4610, so will not be further described in order to avoid unnecessary use details obscure the description of the present invention.

[0542] 在一些实施例中,当远程移动设备接收到来自本地双照相机移动设备的不同照相机的图像(即,本地双照相机移动设备切换照相机)时,远程移动设备也进行切换照相机动画,以显示在来自双照相机移动设备的一部照相机的图像的显示和来自本地双照相机移动设备的另一部照相机的图像的显示之间的转变。 [0542] In some embodiments, when the remote mobile device receives an image different cameras from a local dual camera mobile device (i.e., a local dual camera mobile device to switch the camera), the remote mobile device can switch the camera animation to show transition between the display image from the dual camera mobile device of a camera and the other camera image from the local dual camera mobile device. 图48按照Π 4805的五个操作阶段4810、 4815、4820、4825和4830,图解说明这种切换照相机动画之一的例子。 FIG 48 according to the five stages of operation Π 4805 4810, 4815,4820,4825 and 4830, illustrates an example of one such switching of the animation camera. 图48表示远程移动设备4800上的示例切换照相机动画。 48 shows an example of a mobile device 4800 on the remote switch camera animation. 各个操作阶段与图46的示例动画相同,除了是对显示在显示区4835中的图像进行动画之外,显示区4835是在远程移动设备4800上显示来自本地双照相机移动设备的图像的地方。 The various stages of operation of the example of FIG. 46 is the same animation, in addition to the display is the animation in the image display region 4835, 4835 is a display area where an image from a local dual camera mobile device 4800 on the remote mobile device. 因而,显示在显示区4835中的男士的图像被动画演示成仿佛在位于显示区4850中央的垂直轴4855上旋转180°,以表示在显示区4835中的男士的图像的显示和汽车4870的图像的显示之间的转变。 Thus, a display image of men in the display region 4835 is animation as if rotated by 180 ° in the 4850 vertical axis 4855 the center of the display area to show the image display images in the display area 4835 men and car 4870 the transition between the display. 一些实施例的切换照相机动画的实现和上面说明的动画的实现相同。 Some animation camera to achieve the same switching embodiments described above and to achieve animation.

[0543] 上面的例子举例说明在具有特定用户界面布局的远程设备上的切换照相机动画。 [0543] The above example illustrates a switch camera animation on a remote device having a specific layout of the user interface. 其它实施例可在具有不同的用户界面布局的远程设备上实现该切换照相机动画。 Other embodiments may implement the switch camera animation on a remote device with a different user interface layouts. 例如,图49图解说明具有不同的用户界面布局4805的远程设备4800的一个这种例子。 For example, FIG. 49 illustrates one such example having different user interface 4800 of the layout of the remote device 4805. 特别地,图49的Π 4805具有包括在显示区1155中的静音按钮1285、结束会议按钮1287和切换照相机按钮1289,在视频会议期间,所述显示区1155水久地显示在合成显示4850的一侧。 In particular, FIG. 49 Π 4805 having a mute button 1285 included in the display region 1155, the end of the session and a switch button 1287 button 1289 a camera, during a video conference, the display area 1155 shows a long side of the water 4850 is displayed in the synthesis of . 上面参考图48说明了这三个按钮的布局。 Above with reference to FIG. 48 illustrates the layout of the three buttons. 除了不同的用户界面布局之外,图49的五个阶段4810、4815、4820、4825 和4830 与图48 的五个阶段4810、4815、4820、4825 和4830 相同。 In addition to the different user interface layouts, 4810,4815,4820,4825 4810,4815,4820,4825 same five stages 4830 and 4830 of FIG. 48 and FIG. 49 of five stages.

[0544] 2.远程切换照相机 [0544] 2. Remote switch camera

[0545] 图50图解说明在视频会议期间,切换远程双照相机设备的两部照相机的处理5000。 [0545] FIG. 50 illustrates during a video conference, remote switching processing two cameras 5000 dual camera device. 处理5000由包括至少一部照相机的设备的视频会议管理器执行。 Process 5000 performed by the camera device including at least one video conference manager. 在下面的讨论中, 用户通过其指令远程切换照相机的设备被称为本地设备,而切换其两部照相机的设备被称为远程设备。 In the following discussion, the user switches the camera device is called a local device through which a remote command, the device switches its two cameras is called a remote device. 另外,在下面的讨论中,远程设备被认为在其正面照相机(或者说照相机1) 和其背面照相机(或者说照相机2)之间进行切换。 In the following discussion, the remote device is considered on its front camera (or a camera) to switch between the camera and the back surface thereof (or the camera 2).

[0546] 下面参考图51、52、53和54,说明图50的处理5000。 [0546] Next, with reference to FIGS. 52, 53 and 54, process 5000 in FIG. 50 is described. 图51图解说明本地设备5100 的Π 5105,在视频会议期间,用户通过本地设备5100请求远程设备在其两部照相机之间进行切换。 Figure 51 illustrates a device 5100 local Π 5105, during a video conference, a user requests a remote device via the local device 5100 to switch between the two cameras. 图51图解说明Π 5105的八个不同的操作阶段5110、5115、5120、5125、5130、 5135、5140和5145。 FIG 51 illustrates Π 5105 of eight different phases of operation 5110,5115,5120,5125,5130, 5135,5140, and 5145. 图54图解说明从本地设备5100接收切换照相机请求的远程设备5400 的Π 5405。 FIG. 54 illustrates a reception switch 5400 Π 5405 a camera remote device requested from the local device 5100. 图54图解说明Π 5405的六个不同的操作阶段5410、5415、5420、5425、5430 和5435。 FIG 54 illustrates Π 5405 six different phases of operation 5410,5415,5420,5425,5430 and 5435.

[0547] 如图50中所示,通过在本地设备和远程设备之间启动(在5005)视频会议,开始处理5000。 [0547] As shown in FIG. 50, the start (at 5005) a video conference between the local and remote devices, process 5000 starts. 处理5000随后(在5010)接收来自每个设备的一部照相机(例如,来自每个设备的正面照相机)的图像,根据这些图像生成视频会议的合成图像。 Process 5000 receives an image from each camera device (e.g., from each of the front camera device) then (in 5010), generating a composite image according to the video conference images. 在5010,处理5000还把来自本地设备的视频图像发送给远程设备。 In 5010, the process 5000 also video image from the local device to the remote device.

[0548] 随后,处理5000确定(在5015)是否接收到结束视频会议的请求。 [0548] Subsequently, the process 5000 determines (at 5015) whether a request to end the video conference. 如上所述,在一些实施例中,可应本地或远程设备的用户的请求,结束视频会议。 As described above, in some embodiments, the request may be local or remote user of the device, the end of the video conference. 当处理5000接收到结束视频会议的请求时,处理5000结束。 When receiving a request process 5000 to the end of the video conference, the process 5000 ends.

[0549] 当处理5000未接收到结束视频会议的请求时,处理5000随后确定(在5020)在其上执行处理5000的设备的用户(即,本地设备的用户)是否指令该设备请求远程设备在其用于视频会议的照相机之间切换。 [0549] When the process 5000 does not receive a request to end the video conference, the process 5000 then determines (at 5020) in which a user (i.e., user of the local device) device process 5000 is performed on whether an instruction to the device requesting the remote device switching between the camera for the video conference. 当处理5000确定(在5020)它未被指令发起远程切换照相机时,处理5000返回操作5010。 When the process 5000 determines handover to initiate a remote camera (5020) it is not an instruction, the process 5000 returns to operation 5010. 当处理5000确定(在5020)它已被指令发起远程切换照相机时,处理5000进入下面进一步说明的操作5025。 When the process 5000 determines (at 5020) that it has a remote switch camera command is initiated, the process 5000 proceeds to operation 5025 described further below.

[0550] 图51的Π 5105的前四个阶段5110、5115、5120和5125图解说明接收用户的切换远程设备的照相机的请求的例子。 Π [0550] FIG. 51 the first four stages of 5110,5115,5120 and 5105 5125 illustrates an example of handover requests of a camera device receives the remote user. 第一和第二阶段5100和5115与图46的第一和第二阶段4610和4615相同。 4610 and 4615 the same as the first and second stages and the first and second stages 5100 and 5115 of FIG. 46. 第三阶段5120和第三阶段4620相同,除了第三阶段5120不仅包括请求本地设备5100切换照相机的可选Π项目5175,而且包括请求远程设备5100切换照相机的可选Π项目5180之外。 The same as the third stage 5120 and the third stage 4620, in addition to the third stage 5120 comprises only device 5100 requesting the local switch selectable items Π camera 5175, and a remote device 5100 includes a request to switch selectable items Π 5180 a camera outside. 第四阶段5125图解说明本地设备5100的用户选择请求远程设备切换照相机的Π项目5180 (例如,通过可选Π项目5180的单指轻点5170)。 The fourth stage 5125 illustrates the local device 5100 a user selection item 5180 Π requesting remote device to switch cameras (e.g., via an optional item Π of a single finger tap 5180 5170). 通过突出显示可选Π项目5180,指出所述选择。 Optional Π project by highlighting 5180, points out the selection. 图51表示进行这种操作的一个例子,不过其它实施例可不同地进行请求远程设备切换照相机的操作。 51 shows an example of such operation is performed, although other embodiments may operate differently from the remote device a request to switch cameras.

[0551] 上面参考图51说明的例子通过远程切换照相机用户界面,调用远程切换照相机操作。 [0551] Examples described above with reference to FIG. 51 through the remote switching camera user interface, remote switching call camera operation. 其它实施例不同地调用远程切换照相机操作。 Other embodiments variously invoke a remote operation switch of the camera. 例如,一些实施例通过使切换照相机可选Π项目永久地显示在视频会议期间的Π,例如图52的Π 5105上,调用切换照相机操作。 For example, some embodiments of the camera by switching selectable items displayed permanently [pi [pi during a video conference, for example, a Π 5105 of FIG. 52, camera operation switch calls. 在图52中,远程切换照相机按钮5288连同静音按钮5282、结束会议按钮5284和本地切换照相机按钮5286 -起被显示在显示区1155中。 In FIG 52, the remote switch camera button 5288 along with mute button 5282, the conference button 5284 and a local switch camera button 5286-- from being displayed in the display area 1155.

[0552] 图52按照六个不同的阶段5110、5290、5130、5135、5140和5145,图解说明设备5100的Π 5105的远程切换照相机操作。 [0552] FIG. 52 according to six different stages 5110,5290,5130,5135,5140 and 5145, illustrated remote device 5100 Π 5105 camera switching operation. 图52的第一阶段5110类似于图51的第一阶段5110,除了显示区1155的布局显示静音按钮5282、本地切换照相机按钮5286、远程切换照相机按钮5288和结束会议按钮5284之外。 The first stage 5110 of FIG. 52 is similar to FIG. 51 of the first stage 5110, in addition to the layout of the display area 1155 displays a mute button 5282, the local switch camera button 5286, a camera outside remote switching button 5288 and an end button 5284 meeting. 第二阶段5290图解说明在本地设备5100的用户选择(例如,通过单指轻点5170)远程切换照相机可选Π项目5288之后的Π 1105。 The second stage 5290 illustrates remote switch camera selectable items [pi Π 1105 5288 after the user selects the local device 5100 (e.g., by a single finger tap 5170). 图52的后四个阶段类似于图51的后四个阶段,除了显示区1155的布局和上面在第一阶段5110中描述的布局相同之外,于是不再进一步说明,以避免用不必要的详情模糊本发明的描述。 After four stages in FIG. 52 is similar to FIG. 51, after four stages, in addition to displaying the same layout and the layout area 1155 described above, in the first stage 5110, so will not be further described in order to avoid unnecessary use details obscure the description of the present invention.

[0553] -些实施例提供和图52中图解说明的布局类似的布局,除了远程切换照相机可选Π项目显示在PIP显示5165中,而不是显示在显示区1155中之外。 [0553] - Some embodiments provide a layout similar to the layout described and illustrated in Figure 52, except the camera remote switching Π selectable items displayed in the PIP 5165, and not displayed outside the display region 1155. 图53图解说明这样的布局5105。 FIG 53 illustrates such an arrangement 5105. 具体地说,图53表示具有远程切换照相机可选Π项目5180的PIP显示, 和只具有静音按钮5282、本地切换照相机按钮5286和结束会议按钮5284的显示区1155。 More specifically, FIG. 53 indicates optional items Π remote switching camera having a PIP display 5180, and only a mute button 5282, the local switch camera button 5286 button display region and the meeting of 11,555,284.

[0554] 如上所述,当用户请求远程切换照相机时,处理5000进入操作5025。 [0554] As described above, when the user requests a remote switch camera, process 5000 proceeds to operation 5025. 在操作5025, 处理5000把切换照相机的请求发送给远程设备。 At operation 5025, process 5000 requesting to switch the camera to the remote device. 在一些实施例中,该请求是通过如上所述,与音频通道和视频通道一起被VTP管理器多路复用的视频会议控制通道发送的。 In some embodiments, the request is as described above, video and audio channels and a control channel are transmitted together VTP channel multiplexed video session manager.

[0555] 在切换照相机的请求被接收之后,处理5000确定(在5030)远程设备是否响应了切换照相机的请求。 [0555] After the camera switching request is received, process 5000 determines (at 5030) whether the remote device in response to a request to switch cameras. 在一些实施例中,远程设备自动通过视频会议控制通道,向本地设备发送接受响应(即,发送确认)。 In some embodiments, the remote control device automatically via video conference channel, sends an acceptance response (i.e., transmits an acknowledgment) to the local device. 不过,在其它实施例中,远程设备的用户必须通过远程设备的用户界面接受该请求。 However, in other embodiments, the remote device user must accept the request through the user interface of the remote device.

[0556] 图54的Π 5405的前两个阶段5410和5415图解说明远程用户接受切换远程设备5400的照相机的请求的例子。 Π [0556] FIG. 54 the first two stages 5410 and 5415 illustrate an explanatory example of the remote user 5405 accepts the request of switching the camera remote device 5400. 第一阶段5410显示(1)用于显示把所述请求通知远程用户的文本的显示区5440,(2)用于接受切换远程设备的照相机的请求的可选UI项目5465(例如,"允许(Allow)"按钮5465),和(3)用于拒绝切换远程设备的照相机的请求的可选Π项目5470(例如,"拒绝(Reject)"按钮5470)。 The first stage 5410 display (1) for requesting the display area to inform the remote user text 5440, (2) for optional switching the camera UI item 5465 accepts a request of the remote device (e.g., "allow ( the Allow) "button 5465), and (3) for rejecting the handover request 5470 selectable items Π camera remote device (e.g.," reject (reject) "button 5470). 第二阶段5415随后图解说明在远程设备的用户选择(例如,通过单指轻点5480)用于接受切换照相机的请求的Π项目5465之后的Π 5405,通过突出显示可选Π项目5465,指示用户的所述选择。 Then the second stage 5415 illustrates the remote device selected by the user (e.g., by a single finger tap 5480) Π 5405 after receiving a request to switch cameras for the [pi project 5465, by highlighting [pi selectable items 5465, indicating that the user the selection.

[0557] 当处理5000确定(在5030)它未接收到来自远程设备的响应时,处理5000确定(在5035)是否接收到结束视频会议的请求。 [0557] When the process 5000 determines (at 5030) it does not receive a response from the remote device, process 5000 determines (at 5035) whether a request to end the video conference. 如果是,那么处理5000结束。 If so, then the process 5000 ends. 否则,处理接收(在5040)来自远程设备和本地设备的当前所用照相机的图像,根据这些图像产生视频会议的合成图像,把本地设备的视频图像传送给远程设备,以及随后返回操作5030。 Otherwise, the process receives (at 5040) with a camera image from the remote and local devices in the current composite image generated from these images in the video conference, the video images of the local device to the remote device, and then returns to operation 5030.

[0558] 当处理5000确定(在5030)它接收到了来自远程设备的响应时,处理5000确定(在5045)远程设备是否接受切换照相机的请求。 [0558] When the process 5000 determines (at 5030) that it received the response from the remote device, process 5000 determines (at 5045) whether to accept the request the remote device to switch cameras. 如果否,那么处理5000返回操作5010,继续接收来自另一个设备的所述照相机的图像。 If not, the process 5000 returns to operation 5010, the image of the camera continues to receive from another device. 否则,处理5100接收(在5050)来自远程设备的另一部照相机的图像,随后在本地设备上进行(在5055)切换照相机动画,以显示先前利用的远程照相机的视频和当前利用的远程照相机的视频(即,在操作5050的接收图像) 之间的转变。 Otherwise, the process 5100 receives (at 5050) an image of another portion of the camera from a remote device, followed on the local device (at 5055) a switch camera animation to display previously utilized remote video camera and a camera remote current utilization video (i.e., the image received in operation 5050) transitions between. 在操作5055之后,处理返回上面说明的操作5010。 After operation 5055, the process returns to operation 5010 described above.

[0559] 关于图51中的Π 5105图解说明的后四个操作阶段5130、5135、5140和5145图解说明本地设备5100上的这种远程切换照相机动画的一个例子。 [0559] After Π 5105 on FIG. 51 illustrates the four phases of operation described 5130,5135,5140 and 5145 illustrate such remote switch 5100 on the local device example of a camera animation. 该示例动画与在图48的阶段4815、4820、4825和4830中图解说明的示例动画类似,除了图51在显示区5150中,显示用由远程设备的背面照相机拍摄的树的视频,替换由远程设备的正面照相机拍摄的女士的视频的动画之外。 The example illustrated in the example of an animation stage 4815,4820,4825 and 4830 of FIG. 48 described animation similar to Figure 51 except that in the display region 5150, a tree display captured by the back camera of the remote video device, a remote replaced by Ms. animation outside of the front camera video taken by the device. 图52和图53的后四个阶段图解说明与图51中的动画相同的动画,除了图52和53的显示区1155包含与图51中的显示区1155不同的可选Π项目之外。 Figure 52 and 53 illustrate four stages in FIG. 51 and the same animation movie, in addition to FIGS. 52 and 53, the display area 1155 includes a display region 1155 in FIG. 51 projects different selectable Π outside.

[0560] 在一些实施例中,当远程设备切换照相机时,远程设备的Π也进行切换照相机动画,以显示两部照相机之间的转变。 [0560] In some embodiments, the remote device when the camera is switched, the remote device may also be [pi switch camera animation to show the transition between the two cameras. 关于图54中的Π 5405图解说明的后四个操作阶段5420、5425、5430和5435图解说明当远程设备5400切换照相机时,显示在远程设备5400上的切换照相机动画的例子。 After about Π 5405 illustrated in Figure 54 four phases of operation described in 5420,5425,5430 and 5435 illustrate the camera when the remote device when the switch 5400, a switch camera display example on a remote device 5400 animation. 该动画与在图46的阶段4630、4635、4640和4645中图解说明的动画类似,除了显示区5445中的动画用由远程设备5400的背面照相机拍摄的树的视频, 替换由远程设备5400的正面照相机拍摄的女士的视频之外。 The animation and animation stage 46 of FIG. 4630,4635,4640 and 4645 are illustrated in a similar, except that the animation display area 5445 a tree captured by the back camera of the remote video device 5400, replaced by a remote device 5400 of the front Ms. video camera outside shooting.

[0561] 如上所述,图46、47、48、49、51、52、53和54表示在用户界面上执行的切换照相机动画的各种例子。 [0561] As described above, FIG. 46,47,48,49,51,52,53 and 54 show various examples of executing the animation switch camera on the user interface. 在一些实施例中,切换照相机动画导致相应双照相机移动设备的图像处理操作的变化,比如可由视频会议管理器1604和图像处理管理器1608执行的比例缩放、合成和透视失真。 In some embodiments, switch camera animation causes a change in image processing corresponding dual camera mobile device operation, such as by scaling the video conference manager 1604 and the image processing execution manager 1608, the synthesis and perspective distortion.

[0562] C.曝光调整 [0562] C. exposure adjustment

[0563] 在双照相机移动设备和另一个移动设备之间的视频会议期间,不同的实施例提供调整由任意一个移动设备的照相机拍摄的图像的曝光的不同技术。 [0563] During a video conference between the dual camera mobile device and another mobile device, various embodiments provide different techniques to adjust the exposure of a mobile device from any of the images taken by the camera. 一些实施例向双照相机移动设备的用户提供调整由另一个设备的照相机拍摄的图像的曝光的技术,而其它实施例向用户提供调整由双照相机移动设备的照相机拍摄的图像的曝光的技术。 Some embodiments provide an exposure adjustment device captured by the camera an image of another user in the art dual camera mobile device, while other embodiments provide an exposure adjustment captured by the camera in the art dual camera image of a mobile device to a user. 下面将详细说明一些示例性的例子。 Here are some illustrative examples will be described in detail.

[0564] 图55图解说明在视频会议期间,在一些实施例的双照相机移动设备上进行远程曝光调整操作的处理5500。 [0564] FIG. 55 illustrates during a video conference, remote exposure adjustment processing 5500 operation on a mobile device dual camera embodiment. 在下面的讨论中,用户通过其指令远程设备调整其曝光度的设备被称为本地设备。 In the following discussion, the user adjusts the exposure apparatus which is referred to as the local device through which instructs the remote device. 在一些实施例中,处理5500由本地设备的视频会议管理器执行。 In some embodiments, process 5500 performed by a video conference manager of the local device. 另外, 将参考图56、57和58说明处理5500,图56、57和58举例说明本地设备的用户请求远程设备进行曝光调整操作的各种方式。 Further, FIGS. 56, 57 and 58 will be described with reference to process 5500, FIGS. 56, 57 and 58 illustrate the local user device requesting remote device exposure adjustment operation in various ways.

[0565] 如图55中所示,通过在本地设备和远程设备之间启动(在5505)视频会议,开始处理5500。 [0565] As shown in FIG. 55, the start (at 5505) a video conference between the local and remote devices, process 5500 starts. 处理5500随后接收(在5510)来自远程设备的视频,以在本地设备的显示屏上显示。 Process 5500 then receives (at 5510) the video from the remote device to the local device on a display screen of. 随后,处理5500确定(在5515)是否接收到结束视频会议的请求。 Then, the process 5500 determines (at 5515) whether a request to end the video conference. 如上所述,一些实施例能够从本地或远程设备的用户接收结束视频会议的请求。 As described above, some embodiments of a receiving end of the video conference request from a local or remote user device. 当处理5500接收到结束视频会议的请求时,处理5500结束。 When receiving the request process 5500 ends the video conference, the process 5500 ends.

[0566] 不过,当处理5500未接收到结束视频会议的请求时,处理5500随后确定(在5520)是否接收到调整远程设备的照相机的曝光的请求。 [0566] However, when the processing request is not received the end of the 5500 video conference, then the process 5500 determines (at 5520) whether a request for the remote device to adjust the exposure of the camera. 当处理5500确定未接收到调整远程设备的照相机的曝光的请求时,处理5500返回操作5510,接收从远程设备拍摄的另外的视频。 When the process 5500 determines the request is not received the remote device to adjust the exposure of the camera, the process 5500 returns to operation 5510, receive additional captured video from the remote device. 图56、57和58图解说明向用户提供产生这种请求的三个不同例子。 56, 57 and 58 illustrate users to produce three different examples of such a request. 在图56、57和58中,第一阶段5610、5710和5810都显示本地设备5600、5700和5800的PIP显示5625、 5750和5835, PIP显示5625、5750和5835显示两个视频:由本地设备的照相机拍摄的视频,和由远程设备的照相机拍摄的另一个视频。 In FIGS. 56, 57 and 58, the first stage of 5610,5710 and 5810 are 5600,5700 and 5800 the local device display PIP display 5625, 5750 and 5835, 5625,5750 and 5835 show the PIP display two video: by the local device the video taken by the camera, and another video captured by a camera remote device. 在第一阶段5610、5710和5810中,背景显示5635、5760和5845中的男士较暗黑,指示该男士未被恰当曝光。 In the first stage 5810 and 5610,5710, 5635,5760 and 5845 background display in the dark than men, indicating that the man has not been properly exposed.

[0567] 图56的第二阶段561图解说明通过选择远程设备的视频(例如,通过单击背景显示5635),本地设备5600的用户请求远程设备进行曝光调整的一种方式。 The second phase [0567] 561 of FIG. 56 illustrates a video by selecting a remote device, the local device 5600 requesting remote device user a way of exposure adjustment (e.g., by clicking on the background display 5635). 按照这种方式, Π 5605自动使用户对由方框5645限定的感兴趣区域的选择与用户的指令远程设备对感兴趣区域进行曝光调整的期望关联起来,从而指令本地设备的视频会议管理器联系远程设备,以进行曝光调整操作。 In this manner, Π 5605 allows the user to automatically select a user defined region of interest by the instruction block 5645 the remote device to a desired region of interest associated exposure adjustment, such that the instructions local video conference apparatus Information Manager the remote device to perform exposure adjustment operation. 定义的感兴趣区域被远程设备用于曝光调整的计算。 Definition of region of interest is used to calculate the exposure adjustment remote device.

[0568] 类似于图56的第二阶段5615,图57的第二阶段5715表示本地用户对远程设备的视频的选择,除了该选择指令Π 5705显示如第三阶段5720中所示的可选Π项目5770之夕卜。 The second phase [0568] FIG. 56 is similar to the second stage 5615, 5715 of FIG. 57 indicates that the local user selection of a remote video device, in addition to the display selection instruction optional [pi Π 5705 as shown in the third stage 5720 Xi Bu project of 5770. 第四阶段5725图解说明本地设备的用户选择可选Π项目5770,以指令远程设备如上所述进行曝光调整操作。 The fourth stage 5725 illustrates a user selectable local device selection item Π 5770, a remote device to instruct the exposure adjustment operation described above.

[0569] 图58的第二阶段5815类似于图57的第二阶段,不过代替用户对远程设备的视频的选择指令Π显示单一的可选Π项目,用户的选择指令Π 5805显示可选Π项目5855、 5860、5865和580的菜单,如第三阶段5820中所示。 The second phase [0569] FIG. 58 is similar to the second stage 57 5815, but instead of the user of the remote video device displaying single selection instruction [pi [pi selectable items, a user's selection instruction [pi selectable items displayed Π 5805 5855, 5860,5865, and 580 of the menu, as shown in 5820 in the third stage. 可选Π项目包括"自动聚焦(Auto Focus) "项目5855、" 自动曝光(Auto Exposure) "项目5860、"切换照相机(Switch Camera) " 项目5865、和"取消(Cancel) "项目5870。 Π selectable items include "Auto Focus (Auto Focus)" item 5855, "AE (Auto Exposure)" item 5860, "camera switch (Switch Camera)" item 5865, and "Cancel (the Cancel)" item 5870. 在一些实施例中,"切换照相机"可选Π项目5865 被用于请求本地切换照相机操作,而在其它实施例中,"切换照相机"可选Π项目5865被用于请求远程切换照相机操作。 In some embodiments, "camera switch" optionally Π item 5865 is used to request the local camera operation switch, while in other embodiments, "camera switch" optionally Π item 5865 is used to request a remote operation switch of the camera. 第四阶段5825图解说明用户选择"自动曝光,,项目5860,以指令远程设备如上所述进行曝光调整操作。 The fourth stage 5825 illustrates the user selects "AE ,, item 5860, a remote device to instruct the exposure adjustment operation described above.

[0570] 当处理5500确定(在5520)本地用户指令本地设备请求曝光调整操作时,处理5500通过视频会议控制通道向远程设备发送(在5525)命令,以调整由目前拍摄并向本地设备传送视频的照相机拍摄的视频的曝光。 [0570] When the process 5500 determines the local device requests the exposure adjustment operation instruction (in 5520) a local user, the control process 5500 through the channel transmitting device to the remote video conference (at 5525) a command to adjust the current photographing device transmits to the local video cameras captured video of the exposure. 在操作5525之后,处理5500返回上面说明的操作5510。 After operation 5525, the process 5500 returns to operation 5510 described above.

[0571] 在一些实施例中,在远程设备进行曝光调整操作之前,要求远程设备的用户提供许可,而在其它实施例中,当从本地设备接收到请求后,远程设备自动进行曝光调整操作。 Before [0571] In some embodiments, the exposure adjustment operation at a remote device, require the user of the remote device providing the license, while in other embodiments, after receiving from the local device to request the remote device to automatically perform exposure adjustment operation. 此外,在一些实施例中,一些视频会议功能由视频会议管理器1604实现。 Further, in some embodiments, some of the video conference function 1604 is implemented by the video session manager. 在一些这样的实施例中,视频会议管理器1604通过指令CIPU 1650调整正在使用的远程设备照相机的传感器的曝光设置,执行曝光调整操作。 In some such embodiments, the video conference manager 1604 through the command sensor CIPU exposure setting of the camera remote device being used to adjust 1650, the exposure adjustment operation.

[0572] 图56、57和58的最后阶段5620、5730和5830更明亮地显示远程设备的视频,这指示该男士被恰当曝光。 [0572] FIGS. 56, 57 and 58 of the final stage of 5620,5730 and 5830 more brightly display the remote video device, which indicates that the man is properly exposed. 尽管图56、57和58提供接收校正远程设备的曝光的曝光调整请求的例子,不过一些实施例向本地设备的用户提供请求本地设备调整本地设备的照相机的曝光的方式。 Although FIGS. 56, 57 and 58 provide examples of the remote device receives the exposure correction exposure adjustment request, although some embodiments provide a way to request a local device to adjust the exposure of the camera device to a local user of the local device. 可与在图56、57和58中图解说明的请求远程设备调整其照相机的曝光的方式类似地产生这样的请求。 Can adjust the exposure of the camera with the requesting remote device 58 illustrated in FIGS. 56 and 57 and described in a similar manner of generating such a request.

[0573] 上面说明的图56-58表示用于进行曝光调整操作的几种用户界面。 [0573] FIGS. 56-58 described above showing a user interface for performing several exposure adjustment operation. 在一些实施例中,曝光调整操作能够导致双照相机移动设备的图像处理操作的改变,比如调用下面进一步详细说明的曝光调整处理5900。 In some embodiments, the exposure adjustment operation can lead to changes in the image processing operation of the dual camera mobile device, such as is described in further detail below call exposure adjustment processing 5900. 曝光调整操作还能够造成拍摄视频的双照相机移动设备的照相机的操作的改变,比如改变照相机的曝光度设置。 Exposure adjustment operation can also cause a change in the operation of the camera of the dual camera shooting video of the mobile device, such as changing the exposure setting of the camera.

[0574] 1.曝光调整方法 [0574] 1. The method of exposure adjustment

[0575] 图59概念地图解说明由一些实施例的图像处理管理器,比如图16中图解说明的图像处理管理器执行的曝光调整处理5900。 [0575] Figure 59 conceptually illustrates an example of image processing by the manager of some embodiments, such exposure illustrated in FIG. 16 of the image processing manager 5900 performs the adjustment process. 在一些实施例中,处理5900是上面参考图55、 56、57和58说明的曝光调整操作的一部分。 In some embodiments, the process 5900, the part above with reference to FIGS. 56, 57, 55 and 58 illustrate the operation of the exposure adjustment. 在一些这样的实施例中,图像处理管理器1608 执行处理5900,并通过向视频会议管理器1604发送指令调整照相机的曝光设置,所述视频会议管理器1604指令CIPU 1650调整照相机传感器405a或405b,如上所述。 In some such embodiments, the image processing manager 1608 performs processing 5900, and adjust the exposure settings of the camera by sending commands to the 1604 video conference manager, the video conference manager 1604 CIPU 1650 instructions to adjust the camera sensor 405a or 405b, as described above.

[0576] 在一些实施例中,处理5900由图9中所示的图像处理层930执行,而在其它实施例中,处理5900由图4中所示的统计引擎465执行。 [0576] In some embodiments, the processing performed by the image processing layer 5900 shown in FIG. 9, 930, while in other embodiments, process 5900 performed by a statistics engine 465 shown in FIG. 4. 一些实施例对由视频会议中的(本地或远程)设备的照相机拍摄的图像进行处理5900,而其它实施例以图21中图解说明的处理2100的一部分(例如,操作2110)的形式执行处理5900。 Some forms of embodiments of the image photographed by the camera in a video conference (local or remote) device processing 5900, and the other part of the process 2100 are presented to illustrated in FIG. 21 described embodiment (e.g., operation 2110) execution process 5900 . 一些实施例进行曝光调整操作, 以曝光由双照相机移动设备的照相机拍摄的不是太亮,也不是太暗的图像。 Some embodiments perform exposure adjustment operation, to expose dual camera captured by a camera of the mobile device is not too bright nor too dark image. 换句话说,执行处理5900,以按照使细节的数量尽可能最大化的方式拍摄图像。 In other words, the process of 5900, according to the details in order to maximize the number of possible ways of taking a picture.

[0577] 通过接收(在5905)由双照相机移动设备的照相机拍摄的图像,开始处理5900。 [0577] by the image captured by the camera of the dual camera mobile device receives (at 5905), the process 5900 starts. 在一些实施例中,当接收到的图像是由视频会议中的设备的照相机拍摄的第一帧图像时, 不对第一帧图像进行处理5900(即,在第一帧图像之前,不存在根据其确定曝光值的任何图像)。 In some embodiments, when the received image is a first frame image captured by the camera in a video conference device, the image is not the first processing frame 5900 (i.e., prior to the first frame image, which is not present in accordance with determining any image exposure value). 处理5900随后读取(在5910)接收图像中的规定区域的像素值。 Process 5900 then read (in 5910) receiving pixel values ​​in a predetermined region of the image. 不同的实施例不同地定义所述区域。 Different embodiments different from the defined area. 一些这样的实施例定义不同形状的区域,比如正方形、矩形、三角形、圆形等,而其它的这种实施例在图像的不同位置,比如中心、正上方、正下方等定义所述区域。 Some such embodiment defines regions of different shapes, such as square, rectangular, triangular, circular, etc., and such other embodiments different positions of the image, such as the center, right above, defining the area immediately below the other.

[0578] 随后,处理5900计算(在5915)图像的规定区域中的像素值的平均值。 [0578] Then, the process 5900 calculates an average value of pixel values ​​(in 5915) in a predetermined area of ​​the image. 处理5900 确定(在5920)像素值的计算平均值是否等于特定的规定值。 Process 5900 determines (at 5920) whether the calculated average pixel value equal to a certain predetermined value. 不同的实施例定义不同的特定值。 Specific values ​​are defined according to various different embodiments. 例如,一些实施例把所述特定值定义为图像的动态范围的中位数像素值。 For example, some of the specific values ​​defining the dynamic range of the image pixel values ​​median embodiments. 在一些实施例中,定义数值的范围,而不是单个值。 In some embodiments, the defined range of values, rather than a single value. 在这样的实施例中,处理5900确定(在5920)像素值的计算平均值是否在数值的规定范围之内。 In such embodiments within embodiment, the process 5900 determines (at 5920) whether the calculated average value of pixel values ​​in a predetermined range of values.

[0579] 当像素值的计算平均值不等于特定的规定值时,处理5900根据计算平均值调整(在5925)曝光值。 [0579] When calculating the average pixel value is not equal to a certain predetermined value, the process 5900 (in 5925) to adjust the exposure value is calculated according to the average value. 当像素值的计算平均值等于特定的规定值时,处理5900结束。 When the calculated average pixel value equal to a certain predetermined value, the process 5900 ends. 在一些实施例中,曝光值代表照相机传感器被曝光的时间量。 In some embodiments, the exposure value representative of the exposure time of the camera sensor. 在一些实施例中,调整后的曝光值被用于曝光将由拍摄接收图像的照相机拍摄的下一帧图像。 In the next frame some embodiments, the exposure value is used to adjust the exposure shooting is received by the imaging camera image. 在根据计算的平均值调整曝光值之后,处理5900结束。 After adjusting the exposure value calculated based on the average, process 5900 ends.

[0580] 在一些实施例中,重复执行处理5900,直到像素值的计算平均值等于特定的规定值(或者在数值的规定范围内)为止。 [0580] In some embodiments, process 5900 is repeatedly performed until the calculated average pixel value equal to a specific predetermined value (or within a predetermined range of values) so far. 一些实施例在视频会议期间,不断进行处理5900, 而其它实施例在视频会议期间,以规定的时间间隔(例如,5秒、10秒、30秒等)进行处理5900。 In some embodiments, during a video conference, the ongoing process 5900, while other embodiments during a video conference, at predetermined time intervals (e.g., 5 seconds, 10 seconds, 30 seconds, etc.) 5900 for processing. 此外,在视频会议期间,一些实施例的处理5900在进行处理5900之前,动态重新定义特定的像素值。 Further, during a video conference, the processing of some embodiments of the process 5900 is performed prior to 5900, dynamically redefine specific pixel value.

[0581] 图60概念地图解说明一些实施例的曝光调整操作的例子。 [0581] Figure 60 conceptually illustrates an example of some embodiments of the exposure adjustment operation. 例子6000、6010和6015 都把由双照相机移动设备的照相机拍摄的图像6020显示在左侧。 Examples regarded 6000,6010 and 6015 captured by a camera of the dual camera mobile device 6020 displays the image on the left. 具体地说,图像6020显示背对太阳的暗黑人物。 Specifically, the image display 6020 back to the sun's dark figure. 暗黑的人物指示图像的曝光度未高到足以曝光人物的脸部或身体。 Dark figures indicating the image exposure is not high enough exposure character's face or body. 每个例子6000、6010和6015的右侧分别表示在图像6020之后拍摄的图像6025、6030 和6035。 Each example 6000,6010 and 6015 respectively represent the right images 6025,6030 and 6035 after the image captured 6020. 在一些实施例中,图像6020和右侧的图像是由双照相机移动设备的照相机拍摄的视频的图像。 In some embodiments, an image 6020 and the right image is an image captured by a video camera of the dual camera mobile device. 在其它实施例中,图像620和右侧的图像是由双照相机移动设备的照相机在不同时刻拍摄的静止图像。 In other embodiments, the image 620 and the right image by the camera of the dual camera mobile device photographed still image at different times.

[0582] 第一个例子6000图解说明没有任何曝光调整的操作。 [0582] The first example illustrates the 6000 does not have any exposure adjustment operation. 因而,图像6025显得和图像6020相同。 Thus, the image appears 6025 and 6020 the same image. 由于没有进行任何曝光调整,因此和图像6020中的人物类似,图像6025中的人物仍然暗黑。 In the absence of any exposure adjustment, and therefore the image of the characters is similar to 6020, 6025 image of the characters still dark.

[0583] 在第二个例子6010中,对图像6020进行曝光调整操作。 [0583] In the second example 6010 of an exposure adjustment image 6020 operation. 在一些实施例中,曝光调整操作是由处理5900利用规定区域6040进行的。 In some embodiments, the exposure adjustment operation is performed by the processing using a predetermined region 5900 6040. 根据曝光调整操作,照相机的曝光度被调整,照相机利用调整后的曝光度拍摄图像6030。 The exposure adjustment operation, the exposure of the camera is adjusted, the exposure adjustment using the camera captured image 6030. 如图60中所示,图像6030中的人物不如图像6025中暗黑。 As shown in FIG. 60, the character image 6030 of the image 6025 not as dark. 不过,图像6030中人物的脸部和身体仍然不清晰。 However, the person in the image of the face and body 6030 is still not clear.

[0584] 第三个例子6015表示对图像6020进行的曝光调整操作。 [0584] A third example 6015 indicates exposure adjustment operation performed on the image 6020. 类似于第二个例子6010, 一些实施例的例子6015的曝光调整操作是由处理5900利用规定区域6045进行的。 6010 similar to the second example, some examples of embodiments of operation 6015 of the exposure adjustment processing carried out by the use of a predetermined area 6045 of 5900. 根据曝光调整操作,照相机的曝光度被调整,照相机利用调整后的曝光度拍摄图像6035。 The exposure adjustment operation, the exposure of the camera is adjusted, the exposure adjustment using the camera captured image 6035. 如图60中所示,图像6035中的人物被正确曝光,因为人物的脸部和身体都可见。 As shown in FIG 60, the image 6035 is correctly exposed person, because people are visible face and body.

[0585] 在一些实施例中,规定区域的选择可由双照相机移动设备的用户进行。 [0585] In some embodiments, the predetermined region selected by a user of a mobile device dual camera. 设备本身可以通过上面在CIPU 400中提及的曝光调整的反馈回路,自动调整其用于曝光调整操作的规定区域。 The device itself can be adjusted by the exposure in the above-mentioned feedback loop CIPU 400, which automatically adjusts the predetermined region for the exposure adjustment operation. 图4中的统计引擎465可收集数据,以确定曝光度是否适合于拍摄的图像,并据此调整照相机传感器(例如,通过与传感器模块415的直接连接)。 FIG 4 statistics engine 465 can collect data to determine whether the exposure of the captured image is suitable for, and adjusts the camera sensor (e.g., by direct connection with the sensor module 415).

[0586] D.焦距调整 [0586] D. focus adjustment

[0587] 图61图解说明在视频会议期间,调整双照相机移动设备的焦距的处理6100。 [0587] FIG. 61 illustrates during a video conference, the focal length adjustment process 6100 dual camera mobile device. 在下面的讨论中,用户通过它指令远程设备调整远程设备的照相机焦距的设备被称为本地设备。 In the following discussion, it instructs the user via the remote device remote device to adjust the focal length of the camera device is called a local device. 在一些实施例中,图61的处理6100由本地设备的视频会议管理器1604执行。 In some embodiments, process 6100 in FIG. 61 is performed by the local device 1604 of the video conference manager. 另外, 下面将参考图62和63说明处理6100,图62和63为本地设备的用户提供请求远程设备执行焦距调整操作的两种示例方式。 Furthermore, below with reference to FIGS. 62 and 63 illustrate the processing 6100 in Fig. 62 and 63 provides two exemplary ways the requesting remote device performs a focus adjustment operation for the local user equipment.

[0588] 如图61中所示,通过在本地设备和远程设备之间启动(在6105)视频会议,开始处理6100。 [0588] As shown in FIG. 61, the start (at 6105) a video conference between the local and remote devices, process 6100 starts. 处理6100随后接收(在6110)来自远程设备的视频,以便在本地设备的显示屏上显示。 Process 6100 then receives (at 6110) the video from the remote device, a display screen for display on the local device. 随后,在6115,处理6100确定是否接收到结束视频会议的请求。 Subsequently, in 6115, the process 6100 determines whether a request to end the video conference. 如上所述,在一些实施例中,可应本地或远程设备的用户的请求,结束视频会议。 As described above, in some embodiments, the request may be local or remote user of the device, the end of the video conference. 当处理6100接收到结束视频会议的请求时,处理6100结束。 When the process 6100 receives a request to end the video conference, the process 6100 ends.

[0589] 否则,处理6100确定(在6120)它是否接收到调整远程设备的远程照相机的焦距的请求。 [0589] Otherwise, the process 6100 determines (at 6120) if it receives a request to a remote device remote focus adjustment of the camera. 当处理6100确定它未接收到调整远程设备的远程照相机的焦距的请求时,处理6100返回操作6110,接收来自远程设备的另外的视频。 When the process 6100 determines that it has not received a request for the remote device to adjust the focus of a remote camera, process 6100 returns to operation 6110, additional video received from the remote device. 图62、63和64图解说明不同实施例向用户提供的产生这种请求的三种不同方式。 FIGS. 62, 63 and 64 illustrate three different ways in different embodiments of this embodiment generates a request to the user. 在图62、63和64中,第一阶段6210、6310 和6472都显示本地设备6200、6300和6471的PIP显示6225、6335和6482,PIP显示6225、 6335和6482都显示两个视频,由本地设备拍摄的视频,和由远程设备拍摄的另一个视频。 In FIGS. 62, 63 and 64, the first stage of 6210,6310 and 6472 are 6200,6300 and 6471 the local device display PIP display 6225,6335 and 6482, the PIP display 6225, 6335 and 6482 show two videos, by the local video equipment captured, and another video captured by the remote device. 图62和63中的显示区1155和1155显示结束会议按钮。 Figures 62 and 63 show area in 1155 and 1155 show the conference button. 不过,在图64中,显示区1155的布局与上面说明的图12的显示区1155的布局相同。 However, in FIG. 64, region 1155 of the display area and the layout of FIG. 12 described above is the same layout 1155. 此外,可以选择在显示区1155中所示的切换照相机按钮6488,以在一些实施例中调用本地切换照相机操作,或者在其它实施例中调用远程切换照相机操作。 In addition, the option to switch the camera display button 1155 shown in region 6488 to call the local In some embodiments, the switching operation of the camera, or by calling the remote camera operation switch in other embodiments. 如第一阶段6210、6310和6472中所示,显示在背景显示6235、 6345和6480中的远程设备的视频是模糊的。 As shown in the first stage of 6210,6310 and 6472, a video display 6235, 6345 and 6480 of the remote device in the background is blurred.

[0590] 图62的第二阶段6215图解说明通过简单地选择远程设备的视频(例如,通过在远程设备的视频上的单点6240),本地设备的用户向远程设备请求焦距调整的方法。 The second phase [0590] 6215 FIG. 62 illustrates a video by simply selecting the remote device (e.g., by a single point 6240 on the remote video device), a user equipment method of local focus adjustment request to the remote device. 按照这种方法,UI 6205自动把用户对由方框6245限定的感兴趣区域的选择与用户的指令远程设备对所述感兴趣区域执行操作(例如,焦距调整操作)的期望关联起来,从而指令本地设备6200的视频会议管理器1604联系远程设备,以执行调整操作(比如,焦距调整操作)。 According to this method, UI 6205 automatically associated with the user defined region of interest by a block 6245 selects a desired remote device user's instruction to perform an operation of interest (e.g., a focus adjustment operation) regions together, such that the instructions local 6200 of the video conference equipment manager 1604 Contact the remote device to perform the adjustment operation (for example, adjusting the focal length of the operation). 定义的感兴趣区域被远程设备用于焦距调整的计算。 Defined region of interest focus adjustment is used to calculate a remote device.

[0591] 图63的第二阶段6315类似地表示本地用户对远程视频的选择(例如,通过用户对远程设备的视频的轻点)。 The second stage 6315 [0591] FIG. 63 is similarly represented the local user selection of a remote video (e.g., by a tap on a user remote video device). 不过,不同于图62中图解说明的例子,图63中的这种选择指令Π 6305显示如在第三阶段6320中所示的可选Π项目6355、6360、6365和6370 (它们可被实现成可选的按钮)的菜单。 However, unlike the example illustrated in FIG. 62 and FIG. 63 such as a display selection instruction Π 6305 [pi selectable items shown in the third stage 6320 in 6355,6360,6365 and 6370 (which may be implemented as optional button) menu. 这些可选Π项目包括"自动聚焦(Auto Focus)"项目6360、" 自动曝光(Auto Exposure)"项目6365、"切换照相机(Switch Camera)"项目6370 和"取消(Cancel) "项目6355。 These optional items include Π "Auto Focus (Auto Focus)" item 6360, "AE (Auto Exposure)" item 6365, "camera switch (Switch Camera)" and item 6370 "Cancel (the Cancel)" item 6355. 在一些实施例中,"切换照相机"可选Π项目6370被用于请求本地切换照相机操作,而在其它实施例中,"切换照相机"可选Π项目6370被用于请求远程切换照相机操作。 In some embodiments, "camera switch" optionally Π item 6370 is used to request the local camera operation switch, while in other embodiments, "camera switch" optionally Π item 6370 is used to request a remote operation switch of the camera. 第四阶段6325随后图解说明本地用户选择自动聚焦项目6360。 6325 illustrates the fourth stage then selected a local user autofocus item 6360.

[0592] 图64的第二阶段6474再次类似地表示本地用户对远程视频的选择(例如,通过用户对远程设备的视频的轻点)。 The second stage 6474 [0592] FIG 64 again similarly to the local user, a selection of remote video (e.g., by a tap on a user remote video device). 不过,不同于在图63中图解说明的例子,图64中的这种选择指令Π 6478请求焦距调整操作(即,在第二阶段6474中)。 However, in the example illustrated in FIG. 63 differs from the description, in FIG. 64 this selection instruction Π 6478 requests a focus adjustment operation (i.e., in the second stage 6474). 在结束焦距调整操作之后,Π 6478显示可选Π项目6484和6486的菜单(即,在第三阶段6476中),可选Π项目6484和6486可被实现成可选按钮。 After the end of the focus adjustment operation, Π 6478 [pi selectable items displayed menu 6486 and 6484 (i.e., in the third stage in 6476), [pi selectable items 6484 and 6486 may be implemented as selectable buttons. 这些可选Π项目包括"自动曝光"项目6486和"取消"项目6484。 These optional items include Π "AE" Project 6486 and "Cancel" Project 6484.

[0593] 当处理6100确定(在6120)本地用户指令本地设备请求焦距调整操作时,处理6100通过视频会议控制通道,向远程设备发送(在6140)命令,以调整远程设备目前捕获和传送其视频的照相机的焦距。 [0593] When the process 6100 determines the local device requests a focus adjustment operation command (in 6120) a local user, the process 6100 the control channel through the video conference, sends (at 6140) a command to the remote device, to adjust the remote device is currently capturing and transmitting its video the focal length of the camera. 在6140之后,处理返回上面说明的操作6110。 After 6140, the process returns to operation 6110 described above.

[0594] 在一些实施例中,在远程设备进行这种操作之前,远程设备的用户必须提供允许, 而在其它实施例中,当接收到本地设备的请求后,远程设备自动执行这种操作。 Before [0594] In some embodiments, this operation in the remote device, the user must be provided to allow the remote device, while in other embodiments, after receiving a request for the local device, the remote device automatically performs this operation. 另外,在一些实施例中,焦距调整操作调整远程设备的在视频会议期间使用的照相机的焦距设置。 Further, in some embodiments, the focal length focus adjustment operation of the camera during a video conference using the remote adjustment device is provided. 在一些这样的实施例中,如上所述,一些视频会议功能由视频会议模块1602实现。 In some such embodiments, as described above, a number of video conferencing module 1602 is implemented by the video conference. 在这些实施例中,视频会议管理器1604指令CIPU 1650调整正在使用的远程设备照相机的传感器。 In these embodiments, the video conference manager 1604 instructs the sensor device remote camera adjustment CIPU 1650 being used.

[0595] 图62、63和64的最后阶段6220、6330和6476表示恰当聚焦的远程设备的视频。 The last stage of [0595] FIGS. 62, 63 and 64, 6220,6330 and 6476 represent remote video appropriate focusing apparatus. 尽管图62、63和64提供接收校正远程设备的焦距的焦距调整请求的例子,不过,一些实施例允许本地设备的用户请求本地设备调整本地设备的照相机的焦距。 Although the examples of FIGS. 62, 63 and 64 provide focal correction remote device receiving the request for focus adjustment, however, some users embodiments allow the local device requesting the local device to adjust the local device to the focal length of the camera. 可类似于在图62、63 和64中所示的请求远程设备调整其照相机的焦距的方法,产生这样的请求。 May be similar in FIGS. 62, 63 and 64 of the requesting remote device shown in the method of adjusting the focal length of the camera, generating such a request.

[0596] 图62、63和64图解说明允许用户进行焦距调整操作的三种示例用户界面。 [0596] FIGS. 62, 63 and 64 illustrate focal length adjustment allows the user to operate three exemplary user interface. 在一些实施例中,焦距调整操作导致拍摄显示在Π中的视频的双照相机移动设备的照相机的操作的改变,比如改变照相机的焦距。 In some embodiments, the adjustment operation results in a focal length of the shooting display change operation of a camera video dual camera mobile device in Π is, for example changing the focal length of the camera.

[0597] 如上在图56和62中所述,规定的感兴趣区域被远程移动设备分别用于视频的曝光调整和焦距调整的计算。 [0597] As described in FIGS. 56 and 62, a predetermined region of interest is calculated for each remote mobile device to adjust the exposure and focus adjustment of the video. 不过,在一些其它实施例中,用户对感兴趣区域的选择可被用于指令远程设备进行一个或多个操作。 However, in some other embodiments, the user selects a region of interest may be used to command a remote device or more operations. 例如,在一些实施例中,可根据定义的感兴趣区域,进行曝光调整和焦距调整,从而指令远程设备进行两个操作。 For example, in some embodiments, it may be defined according to the area of ​​interest, focus adjustment and exposure adjustment, such that the instructions operate two remote device.

[0598] E.帧速率控制 [0598] E. frame rate control

[0599] 在视频会议期间,一些实施例可能希望调整或保持向视频会议中的另一个设备传送由双照相机移动设备的照相机拍摄的视频图像的速率(即,帧速率)。 [0599] During a video conference, some embodiments rate (i.e., the frame rate) may wish to adjust or maintain the video image photographed by a camera of the dual camera mobile device to transmit the video conference apparatus according to another embodiment. 例如,假定带宽固定,一些这样的实施例降低视频的帧速率,以提高视频图像的画面质量,而其它这样的实施例增大视频的帧速率,以平滑视频(即,减少抖动)。 For example, assume that a fixed bandwidth, some such embodiments reduce the video frame rate, video images to improve picture quality, while other such embodiments increases the frame rate of the video to smooth video (i.e., to reduce jitter).

[0600] 不同的实施例提供在视频会议期间,控制视频图像的帧速率的不同技术。 [0600] The various embodiments provide different techniques during the video conference, to control the video image frame rate. 前面说明的一个例子调整照相机的传感器模块415的VBI,以控制处理由照相机拍摄的图像的速率。 One example of previously described adjustment of the camera sensor module 415 VBI, in order to control the rate of processing of an image photographed by the camera. 作为另一个例子,图9中所示的视频会议模块925的管理层935的一些实施例通过丢弃图像来控制帧速率。 As another example, a video conference shown in FIG. 9 management module 925 of some embodiments 935 to control the frame rate of the image by dropping. 类似地,图像处理层930的一些实施例通过丢弃图像来控制帧速率。 Similarly, some embodiments of the image processing layer 930 is controlled by dropping the image frame rate. 一些实施例提供除此之外的其它技术来控制帧速率,比如丢弃通用传输缓冲器3120中的帧。 Example embodiments provide some other technique other than the frame rate to control, such as the universal transmission buffer 3120 discards the frame.

[0601] V.双照相机 [0601] V. dual camera

[0602] A.组合视图 [0602] A. Combination View

[0603] 1.画中画:两个远程照相机的显示 [0603] 1. PIP: two cameras remote display

[0604] 一些实施例允许双照相机移动设备按照几种显示排列中的任意一种,显示从在视频会议期间,从该移动设备拍摄的视频和从另一个双照相机移动设备拍摄的视频。 [0604] During a video conference from a number of video captured from the mobile device and taken from another mobile device video camera bis embodiment allows dual camera mobile device in accordance with any one of several display arrangement, the display. 图65图解说明从一个或多个双照相机移动设备拍摄的视频的不同显示排列的例子。 FIG 65 illustrates an example of arrangement of different video captured from one or more double camera mobile device display. 在图65中,双照相机移动设备6500(设备A)的用户和另一个双照相机移动设备6505(设备B)的另一位用户正在相互进行视频会议。 In Figure 65, dual camera mobile device 6500 (apparatus A) and another user 6505 double camera mobile device (device B) is another user of another video conference.

[0605] 图65把设备A的显示排列的四种例子表示在左侧。 Example shows four kinds of [0605] FIG 65 A device arrangement is shown in the left. 设备A的四种显示排列是第一视图6510、第二视图6515、第三视图6520和第四视图6525。 A four configurations of the display array 6510 is a first view, the second view 6515, the view of the third and fourth view 6520 6525. 另外,图65还把设备B的显示排列的四种例子表示在右侧。 Further, four kinds of the example shown in FIG. 65 also indicates the device B arranged on the right side. 设备B的四种显示排列是第一视图6565、第二视图6570、 第三视图6575和第四视图6580。 Four kinds of display array device B 6565 is a first view, the second view 6570, the view of the third and fourth view 6575 6580. 在这个例子中,设备A只显示从设备A的照相机拍摄的两个视频,而设备B显示从设备A的照相机拍摄的两个视频,而且显示从设备B的照相机拍摄的一个或两个视频。 In this example, only two display devices A video camera captured from device A, device B displays the two video captured from the camera apparatus A, and displays a video captured from two or camera device B.

[0606] 在第一视图6510中,设备A的UI 6585提供合成显不6512。 [0606] In the first view 6510, UI 6585 provides a synthesis apparatus A 6512 is not significant. 合成显不6512包括两个显示区:显示从设备A的背面照相机拍摄的视频的显示区6530,和显示从设备A的正面照相机拍摄的视频的显示区6535。 Synthesis 6512 does not include significant two display areas: a display area captured from the back of the device A video camera 6530, and display area photographed from the front A video camera device 6535. 在这个例子中,显示区6530位于合成显示6512的上半部,而显示区6535位于合成显示6512的下半部。 In this example, the display area 6530 positioned synthesized and displayed on the half 6512, while the display area 6535 displays located in the bottom half 6512 of synthesis. 在第一视图6510中,这两个显示区大小相同。 In the first view 6510, the same size of the two display regions. 上部的显示区6530正显示山脉的视频,所述山脉假定是正由设备A的背面照相机拍摄的山脉。 The upper portion 6530 of the display area being displayed video mountain, the mountain is assumed to be positive captured by the back camera apparatus A mountains. 显示区6535正显示树和戴帽男士,所述树和戴帽男士假定是正由设备A的正面照相机拍摄的树和戴帽男士 Display region 6535 displays a tree and positive for men wearing a hat, and the tree is assumed to be positive men capped captured by the front camera device A tree men and hats

[0607] 第二视图6515中的UI 6585提供合成显不6517,合成显不6517包括和第一视图6510相同的两个显示区,除了显示区6535 (显示从设备A的正面照相机拍摄的视频)正在位于合成显示6517的上半部,显示区6530 (显示从设备A的背面照相机拍摄的视频)位于合成显示6517的下半部之外。 [0607] UI 6585 in the second view 6515 is not provided significant Synthesis 6517, 6517 does not include significant synthetic view of the same first two display areas 6510, 6535 in addition to the display area (display video captured from the front of the camera device A) the upper half being situated display synthesis 6517, a display area 6530 (shown from the rear face captured video camera device a) is located outside the lower half of the composite display 6517.

[0608] 在第三视图6520中,Π 6585提供PIP显示6595。 [0608] In the third view 6520, Π 6585 provides PIP display 6595. PIP显示6595包括两个显示区:作为背景显示区的、显示从设备A的正面照相机拍摄的视频的显示区6535,和作为前景插入显示区的、显示从设备A的背面照相机拍摄的视频的显示区6530。 PIP display 6595 includes two display areas: as a background display area, display area photographed from the front A video camera device 6535, and a foreground inset display area, displaying a video taken from the back camera of the display device A District 6530. 在这个视图中,背景显示区6535占据大部分的PIP显示6595,而插入显示区6530较小且重叠在一部分的背景显示区6535上。 In this view, the background display area 6535 occupies most of the PIP display 6595, display area 6530 is inserted into the smaller and overlaid on a portion of the background display region 6535.

[0609] 第四视图6525中的Π 6585也呈现包括如第三视图6520中所示的显示区6530 和6535的PIP显示6598。 [0609] Π 6585 in fourth view 6525 also presents a display as shown in the third view includes a region of 6520 6530 and 6598 6535 PIP display. 不同于PIP显示6595,PIP显示6598包括作为背景主显示的显示区6530 (拍摄自设备A的背面照相机),和作为前景插入显示的显示区6535 (拍摄自设备A的正面照相机)。 Unlike PIP display 6595, the PIP display 6598 comprises a display area 6530 as the background main (photographed from the back of the camera device A), and inserted into the foreground display region 6535 as shown in (self-photographing front camera apparatus A). 另外,PIP显示6598是按横向视图呈现的(即,PIP显示598的宽度大于高度)。 Further, PIP display 6598 is presented in landscape view (i.e., PIP display width greater than the height 598).

[0610] 上面的例子图解说明设备A的Π的四种可能的合成视图-其中显示第一设备的两部照相机的视频的两个显示区6530和6535纵向层叠的两个视图,和两个PIP视图。 [0610] The above examples illustrate four Π apparatus A possible synthetic view - showing two views of two display areas of the two video cameras 6530 and 6535 of the first device longitudinally stacked, and two PIP view. 对设备A的Π来说,其它视图也是可能的。 Π of device A, the other views are also possible. 例如,两个显示区可以水平或对角层叠,或者可以使用不同的PIP视图。 For example, two display areas may be stacked horizontally or diagonally, or may use different PIP view.

[0611] 关于设备B图解说明的各个视图表示设备B的Π可以采用不同的视图。 [0611] Device B illustrate various views of about Π illustration showing different views of the apparatus B may be employed. 这些视图包括从设备A的两部照相机,以及设备B的一部或多部照相机拍摄的视频。 These views include a video captured from the two A camera device, and a camera device or a part of the B. 在设备B的第一视图6565中,设备B的Π6590提供PIP显示6568。 In the first view 6565 of the device B, Π6590 apparatus B provides PIP display 6568. PIP显示6568包括与显示在设备A上的合成显不6512相同的合成显不区6569,以及显不由设备B的照相机之一(例如,正面照相机)拍摄的视频的插入显示区6550。 PIP display 6568 comprises a video capture and display, and the one of the camera A synthesis on the device 6512 were not substantially the same synthetic region 6569 does not help significantly device B (e.g., the front camera) in display area 6550 is inserted. 合成显示区6569包括显示从设备A的背面照相机拍摄的视频的显示区6531,和显示从设备B的正面照相机拍摄的视频的显示区6536。 Synthesis of the display area 6569 includes a display region taken from the back of the device A video camera 6531, and display area photographed from the front of the video camera device B 6536. 显示设备A的视频的合成显示6569占据大部分的PIP显示6568,而插入显示区6550较小且重叠在合成显示6569上。 A composite video display device 6569 to display the PIP display occupies most 6568, inserted into a small display area 6550 and displayed superimposed on the synthesized 6569. 显示区6550显示笑脸的视频,所述笑脸假定是正由设备B的正面照相机拍摄其视频的笑脸。 A video display area 6550 displays a smiling face, the smiling face is assumed to be positive captured by the front video camera device B which faces.

[0612] 第二视图6570中的设备B的Π 6590提供PIP显示6572。 [0612] Π 6590 B apparatus 6570 in the second view 6572 provides PIP display. PIP显示6572包括显示区6550 (显示从设备B的正面照相机拍摄的视频),和具有显示从设备A的照相机拍摄的视频的显示区6531和6536的合成显示6573。 Synthesis PIP display area 6572 includes a display 6550 (shown from the front shooting video camera apparatus B), and having a video display from the captured camera device A 6531 and a display area 6536 to display 6573. 合成显示6573与设备A的第二视图6515 中的合成显示6517相同,并占据大部分的PIP显示6572。 Synthesis Synthesis of the second display device view 65156573 A in the same display 6517, and 6572 occupies most of the PIP display. 和第一视图6565中的PIP显示6568中一样,显示区6550较小且重叠在合成显示6573上。 And a first view of the PIP display 6565 as in 6568, and a small display area 6550 superimposed on the composite display 6573. 具体地说,在这两个视图中,显示区6550重叠在显示从设备A的背面照相机拍摄的视频的显示区6531的一部分上。 Specifically, in both views, a portion of the display area 6550 superimposed on the video display area captured from the back surface of the camera device 6531 of the A.

[0613] 在第三视图6575中,Π 6590提供与设备A的第三视图6520中的PIP显示6595 类似的PIP显示6577。 [0613] In the third view 6575, Π 6590 provides apparatus A third PIP display view 6520 6595 6577 similar to the PIP display. PIP显示6577还包括另外的显示区6550,作为第二插入显示区,第二插入显示区6550重叠在背景显示区6536上。 PIP display 6577 further includes a further display area 6550, the second display area is inserted, is inserted into the second display area 6550 superimposed on the background display region 6536. 两个插入显示区6531和6550水平平铺在背景主显示区6536的底部。 Insert two display areas 6531 and 6550 tiled background level of the bottom region 6536 of the main display.

[0614] 第四视图6580中的Π 6590提供合成显示6582。 Π 6590 [0614] 6580 in fourth view synthesis provides a display 6582. 合成显示6582包括三个显示: PIP显示6583、显示区6550和显示区6540(例如,用于显示由设备B的背面照相机拍摄的视频)。 Composite display 6582 comprises three display: PIP display 6583, a display area 6550, and a display region 6540 (e.g., for displaying video captured by the back camera device B). PIP显示6583与设备A的第四视图6525中的PIP显示6598相同,并占据大部分的合成显示区6582。 65256583 fourth view PIP display device A in the same PIP display 6598, and occupies most of the composite display area 6582. 显示区6540和6550较小,并且水平平铺在PIP显示区6583之下。 6540 and 6550 a small display area, and level tile under PIP display area 6583.

[0615] 尽管图65图解说明了设备B的四种可能视图,不过其它许多视图也是可能的。 [0615] Although FIG. 65 illustrates a view of the device B of the four possible, but many other views are possible. 设备A的视频的背景合成显示可以水平平铺,而不是垂直平铺,插入显示区可重叠在设备A 的正面照相机显示区上,而不是重叠在背面照相机显示区上,较大的显示区可以显示设备B 的照相机的视频,而不是设备A的照相机的视频,插入显示区可以位于不同的位置,等等。 BACKGROUND A video synthesizing apparatus can display tile horizontally, instead of vertically tiled display area may overlap inserted in front of the camera device on the display area A, and not overlapping the display region on the back surface of the camera, the display region may be larger the video display apparatus B of the camera, and the camera is not a video apparatus a, the display region may be inserted at different positions, and the like.

[0616] 源于设备A的每个视图的每组箭头6560表明不存在对表示在设备A上的显示和表示在设备B上的显示之间的关联性的要求。 [0616] A view of the device from each of each arrow indicates the absence of 6560 denotes a display device on the requirements of A and represents the association between the display device on the B. 例如,即使设备A正在按视图6510的排列显示区视频(例如,按照设备A的用户选择的排列),设备B也可按图解说明的四种排列,或者按图65中未示出的多种其它排列任意之一显示视频(例如,按照设备B的用户选择的排列)。 For example, even if the apparatus A is in the arrangement view of a video display area 6510 (e.g., the user selects a device in the arrangement of A), B can also be the device illustrated arrangement of four or more in press 65 is not shown in FIG. arrangement of any one of the other video display (e.g., a user selection device according to the arrangement B). 换句话说,设备A的显示排列与设备B的显示排列无关。 In other words, the display device with the arrangement of the apparatus A B regardless of the arrangement. 一些实施例不把显示区从一个设备传送给另一个设备,而只是传送视频(例如,以编码的形式),所述视频被设备显示在其对应的显示区中。 Some embodiments do not display region from one device to the other device, but only transmit video (e.g., in the form of coding), the video is displayed on the display device in its corresponding region.

[0617] 2.专用PIP [0617] 2. Dedicated PIP

[0618] 一些实施例允许双照相机移动设备的用户在视频会议期间,在PIP显示中把视频的前景叠加在另一个视频上。 [0618] Some embodiments allow a user double camera mobile device during the video conference, the PIP display the foreground video is superimposed on another video. 在一些实施例中,视频的前景按照这样的方式混合到另一个视频中,使得它们表现为由单一的照相机拍摄的单一视频的显示。 In some embodiments, the foreground video mixing in such a manner to another video, so that they exhibit a single video display by a single camera. 图66图解说明在PIP显示中,插入视频的前景在背景视频上的这种叠加的例子。 Figure 66 illustrates the PIP display, insert video prospect Examples of such superimposed on the background video.

[0619] 图66 按照Π 6670 的七个操作阶段6620、6625、6630、6635、6640、6660 和6665, 图解说明这种视频叠加操作。 [0619] FIG. 66 according to the seven stages of operation 6620,6625,6630,6635,6640,6660 Π 6670 and 6665, illustrates such a video overlay operations. 第一阶段6620图解说明在与远程设备的视频会议期间,具有PIP显示6682的双照相机移动设备6600的Π 6670。 The first stage 6620 is illustrated with remote devices during a video conference, having a dual camera PIP display Π 6670 6600 6682 of the mobile device. 如第一阶段6620中所示,PIP显示6682包括两个视频显示:背景主显示6610和前景插入显示6605。 As shown in the first stage 6620, PIP display two video display 6682 includes: the foreground and background main display 6610 display 6605 is inserted. 背景主显示6610占据大部分的Π 6670,而前景插入显示6605较小且重叠在背景主显示6610上。 BACKGROUND main display occupies most 6610 Π 6670, 6605 whereas the smaller inset display the foreground and background superimposed on the main display 6610.

[0620] 在这个例子中,背景显示区6610显示山脉的视频,所述山脉假定是由远程设备的照相机之一拍摄的山脉。 [0620] In this example, the background display area 6610 displays a video mountain, the mountain assumed to be captured by one camera remote device mountains. 前景插入显示区6605显示戴帽人物的视频,所述人物假定是由本地设备的照相机之一拍摄其视频的人物。 The foreground inset display area 6605 displays the video wearing a hat person, the person assumed to be captured by one of its video camera of a local character device. PIP显示6682下面是标记为"End Conference (结束会议)"的可选Π项目6685 (例如,按钮6685),所述可选Π项目6685允许用户通过选择该项目(例如,通过单击或双击该按钮),结束视频会议。 The following is the PIP display 6682 labeled "End Conference (the meeting)" Π selectable item 6685 (e.g., button 6685), the selectable Π item 6685 allows the user to select the item (e.g., by clicking or double-clicking the button), the end of the video conference.

[0621] 第二阶段6625图解说明可选菜单6675的调用。 [0621] The second stage 6625 illustrates an optional menu call of 6675. 在一些实施例中,通过选择(例如触摸)PIP显示区6682,可以调用可选Π项目6675的菜单。 In some embodiments, by selecting (e.g. touching) the PIP display area 6682, 6675 can call a menu item selectable Π. 代替这种调用操作,或者与这种调用操作结合,一些实施例还允许用户通过其它操作,比如通过不同的触屏操作或者利用设备的一个或多个其它物理输入,调用可选UI项目6675的菜单。 Instead of such a call operation or operation in conjunction with such calls, some embodiments also allow the user to other operations, such as by different touchscreen operation or one or more other physical input device use, the call selectable UI item 6675 menu.

[0622] 第三阶段6630显示Π 6670, Π 6670具有调用的一组用于选择视频叠加操作的可选Π项目。 [0622] The third stage 6630 display Π 6670, Π 6670 has a set of operations for selecting a video overlay selectable items [pi call. 在这个例子中,在PIP显示6682上显示具有几个可选Π项目的弹出式菜单6675。 In this example, a display having several optional pop-up menu item 6675 Π 6682 on the PIP. 可选Π项目的菜单6675包括"Flip PIP (翻转PIP)"可选Π项目6640(例如,按钮6640),"Specialized PIP (专用PIP) "可选Π项目6645 (例如,按钮6645),和"Cancel (取消)"可选Π项目6690(例如,按钮6690)。 Π selectable menu items 6675 including "Flip PIP (the PIP flip)" Π selectable item 6640 (e.g., button 6640), "Specialized PIP (the PIP-specific)" Π selectable item 6645 (e.g., button 6645), and " cancel (cancel) "optional Π item 6690 (eg button 6690). 在这个例子中,选择"Flip PIP"按钮6640会导致Π 6670交换背景显示6610和插入显示6605 (在下一节中将详细讨论),选择"Specialized PIP"按钮6645会导致Π 6670开始视频叠加的操作,以及选择"Cancel" 按钮6690会从PIP显示6682中除去弹出式菜单6675。 In this case, select "Flip PIP" button 6640 will result in switching the background display 6610 Π 6670 and 6605 show insert (discussed in detail in the next section), select "Specialized PIP" button 6645 will cause the video overlay Π 6670 starts operation and selecting "Cancel" button 6690 will pop-up menu is removed from the PIP display 66,756,682. 其它实施例在PIP弹出式菜单6675 中包括不同的或者更多的项目。 Other embodiments include a different or more items in the pop-up menu 6675 in the PIP.

[0623] 第四阶段6635图解说明在用户选择了"Specialized PIP"按钮6645(例如通过用他的手指6695轻点按钮6645)之后的Π 6670。 [0623] In the fourth stage 6635 illustrates the user has selected the "Specialized PIP" button 6645 (e.g. 6695 with his finger by tapping the button 6645) after Π 6670. 通过在Π显示6670上突出显示按钮6645,指示这种选择。 By highlighting a button 6645 on 6670, indicating that the selected display Π. 一些实施例使用不同的指示显示(例如,突出显示所选项目的边框或者所选项目中的文本)。 Some embodiments use different indication display (e.g., text highlighting the selected item or selected items frame).

[0624] 第五阶段6640表示在开始视频叠加操作之后的Π 6670。 [0624] The fifth stage 6640 represents Π 6670 after the start of the video overlay operations. 在这个阶段,Π 6670 允许用户选择他想要从哪个视频进行提取以作为前景,和他想要使用哪个视频作为叠加视频中的背景。 At this stage, Π 6670 allows users to choose what he wants to be extracted from the video as a prospect, and which he wants to use as a background video overlay video. Π 6670通过显示在PIP显示6682上的具有几个可选Π项目的弹出式菜单6680,提供各种选项。 Π 6670 by displaying pop-up menu with several optional items on Π 66,826,680, offers a variety of options displayed in the PIP. 可选UI项目的弹出式菜单6680包括"Select Inset (选择插入显示)"可选Π项目6655 (例如,按钮6655),"Select Main (选择主显示)"可选Π项目6650 (例如,按钮6650),和"Cancel"可选Π项目6692 (例如,按钮6692)。 Selectable UI item pop-up menu 6680 includes a "Select Inset (inset display selection)" Π selectable item 6655 (e.g., button 6655), "Select Main (main display selection)" Π selectable item 6650 (e.g., button 6650 ), and "Cancel" optional Π item 6692 (eg button 6692).

[0625] 选择"Select Inset"按钮6655会导致UI 6670把来自本地设备的照相机的插入视频6605的前景(S卩,戴帽的男士)叠加在来自远程设备的照相机的背景主视频6610上。 [0625] Select "Select Inset" button 6655 leads to insertion foreground video 6605 UI 6670 put the camera from the local device (S Jie, capped men) superimposed on the camera from a remote device 6610 background main video. 另一方面,选择"Select Main"按钮6650会导致UI 6670把来自远程设备的照相机的背景主视频6610的前景(S卩,山脉)叠加在来自本地设备的照相机上的插入视频6605上。 On the other hand, choose "Select Main" button 6650 will lead to the foreground the background of the main video 6610 UI 6670 Ba camera from a remote device (S Jie, mountains) superimposed on the video is inserted in the camera 6605 from the local device. 在一些实施例中,这导致两个视频推送(feed)的切换,使得目前在插入显示区6605中的视频将占据大部分的Π 6670,目前在主显示区6610中的视频将被叠加在马上的主视频上。 In some embodiments, this results in two video push (Feed) handover, such that the currently inserted in the video display area 6605 occupies most of the Π 6670, the current in the main video display area 6610 will now be superimposed the main video. 选择"Cancel"按钮6692会放弃视频叠加操作,并从PIP显示区6682去除弹出式菜单6680。 Select "Cancel" button 6692 will give up video overlay operations, and remove the District 6682 6680 pop-up menu from the PIP display.

[0626] 第六阶段6600图解说明在用户选择"Select Inset"按钮6655 (例如,通过用他的手指6695轻点按钮6655)之后的Π 6670。 [0626] In the sixth stage 6600 illustrates the user selects "Select Inset" button 6655 (e.g., with his finger 6695 by tapping the button 6655) Π 6670 after. 通过在Π显示6670上突出显示按钮6655, 指示这种选择。 By highlighting a button 6655 on 6670, indicating that the selected display Π. 一些实施例利用不同的指示显示(例如,突出显示所选项目的边框或者所选项目中的文本)。 Some embodiments utilize different instructs the display (e.g., text highlighting the selected item or selected items frame).

[0627] 第七阶段6665图解说明在完成视频叠加操作之后的Π 6670。 [0627] 6665 illustrates the seventh stage Π 6670 after completion of the operation of the video overlay. 如Π 6670中所示,从显示区6605中提取出插入显示区6605的前景(即,戴帽的男士)。 As shown in Π 6670 extracts the foreground (i.e., capped men) into the display area from the display region 6605 in 6605. 另外从屏幕中消除插入显示6605的窗框和背景(S卩,除所述前景外的其它一切)。 Further elimination of the insertion frame and the background display 6605 (S Jie, everything else except the foreground) from the screen. 最后,按照这样的方式把前景(即,戴帽的男士)混合到背景视频6610中,使得它表现为单一视频。 Finally, in such a way foreground (ie, wearing a hat men) blend into the background video in 6610, so that it behaves as a single video. 可以使用各种不同的技术去除插入视频的背景。 Various techniques can be used to insert a video background removal. 一些实施例识别相对于其它像素不移动的像素、寻找恒定不变的图案或颜色、使用与包括前景的图像相比的基线图像、并扣除差异、或者使用不同的技术。 Some embodiments identify pixels with respect to other embodiments not moving pixels, looking for a constant color or pattern, as compared to baseline image using an image including the foreground, and subtract the difference, or using different techniques.

[0628] 尽管图66的例子图解说明当被叠加在背景显示区6610上时,插入显示区6605的前景保留在Π中的相同位置,不过这只是叠加能够如何实现的一个例子。 [0628] Although FIG. 66 illustrates an example of description when superimposed on the background display region 6610, inserted into the foreground display region 6605 remains in the same position in Π, but this is only capable of superimposing an example of how to implement. 一些实施例把前景视频移动到Π 6670中的特定位置(例如,中心、角落之一,等等)。 Some embodiments of the foreground video Π 6670 moves to a particular position (e.g., center, one corner, etc.). 类似于在IV.A.1和IV. A. 3节中所示的特征,一些实施例允许本地设备的用户在Π中来回拖动叠加的前景视频,或者改变叠加的前景视频的大小。 And similar IV IV.A.1. Features shown in Section 3 A., some embodiments allow the user drags a local device back and forth Π superimposed foreground video, or change the size of the foreground video overlay.

[0629] 可以使用不同的技术来确定视频图像的哪个或哪些部分是如上所述的视频叠加操作的"前景"。 [0629] Different techniques may be used to determine which portion of the video image or what is the "foreground" video overlay operation as described above. 一些实施例的一个这种方法确定视频图像的哪个或哪些部分(如果有的话)是动态的。 Some such a method embodiment determines which of the video image portion (if any) is dynamic. 动态部分被认为是"前景",因为视频图像的背景通常是静态的(即,无运动)。 Dynamic section is considered a "prospect", because the background of the video image is generally static (ie, no movement). 在这样的实施例中,在特定的一段时间内分析视频图像。 In such an embodiment, the analysis of the video image within a specific period of time. 如果所述一段时间内,特定像素的值之间的差异不大于规定的阈值(例如,),那么认为所述特定像素是静态像素。 If within the period of time, the difference between the value of a particular pixel is not larger than a predetermined threshold value (e.g.,), so that the particular pixel is a static pixel. 在分析了视频图像中的每个像素之后,视频图像的动态像素(即,不是静态的像素)被认为是视频图像的"前景"。 After analyzing each pixel in the video image, dynamic pixel (ie, not static pixel) video image is considered a "prospect" of the video image.

[0630] 图67图解说明能够由视频会议管理器1604或者图像处理管理器1608执行的、确定视频图像的前景的这种技术的一个例子。 [0630] An example of such technique to determine foreground video image of FIG. 67 illustrates a video conference can be performed by the manager or the image processing manager 1604 1608. 具体地说,图67图解说明显示戴帽的人物和树的视频的一系列六个图像6705-6730。 Specifically, Figure 67 illustrates the display of a series of six video images 6705-6730 figure wearing a hat and trees. 在这个例子中,假定所述人物不是完全静止地站立着的,可能正在说话。 In this example, assume that the character is not standing perfectly still, and might speak. 如上所述,分析视频图像中的每个像素,以确定像素是动态的还是静态的。 As described above, analysis of the video image for each pixel, the pixel is determined to be dynamic or static. 例如,确定图像6705-6730中的像素6735的值之间的差异是否大于规定的阈值。 For example, it is determined whether the difference between the values ​​of pixels in the image 6705-6730 6735 is greater than a predetermined threshold value. 这里,由于像素6735代表除人物之外的部分地面,因此认为像素6735是静态的。 Here, since the pixel section 6735 represents the ground in addition to the figures, it is considered that the pixel 6735 is static. 在分析了图像6705-6730中的所有像素之后,确定图像中的人物是动态的,以及图像的其余部分是静态的。 After analyzing all the pixels in the image 6705-6730, it is dynamic, and the rest of the image is static and determined person in the image. 因而,所述人物是将由上面参考图66说明的操作提取的"前景"。 Thus, the person operating by 66 explained above with reference to FIG extracted "Prospects."

[0631] 3.交换画中画显示中的视频 [0631] 3. The video PIP display is switched

[0632] 一些实施例允许双照相机移动设备的用户在视频会议期间,交换PIP显示中的两个显示区(即,在PIP显示中,插入显示区变成背景显示区,背景显示区变成插入显示区)。 [0632] Some embodiments allow a user of a dual camera mobile device during the video conference, the exchange PIP display two display areas (i.e., in a PIP display, the display area is inserted into the background display area, the background display area is inserted into display area). 图68图解说明在视频会议期间,交换PIP显示6682中的插入显示区6605和背景显示区6610的例子。 Figure 68 illustrates during a video conference, an example of switching PIP display area 6610 is inserted into the display area 6682 and the background display 6605.

[0633] 图68按照图66中的设备6800的Π 6670的八个操作阶段,图解说明交换PIP操作。 [0633] FIG. 68 according to the eight stages of operation of the apparatus 66 of FIG Π 6670 6800, illustrates the exchange of PIP operation. 图68中的前三个阶段与图66中的前三个阶段相同。 FIG 68 is the same as the first three phases of the first three stages in FIG. 66. 在这些阶段中,通过利用本地设备的触摸屏进行选择,用户使菜单6675出现在Π 6670内。 In these stages, the local device is selected by using a touch screen, a user makes a menu 6675 appears in Π 6670.

[0634] 图68的第四阶段6840图解说明在用户选择"Flip PIP"按钮6640(例如,通过用他的手指6695轻点按钮6640)之后的Π 6670。 Phase IV [0634] 6840 FIG. 68 illustrates the user has selected "Flip PIP" button 6640 (e.g., with his finger by tapping the button 6695 6640) after Π 6670. 通过在Π显示6670上突出显示按钮6640,指示这种选择。 By highlighting a button 6640 on 6670, indicating that the selected display Π. 一些实施例利用不同的指示显示(例如,突出显示所选项目的边框或者所选项目中的文本)。 Some embodiments utilize different instructs the display (e.g., text highlighting the selected item or selected items frame).

[0635] 第五阶段6845图解说明在开始交换PIP操作之后的Π 6670。 [0635] 6845 illustrates the fifth stage after the start of the exchange Π 6670 PIP operation. 一些实施例通过翻转动作,动画演示插入显示6605和背景显示6610的交换。 Some embodiments by reversing the operation, inserting animation display and the background display 6605 6610 exchange. 图68图解说明这种动画演示的一个例子。 FIG 68 illustrates an example of such an animated presentation. 在这个例子中,通过PIP显示6682(在执行交换操作之前)在其一面上,新的PIP显示6684(在执行交换操作之后)在其另一面上的观察窗格的翻转,能够说明这种动画。 In this example, the PIP display 6682 (before performing the swap operation) on one side, the new PIP display 6684 (after performing the exchange operation) of the other flip viewing pane surface, such animation can be described . 观察窗格围绕位于PIP显不6682的中心的垂直轴6686旋转180°。 Viewing pane 180 ° rotation about a vertical axis without significantly PIP 6682 6686 located in the center. 在第五阶段6845, 观察窗格开始围绕垂直轴6686旋转。 In a fifth stage 6845, viewing pane 6686 starts rotation about a vertical axis.

[0636] 在第六阶段6850,观察窗格被表示成已旋转大约90°。 [0636] In a sixth stage 6850, it is shown as a viewing pane has been rotated about 90 °. 这是用显示在屏幕中心的细线6688 ( S卩,观察窗格的边缘)指示的。 This is a display 6688 (S Jie, observe the edge of the pane) in the center of the screen indicated by a thin line. 第七阶段6855图解说明观察窗格的旋转接近于完成。 Seventh stage 6855 illustrates the rotation of the viewing pane nearing completion. 新的PIP显示6684开始从观察窗格的另一面出现,并水平扩展到充满设备的屏幕。 The new PIP display 6684 began to emerge from the other side of the viewing pane, and extends horizontally to fill the screen of the device. PIP显示6684包括在进行交换操作之后的两个显示区6605和6610。 PIP display 6684 includes two display areas is performed after the 6605 and 6610 switching operations. 呈现戴帽男士的视频(来自本地设备的照相机)的显示区6605现在在PIP显示6684的背景中,呈现山脉的视频(来自远程设备的照相机)的显示6610现在在PIP显示6684的前景中,所述前景重叠在显示6605上。 Render the video (camera from the local device) capped men display area 6605 is now in the PIP display background 6684's, presented Mountains video (camera from the remote device) display 6610 now shows prospects 6684's, as in PIP said superimposed foreground on the display 6605. 第八阶段6860表示交换显示操作的完成。 The eighth stage 6860 represents a display switching operation is completed.

[0637] 普通技术人员会认识到图68中所示的动画只是PIP插入/背景交换操作的多种可能动画中的一种。 [0637] ordinary skill in the art will recognize that the animation shown in FIG. 68 only PIP insert / animation plurality of possible switching operations background. 例如,不同的实施例可以沿着水平轴旋转观察窗格、瞬间交换这两个显示区、扩展一个显示区同时缩小另一个显示区,等等。 For example, different embodiments can be viewed along a horizontal axis of rotation pane, exchange instant the two display areas, the display area while reducing a spreading another display area, and the like. 一些实施例提供一种总是用于交换操作的动画,而其它实施例允许用户从几种动画中选择,或者使用不同的动画(例如,通过随机选择)。 Some embodiments provide a swap operation is always used for animation, while other embodiments allow a user to select from among several animations, or use a different animation (e.g., by random selection). 此外,交换操作能够导致双照相机移动设备的图像处理操作的变化,比如导致视频会议管理器1604响应用户的输入,改变视频的比例缩放和合成。 Further, the switching operation can cause changes in the image processing operation of the dual camera mobile device, such as a video conference manager 1604 results in response to user input, change the video scaling and synthesis.

[0638] 4.锁定到角落 [0638] 4. The corner lock to

[0639] 本发明的一些实施例允许双照相机移动设备的用户通过来回移动形成合成显示的一个或多个显示区,修改合成显示。 [0639] Some embodiments allow a user double camera mobile device according to the present invention for forming one or more display regions displayed by the synthesized moving back and forth, modified composite display. 上面在IV. A.1节中说明了这种移动的一个例子。 The above illustrates an example of such movement in IV. A.1 section. 当PIP显示包括多于一个的插入显示区时,插入显示的这种移动也是可能的。 When PIP display includes more than one display area is inserted, this insertion movement display is also possible.

[0640] 图69图解说明在视频会议期间执行的这种例子。 [0640] FIG. 69 illustrates an example of this is performed during a video conference. 在图69中图解说明的例子与在图3中图解说明的例子类似,除了图69图解说明来回移动包括两个插入显示区6905和6910,而不是仅仅一个这种插入显不区的PIP显不6965的插入显不区6910之外。 In the example illustrated in Figure 3 similar to that described in the example illustrated in FIG. 69, except that FIG. 69 illustrates a traverse comprises two insertion display areas 6905 and 6910, rather than just one such insertion region not significant not significant PIP 6965 were not inserted into the region 6910 outside.

[0641] 在图69中,在与另一个设备的远程用户的视频会议期间,移动设备6900的Π 6960呈现PIP显示6965。 [0641] In FIG. 69, the remote user during a video conference with another device, the mobile device presents Π 6960 6900 6965 PIP display. 图69中的PIP显示6965包括三个视频显示:背景主显示6915 及两个前景插入显示6905和6910。 FIG 69 PIP display 6965 includes three video display: background main display 6915 and 6905 and two foreground inset display 6910. 在这个例子中,背景主显示6915呈现边唱边弹吉它的人物的视频,所述视频被假定是由远程设备的背面照相机拍摄的视频。 In this example, the background main display 6915 presents a video of a person singing play the guitar, the video is assumed to be a video captured by the back camera of the remote device. 前景插入显示6905 呈现拿着球拍的人物的视频,在这个例子中,该视频被假定是由本地设备的背面照相机拍摄的视频。 Foreground inset display 6905 presents a video of a person holding a racket, in this case, the video is assumed to be captured by the video camera on the back of the local device. 另一个前景插入显示6910呈现戴着帽子的人物的视频,在这个例子中,该人物假定是由本地设备的正面照相机拍摄其视频的人物。 Another foreground inset display 6910 presents a video of a person wearing a hat, in this case, the person is assumed to be captured by the local device's front camera video of their characters. PIP显示6965下面是标记为"End Conference"的可选Π项目6970 (例如,按钮6970),可选Π项目6970允许用户通过选择它而结束视频会议。 PIP display 6965 Here is labeled "End Conference" optional Π item 6970 (eg button 6970), optional Π Project 6970 allows users to select it and end the video conference.

[0642] PIP显示6965只是呈现由远程设备和本地设备拍摄的视频的合成视图的一种方式。 [0642] PIP display 6965 is only one manner of presenting a composite view captured by the remote and local devices video. 一些实施例可以提供其它的合成视图。 Some embodiments may provide other composite views. 例如,代替具有来自远程设备的视频的较大背景显示6915,较大的背景显示6915可以是来自本地设备的视频,较小的前景插入显示6905 和6910可以是来自远程设备的视频。 For example, instead of having large background video from the remote display device 6915, a larger background display 6915 may be a video from a local device, smaller foreground inset displays 6905 and 6910 may be a video from the remote device. 另外,一些实施例允许本地视频和远程视频出现在插入显示6905和6910在一侧,背景显示6915在另一侧,或者所有这三个显示并排排列的Π 6910中。 Further, some embodiments allow the local and remote video appears in the video display 6905 and 6910 on the insertion side, on the other side of the background display 6915, three or all of [pi 6910 arranged side by side in the display. 在其它实施例中,PIP显示6965可包含较大的背景显示6915和/或较小的前景插入显示。 In other embodiments, PIP display 6965 may comprise a larger background display 6915 and / or smaller foreground inset display. 在一些实施例中,PIP显示6965或者默认显示模式的方式可由用户指定。 In some embodiments, PIP display 6965 or the default display mode specified by the user mode.

[0643] 图69通过参考五个不同的操作阶段6920、6925、6930、6935和6940,图解说明设备6900的Π 6960中的两个插入显示区之一的移动。 [0643] FIG. 69 by reference to five different stages of operation 6920,6925,6930,6935 and 6940, illustrating two of the display area moves into one of Π 6960 in the apparatus 6900. 第一阶段6920图解说明在设备6900的本地用户和远程设备的远程用户之间的视频会议期间的Π 6960。 The first stage 6920 illustrates Π 6960 during a video conference between the local user and the remote user device 6900 of the remote device's.

[0644] 第二阶段6925图解说明通过选择插入显示区6910,用户开始锁定到角落操作。 [0644] 6925 illustrates the second stage 6910, the user starts by selecting the corner lock to insert display area. 在这个例子中,通过把手指6950放在插入显示区6910内的任何地方,做出选择。 In this example, by the insertion of a finger 6950 on anywhere within the display area 6910, to make a choice. 如图所示, 用插入显示区6910的粗边框6962显示这种选择。 As shown, the display area with thick border inserted 69626910 display this choice. 不同的实施例可用不同的方式指出这样的选择,比如通过突出显示插入显示区6910,通过使插入显示区6910摆动,等等。 Various embodiments may be indicated in different ways such selection, such as by highlighting the display region 6910 is inserted, by swinging the insertion display area 6910, and the like.

[0645] 第三阶段6930图解说明在用户开始把PIP显示6965的插入显示区6910从PIP 显示6965中的一个区域移动到PIP显示6965中的另一个区域之后的Π 6960。 [0645] In the third stage 6930 illustrates the user begins to insert the display PIP display area 6965 6910 6965 in a region of the mobile Π 6960 to the PIP display area 6965 after another from the PIP display. 在这个例子中,插入显示区6910已开始从PIP显示6965的右下角向PIP显示6965的右上角移动, 如箭头6955所示。 In this example, the display area 6910 has been inserted into the PIP display starts from the upper right corner of the lower right corner 6965 of the display 6965 moves, as indicated by arrow 6955 to the PIP. 在选择插入显示区6910之后,用户通过朝着PIP显示6965的右上角拖动其手指6950,移动插入显示区6910。 After selecting the inset display area 6910, the user 6965 toward the upper right corner of the PIP display drag their finger 6950, 6910 move the insertion display area. 一些实施例提供在PIP显示6965中来回移动插入显示区6910的其它技术。 Some embodiments provide the PIP display 6965 is inserted moves back and forth in the display area 6910 other techniques.

[0646] 第四阶段6935图解说明在用户已从设备6900的屏幕移开其手指6950之后的状态下的Π 6960。 [0646] The fourth stage 6935 illustrates Π 6960 in the state of the device after the user has removed his finger screen 6900 6950. 在这种状态下,插入显示区6910仍然朝着根据第三阶段中的用户手指移动而识别的PIP显示6965的右上角移动。 In this state, the display area 6910 is still inserted into the upper right corner 6965 moves toward finger of the user identified in the third stage is moved according to PIP. 换句话说,在手指6950启动插入显示区6910朝着PIP显示6965的右上角的移动之后,即使在移开手指6950之后,UI 6960仍然保持这种移动。 In other words, the fingers start inset display area 6950 6910 6965 After displaying moving toward the upper right corner of the PIP, even after the removal of finger 6950, UI 6960 remains such movement. 为了保持所述移动,一些实施例的UI 6960要求在用户移开其手指之前,用户的拖动操作要大于特定的阈值量(例如,大于特定的距离,或者长于特定的时间);否则,这些实施例会在稍微移动插入显示区之后,使该插入显示区保持在其初始的右下角位置,或者根本不移动该插入显示区。 In order to maintain the movement, UI 6960 of some embodiments of the requirements before the user removes his finger, the user's drag operation to be greater than a certain threshold amount (e.g., greater than a certain distance or longer than a certain time); otherwise, the embodiments will move slightly after insertion display area, display area so that the insert remains in its original position of the lower right corner, or not move the insertion display area.

[0647] 不过,尽管即使用户在插入显示区到达其新位置之前,停止了他的拖动操作,一些实施例仍然允许插入显示区继续移动,不过其它实施例要求用户保持拖动操作,直到插入显示区到达其新位置为止。 [0647] However, although even before the user inserted a new display area has reached its position, stopped his drag operation, some embodiments permit insertion still continues to move the display area, although other embodiments require the user to maintain a drag operation, until the insertion display area reaches its new location. 一些实施例提供移动插入显示区的其它技术。 Some embodiments provide mobile insertion display area other techniques. 例如,一些实施例要求用户在显示区6910实际开始移动之前,指定要把显示区6910引导到何处,等等。 For example, some embodiments require the user before actual starts to move the display area 6910, a display area 6910 should specify where to boot, and the like. 一些实施例还通过仅仅以不同角度倾斜移动设备,使显示区可以滑动和锁定到角落。 Some embodiments are also inclined at different angles by simply mobile device, the display area may slide and lock into the corner.

[0648] 第五阶段6940图解说明在插入显示区6910到达其位于PIP显示区6965的右上角的新位置之后的Π 6960。 [0648] In the fifth stage 6940 illustrates the insertion display area 6910 positioned Π 6960 after reaching the new position of the top right corner of the PIP display area 6965. 第五阶段中粗边框6962的消除指示锁定到角落操作已完成。 The fifth stage of the 6962 elimination of the thick border indicates the lock to the corner of the operation has been completed.

[0649] 为了使在上述第三、第四和第五阶段6930、6935和6940中图解说明的移动更容易,一些实施例的Π 6960采用一旦用户使插入显示区6910朝着PIP显示6965的某个角落移动,就允许插入显示区6910快速锁定到该角落的锁定规则。 [0649] In order for the third, fourth and fifth stage moving 6930,6935 and 6940 illustrated easier Π described embodiments, once the user 6960 using the insertion toward the display area 6910 displaying the PIP, a 6965 corner moves, to allow insertion of the locking quickly display area 6910 to the rules of the corner lock. 例如,当用户朝着特定角落拖动插入显示区6910超过阈值量时,一些实施例的Π 6960识别插入显示区6910的运动方向,确定所述运动已超过阈值量,从而随后自动移动插入显示区6910,而不用用户进一步输入插入显示区6910可被锁定到的Π 6960中的下一个网格点。 For example, when the user drags a particular corner of the insert toward the display area 6910 exceeds a threshold amount, the identification number Π 6960 Example insertion movement direction of the display area 6910, determining the amount of motion exceeds a threshold, for subsequent inserted automatically move the display area 6910, without further user input to the next grid points of the display area 6910 may be inserted to be locked in Π 6960. 在一些实施例中,为锁定插入显示区6910而提供的唯一网格点是位于PIP显示6965的四个角落的网格点。 In some embodiments, the only grid point locking insert provided display area 6910 is located in the PIP display grid points of four corners 6965. 其它实施例提供插入显示区6910可被锁定到的Π 6960中(例如,PIP显示6965中)的其它网格点。 Other embodiments provide a display area 6910 can be inserted into the locked Π 6960 (e.g., PIP display in 6965) the other mesh points.

[0650] 另外的实施例可以不采用网格点,使得插入显示区6910可被置于PIP显示中的任意点。 [0650] Further embodiments may not employ a grid point, so that the insert can be placed in the display area 6910 at any point in the PIP display. 再另外的实施例提供允许用户打开或关闭UI的锁定到网格点特征。 Yet other embodiments provide a UI allowing a user to open or close to the grid point lock features. 此外,除了从设备拍摄的视频之外,不同的实施例可允许用户对各种项目,比如图标等执行锁定到角落操作。 Further, in addition to the video capture device from, different embodiments may allow the user to various items, such as icons, performing the locking operation into the corner. 如上所述,合成显示的一个或多个显示区的移动会导致双照相机移动设备的图像处理操作的变化,比如导致视频会议管理器1604响应用户的输入,重新合成合成显示中的显示区。 As described above, moving the one or more display area lead to variations in the composite display image processing operation of the dual camera mobile device, such as a video conference manager 1604 results in response to user input, the display area resynthesized in the composite display.

[0651] 5.推送和锁定 [0651] The push lock and

[0652] 图69中的例子图解说明允许双照相机移动设备的用户把两个插入显示区之一, 从PIP显示的一个角落移动到未被插入显示占据的另一个角落的锁定到角落操作。 In the illustrated example [0652] FIG 69 described dual camera allows the user of the mobile device into one of two display areas, a corner of the displayed moving from the PIP display is not inserted into the lock to the other corner of the corner occupied operation. 一些实施例能够实现把第一插入显示移动到第二插入显示的位置,还把第二插入显示推送到新位置的推送特征。 Some embodiments enable the insertion of the first insertion movement of the display to a second display position, the display also pushed to push the second insertion position of the new feature. 图70图解说明在视频会议期间执行的一个这种例子。 FIG 70 illustrates an example of one such execution during a video conference.

[0653] 图70通过参考Π 6960的六个不同阶段7020、7025、7030、7035、7040和7045,图解说明插入显示从PIP显示的一个角落到未被另一个插入显示占据的PIP显示的另一个角落的移动。 Another [0653] FIG. 70 by reference to six different stages Π 6960 7020,7025,7030,7035,7040 and 7045, illustrates another is not inserted into the insertion display from the display occupies a corner of the PIP display PIP display moving corner. 第一阶段7020图解说明在该设备的本地用户和远程设备的远程用户之间的视频会议期间的Π 6960。 The first stage 7020 illustrates Π 6960 during a video conference between the local user and remote user equipment of the remote device. 图70中的Π 6960表示和在开始视频会议之后,在图69的第一阶段中所示的PIP相同的PIP显示6965。 FIG 70 Π 6960 represents the start and after the video conference, the same PIP PIP shown in the first stage shown in FIG. 69 6965. 在这个例子中,由本地用户的设备拍摄的视频被显示在插入显示区6905和6910中,由远程用户的设备拍摄的视频被显示在背景显示区6915 中。 In this example, the video captured by the local user device is displayed in the display area 6905 and 6910 inserted in the video captured by the remote user device is displayed in the background display region 6915.

[0654] 第二阶段7025图解说明用户通过选择插入显示区6905,开始锁定到角落操作。 [0654] The second stage 7025 illustrates the selection display area 6905 is inserted, locked to the corner starts operation by the user. 在这个例子中,通过把手指7055放在插入显示区6905内的任意地方,进行选择。 In this example, by inserting a finger in 7055 anywhere within the display area 6905, are selected. 如图所示, 用插入显示区6905的粗边框7065显示这种选择。 As shown, the display area with thick border inserted 70656905 display this choice. 不同的实施例用不同的方式指示这样的选择,比如通过突出显示区6905,通过使显示区6905摆动,等等。 Different embodiments of such a selection indication in different ways, such as 6905, by the display section is swung by highlight region 6905, and the like.

[0655] 第三阶段7030图解说明如箭头7050所示,在用户开始把插入显示区6905从PIP 显示6965的左下角移向PIP显示6965的右下角(通过在第三阶段中,在选择插入显示区6905之后,朝着PIP显示6965的右下角拖动他的手指7055)之后的Π 6960。 [0655] 7030 illustrates the third stage as shown by arrow 7050, the user starts the display area 6905 is inserted from the lower left corner of the display PIP PIP display 6965 is moved toward the lower right corner 6965 (displayed by inserting in the third stage, the choice after District 6905, toward the lower right corner of the PIP display 6965 of dragging his finger Π 6960) after 7055. 一些实施例提供在PIP显示6965中来回移动插入显示区6905的其它技术。 Some embodiments provide the PIP display 6965 is inserted moves back and forth in the display area 6905 other techniques.

[0656] 第四阶段7035图解说明在插入显示区6905接触插入显示区6910之后的Π 6960。 [0656] In the fourth stage 7035 illustrates the contact 6905 is inserted is inserted into the display region display area 6910 after Π 6960. 当接触时,插入显示区6910朝着下一个最近的角落移动。 When contacted, the inset display area 6910 moves towards the next nearest corner. 在这个例子中,插入显示区6910开始沿PIP显示6965的右上角的方向(如箭头7075所示)移动。 In this example, the display area 6910 starts displaying the insertion movement (as indicated by arrow 7075) along the direction of the upper right corner 6965 PIP. 用插入显示区6910的粗边框7070,显示这种推送操作的激活。 Slot 70706910 thick border display area, the display of this activation push operation. 不同的实施例可用不同的方式指示这种激活,比如通过突出显示区6910,等等。 Various embodiments may in different ways indicating this activation, such as by highlight region 6910, and the like.

[0657] 第五阶段7040图解说明在插入显示区6905被锁定到先前被插入显示区6910占据的右下角之后的状态下的Π。 [0657] In the fifth stage 7040 illustrates the display area 6905 is inserted into Π locked in a state after the previously inserted into the lower right corner of the display area 6910 occupies. 在这个例子中,该插入显示区仍然朝着PIP显示6965的右上角移动。 In this example, the display area is still inserted into the upper right corner 6965 moves toward the PIP. 另外,不再显示粗边框7065。 In addition, 7065 is no longer displayed thick border. 只要第三阶段7030的用户拖动操作大于使插入显示区6905锁定到右角落的阈值,就把插入显示区6910从其角落移开,并一直锁定到下一个最低的角落。 As long as the user drags the third stage 7030 is greater than the insertion operation display area 6905 is locked to the right corner of the threshold value, put the insert removed from its corner of the display area 6910, and has been locked to the next lowest corner.

[0658] 一些实施例包括确定采用哪种方式推送第二插入显示区6910的一组规则。 [0658] Some embodiments include determining which way to push a set of rules second display area 6910 is inserted. 在图70中图解说明的情况下,一些实施例设法继续保持插入显示区的旋转。 In the case illustrated in FIG. 70 described some embodiments seek to keep the rotation display area is inserted. 即,由于插入显示区6905是沿逆时针方向移动的,因此显示区6910也被逆时针移动。 That is, since the display area 6905 is inserted in the counterclockwise direction, so that the display area 6910 also moves counterclockwise. 一些实施例提供被推送的插入显示区6910能够移动到的可能位置的分级,并选择该列表上的第一个未被占据的位置。 Some embodiments provide a hierarchy may be moved into position to be inserted pushed display area 6910, and select the first on the list of unoccupied positions. 例如,当位于右下角的插入显示区被来自左下角的插入显示区推送时,右上角可能是这种列表中的第一个位置。 For example, when the lower-right corner of the display area is pushed insert inserted from the lower left corner of the display area, the upper right corner of this may be the first position in the list. 不过,如果第三插入显示区已位于右上角,那么一些实施例会移动到列表上的下一个选项(例如,左上角、中心、或者左下角)。 However, if the third display area has been inserted into the upper right corner, a number of embodiments will then move to the next option (e.g., the top left, center, or lower left) on the list. 其它实施例可随着第二插入显示区一起推送第三插入显示区,使得设备不需要确定第三插入显示区的新位置。 Other embodiments may be pushed together as the second display area is inserted into the third insertion display area, so that the device does not need to determine a new position of the third display area of ​​the insert.

[0659] 第六阶段7045图解说明在插入显示区6910到达其位于PIP显示区6965的右上角的新位置之后的Π 6960。 [0659] In the sixth stage 7045 illustrates the insertion display area 6910 positioned Π 6960 after reaching the new position of the top right corner of the PIP display area 6965. 在该阶段中,粗边框7070的消除指示锁定到角落-推送操作已完成。 At this stage, the elimination of the thick border 7070 indicates locked into a corner - push operation has been completed. 类似于参考图68说明的推送到角落操作,合成显示的一个或多个显示区的移动能够导致双照相机移动设备的图像处理操作的变化,比如导致视频会议管理器1604响应用户的输入,重新合成合成显示中的显示区。 Described with reference to FIG. 68 is similar to the corner push operation, moving one or more display areas can cause a change in the composite display image processing operation of the dual camera mobile device, such as a video conference manager 1604 results in response to user input, resynthesis synthesis of the display region in the display.

[0660] 6.旋转 [0660] 6. Turn

[0661] 当用于视频会议的移动设备的用户在会议期间旋转该设备时,一些实施例旋转在视频会议期间呈现的PIP显示。 [0661] When the user mobile device for rotating the video conference apparatus during the meeting, some embodiments of rotation during a video conference embodiment presented PIP display. 图71图解说明当设备7100从垂直位置被旋转到水平位置时,该设备的Π显示7175的旋转。 Figure 71 illustrates when the device is rotated 7100 from the vertical position to a horizontal position, Π of the rotation of the display device 7175. 当屏幕的长边是垂直的时,设备7100被坚握,而当屏幕的长边水平时,设备7100被横握。 When the screen is perpendicular to the long side of the device 7100 grip is firm, and when the long side of the screen horizontally, the device 7100 is horizontal grip. 在图71中图解说明的例子中,Π显示7175从为设备的坚握而优化的纵向视图旋转到为设备7100的横握而优化的横向视图。 In the example illustrated in FIG. 71, Π rotated from the portrait display view 7175 firm grip device optimized to a lateral view of the device 7100 is held horizontally optimized. 这种旋转功能使得当移动设备7100被坚握或横握时,用户都能够观看以直立位置显示的Π 7175。 This rotation when the mobile device 7100 such that when firm grip or lateral grip, the user can view the display in an upright position Π 7175. 图71中图解说明的例子类似于图34中图解说明的例子,除了图71图解说明旋转包括两个插入显示区而不只是一个插入显示区的PIP显示。 The example illustrated in Figure 71 is similar to the described example illustrated in FIG. 34, except that FIG. 71 illustrates a rotary insert comprises two display areas rather than just a display area inserted PIP display.

[0662] 在图71中,在与另一个移动设备的远程用户的视频会议期间,移动设备的Π 7175呈现PIP显示7180。 [0662] In FIG. 71, during a remote user of another mobile device in a video conference, Π 7175 presenting the PIP display of the mobile device 7180. 图71中的PIP显示7180包括三个视频显示:背景主显示7115 和两个前景插入显示7110和7160。 PIP display of FIG. 71 includes three video display 7180: 7115 and two background main display foreground inset display 7110 and 7160. 在这个例子中,背景主显示7115呈现山脉的视频,所述视频假定是由远程设备的正面或背面照相机拍摄的视频。 In this example, the main display background video presentation mountains 7115, the video captured by the video is assumed to be a remote front or back of the camera device. 前景插入显示7110呈现房间中的笑脸的视频,所述视频假定由本地设备的正面或背面照相机拍摄。 Foreground inset display 7110 presents a video of a smiling face in the room, assuming that the video taken from the front or the back of a local camera equipment. 另一个前景插入显示7160呈现唱歌的吉它演奏者的视频,所述吉它演奏者假定是正由本地设备的另一部照相机拍摄其视频的吉它演奏者。 Another foreground inset display 7160 presents a video of a guitar player singing, the guitar player assumes are being captured by another camera device local guitar player whose video of it. PIP显示7180下面是"结束会议"按钮7155,用户可以选择该按钮以结束视频会议(例如,通过单指轻点)。 7180 The following is the PIP display "end session" button 7155, the user may select the button to end the video conference (e.g., a single finger tap). 该PIP显示只是呈现由远程设备和本地设备拍摄的视频的合成视图的一种方式。 The PIP display is only one manner of presenting the synthesized video view captured by a remote device and a local device. 一些实施例可提供其它合成视图,比如平铺视图或者不同的PIP显示。 Some embodiments may provide other synthetic views, such as view or a different tile PIP display.

[0663] 图71按照六个不同的操作阶段7120、7125、7130、7135、7140和7145,图解说明Π 7175的旋转。 [0663] FIG. 71 according to six different operating phases 7120,7125,7130,7135,7140 and 7145, 7175 illustrate Π rotation. 第一阶段7120图解说明在该设备的本地用户和远程设备的远程用户之间的视频会议期间的Π7175。 The first stage 7120 illustrates Π7175 during a video conference between a local user of the remote user device and the remote device.

[0664] 第二阶段7125图解说明在用户开始向一边倾斜设备7100之后的Π 7175。 [0664] The second stage 7125 illustrates Π 7175 after the user starts to tilt apparatus 7100 side. 在这个例子中,设备7100已开始把设备7100从坚握倾斜到被横握,如箭头7185所示。 In this example, the device 7100 has begun to grip the device 7100 to be inclined from the horizontal firm grip, as indicated by arrow 7185 in FIG. Π 7175 的外观没有变化。 Π 7175 appearance has not changed. 在其它情况下,用户可能改为想把设备7100从被横握倾斜到被坚握,在这些情况下,Π显示7175从水平优化的视图切换到垂直优化的视图。 In other cases, the user may want to grip the device from the firm 7100 is inclined to the horizontal grip, in these cases, the display 7175 [pi optimized switching from a horizontal to a vertical view of an optimized view.

[0665] 第三阶段7130图解说明在设备7100已从被坚握倾斜到被横握之后的状态下的Π 7175。 [0665] In the third stage 7130 illustrates the apparatus 7100 from being held inclined to Π 7175 is in a state after the firm lateral grip. 在这种状态下,Π显示7175的外观仍然没有变化。 In this state, Π show the appearance of 7175 remains unchanged. 在一些实施例中,在设备7100被倾斜超过阈值量,并保持超过阈值量一段时间之后,触发旋转操作。 In some embodiments, the apparatus 7100 is tilted beyond a threshold amount, and then holding period exceeds a threshold amount, rotational operation is triggered. 在图71中图解说明的例子中,假定阈值量和旋转速度不会使Π显示7175旋转,直到设备已放置在水平位置之后的较短时间间隔为止。 In the example illustrated in Figure 71 described, it is assumed threshold amount and the rotation speed does not make the display Π 7175 rotated until the device has been placed in a horizontal position after a short time interval so far. 不同的实施例具有用于触发旋转操作的不同阈值量和等待时间。 Different embodiments having different threshold values ​​for triggering the amount of rotational operation and the waiting time. 例如,一些实施例可具有如此低的触发旋转操作的阈值,使得不管设备7100的定向,都使Π 7175显得好像它总是以直立位置显示似的。 For example, some embodiments may have a threshold to trigger such a low rotation operation, so that regardless of the orientation of the device 7100, so that Π 7175 are always displayed as if it seems like an upright position. 在其它实施例中,设备7100的用户可以规定何时可以触发旋转操作(例如,通过菜单优先选择设置)。 In other embodiments, the user device 7100 may be triggered when a predetermined rotational operation (e.g., setting preferences menu). 另外,一些实施例可在设备被倾斜超过阈值量之后,不延迟所述旋转。 Further, some embodiments may be after the device is tilted more than a threshold amount, the rotation is not delayed. 此外,不同的实施例可以允许用不同的方式触发旋转操作,比如通过切换移动设备上的开关、通过发出话音命令、依据通过菜单进行的选择,等等。 Further, different embodiments may allow rotating operation triggered in different ways, such as by the changeover switch on the mobile device, by issuing a voice command, menu selection by the basis, and the like.

[0666] 第四阶段7135图解说明在开始旋转操作之后的Π 7175。 [0666] The fourth stage 7135 illustrates Π 7175 after the start of the rotating operation. 一些实施例动画演示旋转显示区,以向用户提供关于旋转操作的反馈。 Some embodiments rotate the display area animation embodiment, to provide feedback to the user about the rotational operation. 图71图解说明这样的一种动画的例子。 FIG 71 illustrates an example of such an animation. 具体地说,图71在其第四阶段7135表现显示区7110、7115和7160 -起开始旋转。 Specifically, FIG. 717 135 and 7160 7110,7115 expressing a display region in its fourth phase - from start to rotate. 显示区7110、7115和7160围绕经过Π显示7175的中心的轴7165 (即,z轴)旋转。 Display areas 7110,7115 and 7160 about axis 7165 through the center of the display Π 7175 (i.e., z-axis). 显示区7110、 7115和7160被旋转相同的数量,不过沿与设备7100的旋转(例如,通过设备7100的倾斜) 相反的方向旋转。 Display area 7110, 7115 and 7160 are rotated by the same amount, but the direction of rotation of the device 7100 (e.g., by tilting the device 7100) rotate in opposite directions. 在这个例子中,由于设备7100沿顺时针方向旋转了90° (通过从坚握变成横握),因此旋转操作会使显示区7110、7115和7160沿逆时针方向旋转90°。 In this example, since the device 7100 is rotated clockwise 90 ° (from the firm grip by grip becomes horizontal), so that the rotation action causes display areas 7110,7115 and 7160 rotated counterclockwise 90 °. 当显示区7110、7115和7160旋转时,显示区7110、7115和7160按比例地缩小,以适合Π显示7175, 使得显示区7110、7115和7160仍然可以完全出现在Π 7175内。 7110,7115 and 7160, when the rotation display area, display areas 7110,7115 and 7160 reduced proportionally to fit [pi display 7175, 7110,7115 and 7160 so that the display area can still occur entirely in Π 7175. 一些实施例可提供指示设备7100的状态的消息(例如,通过显示字词"Rotating(旋转)")。 Some embodiments may provide an indication of a device status message 7100 (e.g., by displaying the word "Rotating (rotation)").

[0667] 第五阶段7140图解说明在显示区7110、7115和7160从纵向视图到横向视图逆时针旋转90°之后的Π 7175。 [0667] The fifth stage 7140 illustrates the display areas 7110,7115 and 7160 Π 7175 from a lateral longitudinal view to view after rotated counterclockwise by 90 °. 在这个阶段,显示区7110、7115和7160已被旋转,不过还未展开到Π 7175的整个宽度。 At this stage, the display area 7110,7115 and 7160 have been rotated, but has not yet commenced the entire width of Π 7175. 箭头7170指出在第五阶段的结尾,显示区7110、7115和7160 将开始侧向展开,以适合UI 7175的整个宽度。 7170 arrow pointed at the end of the fifth stage of the display areas 7110,7115 and 7160 will begin to spread laterally to fit the entire width of the UI 7175. 不同的实施例可不包括该阶段,因为可以与第四阶段7135中的旋转同时地进行所述展开。 Different embodiments may not include this stage, since the unfolding may be performed simultaneously with the rotation of the fourth stage 7135.

[0668] 第六阶段7145图解说明在显示区7110、7115和7160已被展开,以占据Π 7175 的整个显示之后的Π 7175。 [0668] 7145 illustrates the sixth stage 7110,7115 and 7160 in the display region has been expanded, to occupy the entire display Π 7175 after the Π 7175. 如上所述,其它实施例可以不同地实现这种旋转。 As described above, other embodiments may be implemented differently this rotation. 对一些实施例来说,仅仅把设备的屏幕旋转超过阈值量就可触发所述旋转操作,而不管设备7100的定向。 For some embodiments, the only the screen of the device beyond a threshold amount of rotation of the rotating operation can be triggered, regardless of the orientation of the device 7100.

[0669] 另外,其它实施例可提供用于指示旋转操作的不同动画。 [0669] Further, other embodiments may provide different animations for indicating a rotational operation. 在图71中执行的旋转操作涉及Π显示7175围绕Π显示7175的中心旋转。 Rotating operation performed in FIG. 71 relates to a display 7175 Π Π displayed around the center of rotation of 7175. 另一方面,可以使显示区围绕它们各自显示区的中心轴单独旋转。 On the other hand, the display area can display about their respective central axes of rotation of the individual regions. 图72中表示了一种这样的方法,图72表示动画演示UI 7175 的PIP显示区7180的旋转的备选方法。 FIG 72 shows one such method, FIG. 72 shows the UI PIP 7175 animation display area 7180 of the alternative methods of rotating. 图72中图解说明的PIP显示和图71中图解说明的PIP显示7180相同。 PIP illustrated in FIG. 72 and FIG. 71 is a display illustrated in PIP display the same 7180.

[0670] 图72按照六个不同的操作阶段7120、7125、7130、7220、7225和7230,图解说明PIP显示7180的旋转。 [0670] FIG. 72 according to six different operating phases 7120,7125,7130,7220,7225 and 7230, 7180 illustrate the rotation of the PIP display. Π 7175的前三个阶段的操作与如在图71中的Π 7175中说明的前三个阶段的操作相同。 The first three stages of operation as Π 7175 as described in the Π 7175 in FIG. 71 is the same as in the first three stages. 在图71和72的第三阶段,设备都已从坚握变成横握,并且Π 7175 的旋转还未开始。 In the third stage of FIG. 71 and 72, are held horizontally from the device into firm grip and rotation Π 7175 has not yet started.

[0671] 第四阶段7220图解说明动画演示所述旋转的备选方法。 Alternative method of animation 7220 illustrates the rotation of the [0671] fourth stage. 在第四阶段,旋转操作已开始。 In the fourth stage, the rotating operation has begun. 具体地说,第四阶段7220表现显示区7110、7115和7160的旋转的开始。 Specifically, the fourth stage of the 7220 performance of the display begins to rotate zone 7110,7115 and 7160. 显示区7110、7115和7160分别围绕穿过每个显示区的中心的轴7250( S卩,z轴)旋转。 Display areas 7110,7115 and 7160 respectively, each around an axis through the center of the display area 7250 of (S Jie, z axis). 显示区7110、7115和7160被旋转相同的量,不过沿与设备7100的旋转(例如通过设备7100的倾斜)相反的方向旋转。 Display areas 7110,7115 and 7160 are rotated by the same amount, but the direction of rotation of the device 7100 (e.g., by tilting the device 7100) is rotated in opposite directions. 在这个例子中,由于设备7100沿顺时针方向旋转了90° (通过从坚握变成横握),因此旋转操作会使显示区7110、7115和7160沿逆时针方向旋转90°。 In this example, since the device 7100 is rotated clockwise 90 ° (from the firm grip by grip becomes horizontal), so that the rotation action causes display areas 7110,7115 and 7160 rotated counterclockwise 90 °. 当显示区7110、7115和7160旋转时,它们还按比例缩小,以适合Π显示7175,使得显示区7110、 7115和7160仍然可以完全出现在Π 7175上。 7110,7115 and 7160, when the rotation display area, they are scaled down to fit the [pi display 7175 so that the display area 7110, 7115 and 7160 may still occur on the fully Π 7175.

[0672] 第五阶段7225图解说明在显示区7110、7115和7160都从纵向视图到横向视图逆时针旋转90°之后的Π 7175。 [0672] 7225 illustrates the fifth stage in the display region are 7110,7115 and 7160 to the view from the longitudinal lateral view after Π rotated counterclockwise 90 ° 7175. 在这个阶段,显示区7110、7115和7160已被旋转,不过还未展开到Π 7175的整个宽度,或者到达它们的最终位置。 At this stage, display areas 7110,7115 and 7160 have been rotated, but not yet into the whole width of the Π 7175, or reach their final position. 显示区在PIP显示7115中的最终位置由如在第一阶段7120中所示的显示区在PIP显示7115中的位置确定(例如,插入显示7110在PIP显示7180的左下角,和插入显示7160在PIP显示7180的右下角)。 PIP display area 7115 in the final position of the position determination in 7115 (e.g., inserted into the lower left corner in the PIP display displays 7110 7180, and the insertion area displayed by the display 7120 shown in the first stage in the PIP display 7160 the lower right corner of the PIP display 7180).

[0673] 箭头7170指出在第五阶段的结尾,显示区7115、7110和7160将开始侧向展开,直到主显示区7115适合Π 7175的整个宽度为止。 [0673] indicated by arrow 7170 at the end of the fifth stage, display areas 7115,7110 and 7160 will begin to expand laterally, until the main display area 7115 for the entire width of Π 7175 is reached. 此外,箭头7255指出插入显示区7110和7160将移动,从而到达它们在PIP显示7180中的最终位置。 Further, arrows 7255 and 7110 indicated inset display area 7160 will move to reach their final position in the display 7180 in the PIP. 换句话说,插入显示区7110将向下朝着PIP显示7180的左下角移动,而另一个插入显示区7160移向PIP显示7180的右下角。 In other words, the inset display area 7110 displays a downward movement of the lower left corner toward the PIP 7180, and another display area 7160 is inserted toward the bottom right corner 7180 of the PIP display. 不同的实施例可不同地实现这种动画,例如通过利用在图71中图解说明的锁定和推送操作。 Different embodiments of such an animation can be implemented differently, for example by using the lock illustrated in FIGS. 71 and push operation. 第六阶段7230图解说明在显示区7110、7115和7160已被展开从而占据Π 7175 的整个显示并且已移动到它们的最终位置之后的Π 7175。 Sixth stage 7230 illustrates 7110,7115 and 7160 in the display region has been expanded so as to occupy the entire display Π 7175 and Π 7175 has moved to their final positions after.

[0674] 如上所述,其它实施例可不同地实现这种旋转。 [0674] As described above, other embodiments may be implemented differently this rotation. 例如,与图36和37中图解说明的类似,一些实施例提供一种旋转操作,其中显示由本地设备拍摄的视频的显示区的定向改变,以反映在对本地设备进行旋转操作之后本地设备的定向,一些实施例提供一种旋转操作,其中显示由远程设备拍摄的视频的显示区的定向改变,以反映在对远程设备进行旋转操作之后远程设备的定向,一些实施例提供其中显示区1155保留在相同位置的旋转操作, 一些实施例提供显示区中的不同布局(例如,图12的显示区1155的布局),或者它们的组合。 For example, similar to FIGS. 36 and 37 illustrated below, some embodiments provide a rotary operation, wherein the display changes the orientation of the display area of ​​video captured by the local device to reflect the local device after a rotation operation of the local device orientation Some embodiments provide a rotary operation, wherein the display changes the orientation of the display area of ​​video captured by a remote device to reflect the orientation of the remote device after the remote device is operated to rotate, some embodiments provide a display area 1155 reserved wherein in the same rotational operation position, some embodiments provide a different layout (e.g., layout of the display area 1155 of FIG. 12) in the display area, or a combination thereof.

[0675] 对一些实施例来说,仅仅使设备的屏幕旋转超过阈值量就可触发旋转操作,而不管设备7100的定向。 [0675] For some embodiments, the screen of the device so that only the rotational amount exceeds a threshold rotational operation can be triggered, regardless of the orientation of the device 7100. 另外如上所述,本地设备和远程设备可通过控制通信通道,相互通知对两个设备中的一个设备进行的旋转操作,以允许另一个设备对所述一个设备的视频进行任何对应的修改。 Further described above, the local and remote devices may communicate through the control channel, inform each other of a rotational operation of the device of the two devices is to allow the other device corresponding to any of the modifications to a video device. 另外,旋转操作的动画还会导致照相机的操作或者双照相机移动设备的图像处理操作的变化,比如使视频会议管理器1604重新合成Π 1105中的成不同角度的一个或多个显示区,和比例缩放显示在一个或多个显示区中的图像。 Further, the rotational operation of the animation process also causes a change in operation of the camera image or a dual camera mobile device operation, such that the video conference manager 1604 novo synthesis of a Π different angles of 1105 or more display areas, and the ratio in one or more image display area zoom display.

[0676] 7.选择观看远程视图 [0676] 7. Select View Remote viewing

[0677] 如上所述,一些实施例允许双照相机移动设备的用户在开始视频会议之前,或者在开始视频会议时,选择把哪部照相机用于视频会议。 [0677] As described above, embodiments allow some users dual camera mobile device before the start of the video conference, or at the beginning of the video conference, the selecting unit which the camera for videoconferencing. 代替这种能力,或者与这种能力结合,一些实施例允许该设备的用户在两个视频之间进行选择,所述两个视频是在视频会议中显示的并且或者来自远程设备的两部视频照相机,或者来自用户的本地设备的两部视频照相机。 Instead of this ability, or in conjunction with this capability, a user of some embodiments of the device will be allowed between the two video selection, the two videos are displayed in the video conference and video or two from a remote device two video cameras the camera, or from a user of the local device. 图73图解说明在会议中选择两个远程视频中的一个视频,而图74图解说明在会议中选择两个本地视频中的一个视频。 FIG. 73 illustrates selecting a video in the two remote video conference, and FIG. 74 illustrates selecting a video in the two local video conference.

[0678] 图73按照显示在本地设备7300上的Π 7375的六个操作阶段7335、7340、7345、 7350、7355和7360,图解说明所述远程视频的选择。 [0678] FIG. 73 shows six operating phases according Π 7375 on the local device 7300 7335,7340,7345, 7350,7355 and 7360, illustrates the selection of the remote video. 第一阶段7335图解说明具有在与具有两部照相机的移动设备的远程用户的视频会议期间,呈现的初始PIP显示7390的Π 7375。 The first stage 7335 illustrates having during a remote user with a mobile device having two cameras video conference, the initial presentation of Π 7375 7390 PIP display.

[0679] 如第一阶段7335中所示,初始PIP显示7390包括三个显示:背景主显示7315和两个前景插入显示7305和7310。 [0679] As shown in the first stage 7335, initially comprises three display PIP display 7390: 7315 and two background main display foreground inset display 7305 and 7310. 背景显示7315占据大部分的PIP显示区7390,而前景插入显示7305和7310分别重叠在Π 7375上的部分背景显示7315上。 7315 occupies most of the background display PIP display area 7390, and 7305 and 7310 show the foreground inserted respectively superimposed on the background portion Π 7375 on display 7315. 在这个例子中,背景显示7315呈现在麦克风之前的人物的视频,所述视频假定是由远程设备的背面照相机拍摄的视频。 In this example, the background display 7315 presents the person before the microphone video, the video is assumed to be captured by the back camera remote video device. 第一前景插入显示7305呈现男士的脸部的视频,在这个例子中,该视频假定是由本地设备7300的照相机之一拍摄的视频。 The first foreground inset display 7305 presents a video of the face of men, in this example, assume that the video is a video shot by one of the local camera device 7300. 第二前景插入显示7310呈现戴着帽子的人物的视频,在这个例子中,该视频假定是由远程设备的正面照相机拍摄的视频。 The second foreground inset display 7310 presents a video of a person wearing a hat, in this example, assume that the video is video captured by the front camera remote device.

[0680] 初始PIP显示7390只是呈现由本地设备和远程设备的照相机拍摄的视频的合成视图的一种方式。 [0680] Initial PIP display 7390 is only one manner of presenting the synthesized video view captured by the camera of a local device and a remote device. 一些实施例可以提供其它的合成视图。 Some embodiments may provide other composite views. 例如,背景显示可以呈现来自本地设备的照相机之一的视频,较小的前景插入显示可以呈现来自远程设备的正面照相机和背面照相机的视频。 For example, the background display can present the video camera from one of the local device, the smaller foreground inset display can render video cameras front and back of the camera from a remote device. 另外,在一些情况下,PIP显示可以只包括均来自远程设备的一个背景视频显示和一个前景视频显示。 In addition, in some cases, PIP display may include only have a background video from a remote display device and a video display prospect. 在一些实施例中,PIP显示的方式或者默认显示模式可由用户指定。 In some embodiments, the PIP display mode or the default display mode specified by the user.

[0681] 第二阶段7340图解说明视频选择操作的开始。 [0681] The second stage 7340 illustrates the start of the video selection operation. 在这个例子中,通过调用要显示在PIP显示7390上的一组可选Π项目,发起该操作。 In this example, by calling to display a set of selectable items on Π 7390 in PIP, initiated the operation. 所述一组可选Π项目呈现用于选择显示用远程视频的选项。 Π the set of selectable items presented with the option to choose to display the remote video. 在一些实施例中,通过选择(例如,通过触摸)UI 7375上的播放远程视频的任何显示区,调用所述一组可选Π项目。 In some embodiments, by selecting (e.g., by touching) any play remote video display area on the UI 7375, invoking the set Π selectable items. 在其它实施例中,通过选择(例如,通过触摸)UI 7375上的任意地方,可调用所述各个项目。 In other embodiments, by selecting (e.g., by touching) anywhere on the 7375 UI, you can call the respective project. 代替这种调用操作,或者与这种调用操作结合,一些实施例还允许用户通过其它操作,比如通过不同的触屏操作,或者利用设备的一个或多个其它物理输入,调用所述一组可选Π项目。 Instead of such a call operation, or in conjunction with this calling operation, Some embodiments also allow the user to other operations, such as by the different touch screen operation, or by a physical device or a plurality of other inputs, a set of the call election Π project.

[0682] 第三阶段7345显示Π 7375, Π 7375具有一组调用的用于选择远程视频的可选Π项目7380。 [0682] The third stage 7345 display Π 7375, Π 7375 for selecting a remote video selectable items having a set of calls [pi of 7380. 在这个例子中,呈弹出式菜单形式的一组可选Π项目7380被显示在PIP显示区7390中,重叠在该PIP显示上。 In this example, as a pop-up menu in the form of a set of selectable items Π 7380 is displayed in the PIP display area 7390 superimposed on the PIP display. 所述一组可选Π项目7380 (它们可被实现成可选按钮) 包括"Select R1"可选Π项目7320(例如,按钮7320)/'Select R2"可选Π项目7325(例如,按钮7325),"Select Both"可选Π项目7330 (例如,按钮7330),和"Cancel"可选Π 项目7385(例如,按钮7385)。在这个例子中,选择"Select R1"按钮7320会使UI 7375只显示由远程设备的背面照相机拍摄的视频(呈现在背景显示7315中)。选择"SelectR2" 按钮7325会使Π 7375只显示由远程设备的正面照相机拍摄的视频(呈现在前景插入显示7310中)。选择"Select Both"按钮7330会使Π 7375继续显示由远程设备的正面照相机和背面照相机拍摄的两个视频。选择"Cancel"按钮7385会取消该操作。在一些实施例中,由本地设备拍摄的视频不受在该菜单上进行的选择的影响。 The set of selectable items Π 7380 (which may be implemented as a selectable button) comprises a "Select R1" Π selectable item 7320 (e.g., button 7320) / 'Select R2 "Π selectable item 7325 (e.g., button 7325 ), "select Both" Π selectable item 7330 (e.g., button 7330), and "Cancel" Π selectable item 7385 (e.g., button 7385). in this example, select the "select R1" button 7320 cause the UI 7375 display only the video captured by the camera on the back of the remote device (presented in the background display in 7315). select "SelectR2" button 7325 Π 7375 will display only the video captured by the front camera remote device (inset display 7310 presents in the foreground) choose "select Both" button Π 7375 7330 will continue to show the video captured by two cameras front and back of the camera remote device. select "cancel" button 7385 will cancel the operation. in some embodiments, captured by the local device the video selection is not on the menu effects.

[0683] 第四阶段7350图解说明在用户选择了"Select R1"按钮7320(例如,通过用他的手指7365轻点按钮7320)之后的Π 7375。 [0683] In the fourth stage 7350 illustrates the user selects the "Select R1" button 7320 (e.g., with his finger 7365 by tapping the button 7320) Π 7375 after. 通过突出显示Π 7375上的按钮7320,指示这种选择。 By highlighting a button 7320 on Π 7375, indicating this selection. 一些实施例利用不同的指示显示(例如,突出显示所选项目的边框,或者所选项目中的文本)。 Some embodiments utilize different instructs the display (e.g., highlight the text frame of the selected item or selected items).

[0684] 第五阶段7355图解说明在用户选择了来自R1的视频以便显示之后的Π 7375的动画。 [0684] In the fifth stage 7355 illustrates a user selects animation Π 7375 after the video for display from R1. 在这个例子中,通过如箭头7370所示,使不需要的前景插入显示区7310滑出PIP显示7390的右边缘,UI 7375除去不需要的前景插入显示区7310。 In this example, as shown by arrow 7370, the unnecessary foreground display region 7310 is inserted into the right edge of the slide out of the PIP display 7390, UI 7375 to remove unwanted foreground display region 7310 is inserted. 其它实施例利用不同的动画来消除不需要的插入显示区,比如使该插入显示区淡出或者渐隐、沿不同的方向移动该插入显示区、或者简单地立即消除该插入显示区。 Other embodiments utilize different animations to eliminate unwanted inset display region, the display region such that the insert and out, or fading, moving the display area is inserted in a different direction, or simply inserted immediately eliminate the display area.

[0685] 第六阶段7360显示在结束视频选择操作之后,视频会议期间的Π 7375。 [0685] The sixth stage show in 7360 after the end of the video selection operation, Π 7375 during a video conference. 视频显示区7310不再显示在Π 7375上。 Video display area 7310 is no longer displayed on Π 7375. 在这个阶段,UI7375呈现新的PIP显示7395,PIP显示7395包括作为背景主显示的视频显示区7315,和作为插入显示的视频显示区7305。 At this stage, UI7375 presents a new PIP display 7395, PIP display 7395 includes a video display area as the main background 7315, and 7305 as a video display area of ​​the display insert.

[0686] 在一些实施例中,该视频选择操作还会使远程设备只显示选择的视频,不过在其它实施例中,视频选择操作对远程设备没有任何影响。 [0686] In some embodiments, the video selection operation will enable the remote device to display only the selected video, although in other embodiments, the video selection operation has no effect on the remote device. 在一些实施例中,该视频选择操作会使远程设备停止向本地设备传送不需要的视频。 In some embodiments, the video selection operation will stop transmitting the remote device to the local device does not require a video. 事实上,在一些实施例中,盖视频选择操作会使远程设备的照相机停止拍摄不需要的视频。 Indeed, in some embodiments, the video selection operation will cover the remote device does not require a camera stops shooting video. 在一些实施例中,远程设备的用户可以否决对远程设备的这些影响。 In some embodiments, the user of the remote device may influence on these remote devices rejected.

[0687] 上面的例子举例说明选择的远程视图是已显示在背景主显示中的远程视图的情况。 [0687] The above example illustrates a case where the view selected remote remote view in the background of the main display is displayed. 在一些实施例中,当用户选择显示在插入显示之一中的远程视图时,选择的远程视图被显示在背景主显示中。 In some embodiments, when the user selects one of the remote view of the insertion display, the selected remote view is displayed in the background of the main display. 这种情况下,一些这样的实施例利用和在图68中显示的动画类似的动画。 In this case, some such embodiments utilize animation and 68 shown in FIG similar animation. 此外,远程视频的选择会导致本地双照相机移动设备的图像处理操作的变化,比如使视频会议管理器1604响应用户的输入,在合成显示中只合成选择的一个或多个远程视频。 Further, the selection will cause a change in the remote video image processing local dual camera mobile device operation, such that the video conference manager 1604 in response to user input, one or more remote video synthesis display only the selected synthesis.

[0688] 8.选择观看本地视图 [0688] 8. Choose a local view viewed

[0689] 图74按照显示在本地设备7400上的Π 7475的六个操作阶段7435、7440、7445、 745、7455和7460,图解说明本地视频的选择。 [0689] FIG. 74 shows six operating phases according Π 7475 on the local device 7435,7440,7445 7400, 745,7455 and 7460, local illustrate selected video. 第一阶段7435图解说明Π 7475, Π 7475具有在与具有至少一部照相机的移动设备的远程用户的视频会议期间呈现的初始PIP显示7490。 The first stage 7435 illustrates Π 7475, Π 7475 having an initial PIP during the movement of the camera device having at least one remote user video conference presentation display 7490. PIP显示7490和在图73中的第一阶段7335中的PIP显示类似,除了不同于图73, 背景显示7415呈现由远程设备的照相机拍摄的男士的视频,左侧的前景插入显示7410呈现由本地移动设备的背面照相机拍摄的拿着吉它的人物的视频,右侧的前景插入显示7405 呈现由本地移动设备7400的正面照相机拍摄的戴着帽子的男士的视频之外。 PIP display 7490 and the first stage in FIG. 73, the PIP display 7335 is similar except that differs from FIG. 73, the background display 7415 presents captured by a camera man remote video device, inserted into the left foreground presented by the local display 7410 the back of the camera mobile device photographed holding a guitar figures of video, foreground right side of the inset display 7405 presents a hat outside shooting by the local mobile device's front camera 7400 men video. 从而,只显示一个远程视频,而显示两个本地视频。 Thus, only a remote video display, and two local video display.

[0690] 第二阶段7440图解说明视频选择操作的开始。 [0690] The second stage 7440 illustrates the start of the video selection operation. 在这个例子中,通过调用要显示在PIP显示7490上、用于选择显示用远程视频的一组可选Π项目,发起该操作。 In this example, by calling to the display 7490, for selecting for display a set of selectable items Π remote video initiate the operation shown in the PIP. 在一些实施例中,通过选择(例如,通过触摸)Π显示7475上的播放本地视频的任何显示区,可调用所述一组可选UI项目。 In some embodiments, by selecting (e.g., by touching) [pi show any local video display region on the play 7475, the call may be a set of selectable UI items. 在其它实施例中,通过选择(例如,通过触摸)UI显示7475上的任意地方,可调用所述各个项目。 In other embodiments, by selecting (e.g., by touching) the UI displayed anywhere on 7475, you can call the respective project. 代替这种调用操作,或者与这种调用操作结合,一些实施例还允许用户通过其它操作,比如通过不同的触屏操作,或者利用设备的一个或多个其它物理输入,调用所述一组可选Π项目。 Instead of such a call operation, or in conjunction with this calling operation, Some embodiments also allow the user to other operations, such as by the different touch screen operation, or by a physical device or a plurality of other inputs, a set of the call election Π project.

[0691] 第三阶段7445显示Π 7475, Π 7475具有一组调用的用于选择本地视频的可选Π项目7480。 [0691] The third stage 7445 display Π 7475, Π 7475 for selecting the local video selectable items having a set of calls [pi of 7480. 在这个例子中,呈弹出式菜单形式的一组可选Π项目7480被显示在PIP显示区7490中,重叠在该PIP显示上。 In this example, as a pop-up menu in the form of a set of selectable items Π 7480 is displayed in the PIP display area 7490 superimposed on the PIP display. 所述一组可选Π项目7480包括"Select L1"可选Π 项目7420(例如,按钮7420)/'Select L2"可选Π项目7425(例如,按钮7425)/'Select Both"可选Π项目7430 (例如,按钮7430),和用于取消该操作的"Cancel "可选Π项目7485(例如,按钮7485)。 The set of selectable items 7480 including Π "Select L1" Π selectable item 7420 (e.g., button 7420) / 'Select L2 "Π selectable item 7425 (e.g., button 7425) /' Select Both" selectable items Π 7430 (e.g., button 7430), and for canceling the operation of the "cancel" Π selectable item 7485 (e.g., button 7485). 在这个例子中,选择"Select L1"按钮7420会使Π 7475只显示由本地设备的背面照相机拍摄的视频(呈现在前景插入显示7410中)。 In this example, choose "Select L1" button 7420 Π 7475 will display only the video captured by the local device's back camera (display in the foreground appear inserted in 7410). 选择"SelectL2" 按钮7425会使Π 7475只显示由本地设备的正面照相机拍摄的视频(呈现在前景插入显示7405中)。 Select "SelectL2" button 7425 Π 7475 will display only the video captured by the local device's front camera (inset display 7405 presents in the foreground). 选择"Select Both"按钮7430会使Π 7475继续显示由本地设备的两部照相机拍摄的两个视频,以及选择"Cancel"按钮7485会取消该操作。 Select "Select Both" button 7430 Π 7475 will continue to display two camera video captured by both the local device, and select "Cancel" button 7485 will cancel the operation. 在一些实施例中,由远程设备拍摄的视频不受通过该菜单进行的选择的影响。 In some embodiments, the video captured by the remote device is not affected by the menu selection by.

[0692] 第四阶段7450图解说明在用户选择了"Select L2"按钮7425(例如,通过用他的手指7465轻点按钮7425)之后的Π 7475。 [0692] In the fourth stage 7450 illustrates the user selects the "Select L2" button 7425 (e.g., with his finger 7465 by tapping the button 7425) Π 7475 after. 通过突出显示Π 7475上的按钮7425,指示这种选择。 By highlighting a button 7425 on Π 7475, indicating this selection. 一些实施例利用不同的指示显示(例如,突出显示所选项目的边框,或者所选项目中的文本)。 Some embodiments utilize different instructs the display (e.g., highlight the text frame of the selected item or selected items).

[0693] 第五阶段7455显示在用户选择了来自L2的视频以便显示之后的Π 7475的动画。 [0693] In the fifth stage 7455 the user selects the displayed animation Π 7475 after the video for display from L2. 在这个例子中,通过如箭头7470所示,使不需要的前景插入显示7410滑出PIP显示7490的左边缘,UI 7475除去不需要的前景插入显示7410。 In this example, as shown by arrow 7470, the unnecessary slipping foreground inset display 7410 7490 PIP display left edge, UI 7475 to remove unwanted foreground inset display 7410. 其它实施例利用不同的动画来消除不需要的插入显示区,比如使该插入显示区淡出或者渐隐、沿不同的方向移动该插入显示区、或者简单地立即消除该插入显示区。 Other embodiments utilize different animations to eliminate unwanted inset display region, the display region such that the insert and out, or fading, moving the display area is inserted in a different direction, or simply inserted immediately eliminate the display area.

[0694] 第六阶段显示在完成视频选择操作之后,视频会议期间的Π7475。 [0694] After the completion of the sixth stage display video selection operation, Π7475 during a video conference. 视频显示区7410不再在Π 7425上。 Video display area 7410 is no longer on Π 7425. 在这个阶段,UI 7475呈现新的PIP显示7495, PIP显示7495包括作为背景主显示的远程视频显示7415,和作为插入显示的本地视频显示7405。 At this stage, UI 7475 presents a new PIP display 7495, PIP display 7495 includes a remote video display as the background main display 7415, and 7405 as a local video display insert display. 在一些实施例中,这种视频选择操作只影响本地显示,因为两个视频仍然被传送给远程设备。 In some embodiments, such video selection operation affects only the local display, since two still video transmitted to the remote device. 其它实施例停止从除去的照相机拍摄视频。 Other embodiments to stop recording the video from the camera removed.

[0695] 上面的例子举例说明选择的本地视图是已显示在背景主显示中的视图的情况。 [0695] The above example illustrates a case where the selected local view in the background of the main display view is displayed. 在一些实施例中,当用户选择显示在插入显示之一中的本地视图时,选择的本地视图被显示在背景主显示中。 In some embodiments, when the user selects one of the local view of the insertion display, selected local view is displayed in the background of the main display. 这种情况下,一些这样的实施例利用和在图68中显示的动画类似的动画。 In this case, some such embodiments utilize animation and 68 shown in FIG similar animation. 当背景主显示中的本地视图被除去时,其它实施例将利用插入远程视图。 BACKGROUND When a local main display view is removed, by inserting the other embodiments remote view.

[0696] 类似于上面参考图73说明的远程视图选择操作,本地视频的选择会导致本地双照相机移动设备的图像处理操作的变化,比如使视频会议管理器1604响应用户的输入,在合成显示中只合成选择的一个或多个远程视频。 [0696] similar to that described above with reference to FIG. 73 of the remote view selection operation, the selection will lead to local variation video image processing local dual camera mobile device operation, such that the video conference manager 1604 in response to user input, the display in the synthesis synthesis of only selected one or more remote video. 本地视频的选择还会导致本地设备的一个或多个照相机的操作的变化。 Local video selection will cause a change in the operation of a local device or a plurality of cameras. 例如,一些实施例使不需要的视频的照相机停止向远程设备传送不需要的视频,而其它实施例使该照相机停止拍摄不需要的视频。 For example, some embodiments enable the camera does not need to stop transmitting the video to the remote device does not require a video, while other embodiments do not need that the video camera stops recording.

[0697] 9.选择传送本地视图 [0697] 9. The local view selections

[0698] 上面的各个子节举例说明了会议中对视频显示的修改。 [0698] The above respective sub-section illustrates a modification of the video conference display. 一些实施例还允许双照相机移动设备的用户在开始视频会议之前,选择把哪部照相机用于视频会议。 Some embodiments also allow a user double camera mobile device before the start of the video conference, the choice of which camera for videoconferencing unit. 图75图解说明在会议前,从由用户的双照相机移动设备拍摄的两个视频中选择一个视频,用于视频会议的操作。 FIG 75 illustrates a prior session, select a video from a video captured by two camera mobile bis user device, for operating the video conference.

[0699] 图75按照Π 7500的八个操作阶段,图解说明用于视频会议的本地视频的选择。 [0699] FIG. 75 according to the eight stages of operation Π 7500, illustrates a selection of local video for video conferencing. 第一阶段7502图解说明双照相机移动设备7518的Π 7500,Π 7500具有在用户请求开始与移动设备的远程用户的视频会议之后呈现的初始PIP显示7542。 The first stage 7502 illustrates a dual [pi camera mobile device 7518 of 7500, Π 7500 has a user requests to start the user's mobile device after the remote video conference presentation initial PIP display 7542.

[0700] 如第一阶段7502中所示,初始PIP显示7542包括两个视频显示:背景主显示7520 和前景插入显示7522。 As shown in the first stage 7502 [0700], two initial PIP display 7542 includes a video display: the foreground and background main display 7520 display 7522 is inserted. 背景主显示7520和前景插入显示7522。 Background main display 7520 and a foreground inset display 7522. 背景主显示7520占据设备的大部分显示屏幕,而前景插入显示7522较小且重叠在背景主显示上。 BACKGROUND occupies most of the main display screen of the display device 7520, and the foreground inset display 7522 is small and superimposed on a background of the main display. 在这个例子中,背景显示7520呈现拿着吉它的人物的视频,该视频假定是由设备的背面照相机拍摄的视频。 In this example, the background display 7520 presents holding a guitar figures of video that is assumed to be captured by the video camera on the back of the device. 前景插入显示7522呈现戴着帽子的人物的视频,在本例中,该视频假定是由设备的正面照相机拍摄的视频。 Foreground inset display 7522 presents a video of a person wearing a hat, in this case, assuming that the video is video captured by the front camera device.

[0701] 初始PIP显示7542只是呈现由本地设备的照相机拍摄的视频的合成视图的一种方式。 [0701] Initial PIP display 7542 is only one manner of presenting the synthesized video view captured by the local camera device. 一些实施例可以提供其它的合成视图。 Some embodiments may provide other composite views. 例如,背景显示可以呈现来自设备的正面照相机的视频,较小的前景插入显示可呈现来自设备的背面照相机的视频。 For example, the background display can present the video camera from the front of the device, the smaller foreground inset display can present the video camera from the back of the device. 另外,一些实施例允许两个视频出现在Π 7500中的两个并排显示区(例如,左右显示窗口,或者上下显示窗口),或者两个对角排列的显示区中。 Further, some embodiments allow the two video Π 7500 appear in the display area of ​​the two side by side (e.g., left and right display window, the display window or down), or the two display areas are aligned diagonally. 在一些实施例中,PIP显示的方式或者默认显示模式可由用户指定。 In some embodiments, the PIP display mode or the default display mode specified by the user. PIP显示的下面是标记为"End Conference"的可选Π项目7540(例如,按钮7540),可选Π项目7540允许用户通过选择该项目而结束视频会议。 The following PIP display is labeled "End Conference" Π selectable item 7540 (e.g., button 7540), item 7540 allows the user to optionally Π video conference is ended by selecting the item.

[0702] 在第一阶段7502,移动设备7518的用户已请求与远程用户进行视频会议,并且正在等待远程用户响应。 [0702] In the first stage 7502, 7518 of the mobile device user has requested a video conference with a remote user, the remote user and is waiting for a response. 在显示画面的底面的"Preview, Waiting for response...(预览, 正等待响应...)"注解图解说明该等待期。 In the bottom surface of the display screen "Preview, Waiting for response ... (preview, awaiting response ...)" illustrates the annotation waiting period.

[0703] 第二阶段7504图解说明视频选择操作的开始。 [0703] The second stage 7504 illustrates the start of the video selection operation. 在这个例子中,通过调用要显示在PIP显示7542上的一组可选Π项目,发起该操作。 In this example, by calling to display a set of selectable items on Π 7542 in PIP, initiated the operation. 所述一组可选Π项目呈现用于选择本地视频,以传送给视频会议的远程设备的各种选项。 Π the set of selectable items presented for selecting the local video for delivery to a variety of options for remote video conferencing equipment. 在一些实施例中,在等待远程用户响应的同时,通过在会前时间内选择(例如,触摸)Π显示7500上的任意地方,可调用所述一组可选Π项目。 In some embodiments, the remote user while waiting for a response, by selecting the time of the pre (e.g., touch) [pi anywhere on display 7500, the call may be a set of selectable items [pi. 代替这种调用操作,或者与这种调用操作结合,一些实施例还允许用户通过其它操作,比如通过不同的触屏操作,或者利用设备的一个或多个其它物理输入,调用所述一组可选Π项目。 Instead of such a call operation, or in conjunction with this calling operation, Some embodiments also allow the user to other operations, such as by the different touch screen operation, or by a physical device or a plurality of other inputs, a set of the call election Π project.

[0704] 第三阶段7506图解说明具有一组调用的供用户选择视频的可选Π项目7526 的Π 7500。 [0704] The third stage 7506 illustrates having a plurality of selectable call for user selection of a video item [pi Π 7500 7526's. 在这个例子中,呈弹出式菜单形式的一组可选Π项目7526显示在PIP显示区7542中,并重叠在PIP显示上。 In this example, as a pop-up menu in the form of a set of selectable items Π 7526 displayed in the PIP display area 7542, and superimposed on the PIP display. 在这个例子中,所述一组可选UI项目包括'Transmit L1"项目7528(例如,按钮7528) ;"Transmit L2"项目7530(例如,按钮7530) ;"Transmit Both"项目7532(例如,按钮7532);和"Cancel"项目7534(例如,按钮7534)。在这个例子中,选择"Transmit L1"按钮7528会使Π 7500在视频会议期间,只把由设备的背面照相机拍摄的视频传送给远程设备。选择"Transmit L2"按钮7530会使Π 7500在视频会议期间,只把由设备的正面照相机拍摄的视频传送给远程设备。选择"Transmit Both"按钮7532会使Π 7500把由设备的正面和背面照相机拍摄的两个视频传送给视频会议的远程用户,以及选择"Cancel "按钮7534会取消该操作。 In this example, the set of selectable UI items including 'Transmit L1 "item 7528 (e.g., button 7528);" Transmit L2 "item 7530 (e.g., button 7530);" Transmit Both "item 7532 (e.g., button 7532); video transfer and "Cancel" item 7534 (e.g., button 7534) in this case, select "transmit L1" button 7528 causes Π 7500 during a video conference, only the back surface captured by the camera to the remote device device. select "transmit L2" button 7530 will Π 7500 during a video conference, only the video captured by the front camera device to the remote device. select "transmit Both" button will Π 7500 to 7532 by the front of the device and two video taken by the camera on the back is transmitted to the remote user video conferencing, and select "cancel" button 7534 will cancel the operation.

[0705] 第四阶段7508图解说明在用户选择了"Transmit L1"按钮7528(例如,通过用他的手指7524轻点按钮7528)之后的Π 7500。 [0705] In the fourth stage 7508 illustrates the user has selected the "Transmit L1" button 7528 (e.g., with his finger 7524 by tapping the button 7528) Π 7500 after. 通过突出显示PIP显示区7542上的按钮7528,指示这种选择。 7542 PIP button on the display area is highlighted 7528 to indicate this option. 一些实施例利用不同的指示显示(例如,突出显示所选项目的边框或者所选项目中的文本)。 Some embodiments utilize different instructs the display (e.g., text highlighting the selected item or selected items frame).

[0706] 第五阶段7510图解说明在用户选择了设备的背面照相机的视频,以传送给远程设备之后的Π 7500的动画。 [0706] 7510 illustrates the fifth stage in the back of the user selected the video camera apparatus, to transmit to the remote device after the animation Π 7500. 在这个例子中,通过如箭头756所示,使不需要的前景插入显示7522滑出PIP显示7542的右边缘,Π 7500除去不需要的前景插入显示7522。 In this example, as shown by arrow 756, the unnecessary slipping foreground inset display 7522 7542 PIP display of the right edge, Π 7500 to remove unwanted foreground inset display 7522. 在第六阶段7512,插入显示7522已从PIP显示区7542中被完全除去。 In the sixth stage 7512, the display 7522 is inserted into the PIP display area 7542 from being completely removed. 不同的实施例利用不同的动画来除去不需要的显示区,比如使该显示区淡出或者渐隐、沿不同的方向移动该显示区、 或者简单地立即消除该显示区。 Different embodiments use different animations to remove unnecessary display region, the display region such that the fade-out or fade-out, moves the display area in a different direction, or simply the immediate elimination of the display region.

[0707] 第七阶段7514图解说明在远程用户已接受视频会议请求之后的Π 7500的动画。 [0707] The seventh stage 7514 illustrates the animation Π 7500 has accepted the request after a video conference in the remote user. 通过消除显示画面上的"Preview, Waiting forresponse."注释,突出显示视频会议请求的接受。 By eliminating the display "Preview, Waiting forresponse." Notes on the screen, highlight a video conference to accept the request. 在这个阶段,作为来自设备的背面照相机的视频的背景显示区7520逐渐缩小到PIP显示区7542的左下角,如箭头7538所示。 At this stage, as the back of the camera device from the background video display area 7520 is gradually reduced to the lower left corner of the PIP display area 7542, as shown by arrow 7538. 背景显示7520缩小,使得Π 7500能够在显示区7520之后显示包含来自远程用户的照相机的视频的显示区7544。 Reduced the background display 7520 so that Π 7500 7544 display area can be displayed in the video display area 7520 after the camera from a remote user comprising a. 一些实施例把本地照相机缩小到不同的位置,利用显示的两个视频的平铺合成显示,或者使远程视图成为PIP 显示的插入显示区。 Some embodiments of the camera is reduced locally to a different position, the two video display composite display tile, or to be inserted into the display area remote view PIP display.

[0708] 第八阶段7516表示在完成视频选择操作之后的Π 7500。 [0708] The eighth phase indicates Π 7500 7516 After completion of the video selection operation. Π 7500呈现新的PIP 显示7546,PIP显示7546包括从本地设备拍摄的视频的插入显示7520,和从远程设备传送来的视频的背景显示7544。 Π 7500 presents a new PIP display 7546, PIP display 7546 includes a video taken from a local device is inserted into the display 7520, and transmitted to a remote device from the background of the video display 7544.

[0709] B.带宽&帧速率 [0709] B. Bandwidth & frame rate

[0710] 在一些实施例中,视频会议期间对远程移动设备的显示区的大小的调整会使本地移动设备重新分配分配给由本地移动设备的两部照相机(即,正面照相机和背面照相机) 拍摄的每个视频的带宽。 [0710] In some embodiments, the adjustment of the size of the display region remote mobile device will make a local mobile device during the video conference reassigned captured by two cameras assigned to the local mobile device (i.e., front camera and the rear camera) the bandwidth of each video. 图76图解说明在本地设备的两部照相机之间的这种带宽重新分配的两个例子。 FIG 76 illustrates an example of such a bandwidth between two two cameras local device reassigned.

[0711] 图76中的每个例子涉及本地设备的背面照相机传感器7605、本地设备的正面照相机传感器7610、本地设备的视频会议模块7615、和远程移动设备7620的Π 7635。 In each case [0711] FIG. 76 relates to the local device 7605 back camera sensors, camera sensor front local device 7610, a video conferencing module local device 7615, and a remote mobile device Π 7620 to 7635. 背面照相机传感器7605和正面照相机传感器7610捕获来自本地设备的相应背面照相机和正面照相机的视频。 Back camera sensor 7605 and sensor 7610 captures the respective front camera back camera and front camera video from the local device. 捕获的视频被发送给视频会议模块7615,视频会议模块7615处理捕获的视频,并把它们传送给远程设备,以在Π 7635中显示。 Captured video is transmitted to the video conference module 7615, a video conference video capture processing module 7615, and transmits them to a remote device to display in Π 7635.

[0712] 在图76中,远程设备的Π 7635呈现合成显示。 [0712] In FIG. 76, Π 7635 remote rendering composite display device. 合成显示表示由本地设备的正面照相机和背面照相机拍摄的视频。 Synthesis of the display indicates that the video taken by a front camera and the back camera of the local device. 来自正面照相机的视频拍摄树和戴着帽子的男士,而来自背面照相机的视频拍摄山脉风景。 Video shot from the front of the camera and wearing a hat tree man, and the video camera filming from the back of the mountain scenery. 如图76中所示,根据用于显示视频的显示区的排列以及显示区的大小,可用许多不同的方式在Π 7635中显示这两个视频。 As shown in FIG. 76, the arrangement according to the display size and a display area of ​​the video display area, a number of different methods available display both the video Π 7635. 在每个例子中,视频会议模块7615最初按照远程设备中的显示区的相对大小,在每个视频之间分配总的输出带宽。 In each example, a video conference module 7615 according to the relative size of the first display area in a remote device, the total output bandwidth allocation between each of the video. 具体地说,显示在Π 7635中的较大显示区中的视频被分配较大部分的总带宽,以及显示在Π7635中的较小显示区中的视频被分配较小部分的带宽。 Specifically, the display larger display area Π 7635 in the video is allocated a larger portion of the total bandwidth, and displaying the video in a small display area Π7635 in the smaller portion of bandwidth is allocated. 在一些实施例中,当视频被显示在相同大小的显示区中时,在这两个视频之间平分总的输出带宽。 In some embodiments, when a video is displayed on the display region of the same size, between the two video bisects the total output bandwidth.

[0713] 分配给这两个视频中的每个视频的带宽的量会最影响处理每个视频的方式。 [0713] allocated to the amount of each of these two video bandwidth of the video will most affect the processing of each video mode. 例如,视频可能需要比分配给该视频的带宽更大的带宽。 For example, the video may require more bandwidth than that assigned to the video bandwidth. 在这种情况下,视频的帧速率被调整,或者视频图像的大小被比例缩小,以便适应较小的带宽。 In this case, the video frame rate is adjusted, or the size of the video image is a reduced scale, in order to accommodate a smaller bandwidth. 降低视频的帧速率导致视频显得"不连贯",而比例缩小视频图像的大小会减小显示该视频的区域。 Reduce the video frame rate results in the video appear "choppy", and scaling down the size of the video image decreases the video display area. 于是,当视频被分配一定量的带宽时,一些实施例调整该视频的帧速率,比例缩小视频图像的大小,或者进行这两者的组合,以便确保能够在分配的带宽内传送该视频。 Thus, when the video is allocated a certain amount of bandwidth, some embodiments adjust the video frame rate, the size of the video image is scaled down, or by a combination of both, in order to ensure that the video can be transmitted within the allocated bandwidth. 本领域的普通技术人员会认识到帧速率和平均帧大小的调整可变化,以获得最佳的总体视频质量,同时仍然确保能够在分配的带宽内传送该视频。 Those of ordinary skill in the art will recognize that adjusts the frame rate and the average frame size can be varied to achieve the best overall video quality, while still ensuring that the video can be transmitted within the allocated bandwidth.

[0714] 图76的例子(1)用Π 7635的两个操作阶段,举例说明带宽重新分配的一种情形。 Examples (1) with two operating phases Π of 7635, illustrates a situation a bandwidth reallocation [0714] FIG 76. 第一阶段7670中的远程设备7620的Π 7635呈现包含两个显示的合成显示:一个在Π 7635的顶部,另一个在Π 7635的底部。 The first stage 7670 of remote device 7620 comprises two [pi Synthesis 7635 presentation displayed on the display: In a 7635 [pi top, the other at the bottom of the [pi 7635. 在这个例子中,顶部的显示区7625显示由本地设备的正面照相机拍摄的视频,底部的显示区7630显示由本地设备的背面照相机拍摄的视频。 In this example, the top of the display area 7625 displays a video captured by the front camera of the local device, the bottom of the display area 7630 displays a video captured by the back camera of the local device. 如第一阶段7670中所示,顶部的显示区7625大于底部的显示区7630。 As shown in the first stage 7670, 7625 of the top of the display area larger than the bottom of the display area 7630. 从而,来自本地设备的正面照相机的视频被分配80%的带宽,来自本地设备的背面照相机的视频被分配20%的带宽。 Thus, the front camera video from a local device is allocated 80% of the bandwidth, the back surface of the camera video from the local device are allocated bandwidth of 20%. 为了确保能够在分配的带宽内,把来自本地设备的背面照相机的视频从本地设备传送给远程设备,调整该视频的帧速率和/或缩放比例大小。 In order to ensure that the bandwidth allocation, the back surface of the video camera from the local device from the local device to the remote device, to adjust the video frame rate and / or size scaling.

[0715] 第二阶段7675图解说明在远程设备的用户增大底部显示区的大小,使得顶部显示区7625和底部显示区7630的大小近似相同之后的Π 7635。 [0715] The second stage 7675 illustrates the increase in size of the user of the remote device display area of ​​the bottom, so that the top and bottom of the display area 7625 displays Π after approximately the same size region 7630 7635. 结果,每个视频被视频会议模块7615重新分配50%的总带宽。 As a result, each video is redistributed 7615 50% of the total bandwidth of the video conference module.

[0716] 图76的例子(2)用Π 7635的两个操作阶段,举例说明带宽重新分配的另一种情形。 Examples [0716] FIG. 76 (2) Π with two operating phases of 7635, illustrates another case where the bandwidth reallocation. 在例子(2)的第一阶段7680中,Π 7635呈现PIP显示。 In the first stage of the example (2) of 7680, Π 7635 presents a PIP display. 该PIP显示包含两个显示: 背景主显示区7650和前景插入显示区7655。 The PIP display two display comprising: a main display area 7650 and the background of the foreground display region 7655 is inserted. 背景主显示区7650占据大部分的PIP显示, 而前景插入显示区7655较小且重叠在背景主显示区7650上。 BACKGROUND main display area 7650 occupies most of the PIP display, the foreground inset display area 7655 superimposed on the background and a small main display area 7650. 在这个例子中,背景显示区7650呈现由该设备的正面照相机拍摄的视频。 In this example, the background display area 7650 presents a video captured by the front camera of the device. 插入显示区7655呈现由该设备的背面照相机拍摄的视频。 Inset display area 7655 presents a video captured by a camera on the back of the device. 如在这个阶段中所示,背景显示区7650大于插入显示区7655。 As shown in this stage, the background display region 7650 is greater than the display area 7655 is inserted. 从而,来自设备的正面照相机的视频被分配80%的带宽,来自设备的背面照相机的视频被分配20% 的带宽。 Thus, the front camera video from the device is allocated 80% of the bandwidth of the video camera from the rear surface of the device is allocated 20% of the bandwidth. 为了确保在分配的带宽内,能够把来自本地设备的背面照相机的视频从本地设备传送给远程设备,调整视频的帧速率和/或缩放比例大小。 In order to ensure that the bandwidth allocation can be the back surface of the video camera from the local device from the local device to the remote device, to adjust the video frame rate and / or size scaling.

[0717] 第二阶段7685图解说明在远程设备的用户已交换这两个视频的显示之后的Π 7635。 [0717] The second stage 7685 illustrates the user at the remote device has been exchanged Π 7635 after the two video display. 具体地说,背景显示区7660现在呈现由设备的背面照相机拍摄的视频,以及插入显示区7665现在呈现由设备的正面照相机拍摄的视频。 Specifically, the background display area 7660 now render the video captured by the camera on the back of the device, and inset display area 7665 now render the video captured by the front camera device. 由于这两个视频的显示区的大小已改变,因此来自设备的背面照相机的视频被分配80%的带宽,来自设备的正面照相机的视频被分配20%的带宽。 Since the size of the two video display area has changed, so the back video from the camera device is allocated 80% of the bandwidth, the camera video from the front device is allocated 20% of the bandwidth. 因而,来自本地设备的正面照相机的视频的帧速率和/或缩放比例大小将被减小。 Thus, the frame rate of the video camera from the front side of the local device and / or scaling size will be reduced. 本领域的普通技术人员会认识到在图76中说明的带宽分布只是例子,在视频会议期间,在两部照相机之间分配带宽的其它技术也是可能的。 Those of ordinary skill in the art will recognize that the bandwidth of the distribution illustrated in FIG. 76 is only an example, during a video conference, other techniques for allocating bandwidth between the two cameras is also possible.

[0718] 1.帧速率控制 [0718] 1. The frame rate control

[0719] 类似于上面说明的会议中的帧速率控制操作,一些实施例可能希望单独调整或保持在向视频会议中的另一个设备传送由双照相机移动设备的每部照相机拍摄的视频图像的速率。 Rate frame rate meeting [0719] similar to the above-described control operation, some embodiments it may be desirable to adjust or individually held in the video images captured by each camera device to the dual camera mobile device transmits the other video conference . 一些这样的实施例提供上面说明的类似技术。 Some such embodiments provide similar techniques described above. 例如,一些实施例通过调整每部照相机的传感器模块415的VBI,控制每部照相机的帧速率。 For example, some embodiments of the VBI by adjusting each camera sensor module 415 controls the frame rate of each camera. 其它实施例还提供另外的技术, 比如可由每部照相机的传感器模块415和/或通用传输缓冲器3120执行的帧丢弃。 Other embodiments also provide additional techniques, such as by frame each camera sensor module 415 and / or the common transmission buffer 3120 performs discarded.

[0720] 2.通过比例缩放的带宽控制 [0720] 2. By scaling the bandwidth control

[0721] 如上所述,在双照相机移动设备和另一个设备之间的视频会议期间,在一定时间量内,通过一个或多个网络连接能够传送的图像数据的数量(即,网络连接带宽)有限。 [0721] As described above, during a video conference between the dual camera mobile device and another device, a certain amount of time, through one or more networks capable of transmitting data of the number of connected image (i.e., network connection bandwidth) limited. 为了最大化并维持网络连接的吞吐量,双照相机移动设备的不同实施例提供不同的方式控制在特定时间量内,通过网络连接传送的图像数据的数量。 To maximize throughput and maintain network connections, various embodiments of the dual camera mobile devices provide different controlled within a certain amount of time, the number of image data transmitted through a network connection. 在一些实施例中,吞吐量是通过通信通道(例如,网络连接)的成功消息传送的平均速率。 In some embodiments, the average throughput rate is a success message (e.g., network connection) transmitted via the communication channel.

[0722] 当传送由双照相机移动设备的两部照相机拍摄的图像时,一种这样的方法调整来自双照相机移动设备的一部或两部照相机的图像的大小,以控制通过网络连接传送的图像数据的数量。 [0722] When transmitting the image captured by two cameras dual camera mobile device, one such method for adjusting a size of the image from the camera or two double camera mobile device, to control the image transmitted through a network connection the amount of data. 一些实施例比例缩小由双照相机移动设备拍摄的图像的大小,以减少通过网络连接传送的图像数据的数量,而其它实施例比例放大图像的大小,以增大通过网络连接传送的图像数据的数量。 Some embodiments scale down the size of the mobile device captured by the dual camera image to reduce the number of connections transmitting image data through a network, while other embodiments the ratio of the size of an enlarged image in the embodiment, in order to increase the number of connections transmitting image data through a network .

[0723] -些实施例在缩放时,保持图像的高宽比(S卩,均匀缩放)。 [0723] - when scaling some embodiments, to maintain the aspect ratio of the image (S Jie, uniform scaling). 其它实施例缩放图像, 使得缩放后的图像的高宽比不同于原始图像的高宽比(即,变形缩放)。 Other embodiments of the scaled image, such that the scaled image aspect ratio different from the aspect ratio of the original image (i.e., zoom deformation).

[0724] 此外,可在图像处理过程的不同阶段进行比例缩放。 [0724] In addition, scaling may be performed at different stages of image processing. 一些实施例的比例缩放可由照相机传感器进行。 Some embodiments of the zoom ratio by the camera sensor. 在这样的实施例中,照相机传感器可丢弃图像的多行或多列数据(即, 像素值)。 In such an embodiment, the camera sensor may be discarded multiple rows or columns of the image data (i.e., pixel values). 在一些这样的实施例中,剩余的图像数据被插值,以平滑图像的外形。 In some such embodiments, the remaining image data is interpolated to smooth contour image.

[0725] 其它实施例的比例缩放由CIPU 400的定标器模块455执行。 [0725] Other embodiments of the zoom ratio CIPU 400 performed by the scaler module 455. 在一些实施例中,t匕例缩放由视频会议管理器1604执行,如上所述,在其它实施例中,比例缩放由编码器执行。 In some embodiments, t is scaled dagger embodiment performed by a video conference manager 1604, as described above, in other embodiments, the scaling performed by the encoder. 因而,双照相机移动设备的不同实施例不同地进行比例缩放。 Accordingly, various embodiments of the dual camera mobile device be scaled differently.

[0726] 3.比特率控制 [0726] 3. The bit rate control

[0727] -些实施例提供不同的机制来管理对由双照相机移动设备的照相机拍摄的视频编码的比特率。 [0727] - Some embodiments provide mechanisms to manage different bit rate video coding captured by the camera of the dual camera of mobile device. 在一些实施例中,双照相机移动设备包括用于每部照相机的速率控制器。 In some embodiments, dual camera mobile device for each camera includes a rate controller. 一些实施例提供固定比特率管理方案。 Some embodiments provide a fixed bit rate management program. 按照这种方案,每个速率控制器被设定为固定比特率, 使得来自移动设备上的两部照相机的视频的总比特率是恒定的。 According to this embodiment, each rate controller is set as a fixed bit rate, so that the total bit rate video from both cameras on mobile devices is constant. 其它实施例提供优先级方案,其中当要求减小总比特率时,来自设备的照相机的两个视频之一总是获得比另一个视频高的优先级。 Other embodiments provide a priority scheme, wherein when the required total bit rate is reduced, from one of the two video camera apparatus always obtain a high priority than the other video.

[0728] 在一些实施例中,判优器模块管理两部照相机的两个速率控制器。 [0728] In some embodiments, two speed arbiter module management controller two cameras. 图77图解说明这种判优器模块的例子。 FIG 77 illustrates an example of such an arbiter module. 如图77中所示,速率控制器7700设定正面照相机的比特率,以及速率控制器7705设定背面照相机的比特率。 As shown in FIG. 77, the bit rate controller 7700 sets the front side of the camera, and a rate controller 7705 to set the bit rate of the back of the camera. 速率控制器把来自照相机传感器的图像发送给编码器7715。 The rate controller transmits the image from the camera to the encoder sensor 7715. 判优器模块7710与两个速率控制器连接,并根据诸如可用带宽、两个视频中的每个视频的视频大小等,以任意多种方式控制每个速率控制器7700和7705的比特率的设置。 The arbiter module is connected to the two rate controller 7710, such as according to the available bandwidth, two video video size of each video and the like, in any number of ways to control each of the rate controllers 7700 and 7705 bit rate settings. 以确保在可用带宽下,能够把两个视频都传送给远程设备。 In order to ensure the available bandwidth, two videos are able to transmit to the remote device. 另外,判优器7710可被配置成实现上面提及的固定速率方案,或者优先级方案。 Further, the arbiter 7710 may be configured to implement the above-mentioned fixed rate scheme or a priority scheme.

[0729] 在一些其它实施例中,用于两部照相机的两个速率控制器能够相互通信。 [0729] In some other embodiments, the two cameras for the communication can be two each rate controller. 按照这种方案,速率控制器能够交换它们各自视频的信息,并据此设定视频的比特率。 According to this embodiment, the rate controller can exchange information with their respective video, and the video bit rate is set accordingly. 提供了速率控制器速率管理机制的几个例子。 The rate controller provides several examples of rate management mechanism. 不过,许多其它的不同机制也是可能的。 However, many other different mechanisms is also possible.

[0730] 4.视频处理 [0730] 4. Video Processing

[0731] 双照相机移动设备的一些实施例在不同的情况下,不同地处理由双照相机移动设备的两部照相机拍摄的图像。 [0731] Some embodiment of the dual camera mobile device in different situations, different reasons dual camera mobile device is located in the two images taken by the camera. 例如,当处理包括由双照相机移动设备的两部照相机拍摄的图像的PIP合成图像时,一些实施例有选择地对PIP合成图像进行TNR处理2000。 For example, when processing an image captured by two cameras comprise dual camera mobile device of PIP image synthesis, some embodiments of PIP selectively synthesized image processing TNR 2000. 一些这种实施例只对PIP合成图像中的主图像进行TNR处理2000,而其它的这种实施例只对PIP 合成图像中的插入图像进行TNR处理2000。 Some such embodiments, only the main image synthesizing PIP image processing performed TNR 2000, and such other embodiments only the PIP image is inserted in the composite image processing performed TNR 2000.

[0732] 作为处理由移动设备的两部照相机拍摄的图像的另一个例子,一些实施例根据视频会议的各种改变,比如用户对显示视频的显示区的调整(例如,放大PIP显示的插入显示区、在显示的视频中定义感兴趣区域、交换PIP显示的主显示区/插入显示区,等等),对总的可用带宽的改变,等等,比例缩放由双照相机移动设备的两部照相机拍摄的图像。 Insert [0732] As another example of processing images taken by two camera mobile device, a number of changes in accordance with various embodiments of the video conference, the user such as adjustment of displaying video display area (e.g., an enlarged display displayed PIP region, the region of interest defined in a video display, switching the display of the main display area PIP / inset display area, etc.), changes to the total available bandwidth, and the like, by the two pairs of scaled camera mobile device camera the captured image. 一些这样的实施例按照上述方式比例缩放图像。 Some such embodiments the scaling ratio of the image according to the embodiment. 即,图像可由编码器1655、视频会议管理器1604、 定标器模块455和用其拍摄图像的照相机传感器(即,405a或405b)比例缩放。 That is, the image by the encoder 1655, a video conference manager 1604, and a scaling module 455 with the sensor which camera captured image (i.e., 405a or 405b) scaling.

[0733] 5.编码 [0733] ​​The coding

[0734] 如上所述,一些实施例传送来自双照相机移动设备的两部照相机的视频。 [0734] As described above, the two cameras of transmission from the dual camera mobile video device of some embodiments. 因而,这些实施例可对由两部照相机拍摄的视频编码,以便在视频会议期间传送给远程设备。 Accordingly, these embodiments can encode a video captured by two cameras for transmission to a remote device during the video conference. 不同的实施例提供不同的方式来编码视频以便传输。 Various embodiments provide different ways to encode video for transmission. 图78图解说明利用多路复用器(MUX) 7815、编码器模块7825、缓冲器7830和组合模块7835处理视频,以便传输的方法。 Figure 78 illustrates the use of a multiplexer (MUX) 7815, an encoder module 7825, a buffer 7830, and processing a video combining module 7835, a method for transmission.

[0735] 根据选择信号,MUX 7815获得一个输入信号,并把选择的输入信号输出给编码器7825。 [0735] A signal input select signal, MUX 7815 is obtained according to, and outputs the selected input signal to the encoder 7825. 例如,如果选择信号指示MUX 7815获得来自C1的输入信号,那么MUX 7815选择该输入信号,并输出该输入信号。 For example, if the selection signal indicates the input signal from MUX 7815 to obtain C1, then the MUX 7815 to select the input signal, and outputs the input signal. 可用多种方式,比如通过来自视频会议管理器1604的指令, 提供所述选择信号。 A number of ways, such as by an instruction from the video session manager 1604 provides the selection signal. 通过MUX 7815,编码器7825交替地把从MUX7815接收的图像编码成比特流格式,并把编码的图像保存在缓冲器7830中。 Via MUX 7815, the encoder 7825 are alternately MUX7815 received from the image encoding into a bitstream format, and the encoded image stored in the buffer in 7830. 组合模块7835组合(即,多路复用)保存在缓冲器7830中的一个或多个比特流,并输出单一的比特流。 Combining module 7835 combinations (i.e., multiplexing) one or more bit streams stored in the buffer 7830, and outputs the single bitstream.

[0736] 现在按照三个阶段7860、7865和7870,说明这种编码方法的操作。 [0736] Now 7860,7865 and 7870 in accordance with three phases, the operation of this encoding method. 在第一阶段7860, MUX 7815被配置成接收由照相机C1拍摄的图像7805,并输出给编码器7825以便编码。 In the first stage 7860, MUX 7815 is configured to receive an image captured by the camera C1 is 7805, and outputs to the encoder 7825 to encode. 编码器7825对接收的图像编码并产生比特流7850,比特流7850随后被保存在缓冲器7830中。 The encoder 7825 pairs of the received encoded image and generates a bitstream 7850, the bitstream is then stored in the buffer 7850 in 7830. 第二阶段7865类似于第一阶段7860,除了MUX 7815被配置成接收由照相机C2 拍摄的图像7810,并输出给编码器7825,以便编码之外。 The second stage 7865 is similar to the first stage 7860, in addition to MUX 7815 is configured to receive an image captured by the camera C2 7810, and outputs to the encoder 7825 to encode outside. 同样地,编码器对接收的图像编码并产生比特流7855,比特流7855被保存在缓冲器7830中。 Likewise, an encoder for encoding an image and generates a received bit stream 7855, the bit stream is stored in a buffer 7855 in 7830. 在第三阶段870,组合模块7835 从缓冲器7830取回比特流7850和7855,并把它们组合成一个比特流,以便传送给远程设备。 In the third stage 870, the combining module 7835 retrieves 7830 the bit stream from the buffer 7850 and 7855, and combines them into a single bit stream for transmission to a remote device.

[0737] 图79图解说明对来自双照相机移动设备的两个视频编码,以便在视频会议期间传送给远程设备的另一种方法。 [0737] FIG. 79 illustrates a video encoder of the two from the dual camera mobile device, another method for transmission to a remote device during the video conference. 在这种方法中,来自移动设备的第一部照相机的视频帧(即,图像)和来自移动设备的第二部照相机的另一个视频帧被合成为一个视频帧,之后合成的视频帧被编码成比特流,以便发送给远程设备。 In this method, a first portion of a video frame from the camera of the mobile device (i.e., image) and a second portion of the camera from the mobile device to another video frame is synthesized one video frame, after the synthesis of the video frame is encoded into a bit stream for transmission to a remote device. 如图79中所示,这种方法包括合成器7915、缓冲器7920和编码器7925。 As shown, this method includes a synthesizer 7915 in FIG. 79, a buffer 7920 and the encoder 7925.

[0738] 如图所示,合成器7915合成来自第一部照相机的图像7905和来自第二部照相机的图像7910,以形成合成图像7955。 [0738] As shown, combiner 7915 synthetic image 7905 and the image 7910 from the second portion from the first camera portion of the camera, to form a composite image 7955. 不同的实施例不同地合成图像7905和7910。 Different embodiments of the composite image 7905 and 7910 variously. 例如, 一些实施例的合成器7915可通过把两个图像排列成彼此相邻(如图80中所示),合成图像。 For example, some embodiments of the synthesizer 7915 via the two images are arranged adjacent to each other (as shown in FIG. 80), the composite image. 合成图像8030和8035图解说明利用这种技术的两个示例合成图像。 Synthetic images 8030 and 8035 illustrate two examples of this technique using a composite image. 在合成图像8030 中,来自第一部照相机的图像7905被排列在来自第二照相机的图像7910之上。 In the composite image 8030, an image 7905 from the first camera 7910 are arranged over the image from the second camera. 而合成图像8035表示图像7905排列在图像7910的左侧。 8035 represents a composite image 7905 arranged in the image of the left image 7910.

[0739] 在一些其它的实施例中,合成器7915可通过把两个图像7905和7910叠加在更大的背景图像的上面,合成两个图像7905和7910。 [0739] In some other embodiments, the synthesizer 7915 through 7905 and 7910 the two images superimposed on the background image above a larger, two synthetic images 7905 and 7910. 图80的合成图像8040图解说明利用这种技术的示例合成图像。 FIG 80 illustrates a synthetic image 8040 of the techniques described by way of example of such a composite image. 在合成图像8040中,图像7905和7910对角排列,并叠加在空白图像上(即,图像7905位于背景图像的左上角,而图像7910位于背景图像的右下角)。 In the composite image 8040, the images 7905 and 7910 are arranged diagonally, and superimposed on the blank image (i.e., image 7905 located in the upper left corner of the background image, the background image 7910 in the bottom right of the image). 在一些实施例中,照相机传感器可能大小不同,从而拍摄具有不同像素分辨率的图像。 In some embodiments, the camera sensor may be a different size, so that the photographed image having a different pixel resolution. 在这种实施例中,合成器7915可按照如由图80的合成图像8045图解说明的相似方式合成图像7905 和7910。 Embodiment, the synthesizer 7915 and 7910 may be synthetic images 7905 in a similar manner as illustrated by the synthetic image 8045 illustrated in FIG. 80. In such an embodiment. 在合成这两个图像之后,合成器7915把合成图像保存在缓冲器7920中。 After synthesis of these two images, a composite image synthesis unit 7915 is stored in the buffer 7920. 编码器7925从缓冲器7920取回合成图像,把合成图像编码成比特流,并将其发送给视频会议的远程设备。 The encoder 7925 is retrieved from the buffer 7920 synthetic image, the composite image into a bitstream, and sends it to the remote video conference equipment.

[0740] 现在参考图79中图解说明的合成器7915、缓冲器7920和编码器7925,说明各个操作。 [0740] Referring now to FIG. 79 illustrates the synthesis of 7915, a buffer 7920 and the encoder 7925, described respective operations. 首先,第一部照相机把作为视频中的一系列图像的一部分的图像7905发送给合成器7915。 First, the first camera portion 7915 sends to the synthesizer as a part of a series of images of the video image 7905. 同时,第二部照相机把作为视频中的一系列图像的一部分的图像7910发送给合成器7915。 Meanwhile, the second camera portion 7915 sends to the synthesizer as a part of the video image 7910 of a series of images. 合成器7915随后按照上述方式合成图像7905和7910,以形成合成图像7955。 Synthesis 7915 as described above is then synthesized images 7905 and 7910, 7955 to form a composite image. 之后,合成器7915把合成图像7955发送给缓冲器7920。 Thereafter, the composite image synthesizer 7915 7955 7920 sent to the buffer. 缓冲器7920随后保存合成图像,之后把合成图像发送给编码器7925。 Then the synthesized image storage buffer 7920, then transmits the combined image to the encoder 7925. 最后,编码器7925把合成图像编码成比特流,并将其发送给视频会议的远程设备。 Finally, the composite image encoder 7925 is encoded into a bit stream, and sends it to the remote video conference equipment.

[0741] 图81图解说明对来自双照相机移动设备的两个视频编码,以便在视频会议期间传送给远程设备的再一种方法。 [0741] FIG. 81 illustrates a video encoder of the two from the dual camera mobile device, a method for re-transmitting to the remote device during a video conference. 在这种方法中,来自该设备的两个视频以合成显示的形式被显示,获得合成显示的屏幕截图,并将其编码成比特流,以便发送给远程设备。 In this method, two video from the device is displayed in the form of a composite display, a screenshot of the composite display, and encode it into a bit stream for transmission to a remote device. 如图81中所示,这种方法包括编码器8115。 As shown in FIG. 81, this method includes an encoder 8115. 在一些实施例中,编码器8115对合成图像编码,并发送给远程设备。 In some embodiments, the encoder 8115 pairs of the synthetic image encoding, and sent to the remote device.

[0742] 现在参考图81中图解说明的编码器8115,说明各个操作。 [0742] Referring now to FIG. 81 illustrates an encoder 8115, described respective operations. 首先,来自双照相机移动设备的两部照相机的视频以合成显示的形式被显示在设备的屏幕上。 First, two video cameras from the dual camera mobile device is displayed on the screen of the display device in the form of synthesis. 所述合成显示能够按任意方式呈现视频。 Synthesis of display capable of presenting the video in any way. 例如,一些实施例中的合成显示能够把两个视频呈现在PIP显示,t匕如在图81中图解说明的PIP显示8105中。 For example, synthesis of some embodiments of the two can be displayed in the PIP display video presentation, T dagger PIP as illustrated in FIG. 81 display 8105. 在其它实施例中,合成显示可把两个视频呈现在两个并排的显示区,或者两个对角排列的显示区中。 In other embodiments, the composite display may be presented in two adjacent two video display areas or display areas of the two diagonally aligned. 获得PIP显示8105的屏幕截图,t匕如图像8110,并将其发送给编码器8115。 PIP display is obtained screenshot 8105, t dagger image 8110, and sends it to the encoder 8115. 编码器随后把一系列的屏幕截图编码成比特流8120,之后将其发送给视频会议的远程设备。 The encoder then sends a series of screen shots into a bitstream 8120, then send it to the remote video conference equipment. 尽管上面说明了对两个视频编码的几种不同方法,不过其它方法也是可能的。 Although several different methods described above for two video encoding, but other methods are also possible.

[0743] 6.解码 [0743] 6. The decoding

[0744] 双照相机移动设备的一些实施例可接收用上面参考图78-81说明的方法编码的比特流。 [0744] Some embodiments of the dual camera mobile device may receive a method of encoding described above with reference to FIGS. 78-81 bitstream. 在这样的实施例中,双照相机移动设备可接收(例如,通过视频会议控制通道)指示用于对视频编码的方法的信息。 In such an embodiment, the dual camera mobile device may receive information indicative of a video coding method (e.g., via video conference control channel). 图82图解说明一种对通过通信网络,从另一个设备接收的两个视频的比特流解码,以便在视频会议期间,显示在双照相机移动设备上的方法。 FIG 82 illustrates a method for communication via a network, from the two video bit stream decoding device receives the other, so that during a video conference, displayed on the dual camera mobile device. 具体地说,这种方法用于对用上面参考图78说明的编码方法编码的比特流解码。 In particular, such a method for decoding a bitstream encoded using the encoding method described above with reference to FIG. 78.

[0745] 如图82中所示,这种方案使用分离模块8235、缓冲器8230和8290、以及解码器模块8225。 [0745] As shown in FIG. 82, such a scheme uses separation module 8235, a buffer 8230, and 8290, and a decoder module 8225. 分离模块8235把比特流分离(即,多路分解)成一个或多个比特流,并把比特流保存在缓冲器8230中。 8235 separation module separating the bit stream (i.e., demultiplexing) into one or more bit streams, and the bit stream stored in the buffer in 8230. 解码器8225取回编码的比特流,对它们解码以产生视频,随后把视频保存在缓冲器8290中。 The decoder 8225 retrieves the encoded bit stream, decodes them to produce a video, and then stored in the buffer 8290 to the video.

[0746] 现在参考图82中图解说明的分离模块8235、缓冲器8230和8290、以及解码器模块8225,说明这种方法的操作。 [0746] Referring now to FIG. 82 illustrates the separation module 8235, a buffer 8230, and 8290, and a decoder module 8225 described, the operation of this method. 首先,双照相机移动设备通过通信网络,从视频会议中的另一个设备接收比特流7845(即,在连网管理器1614)。 First, dual camera mobile device through the communication network, from the other video conference apparatus 7845 receives a bit stream (i.e., in the networking manager 1614). 分离模块8235把接收的比特流分离成两个比特流8255和8260,因为接收的比特流是两个比特流的多路复用比特流。 Separation module 8235 receives the bit stream is separated into two bit streams 8255 and 8260, since the bit stream received two bit streams are multiplexed bit stream. 每个编码的比特流代表从设备的两部照相机之一拍摄的视频数据。 Each coded bit stream represents the video data taken from one of two camera apparatus. 随后,分离模块8235把比特流8255和8260保存在缓冲器8230中。 Subsequently, the bit stream separating module 8235 8260 8255 and 8230 are stored in the buffer.

[0747] 之后,解码器8225从缓冲器8230取回比特流8250 (它是两个比特流8255和8260 中的一个),解码器8225对比特流8250解码,以产生视频8280,并把视频8280保存在缓冲器8290中。 After [0747] The decoder 8225 retrieves the bit stream 8250 from the buffer 8230 (which is a two bit streams 8255 and 8260), the decoder 8225 decodes the bit stream 8250, 8280 to generate the video, and the video 8280 stored in the buffer in 8290. 解码器8225还对比特流8255和8260中的另一个解码,并把产生的视频保存在缓冲器8290中。 The decoder 8225 also decodes the bit stream 8255 and the other in 8260, and the generated video stored in the buffer 8290 in. 现在,两个视频可从缓冲器8290被取回,并被保存或显示在双照相机移动设备上。 Now, two video can be retrieved from the buffer 8290 and stored or displayed on a dual camera mobile device.

[0748] 图83图解说明对用参考图79说明的方法编码的比特流解码的方法。 [0748] FIG. 83 illustrates a method of decoding a bit stream encoded by the method described with reference to FIG. 79. 如图83中所示,该方法包括解码器8325、缓冲器8320和去合成器8315。 As shown, the method includes a decoder 8325, a buffer 8320, and to 831,583 in the synthesizer.

[0749] 在一些实施例中,解码器8325接收用图79中图解说明的方法编码的比特流,把比特流解码成一个或多个合成图像,所述合成图像随后被保存在缓冲器8320中。 [0749] In some embodiments, the decoder 8325 receives encoded using the method illustrated in FIG. 79 described bit stream, the bit stream is decoded into one or more composite image, the composite image is then stored in the buffer 8320 . 去合成器8315从每个合成图像中提取这两个图像。 Synthesizer 8315 to extract the two images from each of the composite image. 为了从合成图像中提取这两个图像,去合成器8315还接收指示每个图像在合成图像内的位置的信息(例如,通过视频会议通信控制通道,从视频会议中的合成并编码图像的设备接收的信息)。 In order to extract the two images from the composite image, to the synthesizer 8315 which also receives the indication position of each image within the composite image information (e.g., via video conference communication control channel, the coding device and the synthetic image from a video conference receiving information).

[0750] 现在参考图83中图解说明的解码器8325、缓冲器8320和去合成器8315,说明这种方法的操作。 [0750] Referring now to FIG. 83 illustrates a decoder 8325, a buffer 8320 and the synthesizer 8315 to be described, illustrating the operation of this method. 首先,解码器325从视频会议的另一个移动设备接收视频的比特流,比如用参考图79说明的方法产生的视频的比特流。 First, the decoder 325 receives a bitstream from a video conference video another mobile device, such as a video bit stream produced by the method 79 described with reference to FIG. 解码器8325把该比特流解码成包括合成图像7955的一个或多个合成图像,并把它们保存在缓冲器8320中。 The decoder 8325 decodes the bit stream into a composite image comprising one or more synthetic images 7955, and saves them in the buffer 8320. 缓冲器8320随后保存合成图像,之后把它们发送给去合成器8315。 Then the synthesized image storage buffer 8320, and then to send them to the synthesizer 8315. 当去合成器从缓冲器8320接收到合成图像7955 时,它把合成图像7955分离成与图79中的图像7905和7910相同的两个图像7905和7910。 When receiving from the buffer to the synthesizer 8320 to the synthesized image 7955, which separates the composite image 7955 into the same image 7905 in FIG. 79 7910 7905 and 7910 two images.

[0751] 当从诸如图81中说明的系统之类的系统接收到比特流时,解码器,比如图83中的解码器8325把比特流解码成一系列的屏幕截图。 [0751] When the system such as the system illustrated in FIG 81 receives the bit stream, the decoder, s