CN114845035B - Distributed shooting method, electronic device and medium - Google Patents

Distributed shooting method, electronic device and medium Download PDF

Info

Publication number
CN114845035B
CN114845035B CN202110131870.2A CN202110131870A CN114845035B CN 114845035 B CN114845035 B CN 114845035B CN 202110131870 A CN202110131870 A CN 202110131870A CN 114845035 B CN114845035 B CN 114845035B
Authority
CN
China
Prior art keywords
shooting
mobile phone
camera
image data
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110131870.2A
Other languages
Chinese (zh)
Other versions
CN114845035A (en
Inventor
冯可荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202110131870.2A priority Critical patent/CN114845035B/en
Priority to CN202210973275.8A priority patent/CN115514882B/en
Priority to PCT/CN2021/137917 priority patent/WO2022160985A1/en
Publication of CN114845035A publication Critical patent/CN114845035A/en
Application granted granted Critical
Publication of CN114845035B publication Critical patent/CN114845035B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • H04M1/72415User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories for remote control of appliances
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72439User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for image or video messaging
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Multimedia (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Telephonic Communication Services (AREA)

Abstract

一种分布式拍摄方法,电子设备及介质,可以在实现分布式拍摄过程中,减少远端设备与本端设备进行数据传输所占用的网络带宽。该方法中,第一设备可接收用户选择与第二设备进行同步拍摄的第一操作。响应于该第一操作,第一设备可开始采集第一图像数据,并且第一设备还可以指示第二设备开始采集图像数据。之后,第一设备可接收来自第二设备的第二图像数据,该第二图像数据包括第二设备的摄像头采集的原始图像。最后,第一设备可将第一图像数据对应的第一拍摄画面显示在第一窗口,将第二图像数据对应的第二拍摄画面显示在第二窗口。该第一窗口和第二窗口位于同一显示界面。

A distributed shooting method, electronic device and medium can reduce the network bandwidth occupied by data transmission between a remote device and a local device during the distributed shooting process. In the method, a first device can receive a first operation in which a user selects to perform synchronous shooting with a second device. In response to the first operation, the first device can start to collect first image data, and the first device can also instruct the second device to start collecting image data. Afterwards, the first device can receive second image data from the second device, and the second image data includes the original image collected by the camera of the second device. Finally, the first device can display a first shooting picture corresponding to the first image data in a first window, and display a second shooting picture corresponding to the second image data in a second window. The first window and the second window are located on the same display interface.

Description

一种分布式拍摄方法,电子设备及介质Distributed shooting method, electronic device and medium

技术领域Technical Field

本申请实施例涉及拍照技术领域,尤其涉及一种分布式拍摄方法,电子设备及介质。The embodiments of the present application relate to the field of photographing technology, and in particular to a distributed photographing method, electronic equipment and medium.

背景技术Background technique

随着互联网技术的发展,多个包括摄像头的电子设备可以组成分布式相机系统。例如,该电子设备可以是手机、平板电脑、智能手表或者智能电视机等。在该分布式相机系统中,一个电子设备(称为本端设备)不仅可以为用户呈现该电子设备的摄像头拍摄的画面,还可以获取并显示其他电子设备(称为远端设备)的摄像头拍摄的画面。With the development of Internet technology, multiple electronic devices including cameras can form a distributed camera system. For example, the electronic device can be a mobile phone, a tablet computer, a smart watch or a smart TV. In the distributed camera system, an electronic device (called a local device) can not only present the images taken by the camera of the electronic device to the user, but also obtain and display the images taken by the cameras of other electronic devices (called remote devices).

具体的,本端设备可以通过以下方式获取远端设备的画面:(1)本端设备向远端设备发送控制命令,指示远端设备采集图像并向本端设备返回预览结果;(2)远端设备接收到该控制命令后,采集图像并对采集的图像进行图像信号处理(image signal processing,ISP)得到预览结果;(3)远端设备向本端设备发送该预览结果。Specifically, the local device can obtain the screen of the remote device in the following ways: (1) the local device sends a control command to the remote device, instructing the remote device to capture an image and return a preview result to the local device; (2) after receiving the control command, the remote device captures an image and performs image signal processing (ISP) on the captured image to obtain a preview result; (3) the remote device sends the preview result to the local device.

在上述跨设备的图像数据传输过程中,远端设备向本端设备发送的是经过图像处理得到的成片(即上述预览结果),其数据量加大,需要占用较大的网络带宽,浪费了网络资源。In the above-mentioned image data transmission process across devices, the remote device sends to the local device the finished image (i.e., the above-mentioned preview result) obtained after image processing, and the data volume is increased, which requires a large network bandwidth and wastes network resources.

发明内容Summary of the invention

本申请提供一种分布式拍摄方法,电子设备及介质,可以在实现分布式拍摄过程中,减少远端设备与本端设备进行数据传输所占用的网络带宽。The present application provides a distributed shooting method, an electronic device and a medium, which can reduce the network bandwidth occupied by data transmission between a remote device and a local device during the distributed shooting process.

第一方面,本申请提供一种分布式拍摄方法,该方法应用于第一设备。该方法中,第一设备可以接收用户选择与第二设备进行同步拍摄的第一操作。响应于该第一操作,第一设备可以开始采集第一图像数据,并指示第二设备开始采集图像数据。之后,第一设备可以接收来自第二设备的第二图像数据,该第二图像数据包括原始图像。最后,第一设备可以将第一图像数据对应的第一拍摄画面显示在第一窗口,将第二图像数据对应的第二拍摄画面显示在第二窗口。该第一窗口和第二窗口位于同一显示界面。In a first aspect, the present application provides a distributed shooting method, which is applied to a first device. In the method, the first device can receive a first operation in which a user selects to perform synchronous shooting with a second device. In response to the first operation, the first device can start collecting first image data and instruct the second device to start collecting image data. Afterwards, the first device can receive second image data from the second device, and the second image data includes an original image. Finally, the first device can display a first shooting picture corresponding to the first image data in a first window, and display a second shooting picture corresponding to the second image data in a second window. The first window and the second window are located on the same display interface.

应注意,上述原始图像也可以称为RAW图像。RAW可翻译为“未经加工”。也就是说,上述原始图像是未经过第二设备处理的图像。相比于经过第二设备处理的图像,原始图像的数据量较小。如此,便可以减少跨设备的分布式拍摄业务所占用的网络带宽。It should be noted that the above original image can also be called a RAW image. RAW can be translated as "unprocessed". That is to say, the above original image is an image that has not been processed by the second device. Compared with the image processed by the second device, the data volume of the original image is smaller. In this way, the network bandwidth occupied by the distributed shooting service across devices can be reduced.

在第一方面的一种可能的设计方式中,上述方法还可以包括:第一设备向第二设备发送第一图像数据,该第一图像数据包括原始图像。该原始图像(也称为RAW图像)是未经过第一设备处理的图像。In a possible design of the first aspect, the method may further include: the first device sends first image data to the second device, the first image data including an original image. The original image (also called a RAW image) is an image that has not been processed by the first device.

如此,不仅第一设备可以同时显示第一设备的拍摄画面和第二设备的拍摄画面,第二设备也可以同时显示第一设备的拍摄画面和第二设备的拍摄画面。In this way, not only the first device can simultaneously display the shooting picture of the first device and the shooting picture of the second device, but the second device can also simultaneously display the shooting picture of the first device and the shooting picture of the second device.

在第一方面的另一种可能的设计方式中,在第一设备接收用户选择与第二设备进行同步拍摄的第一操作之后,本申请的方法还可以包括:第一设备获取第二设备的拍摄能力参数,该拍摄能力参数用于指示第二设备支持的图像处理算法。In another possible design of the first aspect, after the first device receives a first operation in which the user selects to perform synchronous shooting with the second device, the method of the present application may further include: the first device obtains a shooting capability parameter of the second device, and the shooting capability parameter is used to indicate an image processing algorithm supported by the second device.

之后,第一设备可以根据第一设备的拍摄能力参数,对第一图像数据进行图像处理,得到第一拍摄画面;根据第二设备的拍摄能力参数,对第二图像数据进行图像处理,得到第二拍摄画面。第一设备的拍摄能力参数用于指示第一设备支持的图像处理算法。Afterwards, the first device may perform image processing on the first image data according to the shooting capability parameter of the first device to obtain a first shooting picture, and perform image processing on the second image data according to the shooting capability parameter of the second device to obtain a second shooting picture. The shooting capability parameter of the first device is used to indicate the image processing algorithm supported by the first device.

应理解,第一设备基于第二设备的算法能力(即第二设备的拍摄能力参数所指示的第二设备支持的图像处理算法),对第二设备采集的RAW图像进行图像处理,可得到与第二设备对该RAW图像进行图像处理相同或相似的效果。如此,可以在第一设备还原第二设备的画面效果。It should be understood that the first device performs image processing on the RAW image captured by the second device based on the algorithm capability of the second device (i.e., the image processing algorithm supported by the second device indicated by the shooting capability parameter of the second device), and can obtain the same or similar effect as the image processing performed by the second device on the RAW image. In this way, the picture effect of the second device can be restored on the first device.

在第一方面的另一种可能的设计方式中,第一设备可以根据所述第一设备的拍摄能力参数,对所述第一图像数据进行图像处理,得到所述第一拍摄画面;根据所述第一设备的拍摄能力参数,对所述第二图像数据进行图像处理,得到所述第二拍摄画面。其中,所述第一设备的拍摄能力参数用于指示所述第一设备支持的图像处理算法。In another possible design manner of the first aspect, the first device may perform image processing on the first image data according to a shooting capability parameter of the first device to obtain the first shooting picture; and perform image processing on the second image data according to the shooting capability parameter of the first device to obtain the second shooting picture. The shooting capability parameter of the first device is used to indicate an image processing algorithm supported by the first device.

应理解,第一设备基于第一设备的算法能力,对第一设备和第二设备采集的RAW图像进行图像处理,可以使第二设备的拍摄画面与第一设备的拍摄画面的图像效果一致。It should be understood that the first device performs image processing on the RAW images captured by the first device and the second device based on the algorithm capability of the first device, so that the image effect of the picture taken by the second device is consistent with that of the picture taken by the first device.

并且,在该设计方式中,第一设备不需要获取并保存第二设备的拍摄能力参数。这样,可以进一步减少跨设备的分布式拍摄业务所占用的网络带宽,还可以节省第一设备的存储空间。Furthermore, in this design, the first device does not need to obtain and save the shooting capability parameters of the second device. In this way, the network bandwidth occupied by the distributed shooting service across devices can be further reduced, and the storage space of the first device can also be saved.

在第一方面的另一种可能的设计方式中,上述第一设备指示第二设备开始采集图像数据,包括:第一设备向第二设备发送拍摄指令。In another possible design manner of the first aspect, the first device instructs the second device to start collecting image data, including: the first device sends a shooting instruction to the second device.

其中,上述拍摄指令中包括预设标记,拍摄指令用于指示第二设备采集图像数据,预设标记用于指示第二设备向第一设备传输第二设备采集的原始图像。第二设备接收到包括上述预设标记的拍摄指令后,则不会对采集的原始图像进行处理;而是将采集的原始图像透传至第一设备。The shooting instruction includes a preset mark, the shooting instruction is used to instruct the second device to collect image data, and the preset mark is used to instruct the second device to transmit the original image collected by the second device to the first device. After the second device receives the shooting instruction including the preset mark, it will not process the collected original image; instead, it will transparently transmit the collected original image to the first device.

在第一方面的另一种可能的设计方式中,在第一设备接收第一操作之前,第一设备可以接收第二操作。该第二操作是用户对用于实现分布式拍摄的功能按钮的点击操作,第一设备的相机应用的预览界面、第一设备的视频通信应用的聊天界面、第一设备的控制中心、下拉菜单或者负一屏中的至少一项包括功能按钮。响应于用于点击功能按钮的第二操作,第一设备在预览界面中显示候选设备列表。该候选设备列表中包括第二设备。其中,第一操作为用户在候选设备列表中选择第二设备的操作。In another possible design of the first aspect, before the first device receives the first operation, the first device may receive a second operation. The second operation is a user clicking operation on a function button for implementing distributed shooting, and at least one of a preview interface of a camera application of the first device, a chat interface of a video communication application of the first device, a control center of the first device, a drop-down menu, or a negative screen includes a function button. In response to the second operation for clicking the function button, the first device displays a list of candidate devices in the preview interface. The list of candidate devices includes the second device. The first operation is an operation of the user selecting the second device in the list of candidate devices.

在第一方面的另一种可能的设计方式中,上述同步拍摄包括同步录像、同步拍照、同步直播或者同步视频通话中的至少一种。也即,本申请的方法可以适用于第一设备录像、拍照、直播或者与其他设备视频通话的过程中。In another possible design of the first aspect, the synchronous shooting includes at least one of synchronous video recording, synchronous photo taking, synchronous live broadcasting or synchronous video calling. That is, the method of the present application can be applied to the process of the first device recording, taking photos, live broadcasting or video calling with other devices.

在第一方面的另一种可能的设计方式中,上述在第一设备接收用户选择与第二设备进行同步拍摄的第一操作之后,本申请的方法还包括:所述第一设备创建所述第一设备与所述第二设备的控制会话和数据会话。In another possible design of the first aspect, after the first device receives the first operation of the user selecting to perform synchronous shooting with the second device, the method of the present application also includes: the first device creates a control session and a data session between the first device and the second device.

其中,上述控制会话用于传输所述第一设备与所述第二设备之间的控制命令,所述控制命令包括用于指示所述第二设备采集图像的拍摄指令,所述数据会话用于传输所述来自所述第二设备的第二图像数据。The control session is used to transmit control commands between the first device and the second device, wherein the control commands include shooting instructions for instructing the second device to capture images, and the data session is used to transmit the second image data from the second device.

其中,第一设备与第二设备执行跨设备的分布式拍摄业务的过程中,可以区分控制流与数据流的传输通路。这样,可以避免控制流与数据流相互挤占带宽的问题。In the process of the first device and the second device executing the cross-device distributed shooting service, the transmission paths of the control flow and the data flow can be distinguished, so as to avoid the problem of the control flow and the data flow occupying each other's bandwidth.

第二方面,本申请提供一种电子设备,该电子设备为第一设备,该电子设备包括:一个或多个摄像头、一个或多个处理器、显示屏、存储器和通信模块。上述摄像头、显示屏、存储器、通信模块和处理器耦合。In a second aspect, the present application provides an electronic device, which is a first device, and includes: one or more cameras, one or more processors, a display screen, a memory, and a communication module. The cameras, display screen, memory, communication module, and processor are coupled.

其中,存储器用于存储计算机程序代码,计算机程序代码包括计算机指令,当计算机指令被电子设备执行时,使得电子设备执行如第一方面及其任一种可能的设计方式所述的方法。The memory is used to store computer program code, which includes computer instructions. When the computer instructions are executed by the electronic device, the electronic device executes the method described in the first aspect and any possible design thereof.

第三方面,本申请提供一种芯片系统,该芯片系统应用于包括显示屏、存储器和通信模块的电子设备;所述芯片系统包括一个或多个接口电路和一个或多个处理器;所述接口电路和所述处理器通过线路互联;所述接口电路用于从所述电子设备的存储器接收信号,并向所述处理器发送所述信号,所述信号包括所述存储器中存储的计算机指令;当所述处理器执行所述计算机指令时,所述电子设备执行如第一方面及其任一种可能的设计方式所述的方法。该电子设备是第一设备。In a third aspect, the present application provides a chip system, which is applied to an electronic device including a display screen, a memory and a communication module; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through a line; the interface circuit is used to receive a signal from the memory of the electronic device and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method as described in the first aspect and any possible design thereof. The electronic device is a first device.

第四方面,本申请提供一种计算机存储介质,该计算机存储介质包括计算机指令,当所述计算机指令在电子设备上运行时,使得电子设备执行如第一方面及其任一种可能的设计方式所述的方法。In a fourth aspect, the present application provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method described in the first aspect and any possible design thereof.

第五方面,本申请提供一种计算机程序产品,当所述计算机程序产品在计算机上运行时,使得所述计算机执行如第一方面及其任一种可能的设计方式所述的方法。In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the first aspect and any possible design thereof.

第六方面,本申请提供一种分布式拍摄系统,该系统包括上述第二方面以及任一种可能设计方式所述的第一设备以及上述第一方面以及任一种可能设计方式涉及的第二设备。In a sixth aspect, the present application provides a distributed shooting system, which includes the first device described in the above-mentioned second aspect and any possible design method and the second device involved in the above-mentioned first aspect and any possible design method.

可以理解地,上述提供的第二方面所述的电子设备,第三方面所述的芯片系统,第四方面所述的计算机存储介质,第五方面所述的计算机程序产品,第六方面所述的分布式拍摄系统所能达到的有益效果,可参考第一方面及其任一种可能的设计方式中的有益效果,此处不再赘述。It can be understood that the beneficial effects that can be achieved by the electronic device described in the second aspect, the chip system described in the third aspect, the computer storage medium described in the fourth aspect, the computer program product described in the fifth aspect, and the distributed shooting system described in the sixth aspect provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1为本申请实施例提供的一种分布式拍摄系统的架构示意图;FIG1 is a schematic diagram of the architecture of a distributed shooting system provided in an embodiment of the present application;

图2A为本申请实施例提供的一种分布式拍摄方法所实现的功能示意图;FIG2A is a functional schematic diagram of a distributed shooting method implemented by an embodiment of the present application;

图2B为本申请实施例提供的一种手机的硬件结构示意图;FIG2B is a schematic diagram of the hardware structure of a mobile phone provided in an embodiment of the present application;

图2C为一种本端设备101或远端设备102的软件架构示意图;FIG2C is a schematic diagram of a software architecture of the local device 101 or the remote device 102;

图2D为一种本端设备101与远端设备102的软件架构示意图;FIG2D is a schematic diagram of a software architecture of the local device 101 and the remote device 102;

图2E为本申请实施例提供的一种本端设备101与远端设备102的软件架构示意图;FIG. 2E is a schematic diagram of a software architecture of a local device 101 and a remote device 102 provided in an embodiment of the present application;

图2F为图2E所示的软件架构的简化图;FIG2F is a simplified diagram of the software architecture shown in FIG2E ;

图3A为本申请实施例提供的一种分布式拍摄界面示意图;FIG3A is a schematic diagram of a distributed shooting interface provided in an embodiment of the present application;

图3B为本申请实施例提供的另一种分布式拍摄界面示意图;FIG3B is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图4为本申请实施例提供的另一种分布式拍摄界面示意图;FIG4 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图5为本申请实施例提供的另一种分布式拍摄界面示意图;FIG5 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图6A为本申请实施例提供的另一种分布式拍摄界面示意图;FIG6A is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图6B为本申请实施例提供的另一种分布式拍摄界面示意图;FIG6B is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图7为本申请实施例提供的另一种分布式拍摄界面示意图;FIG7 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图8为本申请实施例提供的另一种分布式拍摄界面示意图;FIG8 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图9为本申请实施例提供的另一种分布式拍摄界面示意图;FIG9 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图10为本申请实施例提供的另一种分布式拍摄界面示意图;FIG10 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图11为本申请实施例提供的另一种分布式拍摄界面示意图;FIG11 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图12为本申请实施例提供的一种拍摄能力参数的数据结构示意图;FIG12 is a schematic diagram of a data structure of a shooting capability parameter provided in an embodiment of the present application;

图13为本申请实施例提供的一种本端设备与远端设备的数据传输流程示意图;FIG13 is a schematic diagram of a data transmission process between a local device and a remote device provided in an embodiment of the present application;

图14为本申请实施例提供的一种远端设备中的数据处理及传输流程示意图;FIG14 is a schematic diagram of a data processing and transmission process in a remote device provided in an embodiment of the present application;

图15为本申请实施例提供的另一种分布式拍摄界面示意图;FIG15 is a schematic diagram of another distributed shooting interface provided in an embodiment of the present application;

图16为本申请实施例提供的一种本端设备与远端设备的数据传输流程示意图;FIG16 is a schematic diagram of a data transmission process between a local device and a remote device provided in an embodiment of the present application;

图17为本申请实施例提供的一种芯片系统的结构示意图。FIG17 is a schematic diagram of the structure of a chip system provided in an embodiment of the present application.

具体实施方式Detailed ways

以下,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本实施例的描述中,除非另有说明,“多个”的含义是两个或两个以上。In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

本申请实施例提供的一种分布式拍摄方法,可应用于图1所示的分布式拍摄系统100中。如图1所示,该分布式拍摄系统100中可以包括本端设备101和N个远端设备102,N为大于0的整数。本端设备101与任意一个远端设备102之间可通过有线的方式通信,也可以通过无线的方式通信。其中,本端设备101是第一设备,远端设备102是第二设备。A distributed shooting method provided in an embodiment of the present application can be applied to a distributed shooting system 100 shown in FIG1 . As shown in FIG1 , the distributed shooting system 100 may include a local device 101 and N remote devices 102 , where N is an integer greater than 0. The local device 101 can communicate with any remote device 102 via a wired manner or a wireless manner. The local device 101 is a first device, and the remote device 102 is a second device.

示例性的,本端设备101与远端设备102之间可以使用通用串行总线(universalserial bus,USB)建立有线连接。又例如,本端设备101与远端设备102之间可以通过全球移动通讯系统(global system for mobile communications,GSM)、通用分组无线服务(general packet radio service,GPRS)、码分多址接入(code division multipleaccess,CDMA)、宽带码分多址(wideband code division multiple access,WCDMA),时分码分多址(time-division code division multiple access,TD-SCDMA),长期演进(longterm evolution,LTE)、5G以及后续标准协议、蓝牙、无线保真(wireless fidelity,Wi-Fi)、NFC、基于互联网协议的语音通话(voice over Internet protocol,VoIP)、支持网络切片架构的通信协议建立无线连接。Exemplarily, a universal serial bus (USB) can be used to establish a wired connection between the local device 101 and the remote device 102. For another example, a wireless connection can be established between the local device 101 and the remote device 102 through a global system for mobile communications (GSM), a general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long-term evolution (LTE), 5G and subsequent standard protocols, Bluetooth, wireless fidelity (Wi-Fi), NFC, voice over Internet protocol (VoIP), and a communication protocol supporting a network slicing architecture.

在一些实施例中,本端设备101可以采用上述无线连接,直接与远端设备102通信。在另一些实施例中,本端设备101可以采用上述无线连接,通过云服务器与远端设备102通信,本申请实施例对此不作限制。In some embodiments, the local device 101 can use the above wireless connection to communicate directly with the remote device 102. In other embodiments, the local device 101 can use the above wireless connection to communicate with the remote device 102 through a cloud server, which is not limited in this embodiment of the present application.

其中,本端设备101和远端设备102中均可设置一个或多个摄像头。本端设备101可使用自身的摄像头采集图像数据,远端设备102也可以使用自身的摄像头采集图像数据。在本申请实施例中,本端设备101和远端设备102可以同时使用自身的摄像头采集图像数据,并且,远端设备102可将图像数据发送给本端设备101,由本端设备101同时显示来自本端设备101和远端设备102的图像数据,从而实现跨设备的分布式拍摄功能。Among them, one or more cameras may be set in both the local device 101 and the remote device 102. The local device 101 can use its own camera to collect image data, and the remote device 102 can also use its own camera to collect image data. In the embodiment of the present application, the local device 101 and the remote device 102 can use their own cameras to collect image data at the same time, and the remote device 102 can send the image data to the local device 101, and the local device 101 simultaneously displays the image data from the local device 101 and the remote device 102, thereby realizing a distributed shooting function across devices.

如图2A所示,本端设备101可以包括预设应用(application,APP)。本端设备101可以通过该应用,实现上述跨设备的分布式拍摄功能。例如,该预设应用可以是系统相机应用或第三方的相机应用。本申请实施例中,系统相机应用或第三方的相机应用可以统称为相机应用。其中,系统应用也可以称为嵌入式应用,嵌入式应用是作为电子设备(如本端设备101或远端设备102)实现的一部分提供的应用程序。第三方应用也可以称为可下载应用。可下载应用是一个可以提供自己的因特网协议多媒体子系统(internet protocolmultimedia subsystem,IMS)连接的应用程序。该可下载应用可以是预先安装在电子设备中的应用或可以是由用户下载并安装在电子设备的应用。As shown in Figure 2A, the local device 101 may include a preset application (application, APP). The local device 101 can implement the above-mentioned distributed shooting function across devices through the application. For example, the preset application may be a system camera application or a third-party camera application. In an embodiment of the present application, the system camera application or the third-party camera application may be collectively referred to as a camera application. Among them, the system application may also be referred to as an embedded application, and the embedded application is an application provided as part of the implementation of an electronic device (such as the local device 101 or the remote device 102). A third-party application may also be referred to as a downloadable application. A downloadable application is an application that can provide its own Internet protocol multimedia subsystem (IMS) connection. The downloadable application may be an application pre-installed in an electronic device or may be an application downloaded and installed by a user in an electronic device.

具体的,如图2A所示,本端设备101可以连接远端设备102,还可以通过本端设备101与远端设备120之间的连接,控制远端设备102打开远端设备102的摄像头。如图2A所示,本端设备101上可以实现对远端设备102的图像预览,控制预览效果。如图2A所示,本端设备101可以控制远端设备102拍照,并控制远端设备102的拍照效果。如图2A所示,本端设备101可以控制远端设备102录像,并控制远端设备102的录像效果。其中,实现上上述跨设备的分布式拍摄功能,如图2A所示,本端设备101可以显示本端设备101和远端设备102的拍摄画面。如此,用户在本端设备101上不仅可以看到本端设备101的拍摄画面,还可以看到远端设备102的拍摄画面。当然,本端设备101还可以通过本端设备101与远端设备120之间的连接,控制远端设备102关闭远端设备102的摄像头。Specifically, as shown in FIG2A , the local device 101 can be connected to the remote device 102, and can also control the remote device 102 to turn on the camera of the remote device 102 through the connection between the local device 101 and the remote device 120. As shown in FIG2A , the local device 101 can realize image preview of the remote device 102 and control the preview effect. As shown in FIG2A , the local device 101 can control the remote device 102 to take pictures and control the picture taking effect of the remote device 102. As shown in FIG2A , the local device 101 can control the remote device 102 to record and control the video recording effect of the remote device 102. Among them, the above-mentioned distributed shooting function across devices is realized, as shown in FIG2A , the local device 101 can display the shooting pictures of the local device 101 and the remote device 102. In this way, the user can not only see the shooting pictures of the local device 101 on the local device 101, but also see the shooting pictures of the remote device 102. Of course, the local device 101 can also control the remote device 102 to turn off the camera of the remote device 102 through the connection between the local device 101 and the remote device 120.

示例性的,本端设备101(或远端设备102)具体可以为手机、平板电脑、电视(也可称为智能电视、智慧屏或大屏设备)、笔记本电脑、超级移动个人计算机(Ultra-mobilePersonal Computer,UMPC)、手持计算机、照相机(如数码照相机)、摄像机(如数码摄像机)、上网本、个人数字助理(Personal Digital Assistant,PDA)、可穿戴电子设备(例如:智能手表,智能手环,智能眼镜)、车载设备、虚拟现实设备等具有拍摄功能的电子设备,本申请实施例对此不做任何限制。Exemplarily, the local device 101 (or the remote device 102) can be a mobile phone, a tablet computer, a television (also called a smart TV, a smart screen or a large-screen device), a laptop computer, an ultra-mobile personal computer (UMPC), a handheld computer, a camera (such as a digital camera), a camcorder (such as a digital video camera), a netbook, a personal digital assistant (PDA), a wearable electronic device (for example: a smart watch, a smart bracelet, a smart glasses), a vehicle-mounted device, a virtual reality device, and other electronic devices with a shooting function. The embodiments of the present application do not impose any restrictions on this.

示例性的,以手机作为上述分布式拍摄系统100中的本端设备101举例,图2B示出了手机的结构示意图。如图2B所示,手机可以包括处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,天线1,天线2,移动通信模块150,无线通信模块160,音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,传感器模块180等。Exemplarily, taking a mobile phone as the local device 101 in the above-mentioned distributed shooting system 100, FIG2B shows a schematic diagram of the structure of the mobile phone. As shown in FIG2B, the mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, etc.

可以理解的是,本发明实施例示意的结构并不构成对手机的具体限定。在本申请另一些实施例中,手机可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone. In other embodiments of the present application, the mobile phone may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

处理器110可以包括一个或多个处理单元,例如:处理器110可以包括应用处理器(application processor,AP),调制解调处理器,图形处理器(graphics processingunit,GPU),图像信号处理器(image signal processor,ISP),控制器,存储器,视频编解码器,数字信号处理器(digital signal processor,DSP),基带处理器,和/或神经网络处理器(neural-network processing unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从所述存储器中直接调用。避免了重复存取,减少了处理器110的等待时间,因而提高了系统的效率。The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

手机的无线通信功能可以通过天线1,天线2,移动通信模块150,无线通信模块160,调制解调处理器以及基带处理器等实现。天线1和天线2用于发射和接收电磁波信号。The wireless communication function of the mobile phone can be realized by antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

移动通信模块150可以提供应用在手机上的包括2G/3G/4G/5G等无线通信的解决方案。在一些实施例中,移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中,移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。The mobile communication module 150 can provide solutions for wireless communications including 2G/3G/4G/5G, etc., applied on mobile phones. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the same device as at least some modules of the processor 110.

无线通信模块160可以提供应用在手机上的包括无线局域网(wireless localarea networks,WLAN)(如无线保真(wireless fidelity,Wi-Fi)网络),蓝牙(Bluetooth,BT),全球导航卫星系统(global navigation satellite system,GNSS),调频(frequencymodulation,FM),近距离无线通信技术(near field communication,NFC),红外技术(infrared,IR)等无线通信的解决方案。The wireless communication module 160 can provide wireless communication solutions for use on mobile phones, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

在一些实施例中,手机的天线1和移动通信模块150耦合,天线2和无线通信模块160耦合,使得手机可以通过无线通信技术与网络以及其他设备通信。In some embodiments, the antenna 1 of the mobile phone is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone can communicate with the network and other devices through wireless communication technology.

手机通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。The mobile phone implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

显示屏194用于显示图像,视频等。显示屏194包括显示面板。在一些实施例中,手机可以包括1个或N个显示屏194,N为大于1的正整数。The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the mobile phone may include 1 or N display screens 194, where N is a positive integer greater than 1.

手机可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。感光元件把光信号转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,手机可以包括1个或N个摄像头193,N为大于1的正整数。The mobile phone can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor. Camera 193 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the mobile phone may include 1 or N cameras 193, where N is a positive integer greater than 1.

外部存储器接口120可以用于连接外部存储卡,例如Micro SD卡,实现扩展手机的存储能力。外部存储卡通过外部存储器接口120与处理器110通信,实现数据存储功能。例如将音乐,视频等文件保存在外部存储卡中。The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

内部存储器121可以用于存储计算机可执行程序代码,所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令,从而执行手机的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中,存储程序区可存储操作系统,至少一个功能所需的应用程序(比如声音播放功能,图像播放功能等)等。存储数据区可存储手机使用过程中所创建的数据(比如音频数据,电话本等)等。此外,内部存储器121可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件,闪存器件,通用闪存存储器(universal flash storage,UFS)等。The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the mobile phone by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

手机可以通过音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,以及应用处理器等实现音频功能。例如音乐播放,录音等。The mobile phone can realize audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

传感器模块180中可以包括压力传感器,陀螺仪传感器,气压传感器,磁传感器,加速度传感器,距离传感器,接近光传感器,指纹传感器,温度传感器,触摸传感器,环境光传感器,骨传导传感器等。The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

当然,手机还可以包括充电管理模块、电源管理模块、电池、按键、指示器以及1个或多个SIM卡接口等,本申请实施例对此不做任何限制。Of course, the mobile phone may also include a charging management module, a power management module, a battery, buttons, indicators, and one or more SIM card interfaces, etc., and the embodiments of the present application do not impose any restrictions on this.

图2C是本申请实施例的手机的软件结构框图。FIG. 2C is a software structure block diagram of the mobile phone according to an embodiment of the present application.

分层架构将软件分成若干个层,每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中,将Android系统分为五层,从上至下分别为应用程序层,应用程序框架层,安卓运行时(Android runtime)和系统库,HAL(hardware abstractionlayer,硬件抽象层)层以及内核层。应理解:本文以Android系统举例来说明,在其他操作系统中(例如鸿蒙系统,IOS系统等),只要各个功能模块实现的功能和本申请的实施例类似也能实现本申请的方案。The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, the HAL (hardware abstraction layer, hardware abstraction layer) layer, and the kernel layer. It should be understood that: This article uses the Android system as an example to illustrate that in other operating systems (such as Hongmeng system, IOS system, etc.), as long as the functions implemented by each functional module are similar to the embodiments of the present application, the solution of the present application can also be implemented.

应用程序层可以包括一系列应用程序包。The application layer can include a series of application packages.

如图2C所示,应用程序层中可以安装通话,备忘录,浏览器,联系人,图库,日历,地图,蓝牙,音乐,视频,短信息等应用。As shown in FIG. 2C , the application layer may be installed with applications such as calls, memos, browsers, contacts, gallery, calendar, maps, Bluetooth, music, videos, and short messages.

在本申请实施例中,应用程序层中可以安装具有拍摄功能的应用,例如,相机应用。当然,其他应用需要使用拍摄功能时,也可以调用相机应用实现拍摄功能。In the embodiment of the present application, an application with a shooting function, such as a camera application, can be installed in the application layer. Of course, when other applications need to use the shooting function, the camera application can also be called to implement the shooting function.

应用程序框架层为应用程序层的应用程序提供应用编程接口(applicationprogramming interface,API)和编程框架。应用程序框架层包括一些预先定义的函数。The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

例如,应用程序框架层可以包括窗口管理器,内容提供器,视图系统,资源管理器,通知管理器等,本申请实施例对此不做任何限制。For example, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, etc., and the embodiments of the present application do not impose any restrictions on this.

例如,上述窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小,判断是否有状态栏,锁定屏幕,截取屏幕等。上述内容提供器用来存放和获取数据,并使这些数据可以被应用程序访问。所述数据可以包括视频,图像,音频,拨打和接听的电话,浏览历史和书签,电话簿等。上述视图系统可用于构建应用程序的显示界面。每个显示界面可以由一个或多个控件组成。一般而言,控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、微件(Widget)等界面元素。上述资源管理器为应用程序提供各种资源,比如本地化字符串,图标,图片,布局文件,视频文件等等。上述通知管理器使应用程序可以在状态栏中显示通知信息,可以用于传达告知类型的消息,可以短暂停留后自动消失,无需用户交互。比如通知管理器被用于告知下载完成,消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知,例如后台运行的应用程序的通知,还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息,发出提示音,振动,指示灯闪烁等。For example, the above-mentioned window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The above-mentioned content provider is used to store and obtain data, and make these data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The above-mentioned view system can be used to build the display interface of the application. Each display interface can be composed of one or more controls. Generally speaking, controls can include interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. The above-mentioned resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc. The above-mentioned notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, displaying text messages in the status bar, making beep sounds, vibrating, or flashing indicator lights.

如图2C所示,Android runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。As shown in Figure 2C, the Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

核心库包含两部分:一部分是java语言需要调用的功能函数,另一部分是安卓的核心库。The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理,堆栈管理,线程管理,安全和异常的管理,以及垃圾回收等功能。The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

系统库可以包括多个功能模块。例如:表面管理器(surface manager),媒体库(Media Libraries),三维图形处理库(例如:OpenGL ES),2D图形引擎(例如:SGL)等。The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

其中,表面管理器用于对显示子系统进行管理,并且为多个应用程序提供了2D和3D图层的融合。媒体库支持多种常用的音频,视频格式回放和录制,以及静态图像文件等。媒体库可以支持多种音视频编码格式,例如:MPEG4,H.264,MP3,AAC,AMR,JPG,PNG等。三维图形处理库用于实现三维图形绘图,图像渲染,合成,和图层处理等。2D图形引擎是2D绘图的绘图引擎。Among them, the surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

内核层位于HAL之下,是硬件和软件之间的层。内核层至少包含显示驱动,摄像头驱动,音频驱动,传感器驱动等,本申请实施例对此不做任何限制。The kernel layer is located below the HAL and is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, a sensor driver, etc., and the present application embodiment does not impose any restrictions on this.

在本申请实施例中,仍如图2C所示,以相机应用举例,可应用程序框架层中设置有相机服务(Camera Service)。相机应用可通过调用预设的API启动Camera Service。CameraService在运行过程中可以与HAL(hardware abstraction layer,硬件抽象层)中的CameraHAL交互。其中,Camera HAL负责与手机中实现拍摄功能的硬件设备(例如摄像头)进行交互,Camera HAL一方面隐藏了相关硬件设备的实现细节(例如具体的图像处理算法),另一方面可向Android系统提供调用相关硬件设备的接口。In an embodiment of the present application, still as shown in FIG. 2C , taking the camera application as an example, a camera service (Camera Service) can be set in the application framework layer. The camera application can start the Camera Service by calling a preset API. During operation, the CameraService can interact with the CameraHAL in the HAL (hardware abstraction layer). Among them, the Camera HAL is responsible for interacting with the hardware device (such as a camera) that implements the shooting function in the mobile phone. On the one hand, the Camera HAL hides the implementation details of the relevant hardware devices (such as specific image processing algorithms), and on the other hand, it can provide the Android system with an interface for calling the relevant hardware devices.

示例性的,相机应用运行时可将用户下发的相关控制命令(例如预览、放大、拍照或录像指令)发送至Camera Service。一方面,Camera Service可将接收到的控制命令发送至Camera HAL,使得Camera HAL可根据接收到的控制命令调用内核层中的相机驱动,由相机驱动来驱动摄像头等硬件设备响应该控制命令采集图像数据。例如,摄像头可按照一定的帧率,将采集到的每一帧图像数据通过相机驱动传递给Camera HAL。其中,控制命令在操作系统内部的传递过程可参见图2C中控制流的具体传递过程。For example, when the camera application is running, the relevant control commands (such as preview, zoom, photo or video recording instructions) issued by the user can be sent to the Camera Service. On the one hand, the Camera Service can send the received control command to the Camera HAL, so that the Camera HAL can call the camera driver in the kernel layer according to the received control command, and the camera driver drives the camera and other hardware devices to respond to the control command to collect image data. For example, the camera can pass each frame of image data collected to the Camera HAL through the camera driver at a certain frame rate. Among them, the transmission process of the control command within the operating system can refer to the specific transmission process of the control flow in Figure 2C.

另一方面,Camera Service接收到上述控制命令后,可根据接收到的控制命令确定此时的拍摄策略,拍摄策略中设置了需要对采集到的图像数据执行的具体图像处理任务。例如,在预览模式下,Camera Service可在拍摄策略中设置图像处理任务1用于实现人脸检测功能。又例如,如果在预览模式下用户开启了美颜功能,则Camera Service还可以在拍摄策略中设置图像处理任务2用于实现美颜功能。进而,Camera Service可将确定出的拍摄策略发送至Camera HAL。On the other hand, after receiving the above control command, the Camera Service can determine the current shooting strategy based on the received control command. The shooting strategy sets the specific image processing tasks that need to be performed on the collected image data. For example, in preview mode, the Camera Service can set image processing task 1 in the shooting strategy to implement the face detection function. For another example, if the user turns on the beauty function in preview mode, the Camera Service can also set image processing task 2 in the shooting strategy to implement the beauty function. Then, the Camera Service can send the determined shooting strategy to the Camera HAL.

当Camera HAL接收到摄像头采集到的每一帧图像数据后,可根据Camera Service下发的拍摄策略对上述图像数据执行相应的图像处理任务,得到图像处理后的每一帧拍摄画面。例如,Camera HAL可根据拍摄策略1对接收到的每一帧图像数据执行图像处理任务1,得到对应的每一帧拍摄画面。当拍摄策略1更新为拍摄策略2后,Camera HAL可根据拍摄策略2对接收到的每一帧图像数据执行图像处理任务2和图像处理任务3,得到对应的每一帧拍摄画面。When Camera HAL receives each frame of image data collected by the camera, it can perform corresponding image processing tasks on the above image data according to the shooting strategy issued by Camera Service to obtain each frame of captured images after image processing. For example, Camera HAL can perform image processing task 1 on each frame of image data received according to shooting strategy 1 to obtain each corresponding frame of captured images. When shooting strategy 1 is updated to shooting strategy 2, Camera HAL can perform image processing tasks 2 and 3 on each frame of image data received according to shooting strategy 2 to obtain each corresponding frame of captured images.

后续,Camera HAL可将经过图像处理后的每一帧拍摄画面通过Camera Service上报给相机应用,相机应用可将每一帧拍摄画面显示在显示界面中,或者,相机应用以照片或视频的形式将每一帧拍摄画面保存在手机内。其中,上述拍摄画面在操作系统内部的传递过程可参见图2C中数据流的具体传递过程。Subsequently, the Camera HAL can report each frame of the captured image after image processing to the camera application through the Camera Service, and the camera application can display each frame of the captured image in the display interface, or the camera application can save each frame of the captured image in the form of a photo or video in the mobile phone. The transmission process of the above-mentioned captured image within the operating system can be referred to the specific transmission process of the data flow in Figure 2C.

请参考图2D,其示出本地设备101和远端设备102的一种软件架构示意图。本申请实施例这里结合图2D所示的软件架构,说明本地设备101和远端设备102实现跨设备分布式拍摄的方案。Please refer to Figure 2D, which shows a schematic diagram of a software architecture of the local device 101 and the remote device 102. In this embodiment of the application, in conjunction with the software architecture shown in Figure 2D, a solution for the local device 101 and the remote device 102 to implement cross-device distributed shooting is described.

其中,本端设备101具备设备发现和远程注册功能。例如,本端设备101可以发现远端设备102,如图4所示的电视1、手表2和手机3。当用户选择电视1作为远端设备102与本端设备101进行跨设备分布式拍摄(如用户选择图4所示的电视1)后,本端设备101可以将该远端设备102(如电视1)注册到本端设备101上。Among them, the local device 101 has device discovery and remote registration functions. For example, the local device 101 can discover the remote device 102, such as the TV 1, watch 2 and mobile phone 3 shown in Figure 4. When the user selects TV 1 as the remote device 102 to perform cross-device distributed shooting with the local device 101 (such as the user selects TV 1 as shown in Figure 4), the local device 101 can register the remote device 102 (such as TV 1) to the local device 101.

例如,本端设备101可以在本端设备101的硬件抽象层(hardware abstractlayer,HAL),针对远端设备102创建一个分布式移动传感开发平台(Distributed MobileSensing Development Platform,DMSDP)HAL,也可以称为虚拟相机HAL(即虚拟CameraHAL)。其中,与本端设备101中传统的Camera HAL不同的是,DMSDP HAL并不与本端设备101实际的硬件设备相对应,而是与本端设备101当前连接的远端设备102对应。图2D所示的DMSDP HAL是本端设备101按照远端设备102的拍摄能力参数创建的HAL。本端设备101可以通过DMSDP HAL与远端设备102进行数据收发,将远端设备102作为本端设备101的一个虚拟设备,与远端设备102协同完成跨设备的分布式拍摄业务。For example, the local device 101 can create a distributed mobile sensing development platform (Distributed Mobile Sensing Development Platform, DMSDP) HAL, which can also be called a virtual camera HAL (i.e., virtual Camera HAL), for the remote device 102 in the hardware abstract layer (HAL) of the local device 101. Among them, unlike the traditional Camera HAL in the local device 101, the DMSDP HAL does not correspond to the actual hardware device of the local device 101, but corresponds to the remote device 102 currently connected to the local device 101. The DMSDP HAL shown in Figure 2D is a HAL created by the local device 101 according to the shooting capability parameters of the remote device 102. The local device 101 can send and receive data with the remote device 102 through the DMSDP HAL, regard the remote device 102 as a virtual device of the local device 101, and collaborate with the remote device 102 to complete the distributed shooting service across devices.

以下介绍本端设备101与远端设备102协同完成跨设备的分布式拍摄业务的具体流程。如图2D所示,该流程可以包括步骤①-步骤⑩。The following describes a specific process of the local device 101 and the remote device 102 cooperating to complete the cross-device distributed shooting service. As shown in FIG2D , the process may include steps ① to ⑩.

步骤①:本端设备101的应用层向服务层的相机服务下发用于控制远端设备102的摄像头的控制命令。步骤②:本端设备101的服务层向HAL的DMSDP HAL发送该控制命令。步骤③:本端设备101的DMSDP HAL向远端设备102传输控制命令。步骤④:远端设备102接收到控制命令后,向服务层传输控制命令。步骤⑤:远端设备102的服务层向HAL的Camera HAL传输控制命令。其中,远端设备102的Camera HAL与远端设备102实际的硬件设备相对应。如此,远端设备102的HAL便可以调用其底层(如内核层Kernel)根据该控制命令,执行对应的拍摄任务(如启动摄像头、切换摄像头或者调整摄像头的相关参数等)。其中,上述步骤①-步骤⑤中所传输的数据可以称为控制流。Step ①: The application layer of the local device 101 sends a control command for controlling the camera of the remote device 102 to the camera service of the service layer. Step ②: The service layer of the local device 101 sends the control command to the DMSDP HAL of the HAL. Step ③: The DMSDP HAL of the local device 101 transmits the control command to the remote device 102. Step ④: After receiving the control command, the remote device 102 transmits the control command to the service layer. Step ⑤: The service layer of the remote device 102 transmits the control command to the Camera HAL of the HAL. Among them, the Camera HAL of the remote device 102 corresponds to the actual hardware device of the remote device 102. In this way, the HAL of the remote device 102 can call its bottom layer (such as the kernel layer Kernel) according to the control command to perform the corresponding shooting task (such as starting the camera, switching the camera, or adjusting the relevant parameters of the camera, etc.). Among them, the data transmitted in the above steps ① to ⑤ can be called a control flow.

步骤⑥:远端设备101的HAL向服务层的相机服务上传预览流。该预览流包括远端设备102底层的Camera器件采集的,并经过底层的图像信号处理器(image signalprocessor,ISP)处理过的一帧或多帧预览图像。步骤⑦:远端设备101的服务层向远端设备102的应用层传输预览流。步骤⑧:远端设备101的应用层向本端设备101的DMSDP HAL传输预览流。步骤⑨:本端设备101的DMSDP HAL向服务层的相机服务传输预览流。步骤⑩:本端设备101的服务层向应用层上报该预览流。如此,本端设备101的相机应用便可以向用户呈现来自远端设备102的预览流(如上述拍摄画面2、拍摄画面3或拍摄画面c)。当然,本端设备101的相机应用还可以向用户呈现来自本端设备101的预览流,具体流程可参考常规技术的相关介绍,这里不予赘述。其中,上述步骤⑥-步骤⑩中所传输的数据可以称为预览流或者数据流。Step ⑥: The HAL of the remote device 101 uploads the preview stream to the camera service of the service layer. The preview stream includes one or more preview images collected by the bottom-layer Camera device of the remote device 102 and processed by the bottom-layer image signal processor (ISP). Step ⑦: The service layer of the remote device 101 transmits the preview stream to the application layer of the remote device 102. Step ⑧: The application layer of the remote device 101 transmits the preview stream to the DMSDP HAL of the local device 101. Step ⑨: The DMSDP HAL of the local device 101 transmits the preview stream to the camera service of the service layer. Step ⑩: The service layer of the local device 101 reports the preview stream to the application layer. In this way, the camera application of the local device 101 can present the preview stream from the remote device 102 to the user (such as the above-mentioned shooting picture 2, shooting picture 3 or shooting picture c). Of course, the camera application of the local device 101 can also present the preview stream from the local device 101 to the user. The specific process can refer to the relevant introduction of conventional technology, which will not be repeated here. The data transmitted in the above steps ⑥ to ⑩ may be referred to as a preview stream or a data stream.

需要说明的是,如图2D所示,本端设备101或远端设备102的服务层可以包含于框架层,服务层可在框架层中实现。当然,本端设备101或远端设备102的服务层的服务层也可以独立于框架层,本申请实施例对此不作限制。It should be noted that, as shown in Figure 2D, the service layer of the local device 101 or the remote device 102 can be included in the framework layer, and the service layer can be implemented in the framework layer. Of course, the service layer of the service layer of the local device 101 or the remote device 102 can also be independent of the framework layer, and this embodiment of the application does not limit this.

一方面,虽然执行上述流程可完成跨设备的分布式拍摄业务;但是,在上述跨设备的预览流传输过程中,远端设备102向本端设备101传输的预览流是经过图像处理得到的成片(即可直接显示的预览结果),其数据量加大,需要占用较大的网络带宽,浪费了网络资源。On the one hand, although the execution of the above process can complete the distributed shooting service across devices; however, in the above-mentioned preview stream transmission process across devices, the preview stream transmitted by the remote device 102 to the local device 101 is a finished film obtained after image processing (that is, a preview result that can be directly displayed), and its data volume is increased, which requires a larger network bandwidth and wastes network resources.

另一方面,不同设备的进行图像处理的性能不同。例如,相比于本端设备101,远端设备102进行图层处理的性能可能较低。如此,远端设备102的出图效率则较低。实现本端设备101上同时显示来自本端设备101和远端设备102的拍摄画面,则需要较大的时延。并且,上述流程中并不区分控制流与数据流(或预览流)的传输通路,控制流与数据流可能会相互挤占带宽。On the other hand, different devices have different image processing performance. For example, compared with the local device 101, the performance of the remote device 102 in layer processing may be lower. As a result, the efficiency of the remote device 102 in outputting images is lower. To achieve the simultaneous display of the captured images from the local device 101 and the remote device 102 on the local device 101, a larger delay is required. In addition, the above process does not distinguish between the transmission paths of the control stream and the data stream (or preview stream), and the control stream and the data stream may occupy each other's bandwidth.

本申请实施例提供一种分布式拍摄方法,在本端设备101与远端设备102执行跨设备的分布式拍摄业务的过程中,本端设备101可以从远端设备102直接获取到该远端设备102采集的原始图像。该原始图像也可以称为RAW图像。RAW可翻译为“未经加工”。也就是说,上述远端设备102采集的原始图像是未经过远端设备102处理的图像。以下实施例中,采用RAW图像代表原始图像,介绍本申请实施例的方法。The embodiment of the present application provides a distributed shooting method. During the process of the local device 101 and the remote device 102 performing a distributed shooting service across devices, the local device 101 can directly obtain the original image captured by the remote device 102 from the remote device 102. The original image can also be called a RAW image. RAW can be translated as "unprocessed". In other words, the original image captured by the above-mentioned remote device 102 is an image that has not been processed by the remote device 102. In the following embodiments, a RAW image is used to represent the original image to introduce the method of the embodiment of the present application.

其中,相比于经过远端设备102处理的图像,RAW图像的数据量较小。如此,便可以减少跨设备的分布式拍摄业务所占用的网络带宽。Among them, the data volume of the RAW image is smaller than that of the image processed by the remote device 102. In this way, the network bandwidth occupied by the distributed shooting service across devices can be reduced.

并且,由本端设备101对来自远端设备102的RAW图像进行处理,可以避免由于两端设备性能差异较大而增大时延的问题。采用该方案,可以减少分布式拍摄业务的时延。进一步的,本端设备101与远端设备102执行跨设备的分布式拍摄业务的过程中,可以区分控制流与数据流的传输通路。这样,便可以避免控制流与数据流相互挤占带宽的问题。Furthermore, the local device 101 processes the RAW image from the remote device 102, which can avoid the problem of increased latency due to the large performance difference between the devices at both ends. This solution can reduce the latency of the distributed shooting service. Furthermore, during the process of the local device 101 and the remote device 102 performing the cross-device distributed shooting service, the transmission paths of the control stream and the data stream can be distinguished. In this way, the problem of the control stream and the data stream occupying each other's bandwidth can be avoided.

请参考图2E,其示出本申请实施例提供的一种本端设备101和远端设备102的软件架构示意图。如图2E所示,本端设备101可以包括:应用层200、框架层201、服务层201a和HAL202;远端设备102可以包括:应用层210、框架层211、服务层211a和HAL 212。其中,服务层可在框架层中实现。例如,如图2E所示,服务层201a可在框架层201中实现,服务层211a可在框架层211中实现。Please refer to FIG. 2E , which shows a schematic diagram of the software architecture of a local device 101 and a remote device 102 provided in an embodiment of the present application. As shown in FIG. 2E , the local device 101 may include: an application layer 200, a framework layer 201, a service layer 201a, and a HAL 202; the remote device 102 may include: an application layer 210, a framework layer 211, a service layer 211a, and a HAL 212. Among them, the service layer may be implemented in the framework layer. For example, as shown in FIG. 2E , the service layer 201a may be implemented in the framework layer 201, and the service layer 211a may be implemented in the framework layer 211.

其中,应用层可以包括一系列应用程序包。例如,应用层200可以包括多个应用,如系相机应用和Dv应用200a。当然,其他应用需要使用拍摄功能时,也可以调用相机应用实现拍摄功能。又例如,应用层210包括用于支持远端设备102与本端设备101协同完成分布式拍摄业务的相机代理服务210a。该相机代理服务210a也可以称为相机代理应用,本申请实施例对此不作限制。Among them, the application layer may include a series of application packages. For example, the application layer 200 may include multiple applications, such as a camera application and a DV application 200a. Of course, when other applications need to use the shooting function, they can also call the camera application to implement the shooting function. For another example, the application layer 210 includes a camera proxy service 210a for supporting the remote device 102 and the local device 101 to collaborate to complete the distributed shooting service. The camera proxy service 210a can also be called a camera proxy application, which is not limited in the embodiments of the present application.

并且,本申请中,还可以在应用层200中安装用于实现分布式拍摄功能的设备虚拟化(device virtualization,Dv)应用200a。Dv应用200a可作为系统应用常驻在本端设备101中运行。或者,也可将Dv应用200a实现的功能以系统服务的形式常驻在本端设备101中运行。Furthermore, in the present application, a device virtualization (Dv) application 200a for implementing a distributed shooting function may also be installed in the application layer 200. The Dv application 200a may be resident and run in the local device 101 as a system application. Alternatively, the functions implemented by the Dv application 200a may be resident and run in the local device 101 as a system service.

框架层为应用层的应用程序提供API和编程框架。例如,框架层201和框架层211均可以提供Camera API 2.0。框架层201和框架层211的其他功能可以参考本申请实施例对图2C所述的应用程序框架层的详细介绍,这里不予赘述。The framework layer provides an API and programming framework for the application of the application layer. For example, both the framework layer 201 and the framework layer 211 can provide Camera API 2.0. For other functions of the framework layer 201 and the framework layer 211, please refer to the detailed description of the application framework layer described in FIG. 2C in the embodiment of the present application, which will not be repeated here.

如图2E所示,以相机应用举例,框架层201中设置有Camera kit 201c。其中,Camera kit 201c中可以封装多种相机模式。例如,图2E所示的拍照模式、录像模式和双景模式。As shown in FIG2E , taking a camera application as an example, a Camera kit 201c is provided in the framework layer 201. The Camera kit 201c may encapsulate a variety of camera modes, such as the photo mode, video mode, and dual-view mode shown in FIG2E .

如图2E所示,可在服务层201a中设置有相机服务(Camera Service)201a1。相机应用可通过调用预设的API(如Camera API 2.0)启动Camera Service 201a1。HAL 202中设置有Camera HAL 202a和DMSDP HAL 202b。Camera Service 201a1在运行过程中可以与HAL202中的Camera HAL 202a和/或DMSDP HAL 202b交互。As shown in FIG2E , a camera service (Camera Service) 201a1 may be provided in the service layer 201a. The camera application may start the Camera Service 201a1 by calling a preset API (e.g., Camera API 2.0). A Camera HAL 202a and a DMSDP HAL 202b are provided in the HAL 202. The Camera Service 201a1 may interact with the Camera HAL 202a and/or the DMSDP HAL 202b in the HAL 202 during operation.

其中,Camera HAL 202a负责与本端设备101中实现拍摄功能的硬件设备(例如摄像头)进行交互,Camera HAL 202a一方面隐藏了相关硬件设备的实现细节(例如具体的图像处理算法),另一方面可向Android系统提供调用相关硬件设备的接口。Among them, Camera HAL 202a is responsible for interacting with the hardware device (such as a camera) that implements the shooting function in the local device 101. On the one hand, Camera HAL 202a hides the implementation details of the relevant hardware devices (such as specific image processing algorithms), and on the other hand, it can provide the Android system with an interface for calling the relevant hardware devices.

需要说明的是,应用层200中相机应用运行时,应用层200与框架层201、服务层201a、Camera HAL 202a和DMSDP HAL 202b等的交互流程,可以参考上述实施例对图2C中对应软件模块的介绍,本申请实施例这里不予赘述。It should be noted that when the camera application in the application layer 200 is running, the interaction process between the application layer 200 and the framework layer 201, the service layer 201a, the Camera HAL 202a and the DMSDP HAL 202b, etc., can refer to the introduction of the corresponding software modules in Figure 2C in the above embodiment, and the embodiment of the present application will not be repeated here.

其中,上述框架层201还可设置用于实现分布式拍摄功能的Dv套件(kit)201b。Dvkit201b可以由设置在应用层200的Dv应用200a调用,以实现发现并连接远端设备102的功能。或者,还可将Dv应用200a实现的功能以系统服务的形式常驻在手机中运行。The framework layer 201 may also be provided with a Dv kit 201b for implementing a distributed shooting function. The Dvkit 201b may be called by the Dv application 200a provided in the application layer 200 to implement the function of discovering and connecting to the remote device 102. Alternatively, the function implemented by the Dv application 200a may be resident in the mobile phone in the form of a system service.

当手机需要使用其他电子设备的摄像头实现分布式拍摄功能时,在Camera kit201c提供的相机模式(如双景模式)下,本端设备101可以通过Dv kit 201b发现并连接远端设备102。本端设备101与远端设备102建立连接后,本端设备101可以在本端设备101的HAL202,针对远端设备102创建一个DMSDP HAL 202b。即可将远端设备102注册到本端设备101。图2E所示的DMSDP HAL 202b是本端设备101按照远端设备102的拍摄能力参数创建的HAL。When the mobile phone needs to use the camera of other electronic devices to realize the distributed shooting function, in the camera mode (such as dual view mode) provided by Camera kit 201c, the local device 101 can discover and connect to the remote device 102 through Dv kit 201b. After the local device 101 establishes a connection with the remote device 102, the local device 101 can create a DMSDP HAL 202b for the remote device 102 in the HAL 202 of the local device 101. The remote device 102 can be registered with the local device 101. The DMSDP HAL 202b shown in FIG. 2E is a HAL created by the local device 101 according to the shooting capability parameters of the remote device 102.

其中,远端设备102也可以包括Dv应用和Dv kit。该远端设备102中Dv应用和Dvkit的功能,可以参考本申请实施例对Dv应用200a和Dv kit 201b的功能介绍,这里不予赘述。The remote device 102 may also include a Dv application and a Dv kit. The functions of the Dv application and the Dv kit in the remote device 102 can refer to the functional introduction of the Dv application 200a and the Dv kit 201b in the embodiment of the present application, which will not be repeated here.

仍如图2E所示,除了在HAL 202中为远端设备201创建对应的DMSDP HAL 202b之外,还可以将远端设备102的拍摄能力参数发送至Camera Service 201a1进行保存,也即在Camera Service 201a1中注册当前远端设备102的拍摄能力参数(包括设备能力信息和算法能力信息)。其中,设备能力信息用于指示远端设备102拍摄图像的硬件能力,算法能力信息用于指示远端设备102对拍摄的图像进行图像处理的算法能力。远端设备102的能力信息可通过DMSDP HAL 202b注册或映射到Camera Service 201a1。进一步的,DMSDP HAL 202b中还可设置“对端Meta同步映射”功能。对端Meta是指远端设备102中用于指示远端设备102的拍摄能力参数的标签。Still as shown in FIG. 2E , in addition to creating a corresponding DMSDP HAL 202b for the remote device 201 in HAL 202, the shooting capability parameters of the remote device 102 can also be sent to the Camera Service 201a1 for storage, that is, the shooting capability parameters (including device capability information and algorithm capability information) of the current remote device 102 are registered in the Camera Service 201a1. Among them, the device capability information is used to indicate the hardware capability of the remote device 102 for shooting images, and the algorithm capability information is used to indicate the algorithm capability of the remote device 102 for image processing of the captured images. The capability information of the remote device 102 can be registered or mapped to the Camera Service 201a1 through the DMSDP HAL 202b. Furthermore, the "peer Meta synchronization mapping" function can also be set in the DMSDP HAL 202b. Peer Meta refers to a label in the remote device 102 used to indicate the shooting capability parameters of the remote device 102.

需要说明的是,DMSDP HAL 202b也可以称为虚拟Camera HAL或者DMSDP CameraHAL。与传统的Camera HAL不同的是,DMSDP HAL 202b并不与本端设备101实际的硬件设备相对应,而是与远端设备102对应。如图2F所示,远端设备102(如电视、平板电脑、手机等包括摄像头的设备)可作为本端设备101的虚拟相机。其中,本端设备101(如图2F所示的虚拟框架)可通过DMSDP HAL 202b与远端设备102进行数据收发,将远端设备102作为本端设备101的一个虚拟相机,与远端设备102协同完成分布式拍摄场景中的各项业务。It should be noted that DMSDP HAL 202b can also be called virtual Camera HAL or DMSDP CameraHAL. Different from the traditional Camera HAL, DMSDP HAL 202b does not correspond to the actual hardware device of the local device 101, but corresponds to the remote device 102. As shown in FIG2F, the remote device 102 (such as a TV, tablet computer, mobile phone, etc. including a camera) can be used as a virtual camera of the local device 101. Among them, the local device 101 (the virtual framework shown in FIG2F) can send and receive data with the remote device 102 through the DMSDP HAL 202b, and use the remote device 102 as a virtual camera of the local device 101, and cooperate with the remote device 102 to complete various services in the distributed shooting scene.

在一些实施例中,本端设备101的Dv应用200a还可以通过Dv kit 201b获取远端设备102的音频能力参数(例如音频播放时延、音频采样率或声音通道数目等)、显示能力参数(例如屏幕分辨率、显示数据的编解码算法等)。当然,如果远端设备102还具有其他能力(例如打印能力等),则远端设备102也可将相关的能力参数发送给本端设备101的Dv应用200a。此时,DMSDP HAL 202b不仅具有远端设备102的图像处理能力,还具有远端设备102的音频、显示等能力,使得远端设备102可以作为本端设备101的虚拟设备与本端设备101协同完成分布式场景中的各项业务。In some embodiments, the Dv application 200a of the local device 101 can also obtain the audio capability parameters (such as audio playback delay, audio sampling rate or number of sound channels, etc.) and display capability parameters (such as screen resolution, encoding and decoding algorithm of display data, etc.) of the remote device 102 through the Dv kit 201b. Of course, if the remote device 102 also has other capabilities (such as printing capabilities, etc.), the remote device 102 can also send the relevant capability parameters to the Dv application 200a of the local device 101. At this time, the DMSDP HAL 202b not only has the image processing capability of the remote device 102, but also has the audio, display and other capabilities of the remote device 102, so that the remote device 102 can serve as a virtual device of the local device 101 and collaborate with the local device 101 to complete various services in the distributed scenario.

与图2D所示的方案不同的是,本端设备101与远端设备102建立连接后,可分别创建两个相机会话(Camera Session)。例如,本端设备101可以通过DMSDP HAL 202b创建用于传输控制流的控制会话(Control Session)和用于传输数据流(即预览流)的数据会话(Data Session)。Different from the solution shown in FIG. 2D , after the local device 101 establishes a connection with the remote device 102, two camera sessions (Camera Session) can be created respectively. For example, the local device 101 can create a control session (Control Session) for transmitting a control stream and a data session (Data Session) for transmitting a data stream (i.e., a preview stream) through the DMSDP HAL 202b.

其中,除了在HAL中为远端设备102创建对应的DMSDP HAL 202b之外,还可以将远端设备102的拍摄能力参数发送至Camera Service 201a1进行保存,也即在CameraService 201a1中注册当前远端设备102的拍摄能力。In addition to creating a corresponding DMSDP HAL 202b for the remote device 102 in the HAL, the shooting capability parameters of the remote device 102 may also be sent to the Camera Service 201a1 for storage, that is, the shooting capability of the current remote device 102 is registered in the CameraService 201a1.

当手机运行相机应用时,Camera Service 201a1能够根据相机应用下发的控制命令(例如预览、放大、录像等指令),结合远端设备102的拍摄能力实时确定拍摄过程中的拍摄策略。例如,Camera Service可以在根据从设备的拍摄能力参数,在拍摄策略中设置手机需要执行的图像处理任务,以及远端设备102需要执行的图像处理任务。进而,CameraService可使用DMSDP HAL 202b,通过Control Session向远端设备102发送与拍摄策略对应的拍摄指令,触发远端设备102执行对应的图像处理任务。When the mobile phone runs the camera application, the Camera Service 201a1 can determine the shooting strategy in real time during the shooting process according to the control commands (such as preview, zoom, and record instructions) issued by the camera application and the shooting capability of the remote device 102. For example, the Camera Service can set the image processing tasks that the mobile phone needs to perform and the image processing tasks that the remote device 102 needs to perform in the shooting strategy according to the shooting capability parameters of the slave device. Furthermore, the CameraService can use the DMSDP HAL 202b to send the shooting instructions corresponding to the shooting strategy to the remote device 102 through the Control Session, triggering the remote device 102 to perform the corresponding image processing tasks.

之后,拍摄指令传输至远端设备102的Camera HAL 212a后,使得远端设备102的Camera HAL 212a可根据接收到的拍摄指令调用内核层中的相机驱动,驱动摄像头等硬件设备响应该拍摄指令采集图像数据。例如,摄像头可按照一定的帧率,将采集到的每一帧图像数据通过相机驱动传递给Camera HAL。Afterwards, after the shooting instruction is transmitted to the Camera HAL 212a of the remote device 102, the Camera HAL 212a of the remote device 102 can call the camera driver in the kernel layer according to the received shooting instruction, and drive the camera and other hardware devices to respond to the shooting instruction to collect image data. For example, the camera can pass each frame of image data collected to the Camera HAL through the camera driver at a certain frame rate.

当远端设备102的Camera HAL 212a接收到摄像头采集到的每一帧图像数据后,向上层传输包括摄像头采集的RAW图像的数据流。其中,远端设备102的应用层210接收到包括RAW图像的数据流后,可由应用层210的相机代理服务210a(或者也可以称为相机代理应用)通过DMSDP HAL 202b向本端设备101发送包括RAW图像的数据流。例如,相机代理服务210a可以通过DMSDP HAL 202b提供的Data Session向本端设备101发送包括RAW图像的数据流。After the Camera HAL 212a of the remote device 102 receives each frame of image data collected by the camera, it transmits a data stream including the RAW image collected by the camera to the upper layer. After the application layer 210 of the remote device 102 receives the data stream including the RAW image, the camera proxy service 210a (or also referred to as the camera proxy application) of the application layer 210 can send the data stream including the RAW image to the local device 101 through the DMSDP HAL 202b. For example, the camera proxy service 210a can send the data stream including the RAW image to the local device 101 through the Data Session provided by the DMSDP HAL 202b.

本端设备101的DMSDP HAL 202b接收到包括RAW图像的数据流后,可按照CameraService201a1中保存的拍摄能力参数调用相关算法执行后处理(算法),对该RAW图像进行图像处理,得到对应的每一帧拍摄画面。后续,DMSDP HAL 202b可将经过图像处理后的每一帧拍摄画面通过Camera Service 201a1上报给相机应用,相机应用可将每一帧拍摄画面显示在显示界面中。After receiving the data stream including the RAW image, the DMSDP HAL 202b of the local device 101 can call the relevant algorithm to perform post-processing (algorithm) according to the shooting capability parameters saved in the CameraService201a1, perform image processing on the RAW image, and obtain each corresponding frame of the shooting picture. Subsequently, the DMSDP HAL 202b can report each frame of the shooting picture after image processing to the camera application through the Camera Service 201a1, and the camera application can display each frame of the shooting picture in the display interface.

这样一来,本端设备101和远端设备102在实现分布式拍摄功能时,本端设备101可基于上述拍摄策略,按照自身的拍摄能力和远端设备102的拍摄能力对图像数据进行相应的图像处理,在分布式拍摄场景中实现较好的拍摄效果。In this way, when the local device 101 and the remote device 102 implement the distributed shooting function, the local device 101 can perform corresponding image processing on the image data based on the above-mentioned shooting strategy according to its own shooting capabilities and the shooting capabilities of the remote device 102, so as to achieve better shooting effects in the distributed shooting scene.

在一些实施例中,本端设备101也可以向远端设备102发送该本端设备101采集的RAW图像。远端设备102的Camera Service 211a1中也可以保存本端设备101的拍摄能力参数。远端设备102也可以按照本端设备101的拍摄能力参数调用相关的算法,对来自本端设备101的RAW图像进行图像处理,得到对应的每一帧拍摄画面。远端设备102可以按照远端设备102的拍摄能力参数调用相关的算法对远端设备102采集的RAW图像进行图像处理,得到对应的每一帧拍摄画面。如此,远端设备102也可以同时显示本端设备101的拍摄画面和远端设备102的拍摄画面。采用本方案,可以同时在本端设备101和远端设备102上实现上述跨设备的分布式拍摄功能。In some embodiments, the local device 101 may also send a RAW image captured by the local device 101 to the remote device 102. The camera service 211a1 of the remote device 102 may also store the shooting capability parameters of the local device 101. The remote device 102 may also call relevant algorithms according to the shooting capability parameters of the local device 101, perform image processing on the RAW image from the local device 101, and obtain each corresponding frame of the shooting picture. The remote device 102 may call relevant algorithms according to the shooting capability parameters of the remote device 102, and perform image processing on the RAW image captured by the remote device 102, and obtain each corresponding frame of the shooting picture. In this way, the remote device 102 may also display the shooting pictures of the local device 101 and the shooting pictures of the remote device 102 at the same time. By adopting this solution, the above-mentioned distributed shooting function across devices can be implemented on both the local device 101 and the remote device 102.

以下举例说明本申请实施例提供的方法的应用场景。本申请实施例的方法至少可以适用于以下应用场景(1)-应用场景(2)。The following examples illustrate the application scenarios of the method provided in the embodiments of the present application. The method in the embodiments of the present application can be applied to at least the following application scenarios (1) to (2).

应用场景(1):跨设备多景拍摄场景。Application scenario (1): Cross-device multi-scene shooting scenario.

例如,该跨设备多景拍摄场景可以为跨设备双景拍摄场景。其中,跨设备双景拍摄场景可以包括跨设备双景拍照场景和跨设备双景录像场景。以下实施例中,以跨设备双景拍照场景为例,介绍该跨设备双景拍摄场景。For example, the cross-device multi-view shooting scene may be a cross-device dual-view shooting scene. The cross-device dual-view shooting scene may include a cross-device dual-view photo shooting scene and a cross-device dual-view video recording scene. In the following embodiments, the cross-device dual-view photo shooting scene is taken as an example to introduce the cross-device dual-view shooting scene.

示例性的,上述预设应用可以是用于实现拍摄功能的相机应用。以手机0为本端设备101举例,手机0中可以安装有相机应用。手机0(即本端设备101)可以通过相机应用与远端设备102进行跨设备多景拍摄。如图3A所示,检测到用户打开相机应用后,手机0可打开自身的摄像头开始采集图像数据,并根据采集到的图像数据在预览界面301的预览框302中实时显示对应的拍摄画面。Exemplarily, the preset application may be a camera application for realizing the shooting function. Taking mobile phone 0 as the local device 101 as an example, a camera application may be installed in mobile phone 0. Mobile phone 0 (i.e., local device 101) may perform cross-device multi-view shooting with remote device 102 through the camera application. As shown in FIG3A , after detecting that the user opens the camera application, mobile phone 0 may open its own camera to start collecting image data, and display the corresponding shooting picture in real time in the preview box 302 of the preview interface 301 according to the collected image data.

其中,手机0在预览框302中显示的拍摄画面与手机0使用摄像头采集的图像数据可以不同。例如,手机0获取到摄像头采集的图像数据后,可以对采集到的图像数据执行防抖、对焦、柔焦、虚化、滤镜、美颜、人脸检测或AR识别等图像处理任务,得到图像处理后的拍摄画面。进而,手机0可将图像处理后的拍摄画面显示在上述预览框302中。The captured image displayed by the mobile phone 0 in the preview frame 302 may be different from the image data collected by the mobile phone 0 using the camera. For example, after the mobile phone 0 obtains the image data collected by the camera, it may perform image processing tasks such as anti-shake, focus, soft focus, blur, filter, beauty, face detection or AR recognition on the collected image data to obtain the captured image after image processing. Furthermore, the mobile phone 0 may display the captured image after image processing in the preview frame 302.

类似的,远端设备102也可以打开自身的摄像头开始采集图像数据,并对采集到的图像数据执行防抖、对焦、柔焦、虚化、滤镜、美颜、人脸检测或AR识别等图像处理任务,得到图像处理后的拍摄画面,本申请实施例对此不做任何限制。Similarly, the remote device 102 can also turn on its own camera to start collecting image data, and perform image processing tasks such as anti-shake, focus, soft focus, blur, filter, beauty, face detection or AR recognition on the collected image data to obtain the captured picture after image processing. The embodiments of the present application do not impose any restrictions on this.

在本申请实施例中,如图3A所示,手机0可以在相机应用的预览界面301中设置用于跨设备多景拍摄的功能按钮303。当用户希望在手机0上看到手机0与远端设备102拍摄的多景画面时,可点击功能按钮303开启跨设备多景拍摄功能。In the embodiment of the present application, as shown in FIG3A , the mobile phone 0 can set a function button 303 for cross-device multi-view shooting in the preview interface 301 of the camera application. When the user wants to see the multi-view images shot by the mobile phone 0 and the remote device 102 on the mobile phone 0, the function button 303 can be clicked to start the cross-device multi-view shooting function.

或者,手机0还可以将同步拍摄功能的功能按钮304设置在手机的控制中心、下拉菜单、负一屏或其他应用(例如视频通话应用)中,本申请实施例对此不做任何限制。例如,如图3B所示,手机可响应用户打开控制中心的操作显示控制中心305,控制中心305中设置有上述功能按钮304。如果用户希望使用手机和其他电子设备一起进行同步拍摄,可点击功能按钮304。Alternatively, the mobile phone 0 may also set the function button 304 of the synchronous shooting function in the control center, drop-down menu, negative one screen or other application (such as a video call application) of the mobile phone, and the embodiment of the present application does not impose any restrictions on this. For example, as shown in FIG3B , the mobile phone may display the control center 305 in response to the user's operation of opening the control center, and the control center 305 is provided with the above-mentioned function button 304. If the user wants to use the mobile phone and other electronic devices to perform synchronous shooting together, the function button 304 may be clicked.

示例性的,手机0检测到用户点击功能按钮303、功能按钮304后,如图4所示,手机0可在对话框401中显示当前手机0搜索到的可以采集图像数据一个或多个候选设备。Exemplarily, after mobile phone 0 detects that the user clicks function button 303 and function button 304 , as shown in FIG. 4 , mobile phone 0 may display one or more candidate devices that can collect image data that are currently searched by mobile phone 0 in dialog box 401 .

例如,云服务器中可以记录每个电子设备是否具有拍摄功能。那么,手机0可以在云服务器中查询与手机0登录同一账号(例如华为账号)的具有拍摄功能的电子设备。进而,手机0可将查询到的电子设备作为候选设备显示在对话框401中。For example, the cloud server may record whether each electronic device has a shooting function. Then, mobile phone 0 may query the cloud server for electronic devices with a shooting function that are logged in with the same account (e.g., Huawei account) as mobile phone 0. Furthermore, mobile phone 0 may display the queried electronic devices as candidate devices in dialog box 401.

或者,手机0可以搜索与手机0位于同一Wi-Fi网络中的电子设备。进而,手机0可向同一Wi-Fi网络中的各个电子设备发送查询请求,触发接收到查询请求的电子设备可向手机0发送响应消息,响应消息中可以指示自身是否具有拍摄功能。那么,手机0可以根据接收到的响应消息确定出当前Wi-Fi网络中具有拍摄功能的电子设备。进而,手机0可将具有拍摄功能的电子设备作为候选设备显示在对话框401中。Alternatively, mobile phone 0 may search for electronic devices in the same Wi-Fi network as mobile phone 0. Further, mobile phone 0 may send a query request to each electronic device in the same Wi-Fi network, triggering the electronic device that receives the query request to send a response message to mobile phone 0, and the response message may indicate whether it has a shooting function. Then, mobile phone 0 may determine the electronic device with a shooting function in the current Wi-Fi network based on the received response message. Further, mobile phone 0 may display the electronic device with a shooting function as a candidate device in dialog box 401.

又或者,手机0中可安装用于管理家庭内智能家居设备(例如电视、空调、音箱或冰箱等)的应用。以智能家居应用举例,用户可以在智能家居应用中添加一个或多个智能家居设备,使得用户添加的智能家居设备与手机0建立关联。例如,智能家居设备上可以设置包含设备标识等设备信息的二维码,用户使用手机0的智能家居应用扫描该二维码后,可将对应的智能家居设备添加至智能家居应用中,从而建立智能家居设备与手机0的关联关系。在本申请实施例中,当智能家居应用中添加的一个或多个智能家居设备上线时,例如,当手机0检测到已添加智能家居设备发送的Wi-Fi信号时,手机0可将该智能家居设备作为候选设备显示在对话框401中,提示用户选择使用相应的智能家居设备与手机0进行同步拍摄。Alternatively, an application for managing smart home devices (such as televisions, air conditioners, speakers, or refrigerators, etc.) in the home may be installed in mobile phone 0. Taking the smart home application as an example, the user can add one or more smart home devices to the smart home application so that the smart home devices added by the user are associated with mobile phone 0. For example, a QR code containing device information such as a device identifier may be set on the smart home device. After the user scans the QR code using the smart home application of mobile phone 0, the corresponding smart home device can be added to the smart home application, thereby establishing an association between the smart home device and mobile phone 0. In an embodiment of the present application, when one or more smart home devices added to the smart home application are online, for example, when mobile phone 0 detects a Wi-Fi signal sent by the added smart home device, mobile phone 0 may display the smart home device as a candidate device in dialog box 401, prompting the user to choose to use the corresponding smart home device for synchronous shooting with mobile phone 0.

如图4所示,以手机0搜索到的候选设备包括电视1、手表2以及手机3举例,用户可以在电视1、手表2以及手机3中选择本次与手机0进行跨设备多景拍摄的一个或多个远端设备102。本申请实施例中,以跨设备双景拍摄为例。例如,如果检测到用户选择电视1,则手机0可将电视1作为远端设备102,与电视1建立网络连接。例如,手机0可通过路由器与电视1建立Wi-Fi连接,或者,手机0可直接与电视1建立Wi-Fi P2P连接,或者手机0可以直接与电视1建立蓝牙连接;或者,手机0可直接与电视1建立蓝牙连接;或者,手机0可直接与电视1建立短距离无线连接,该短距离无线连接包括但不限于近场通信(Near Field Communication,NFC)连接,红外线连接,超宽带(Ultra Wideband,UWB)连接,紫峰协议(ZigBee)连接;或者,手机0可直接与电视1建立移动网络连接,该移动网络包括但不限于支持2G,3G,4G,5G以及后续标准协议的移动网络。As shown in FIG4 , for example, the candidate devices searched by mobile phone 0 include TV 1, watch 2, and mobile phone 3. The user can select one or more remote devices 102 for cross-device multi-view shooting with mobile phone 0 from TV 1, watch 2, and mobile phone 3. In the embodiment of the present application, cross-device dual-view shooting is taken as an example. For example, if it is detected that the user selects TV 1, mobile phone 0 can use TV 1 as the remote device 102 to establish a network connection with TV 1. For example, mobile phone 0 can establish a Wi-Fi connection with TV 1 through a router, or mobile phone 0 can directly establish a Wi-Fi P2P connection with TV 1, or mobile phone 0 can directly establish a Bluetooth connection with TV 1; or mobile phone 0 can directly establish a Bluetooth connection with TV 1; or mobile phone 0 can directly establish a short-range wireless connection with TV 1, and the short-range wireless connection includes but is not limited to near field communication (NFC) connection, infrared connection, ultra-wideband (Ultra Wideband, UWB) connection, ZigBee connection; or mobile phone 0 can directly establish a mobile network connection with TV 1, and the mobile network includes but is not limited to mobile networks supporting 2G, 3G, 4G, 5G and subsequent standard protocols.

在另一些实施例中,手机0检测到用户点击功能按钮303后,手机可按照上述方法搜索具有拍摄功能的一个或多个电子设备。进而,手机0可自动与搜索到的电子设备建立网路连接。此时,无需用户手动选择与手机0建立网络连接的具体设备。In other embodiments, after the mobile phone 0 detects that the user clicks the function button 303, the mobile phone can search for one or more electronic devices with a shooting function according to the above method. Then, the mobile phone 0 can automatically establish a network connection with the searched electronic devices. In this case, the user does not need to manually select a specific device to establish a network connection with the mobile phone 0.

又或者,在用户打开相机应用之前,手机可能已经与一个或多个具有拍摄功能的电子设备建立了网络连接。例如,用户在手机0中打开相机应用之前,已经与平板电脑建立了蓝牙连接。后续,手机0打开相机应用显示出相机应用的预览界面301后,如果检测到用户点击功能按钮303,则手机0可以不再搜索具有拍摄功能的电子设备,而是执行下述方法。Alternatively, before the user opens the camera application, the mobile phone may have established a network connection with one or more electronic devices with a shooting function. For example, before the user opens the camera application in the mobile phone 0, a Bluetooth connection has been established with the tablet computer. Subsequently, after the mobile phone 0 opens the camera application and displays the preview interface 301 of the camera application, if it is detected that the user clicks the function button 303, the mobile phone 0 may no longer search for electronic devices with a shooting function, but execute the following method.

手机0与电视1建立网络连接后,一方面,如图5所示,手机0可打开自身的摄像头开始采集图像数据,并对采集到的图像数据进行图像处理,得到拍摄画面1。另一方面,如图5所示,手机0可指示电视1打开自身的摄像头开始采集图像数据,并对采集到的图像数据进行图像处理,得到拍摄画面2。后续,电视1可将拍摄画面2发送给手机0。这样,手机0可在相机应用的显示界面中同时显示来自手机0的拍摄画面1以及来自电视1的拍摄画面2。After mobile phone 0 establishes a network connection with TV 1, on one hand, as shown in FIG5 , mobile phone 0 can turn on its own camera to start collecting image data, and perform image processing on the collected image data to obtain shooting picture 1. On the other hand, as shown in FIG5 , mobile phone 0 can instruct TV 1 to turn on its own camera to start collecting image data, and perform image processing on the collected image data to obtain shooting picture 2. Subsequently, TV 1 can send shooting picture 2 to mobile phone 0. In this way, mobile phone 0 can simultaneously display shooting picture 1 from mobile phone 0 and shooting picture 2 from TV 1 in the display interface of the camera application.

如上述实施例所述,在应用场景(1)的第一种情况下,在上述跨设备多景拍摄场景中,可以使用多个摄像头实现对同一拍摄对象的多视角拍摄,可以提升拍摄的趣味性。As described in the above embodiment, in the first case of application scenario (1), in the above cross-device multi-view shooting scenario, multiple cameras can be used to achieve multi-perspective shooting of the same shooting object, which can enhance the fun of shooting.

在应用场景(1)的第二种情况下,在上述跨设备多景拍摄场景中,还可以使用多个摄像头实现对多个拍摄对象的合拍(如亲子合拍或好友合拍等)。例如,假设本端设备101是图5所示的手机0,远端设备102是图6A所示的手机3。其中,手机0或手机3可以通过上述预设应用(如相机应用或者其他可用于实现照片或视频拍摄的应用),实现多个拍摄对象的合拍。In the second case of application scenario (1), in the above-mentioned cross-device multi-scene shooting scenario, multiple cameras can also be used to achieve co-shooting of multiple shooting objects (such as parent-child shooting or friend shooting, etc.). For example, assume that the local device 101 is the mobile phone 0 shown in FIG5, and the remote device 102 is the mobile phone 3 shown in FIG6A. Among them, the mobile phone 0 or the mobile phone 3 can achieve co-shooting of multiple shooting objects through the above-mentioned preset application (such as a camera application or other applications that can be used to achieve photo or video shooting).

例如,以预设应用是相机应用为例。当用户希望在手机0上看到手机0与手机3的合拍画面时,可点击功能按钮303或功能按钮304开启跨设备多景合拍功能。For example, taking the camera application as the preset application, when the user wishes to see the co-photographed picture of mobile phone 0 and mobile phone 3 on mobile phone 0, the user can click function button 303 or function button 304 to start the cross-device multi-scene co-photographing function.

示例性的,手机0检测到用户点击功能按钮303或功能按钮304后,如图4所示,手机0可在对话框401中显示当前手机0搜索到的可以采集图像数据一个或多个候选设备。如图4所示,以手机0搜索到的候选设备包括电视1、手表2以及手机3举例,用户可以在电视1、手表2以及手机3中选择本次与手机0进行跨设备多景合拍的远端设备102。例如,如果检测到用户选择手机3,则手机0可将手机3作为远端设备102,与手机3建立网络连接。Exemplarily, after mobile phone 0 detects that the user clicks function button 303 or function button 304, as shown in FIG4 , mobile phone 0 may display one or more candidate devices that can collect image data that are currently searched by mobile phone 0 in dialog box 401. As shown in FIG4 , taking the example that the candidate devices searched by mobile phone 0 include TV 1, watch 2, and mobile phone 3, the user may select remote device 102 for cross-device multi-scene co-shooting with mobile phone 0 from TV 1, watch 2, and mobile phone 3. For example, if it is detected that the user selects mobile phone 3, mobile phone 0 may use mobile phone 3 as remote device 102 and establish a network connection with mobile phone 3.

手机0与手机3建立网络连接后,一方面,如图6A所示,手机0可打开自身的摄像头开始采集图像数据,并对采集到的图像数据进行图像处理,得到拍摄画面1。另一方面,如图6A所示,手机0可指示手机3打开自身的摄像头开始采集图像数据,并对采集到的图像数据进行图像处理,得到拍摄画面3。后续,手机3可将拍摄画面3发送给手机0。这样,手机0可在相机应用的显示界面中同时显示来自手机0的拍摄画面1以及来自手机3的拍摄画面3。After mobile phone 0 establishes a network connection with mobile phone 3, on one hand, as shown in FIG6A , mobile phone 0 can turn on its own camera to start collecting image data, and perform image processing on the collected image data to obtain shooting picture 1. On the other hand, as shown in FIG6A , mobile phone 0 can instruct mobile phone 3 to turn on its own camera to start collecting image data, and perform image processing on the collected image data to obtain shooting picture 3. Subsequently, mobile phone 3 can send shooting picture 3 to mobile phone 0. In this way, mobile phone 0 can simultaneously display shooting picture 1 from mobile phone 0 and shooting picture 3 from mobile phone 3 in the display interface of the camera application.

需要说明的是,与上述第一种情况不同的是,在进行跨设备多景合拍的过程中,手机0可将拍摄画面1发送给手机3。如图6A所示,手机3也可以同时显示来自手机0的拍摄画面1以及来自手机3的拍摄画面3。此时,手机0可同时作为本端设备和远端设备。具体的,手机0作为本端设备,手机3作为远端设备,手机0可同时显示手机0的拍摄画面1以及来自手机3的拍摄画面3。手机3作为本端设备,手机0作为远端设备,手机3可同时显示手机3的拍摄画面3以及来自手机0的拍摄画面1。It should be noted that, unlike the first case above, in the process of cross-device multi-scene co-shooting, mobile phone 0 can send shooting picture 1 to mobile phone 3. As shown in FIG6A , mobile phone 3 can also simultaneously display shooting picture 1 from mobile phone 0 and shooting picture 3 from mobile phone 3. At this time, mobile phone 0 can be used as a local device and a remote device at the same time. Specifically, mobile phone 0 is used as a local device, mobile phone 3 is used as a remote device, and mobile phone 0 can simultaneously display shooting picture 1 of mobile phone 0 and shooting picture 3 from mobile phone 3. Mobile phone 3 is used as a local device, and mobile phone 0 is used as a remote device, and mobile phone 3 can simultaneously display shooting picture 3 of mobile phone 3 and shooting picture 1 from mobile phone 0.

应注意,上述跨设备多景拍摄场景可以是至少两个设备进行跨设备多景拍摄的场景。示例性的,三个电子设备也可以应用本申请实施例的方法,实现跨设备多景拍摄功能。例如,图6B所示的手机0可以作为本端设备,手机3和电视1作为远端设备;手机0可以与手机3和电视1实现跨设备多景拍摄。如图6B所示,手机0可以同时显示手机0拍摄画面1、来自手机3的拍摄画面3,以及来自电视1的拍摄画面2。It should be noted that the above-mentioned cross-device multi-scene shooting scenario can be a scenario in which at least two devices perform cross-device multi-scene shooting. Exemplarily, three electronic devices can also apply the method of the embodiment of the present application to realize the cross-device multi-scene shooting function. For example, mobile phone 0 shown in Figure 6B can be used as a local device, and mobile phone 3 and TV 1 can be used as remote devices; mobile phone 0 can realize cross-device multi-scene shooting with mobile phone 3 and TV 1. As shown in Figure 6B, mobile phone 0 can simultaneously display the shooting picture 1 of mobile phone 0, the shooting picture 3 from mobile phone 3, and the shooting picture 2 from TV 1.

应用场景(2):本端设备101调用远端设备102进行视频通话、直播、拍照或者录像的场景。其中,本端设备102可以调用远端设备102的摄像头,或者远端设备102的摄像头和显示屏,协助本端设备101行视频通话、直播、拍照或者录像。Application scenario (2): The scenario where the local device 101 calls the remote device 102 to make a video call, live broadcast, take a photo, or record a video. The local device 102 can call the camera of the remote device 102, or the camera and display screen of the remote device 102, to assist the local device 101 in making a video call, live broadcast, take a photo, or record a video.

示例性的,本申请实施例这里以本端设备101调用远端设备102进行视频通话为例,介绍该应用场景(2)。上述预设应用可以是用于实现视频通话的视频通信应用(如微信TM应用)。以手机0为本端设备101,电视1是远端设备102举例,手机0中可以安装有上述视频通信应用。For example, the present application embodiment takes the local device 101 calling the remote device 102 for a video call as an example to introduce the application scenario (2). The above-mentioned preset application can be a video communication application (such as WeChat TM application) for implementing a video call. Taking the mobile phone 0 as the local device 101 and the TV 1 as the remote device 102 as an example, the above-mentioned video communication application can be installed in the mobile phone 0.

在一种实现方式中,手机0可以在与手机4进行视频通话的过程中,调用远端设备102(如电视1)摄像头和显示屏,协助手机0与手机4进行视频通话。例如,如图7所示,手机0显示视频通话界面701,该视频通话界面701中设置有用于调用其他设备协助手机0进行视频通话的功能按钮702。当用户希望手机0调用其他设备协助手机0进行视频通话时,可点击功能按钮702。In one implementation, mobile phone 0 can call the camera and display screen of remote device 102 (such as TV 1) to assist mobile phone 0 in making a video call with mobile phone 4 during a video call. For example, as shown in FIG7 , mobile phone 0 displays a video call interface 701, and a function button 702 for calling other devices to assist mobile phone 0 in making a video call is provided in the video call interface 701. When the user wants mobile phone 0 to call other devices to assist mobile phone 0 in making a video call, the function button 702 can be clicked.

在另一种实现方式中,手机0可以在请求与手机4进行视频通话之前,调用远端设备102(如电视1)摄像头和显示屏,协助手机0与手机4进行视频通话。例如,如图8所示,响应于用户对聊天界面中“视频通话”选项801的点击操作,手机0可显示确认窗口802,该确认窗口802用于请求用户确认是否使用大屏协助手机进行语音通话。当用户希望手机0调用其他设备协助手机0进行视频通话时,可点击确认窗口802中的“是”按钮。In another implementation, before requesting a video call with mobile phone 4, mobile phone 0 can call the camera and display of remote device 102 (such as TV 1) to assist mobile phone 0 in making a video call with mobile phone 4. For example, as shown in FIG8 , in response to a user clicking on the “video call” option 801 in the chat interface, mobile phone 0 can display a confirmation window 802, which is used to request the user to confirm whether to use the large screen to assist the mobile phone in making a voice call. When the user wants mobile phone 0 to call other devices to assist mobile phone 0 in making a video call, the user can click the “Yes” button in the confirmation window 802.

示例性的,手机0检测到用户点击功能按钮702或者确认窗口802中的“是”按钮后,可搜索候选设备。如可显示图4所示的对话框401。其中,手机搜索候选设备,显示对话框401的具体方法,可以参考上述实施例中的详细描述,这里不予赘述。Exemplarily, after the mobile phone 0 detects that the user clicks the function button 702 or the "Yes" button in the confirmation window 802, the candidate device can be searched. For example, the dialog box 401 shown in FIG. 4 can be displayed. The specific method of searching the candidate device and displaying the dialog box 401 by the mobile phone can refer to the detailed description in the above embodiment, which will not be repeated here.

手机0检测到用户选择电视1,则手机0可将电视1作为远端设备102,与电视1建立网络连接。手机0与电视1建立网络连接后,一方面,如图9所示,手机0可接收来自手机4的拍摄画面b,并向电视1传输该拍摄画面b。另一方面,如图9所示,手机0可指示电视1打开自身的摄像头开始采集图像数据,并对采集到的图像数据进行图像处理,得到拍摄画面c。后续,电视1可以将拍摄画面c发送给手机0,由手机0将该拍摄画面c传输至手机4。这样,电视1可同时显示来自手机4的拍摄画面b以及来自电视1的拍摄画面c;手机4也可以同时显示来自电视1的拍摄画面c以及来自手机4的拍摄画面b。When mobile phone 0 detects that the user selects TV 1, mobile phone 0 can use TV 1 as a remote device 102 to establish a network connection with TV 1. After mobile phone 0 establishes a network connection with TV 1, on the one hand, as shown in FIG9 , mobile phone 0 can receive the captured image b from mobile phone 4 and transmit the captured image b to TV 1. On the other hand, as shown in FIG9 , mobile phone 0 can instruct TV 1 to turn on its own camera to start collecting image data, and perform image processing on the collected image data to obtain captured image c. Subsequently, TV 1 can send captured image c to mobile phone 0, and mobile phone 0 will transmit the captured image c to mobile phone 4. In this way, TV 1 can simultaneously display captured image b from mobile phone 4 and captured image c from TV 1; mobile phone 4 can also simultaneously display captured image c from TV 1 and captured image b from mobile phone 4.

在另一些实施例中,手机0调用电视1摄像头和显示屏,协助手机0与手机4进行视频通话的过程中,手机0也可以显示来自手机4的拍摄画面b和/或来自电视1的拍摄画面c。In other embodiments, mobile phone 0 calls the camera and display screen of TV 1 to assist mobile phone 0 in making a video call with mobile phone 4. Mobile phone 0 can also display the captured picture b from mobile phone 4 and/or the captured picture c from TV 1.

或者,手机0调用电视1摄像头和显示屏,协助手机0与手机4进行视频通话的过程中,手机0可以显示预设界面。该预设界面可以是图9所示的主界面901。或者,该预设界面还可以是预先设定的图片或者预先设定的动画等。Alternatively, when mobile phone 0 calls the camera and display screen of TV 1 to assist mobile phone 0 in making a video call with mobile phone 4, mobile phone 0 may display a preset interface. The preset interface may be the main interface 901 shown in FIG. 9 . Alternatively, the preset interface may also be a preset picture or a preset animation.

或者,手机0调用电视1摄像头和显示屏,协助手机0与手机4进行视频通话的过程中,手机0可以黑屏。这样,可以降低手机0的功耗。Alternatively, the screen of mobile phone 0 can be black when mobile phone 0 calls the camera and display screen of TV 1 to assist mobile phone 0 in making a video call with mobile phone 4. In this way, the power consumption of mobile phone 0 can be reduced.

需要说明的是,由于手机0和电视1的拍摄视角不同,因此手机0的拍摄画面与电视1的拍摄画面可能不同。例如,图7所示的拍摄画面a是手机0拍摄的,图9所示的拍摄画面c是电视1拍摄的,拍摄画面a与拍摄画面c不同。It should be noted that, since the shooting angles of mobile phone 0 and TV 1 are different, the picture shot by mobile phone 0 may be different from the picture shot by TV 1. For example, picture shot a shown in FIG7 is shot by mobile phone 0, and picture shot c shown in FIG9 is shot by TV 1, and picture shot a is different from picture shot c.

在上述任一种应用场景中,本端设备101都可以使用并控制远端设备102的摄像头,利用远端设备102的摄像头协助本端设备101完成拍摄任务、视频任务或者其他相关任务。当然,远端设备102也可以使用并控制本端设备101的摄像头,利用本端设备101的摄像头协助远端设备102完成上述任务。例如,如图6A所示,手机3也可以同时显示来自手机0的拍摄画面1以及来自手机3的拍摄画面3。In any of the above application scenarios, the local device 101 can use and control the camera of the remote device 102, and use the camera of the remote device 102 to assist the local device 101 in completing shooting tasks, video tasks or other related tasks. Of course, the remote device 102 can also use and control the camera of the local device 101, and use the camera of the local device 101 to assist the remote device 102 in completing the above tasks. For example, as shown in FIG6A, the mobile phone 3 can also simultaneously display the shooting picture 1 from the mobile phone 0 and the shooting picture 3 from the mobile phone 3.

在本申请实施例中,本端设备101和远端设备102可在分布式拍摄场景下进行拍摄(如多景拍摄)。以下实施例中,以本端设备101是手机为例,详细介绍本申请实施例的方法。In the embodiment of the present application, the local device 101 and the remote device 102 can perform shooting in a distributed shooting scene (such as multi-view shooting). In the following embodiment, the method of the embodiment of the present application is described in detail by taking the local device 101 as a mobile phone as an example.

例如,如上述实施例所示,本端设备101(如手机)可提供双景模式(包括双景拍照模式和双景录像模式)。如图10所示,手机可以在相机应用的预览界面中设置用于跨设备多景拍摄的功能按钮1001和功能按钮1002。具体的,该功能按钮1001用于触发手机进入多景拍照模式。该功能按钮1002用于触发手机进入多景录像模式。当用户希望在手机上看到手机与远端设备1002拍摄的多景画面时,可点击功能按钮1001或功能按钮1002开启跨设备多景拍摄功能。本申请实施例中,用户对功能按钮1001、功能按钮1002或功能按钮303等的点击操作是第二操作。For example, as shown in the above embodiment, the local device 101 (such as a mobile phone) can provide a dual-view mode (including a dual-view photo mode and a dual-view video recording mode). As shown in Figure 10, the mobile phone can set a function button 1001 and a function button 1002 for cross-device multi-view shooting in the preview interface of the camera application. Specifically, the function button 1001 is used to trigger the mobile phone to enter the multi-view photo mode. The function button 1002 is used to trigger the mobile phone to enter the multi-view video recording mode. When the user wants to see the multi-view pictures taken by the mobile phone and the remote device 1002 on the mobile phone, the function button 1001 or the function button 1002 can be clicked to turn on the cross-device multi-view shooting function. In the embodiment of the present application, the user's click operation on the function button 1001, the function button 1002 or the function button 303 is the second operation.

示例性的,手机检测到用户点击上述功能按钮1001后,手机的预设应用(如相机应用)可以触发手机搜索附近具有拍摄功能的一个或多个候选设备。例如,手机的预设应用可以通过Dv kit发现(即搜索)具有拍摄功能的一个或多个候选设备。For example, after the mobile phone detects that the user clicks the above function button 1001, the preset application of the mobile phone (such as the camera application) can trigger the mobile phone to search for one or more candidate devices with a shooting function nearby. For example, the preset application of the mobile phone can discover (i.e. search) one or more candidate devices with a shooting function through a Dv kit.

并且,如图11所示,手机可将搜索到的一个或多个候选设备显示在对话框1101中。例如,手机可在服务器中查询与手机登录同一账号且具有拍摄功能的电子设备,并将查询到的电子设备作为候选设备显示在对话框1101中。本申请实施例中,用户对对话框1101和对话框401(也称为候选设备列表)中候选设备的点击操作是第一操作。Furthermore, as shown in FIG11 , the mobile phone may display one or more searched candidate devices in a dialog box 1101. For example, the mobile phone may query the server for electronic devices that are logged into the same account as the mobile phone and have a camera function, and display the queried electronic devices as candidate devices in the dialog box 1101. In the embodiment of the present application, the user's click operation on the candidate devices in the dialog box 1101 and the dialog box 401 (also referred to as the candidate device list) is the first operation.

以对话框1101中的候选设备包括电视1102、电视1103以及手表1104举例,用户可在对话框1101中选择本次与手机协同实现同步拍摄功能的远端设备。例如,如果手机检测到用户选择对话框1101中的电视1102,说明用户希望使用手机和电视1102同时进行拍摄。此时,手机的Dv kit可将电视1102作为手机的远端设备与手机建立网络连接。For example, if the candidate devices in the dialog box 1101 include TV 1102, TV 1103, and watch 1104, the user can select the remote device that cooperates with the mobile phone to realize the synchronous shooting function in the dialog box 1101. For example, if the mobile phone detects that the user selects TV 1102 in the dialog box 1101, it means that the user wants to use the mobile phone and TV 1102 to shoot at the same time. At this time, the Dv kit of the mobile phone can use TV 1102 as the remote device of the mobile phone to establish a network connection with the mobile phone.

手机与电视1102建立网络连接后,电视1102可注册在手机的HAL。具体的,手机可基于该网络连接,从电视1102获取电视1102的拍摄能力参数;并按照电视1102的拍摄能力参数在HAL创建对应的DMSDP HAL。After the mobile phone establishes a network connection with the TV 1102, the TV 1102 can be registered in the HAL of the mobile phone. Specifically, the mobile phone can obtain the shooting capability parameters of the TV 1102 from the TV 1102 based on the network connection; and create a corresponding DMSDP HAL in the HAL according to the shooting capability parameters of the TV 1102.

其中,电视1002的拍摄能力参数用于反映电视1102的具体拍摄能力。该拍摄能力参数可以包括设备能力信息。该设备能力信息用于指示电视1102拍摄图像的硬件能力。例如,设备能力信息可以指示电视1102中摄像头的数目、摄像头的分辨率或图像处理器的型号等参数。该设备能力信息可用于手机确定电视1102的拍摄策略。The shooting capability parameter of the TV 1002 is used to reflect the specific shooting capability of the TV 1102. The shooting capability parameter may include device capability information. The device capability information is used to indicate the hardware capability of the TV 1102 for shooting images. For example, the device capability information may indicate parameters such as the number of cameras in the TV 1102, the resolution of the cameras, or the model of the image processor. The device capability information may be used by the mobile phone to determine the shooting strategy of the TV 1102.

上述电视1102的拍摄能力参数可以保存在手机的Camera Service中。示例性的,电视1102的拍摄能力参数可以以标签(Tag)的方式,保存在手机的Camera Service中。The shooting capability parameters of the TV 1102 can be stored in the Camera Service of the mobile phone. For example, the shooting capability parameters of the TV 1102 can be stored in the Camera Service of the mobile phone in the form of tags.

本申请实施例这里以设备能力信息为例,介绍电视1102的拍摄能力参数在手机的Camera Service中的存储格式。例如,请参考图12,其示出本申请实施例中,手机的CameraService中保存的电视1102的设备能力信息的一种格式示意图。In this embodiment of the present application, the device capability information is used as an example to introduce the storage format of the shooting capability parameters of the TV 1102 in the Camera Service of the mobile phone. For example, please refer to Figure 12, which shows a format diagram of the device capability information of the TV 1102 stored in the Camera Service of the mobile phone in the embodiment of the present application.

手机的Camera Service中可以采用图12所示的三层数据结构,保存电视1102的设备能力信息。其中,手机可以在Camera Service中分段存储电视1102的每种设备能力信息。图12所示的Section_name为电视1102的存储地址;Tag_name为Section_name的一个地址下多个Tag名称;Tag_index为Tag名称的能力信息的索引。该索引用于指示具体能力信息的存储地址。The mobile phone's Camera Service can use the three-layer data structure shown in FIG12 to save the device capability information of the TV 1102. The mobile phone can store each device capability information of the TV 1102 in segments in the Camera Service. Section_name shown in FIG12 is the storage address of the TV 1102; Tag_name is a plurality of tag names under one address of Section_name; Tag_index is the index of the capability information of the tag name. The index is used to indicate the storage address of the specific capability information.

例如,电视1102的设备能力信息可以包括图12所示的5个Section_name:com.huawei.capture.metadata;com.huawei.device.capabilities;android.huawei.device.parameters;android.huawei.stream.info;android.huawei.stream.parameters。For example, the device capability information of the TV 1102 may include the five Section_names shown in FIG. 12 : com.huawei.capture.metadata; com.huawei.device.capabilities; android.huawei.device.parameters; android.huawei.stream.info; and android.huawei.stream.parameters.

以图12所示的一个Section_name(如com.huawei.device.capabilities)为例。如图12所示,电视1102的设备能力信息还可以包括:该Section_name对应的存储地址中保存的多个Tag的Tag名称,如device_sensor_position、hidden_camera_id、colerBarCheckUnsupport、amoothZoomSupport和tofType等。例如,device_sensor_position代表电视1102的摄像头的传感器。Take a Section_name (such as com.huawei.device.capabilities) as shown in FIG12 as an example. As shown in FIG12, the device capability information of the TV 1102 may also include: Tag names of multiple Tags stored in the storage address corresponding to the Section_name, such as device_sensor_position, hidden_camera_id, colorBarCheckUnsupport, amoothZoomSupport, and tofType, etc. For example, device_sensor_position represents the sensor of the camera of the TV 1102.

以图12所示的一个Tag名称为例。如图12所示,电视1102的设备能力信息还可以包括:该Tag名称的能力信息的索引,如CAMERA_HUAWEI_DEVICE_CAPABILITIES_START和CAMERA_HUAWEI_DEVICE_CAPABILITIES_END。Take a Tag name as shown in Figure 12 as an example. As shown in Figure 12, the device capability information of the TV 1102 may further include: an index of the capability information of the Tag name, such as CAMERA_HUAWEI_DEVICE_CAPABILITIES_START and CAMERA_HUAWEI_DEVICE_CAPABILITIES_END.

其中,CAMERA_HUAWEI_DEVICE_CAPABILITIES_START可指示电视1102中存储一个Tag名称的能力信息的起始地址。CAMERA_HUAWEI_DEVICE_CAPABILITIES_END可指示电视1102中存储一个Tag名称的能力信息的结束地址。根据该起始地址和结束地址,手机便可以向电视1102查询电视1102的各种设备能力。Among them, CAMERA_HUAWEI_DEVICE_CAPABILITIES_START may indicate the starting address of capability information of a tag name stored in the TV 1102. CAMERA_HUAWEI_DEVICE_CAPABILITIES_END may indicate the ending address of capability information of a tag name stored in the TV 1102. According to the starting address and the ending address, the mobile phone can query the TV 1102 for various device capabilities of the TV 1102.

应理解,如图13所示,保存在Camera Service中的TAG(即上述拍摄能力参数),可结合手机中的后处理(算法)模块,对电视1102返回的RAW图进行处理。其中,手机根据上述TAG,结合后处理(算法)模块对电视1102返回的RAW图进行处理的具体方法,在以下实施例中详细介绍,此处不予赘述。It should be understood that, as shown in FIG13 , the TAG (i.e., the above-mentioned shooting capability parameter) stored in the Camera Service can be combined with the post-processing (algorithm) module in the mobile phone to process the RAW image returned by the TV 1102. The specific method in which the mobile phone processes the RAW image returned by the TV 1102 according to the above-mentioned TAG in combination with the post-processing (algorithm) module is described in detail in the following embodiments and will not be described in detail here.

后续,手机在运行相机应用时,一方面可调用自身的摄像头获取每一帧拍摄画面,另一方面,手机可指示电视1102获取RAW图像,并将获取到的RAW图像通过DMSDP HAL 202b发送给手机的相机应用。其中,手机可以按照一定的帧率调用自身的摄像头获取每一帧拍摄画面,并指示电视1102按照同样的帧率获取RAW图像。Subsequently, when the mobile phone runs the camera application, on the one hand, it can call its own camera to obtain each frame of the captured image, and on the other hand, the mobile phone can instruct the TV 1102 to obtain the RAW image, and send the obtained RAW image to the camera application of the mobile phone through the DMSDP HAL 202b. Among them, the mobile phone can call its own camera to obtain each frame of the captured image at a certain frame rate, and instruct the TV 1102 to obtain the RAW image at the same frame rate.

其中,手机可通过DMSDP HAL向电视1102发送拍摄指令,以指示电视1102获取RAW图像。例如,如图13所示,手机的应用层200可通过框架层201和服务层201a向HAL 202的DMSDP HAL 202b传输拍摄指令;HAL 202的DMSDP HAL 202b可执行步骤a向电视1102传输拍摄指令。电视1102接收到手机通过DMSDP HAL 202b传输的拍摄指令后,应用层210的相机代理服务210a可在该拍摄指令中增加预设标记。该预设标记用于指示直接获取底层的Camera器件采集的RAW数据,并将该RAW数据透传至应用层211的相机代理服务210a。例如,如图13所示,应用层210的相机代理服务210a可在该拍摄指令中增加预设标记,并通过框架层211、服务层211a向HAL 212中的Camera HAL 212a传输添加了预设标记的拍摄指令。HAL 212中的Camera HAL 212a接收到该拍摄指令后,可调用内核层213的Camera器件执行该拍摄指令。例如,HAL 212中的Camera HAL 212a可调用内核层213中的Camera器件采集RAW图像,并执行步骤b向将该RAW数据透传至应用层211的相机代理服务210a。然后,由应用层211的相机代理服务210a执行步骤c,向手机的DMSDP HAL 202b传输该RAW图像。Among them, the mobile phone can send a shooting instruction to the TV 1102 through the DMSDP HAL to instruct the TV 1102 to obtain a RAW image. For example, as shown in FIG13, the application layer 200 of the mobile phone can transmit the shooting instruction to the DMSDP HAL 202b of the HAL 202 through the framework layer 201 and the service layer 201a; the DMSDP HAL 202b of the HAL 202 can execute step a to transmit the shooting instruction to the TV 1102. After the TV 1102 receives the shooting instruction transmitted by the mobile phone through the DMSDP HAL 202b, the camera proxy service 210a of the application layer 210 can add a preset mark to the shooting instruction. The preset mark is used to indicate that the RAW data collected by the underlying Camera device is directly obtained, and the RAW data is transparently transmitted to the camera proxy service 210a of the application layer 211. For example, as shown in FIG13 , the camera proxy service 210a of the application layer 210 may add a preset mark to the shooting instruction, and transmit the shooting instruction with the preset mark added to the Camera HAL 212a in the HAL 212 through the framework layer 211 and the service layer 211a. After receiving the shooting instruction, the Camera HAL 212a in the HAL 212 may call the Camera device in the kernel layer 213 to execute the shooting instruction. For example, the Camera HAL 212a in the HAL 212 may call the Camera device in the kernel layer 213 to collect a RAW image, and execute step b to transparently transmit the RAW data to the camera proxy service 210a of the application layer 211. Then, the camera proxy service 210a of the application layer 211 executes step c to transmit the RAW image to the DMSDP HAL 202b of the mobile phone.

其中,内核层213的Camera器件可以包括图14所示的感光器件、数字信号处理器(DSP)和图像处理器(ISP)。该感光器件可以包括镜头和传感器(Sensor)等。该感光器件用于采集RAW图像。数字信号处理器(DSP)用于对感光器件采集的多帧RAW图像进行抽样处理;然后,向图像处理器(ISP)传输抽样后的RAW图像。一般而言,图像处理器(ISP)可以对来自DSP的RAW图像进行图像处理。但是,本申请实施例中,ISP不会对来自DSP的RAW图像进行图像处理,而是将来自DSP的RAW图像透传至HAL 212中的Camera HAL 212a。Among them, the Camera device of the kernel layer 213 may include a photosensitive device, a digital signal processor (DSP) and an image processor (ISP) as shown in Figure 14. The photosensitive device may include a lens and a sensor (Sensor), etc. The photosensitive device is used to collect RAW images. The digital signal processor (DSP) is used to sample and process multiple frames of RAW images collected by the photosensitive device; then, the sampled RAW images are transmitted to the image processor (ISP). Generally speaking, the image processor (ISP) can perform image processing on the RAW images from the DSP. However, in an embodiment of the present application, the ISP does not perform image processing on the RAW images from the DSP, but instead transmits the RAW images from the DSP to the Camera HAL 212a in the HAL 212.

需要说明的是,图13所述步骤b直接由内核层213指向应用层210,只是为了说明手机不会对RAW图像进行图像处理,并不表示该RAW图像不经过HAL 212中的Camera HAL212a、服务层211a和框架层211便可以直接由内核层213传输至应用层210。内核层213中的Camera器件采集RAW图像是需要通过HAL 212中的Camera HAL 212a、服务层211a和框架层211传输至应用层210的。但是,如图14所示,HAL 212中的Camera HAL 212a、服务层211a和框架层211只可以透传该RAW图像,而不会对该RAW图像进行任何处理。It should be noted that the step b in FIG. 13 is directed directly from the kernel layer 213 to the application layer 210 only to illustrate that the mobile phone will not perform image processing on the RAW image, and does not mean that the RAW image can be directly transmitted from the kernel layer 213 to the application layer 210 without passing through the Camera HAL 212a, the service layer 211a and the framework layer 211 in the HAL 212. The Camera device in the kernel layer 213 needs to collect the RAW image and transmit it to the application layer 210 through the Camera HAL 212a, the service layer 211a and the framework layer 211 in the HAL 212. However, as shown in FIG. 14, the Camera HAL 212a, the service layer 211a and the framework layer 211 in the HAL 212 can only pass through the RAW image, and will not perform any processing on the RAW image.

在本申请实施例中,电视1102向手机传输的并不是经过处理的、可直接显示的拍摄画面,而是未经过图像处理的RAW图像。电视1102使用摄像头采集到RAW图像后,不需要对RAW图像进行图像处理,便可以直接向手机传输该RAW图像。该RAW图像是未经过电视1102处理的图像。相比于经过电视1102处理的图像,RAW图像的数据量较小。如此,便可以减少跨设备的分布式拍摄业务所占用的网络带宽。In the embodiment of the present application, what the TV 1102 transmits to the mobile phone is not a processed, directly displayable shooting picture, but a RAW image that has not been image processed. After the TV 1102 uses the camera to capture the RAW image, it does not need to process the RAW image, and can directly transmit the RAW image to the mobile phone. The RAW image is an image that has not been processed by the TV 1102. Compared with the image processed by the TV 1102, the data volume of the RAW image is smaller. In this way, the network bandwidth occupied by the distributed shooting service across devices can be reduced.

然后,手机可对来自电视1102的RAW图像进行图像处理,得到对应的拍摄画面。这样,手机的相机应用不仅可以获取到来自手机的每一帧拍摄画面,还可以获取到电视1102的每一帧拍摄画面。进而,相机应用可将手机的拍摄画面和电视1102的拍摄画面同步显示在相机应用的显示界面中,实现跨设备的分布式拍摄功能。例如,如图15所示,手机的相机应用的预览界面中包括手机的拍摄画面1501和电视1102的拍摄画面1502。Then, the mobile phone can process the RAW image from the TV 1102 to obtain the corresponding shooting picture. In this way, the camera application of the mobile phone can not only obtain each frame of the shooting picture from the mobile phone, but also obtain each frame of the shooting picture of the TV 1102. Furthermore, the camera application can synchronously display the shooting picture of the mobile phone and the shooting picture of the TV 1102 in the display interface of the camera application, realizing the distributed shooting function across devices. For example, as shown in FIG. 15, the preview interface of the camera application of the mobile phone includes the shooting picture 1501 of the mobile phone and the shooting picture 1502 of the TV 1102.

其中,由手机对来自电视1102的RAW图像进行处理,可以避免由于两端设备性能差异较大而增大时延的问题。采用该方案,可以减少分布式拍摄业务的时延。The mobile phone processes the RAW image from the TV 1102, thereby avoiding the problem of increased latency due to the large performance difference of the devices at both ends. This solution can reduce the latency of the distributed shooting service.

并且,本申请实施例中,手机与电视1102建立连接后,可基于图13所示的DMSDPHAL 202b,创建两个相机会话(Camera Session),如用于传输控制流(如上述拍摄指令)的控制会话(Control Session)和用于传输数据流(如上述RAW数据)的数据会话(DataSession)。其中,Control Session和Data Session可对应不同的传输通路(或者称为传输管道)。手机的DMSDP HAL 202b执行图13所示的步骤a,可通过上述Control Session的传输管道向电视1102的相机代理服务210a传输拍摄指令。电视1102的相机代理服务210a执行图13所示的步骤c,可通过上述Data Session的传输管道向手机的DMSDP HAL 202b传输数据流(如上述RAW数据)。Moreover, in the embodiment of the present application, after the mobile phone establishes a connection with the TV 1102, two camera sessions (Camera Session) may be created based on the DMSDPHAL 202b shown in FIG. 13, such as a control session (Control Session) for transmitting a control stream (such as the above-mentioned shooting instruction) and a data session (DataSession) for transmitting a data stream (such as the above-mentioned RAW data). Among them, the Control Session and the Data Session may correspond to different transmission paths (or transmission pipelines). The DMSDP HAL 202b of the mobile phone executes step a shown in FIG. 13, and may transmit the shooting instruction to the camera proxy service 210a of the TV 1102 through the transmission pipeline of the above-mentioned Control Session. The camera proxy service 210a of the TV 1102 executes step c shown in FIG. 13, and may transmit the data stream (such as the above-mentioned RAW data) to the DMSDP HAL 202b of the mobile phone through the transmission pipeline of the above-mentioned Data Session.

其中,手机与电视1102执行跨设备的分布式拍摄业务的过程中,可以区分控制流与数据流的传输通路。这样,可以避免控制流与数据流相互挤占带宽的问题。In the process of executing the cross-device distributed shooting service, the mobile phone and the TV 1102 can distinguish the transmission paths of the control stream and the data stream, so as to avoid the problem of the control stream and the data stream occupying each other's bandwidth.

在一些实施例中,上述电视1102的拍摄能力参数还可以包括算法能力信息。该算法能力信息用于指示电视1102对拍摄的图像进行图像处理的算法能力。例如,算法能力信息可以指示电视1102支持的人脸识别算法、自动对焦算法等一项或多项图像处理算法。也就是说,手机还可以将电视1002的后处理算法同步至手机中。例如,如图13所示,手机的服务层201a中保留了电视机1102的后处理算法。In some embodiments, the shooting capability parameters of the TV 1102 may also include algorithm capability information. The algorithm capability information is used to indicate the algorithm capability of the TV 1102 to process the captured images. For example, the algorithm capability information may indicate one or more image processing algorithms such as face recognition algorithm and autofocus algorithm supported by the TV 1102. In other words, the mobile phone may also synchronize the post-processing algorithm of the TV 1102 to the mobile phone. For example, as shown in FIG13 , the post-processing algorithm of the TV 1102 is retained in the service layer 201a of the mobile phone.

在该实施例中,手机的服务层201a可基于电视1102的算法能力信息,调用保留的电视机1102的后处理算法,对来自DMSDP HAL 202b的RAW图像进行图像处理。之后,手机可基于时间戳,对手机采集的图像和来自电视1102经过后处理的图像进行同步渲染,得到并显示图15所示的手机的拍摄画面1501和电视1102的拍摄画面1502。In this embodiment, the service layer 201a of the mobile phone can call the retained post-processing algorithm of the TV 1102 to process the RAW image from the DMSDP HAL 202b based on the algorithm capability information of the TV 1102. Afterwards, the mobile phone can synchronously render the image captured by the mobile phone and the post-processed image from the TV 1102 based on the timestamp, and obtain and display the shooting picture 1501 of the mobile phone and the shooting picture 1502 of the TV 1102 shown in FIG. 15.

需要说明的是,手机基于电视1102的算法能力信息,调用保留的电视机1102的后处理算法对电视1102采集的RAW图像进行图像处理,可得到与电视1102侧对该RAW图像进行图像处理相同或相似的效果。It should be noted that, based on the algorithm capability information of TV 1102, the mobile phone calls the retained post-processing algorithm of TV 1102 to process the RAW image captured by TV 1102, and can obtain the same or similar effect as the image processing of the RAW image by TV 1102.

其中,即使电视1102采集的RAW图像是在手机侧进行图像处理的;但是,经过图像处理的图像也可以达到与电视1102侧进行图像处理的相同效果。也就是说,采用本申请实施例的方法,不仅可以减少跨设备的分布式拍摄业务所占用的网络带宽,还可以还原远端设备的图像效果。Even if the RAW image captured by the TV 1102 is processed on the mobile phone side, the processed image can achieve the same effect as the image processing on the TV 1102 side. That is, the method of the embodiment of the present application can not only reduce the network bandwidth occupied by the distributed shooting service across devices, but also restore the image effect of the remote device.

在另一些实施例中,上述电视1102的拍摄能力参数可以不包括算法能力信息。也就是说,手机可以不用将电视1002的后处理算法同步至手机中。例如,如图16所示,手机的服务层201a中未保留电视机1102的后处理算法。In other embodiments, the shooting capability parameters of the TV 1102 may not include algorithm capability information. In other words, the mobile phone may not synchronize the post-processing algorithm of the TV 1102 to the mobile phone. For example, as shown in FIG16 , the post-processing algorithm of the TV 1102 is not retained in the service layer 201a of the mobile phone.

在该实施例中,手机的服务层201a可基于手机的算法能力信息,调用手机的后处理算法,对来自DMSDP HAL 202b的RAW图像进行图像处理。之后,手机可基于时间戳,对手机采集的图像和来自电视1102经过后处理的图像进行同步渲染,得到并显示图15所示的手机的拍摄画面1501和电视1102的拍摄画面1502。In this embodiment, the service layer 201a of the mobile phone can call the post-processing algorithm of the mobile phone based on the algorithm capability information of the mobile phone to perform image processing on the RAW image from the DMSDP HAL 202b. Afterwards, the mobile phone can synchronously render the image captured by the mobile phone and the post-processed image from the TV 1102 based on the timestamp, and obtain and display the shooting picture 1501 of the mobile phone and the shooting picture 1502 of the TV 1102 shown in FIG. 15.

需要说明的是,手机调用手机的后处理算法对电视1102采集的RAW图像进行图像处理,虽然无法还原电视1102侧进行图像处理的效果;但是,针对手机采集的图像与电视1102采集的RAW图像,均调用手机的后处理算法进行图像处理,可以使手机的拍摄画面与电视1102的拍摄画面的图像效果一致。It should be noted that the mobile phone calls the post-processing algorithm of the mobile phone to perform image processing on the RAW images captured by the TV 1102, although it is impossible to restore the effect of image processing on the TV 1102 side; however, the post-processing algorithm of the mobile phone is called to perform image processing on both the images captured by the mobile phone and the RAW images captured by the TV 1102, so that the image effects of the pictures taken by the mobile phone and the pictures taken by the TV 1102 can be consistent.

并且,在手机的算法处理能力优于电视1102的算法处理能力的情况下,相比于调用电视1102的后处理算法对电视1102采集的RAW图像进行图像处理,调用手机的后处理算法对该RAW图像进行图像处理,还可以提升手机显示的电视1102的拍摄画面的图像效果。Moreover, when the algorithm processing capability of the mobile phone is better than that of the TV 1102, compared with calling the post-processing algorithm of the TV 1102 to perform image processing on the RAW image captured by the TV 1102, calling the post-processing algorithm of the mobile phone to perform image processing on the RAW image can also improve the image effect of the captured picture of the TV 1102 displayed on the mobile phone.

进一步的,使用手机的后处理算法对电视1102采集的RAW图像进行图像处理,手机中则不需要保留电视1102的后处理算法。这样,可以节省手机的存储空间。Furthermore, the post-processing algorithm of the mobile phone is used to process the RAW image collected by the television 1102, and the mobile phone does not need to retain the post-processing algorithm of the television 1102. In this way, the storage space of the mobile phone can be saved.

另外,上述实施例中是以手机为分布式拍摄场景中的本端设备举例说明的,可以理解的是,分布式拍摄场景中的本端设备还可以是平板电脑、电视等具有上述拍摄功能的电子设备,本申请实施例对此不做任何限制。In addition, the above embodiments are illustrated by taking a mobile phone as the local device in a distributed shooting scenario. It can be understood that the local device in the distributed shooting scenario can also be a tablet computer, a television, or other electronic device with the above-mentioned shooting function, and the embodiments of the present application do not impose any restrictions on this.

需要说明的是,上述实施例中是以Android系统为例阐述的各个功能模块之间实现分布式拍摄功能的具体方法,可以理解的是,也可以在其他操作系统中设置相应的功能模块实现上述方法。只要各个设备和功能模块实现的功能和本申请的实施例类似,即属于本申请权利要求及其等同技术的范围之内。It should be noted that the above embodiment uses the Android system as an example to illustrate the specific method for implementing the distributed shooting function between the various functional modules. It can be understood that corresponding functional modules can also be set in other operating systems to implement the above method. As long as the functions implemented by each device and functional module are similar to the embodiments of the present application, they fall within the scope of the claims of the present application and their equivalent technologies.

本申请实施例中,本端设备101采集的图像数据是第一图像数据,本端设备101的拍摄画面是第一拍摄画面,该第一拍摄画面显示在第一窗口。远端设备102采集的图像数据是第二图像数据,远端设备102的拍摄画面是第二拍摄画面,该第二拍摄画面显示在第二窗口。上述第二图像数据包括RAW图像。In the embodiment of the present application, the image data collected by the local device 101 is the first image data, the shooting screen of the local device 101 is the first shooting screen, and the first shooting screen is displayed in the first window. The image data collected by the remote device 102 is the second image data, the shooting screen of the remote device 102 is the second shooting screen, and the second shooting screen is displayed in the second window. The above-mentioned second image data includes a RAW image.

例如,第一拍摄画面可以是图5所示的拍摄画面1,第二拍摄画面可以是图5所示的拍摄画面2。图5所示的拍摄画面1和拍摄画面2显示在同一显示界面。For example, the first shooting picture may be the shooting picture 1 shown in Fig. 5, and the second shooting picture may be the shooting picture 2 shown in Fig. 5. The shooting picture 1 and the shooting picture 2 shown in Fig. 5 are displayed on the same display interface.

又例如,第一拍摄画面可以是图6A所示的拍摄画面1,第二拍摄画面可以是图6A所示的拍摄画面3。图6A所示的拍摄画面1和拍摄画面2,在手机0和手机1均显示在同一显示界面。For another example, the first shooting picture may be shooting picture 1 shown in Fig. 6A, and the second shooting picture may be shooting picture 3 shown in Fig. 6A. Shooting picture 1 and shooting picture 2 shown in Fig. 6A are displayed on the same display interface of mobile phone 0 and mobile phone 1.

本申请另一些实施例提供了一种电子设备,该电子设备可以包括:上述触摸屏、存储器和一个或多个处理器。该触摸屏、存储器和处理器耦合。该存储器用于存储计算机程序代码,该计算机程序代码包括计算机指令。当处理器执行计算机指令时,电子设备可执行上述方法实施例中手机执行的各个功能或者步骤。该电子设备的结构可以参考图2B所示的手机的结构。Some other embodiments of the present application provide an electronic device, which may include: the above-mentioned touch screen, a memory and one or more processors. The touch screen, the memory and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can execute the various functions or steps executed by the mobile phone in the above-mentioned method embodiment. The structure of the electronic device can refer to the structure of the mobile phone shown in Figure 2B.

本申请实施例还提供一种芯片系统,如图17所示,该芯片系统1700包括至少一个处理器1701和至少一个接口电路1702。处理器1701和接口电路1702可通过线路互联。例如,接口电路1702可用于从其它装置(例如电子设备的存储器)接收信号。又例如,接口电路1702可用于向其它装置(例如处理器1701)发送信号。示例性的,接口电路1702可读取存储器中存储的指令,并将该指令发送给处理器1701。当所述指令被处理器1701执行时,可使得电子设备执行上述实施例中的各个步骤。当然,该芯片系统还可以包含其他分立器件,本申请实施例对此不作具体限定。The embodiment of the present application also provides a chip system, as shown in Figure 17, the chip system 1700 includes at least one processor 1701 and at least one interface circuit 1702. The processor 1701 and the interface circuit 1702 can be interconnected by lines. For example, the interface circuit 1702 can be used to receive signals from other devices (such as the memory of the electronic device). For another example, the interface circuit 1702 can be used to send signals to other devices (such as the processor 1701). Exemplarily, the interface circuit 1702 can read the instructions stored in the memory and send the instructions to the processor 1701. When the instructions are executed by the processor 1701, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

本申请实施例还提供一种计算机存储介质,该计算机存储介质包括计算机指令,当所述计算机指令在上述电子设备上运行时,使得该电子设备执行上述方法实施例中手机执行的各个功能或者步骤。An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the mobile phone in the above-mentioned method embodiment.

本申请实施例还提供一种计算机程序产品,当所述计算机程序产品在计算机上运行时,使得所述计算机执行上述方法实施例中手机执行的各个功能或者步骤。The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the mobile phone in the above method embodiment.

通过以上实施方式的描述,所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

在本申请所提供的几个实施例中,应该理解到,所揭露的装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述模块或单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个装置,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是一个物理单元或多个物理单元,即可以位于一个地方,或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个可读取存储介质中。基于这样的理解,本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该软件产品存储在一个存储介质中,包括若干指令用以使得一个设备(可以是单片机,芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(read only memory,ROM)、随机存取存储器(random access memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

以上内容,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何在本申请揭露的技术范围内的变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (6)

1.一种分布式拍摄方法,其特征在于,包括:1. A distributed shooting method, comprising: 第一设备接收用户选择与第二设备进行同步拍摄的第一操作;The first device receives a first operation of a user selecting to perform synchronous shooting with the second device; 响应于所述第一操作,所述第一设备开始采集第一图像数据,并且,所述第一设备指示所述第二设备开始采集图像数据;In response to the first operation, the first device starts to collect first image data, and the first device instructs the second device to start collecting image data; 所述第一设备接收来自所述第二设备的第二图像数据,所述第二图像数据包括所述第二设备的摄像头采集的原始图像;The first device receives second image data from the second device, where the second image data includes an original image captured by a camera of the second device; 所述第一设备将所述第一图像数据对应的第一拍摄画面显示在第一窗口,将所述第二图像数据对应的第二拍摄画面显示在第二窗口,所述第一窗口和所述第二窗口位于同一显示界面;The first device displays a first captured image corresponding to the first image data in a first window, and displays a second captured image corresponding to the second image data in a second window, wherein the first window and the second window are located on the same display interface; 在第一设备接收用户选择与第二设备进行同步拍摄的第一操作之后,所述方法还包括:After the first device receives a first operation of the user selecting to perform synchronous shooting with the second device, the method further includes: 所述第一设备创建所述第一设备与所述第二设备的控制会话和数据会话;The first device establishes a control session and a data session between the first device and the second device; 其中,所述控制会话用于传输所述第一设备与所述第二设备之间的控制命令,所述控制命令包括用于指示所述第二设备采集图像的拍摄指令,所述数据会话用于传输所述来自所述第二设备的第二图像数据;The control session is used to transmit a control command between the first device and the second device, the control command includes a shooting instruction for instructing the second device to capture an image, and the data session is used to transmit the second image data from the second device; 在第一设备接收用户选择与第二设备进行同步拍摄的第一操作之后,所述方法还包括:After the first device receives a first operation of the user selecting to perform synchronous shooting with the second device, the method further includes: 所述第一设备获取所述第二设备的拍摄能力参数,所述第二设备的拍摄能力参数用于指示所述第二设备支持的图像处理算法;The first device acquires a shooting capability parameter of the second device, where the shooting capability parameter of the second device is used to indicate an image processing algorithm supported by the second device; 其中,在所述第一设备将所述第一图像数据对应的第一拍摄画面显示在第一窗口,将所述第二图像数据对应的第二拍摄画面显示在第二窗口之前,所述方法还包括:Wherein, before the first device displays the first captured picture corresponding to the first image data in the first window and displays the second captured picture corresponding to the second image data in the second window, the method further includes: 所述第一设备根据所述第一设备的拍摄能力参数,对所述第一图像数据进行图像处理,得到所述第一拍摄画面,所述第一设备的拍摄能力参数用于指示所述第一设备支持的图像处理算法;The first device performs image processing on the first image data according to a shooting capability parameter of the first device to obtain the first shooting picture, where the shooting capability parameter of the first device is used to indicate an image processing algorithm supported by the first device; 所述第一设备根据所述第二设备的拍摄能力参数,对所述第二图像数据进行图像处理,得到所述第二拍摄画面;The first device performs image processing on the second image data according to the shooting capability parameter of the second device to obtain the second shooting picture; 所述方法还包括:所述第一设备向所述第二设备发送所述第一图像数据,所述第一图像数据包括所述第一设备的摄像头采集的原始图像。The method also includes: the first device sending the first image data to the second device, where the first image data includes an original image captured by a camera of the first device. 2.根据权利要求1所述的方法,其特征在于,所述第一设备指示所述第二设备开始采集图像数据,包括:2. The method according to claim 1, wherein the first device instructs the second device to start collecting image data, comprising: 所述第一设备向所述第二设备发送拍摄指令;The first device sends a shooting instruction to the second device; 其中,所述拍摄指令中包括预设标记,所述拍摄指令用于指示所述第二设备采集图像数据,所述预设标记用于指示所述第二设备向所述第一设备传输所述第二设备采集的RAW图像。The shooting instruction includes a preset mark, the shooting instruction is used to instruct the second device to collect image data, and the preset mark is used to instruct the second device to transmit the RAW image collected by the second device to the first device. 3.根据权利要求1或2所述的方法,其特征在于,在第一设备接收用户选择与第二设备进行同步拍摄的第一操作之前,所述方法还包括:3. The method according to claim 1 or 2, characterized in that before the first device receives a first operation of the user selecting to perform synchronous shooting with the second device, the method further comprises: 所述第一设备接收第二操作,所述第二操作是用户对用于实现分布式拍摄的功能按钮的点击操作,所述第一设备的相机应用的预览界面、所述第一设备的视频通信应用的聊天界面、所述第一设备的控制中心、下拉菜单或者负一屏中的至少一项包括所述功能按钮;The first device receives a second operation, where the second operation is a user clicking operation on a function button for implementing distributed shooting, and at least one of a preview interface of a camera application of the first device, a chat interface of a video communication application of the first device, a control center of the first device, a drop-down menu, or a negative first screen includes the function button; 响应于用于点击所述功能按钮的第二操作,所述第一设备在所述预览界面中显示候选设备列表,所述候选设备列表中包括所述第二设备;In response to a second operation for clicking the function button, the first device displays a candidate device list in the preview interface, the candidate device list including the second device; 其中,所述第一操作为用户在所述候选设备列表中选择所述第二设备的操作。The first operation is an operation in which a user selects the second device in the candidate device list. 4.根据权利要求1-3中任一项所述的方法,其特征在于,所述同步拍摄包括同步录像、同步拍照、同步直播或者同步视频通话中的至少一种。4. The method according to any one of claims 1-3 is characterized in that the synchronous shooting includes at least one of synchronous video recording, synchronous photo taking, synchronous live broadcast or synchronous video calling. 5.一种电子设备,其特征在于,所述电子设备为第一设备,所述电子设备包括:一个或多个摄像头、一个或多个处理器、显示屏、存储器和通信模块;所述摄像头、所述显示屏、所述存储器、所述通信模块和所述处理器耦合;5. An electronic device, characterized in that the electronic device is a first device, and the electronic device comprises: one or more cameras, one or more processors, a display screen, a memory, and a communication module; the camera, the display screen, the memory, the communication module, and the processor are coupled; 其中,所述存储器用于存储计算机程序代码,所述计算机程序代码包括计算机指令,当所述计算机指令被所述电子设备执行时,使得所述电子设备执行如权利要求1-4中任一项所述的方法。The memory is used to store computer program codes, and the computer program codes include computer instructions. When the computer instructions are executed by the electronic device, the electronic device executes the method as described in any one of claims 1 to 4. 6.一种计算机可读存储介质,所述计算机可读存储介质中存储有指令,其特征在于,当所述指令在电子设备上运行时,使得所述电子设备执行如权利要求1-4中任一项所述的方法。6. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, wherein when the instructions are executed on an electronic device, the electronic device executes the method as claimed in any one of claims 1 to 4.
CN202110131870.2A 2021-01-30 2021-01-30 Distributed shooting method, electronic device and medium Active CN114845035B (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202110131870.2A CN114845035B (en) 2021-01-30 2021-01-30 Distributed shooting method, electronic device and medium
CN202210973275.8A CN115514882B (en) 2021-01-30 2021-01-30 Distributed shooting method, electronic equipment and medium
PCT/CN2021/137917 WO2022160985A1 (en) 2021-01-30 2021-12-14 Distributed photographing method, electronic device, and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110131870.2A CN114845035B (en) 2021-01-30 2021-01-30 Distributed shooting method, electronic device and medium

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202210973275.8A Division CN115514882B (en) 2021-01-30 2021-01-30 Distributed shooting method, electronic equipment and medium

Publications (2)

Publication Number Publication Date
CN114845035A CN114845035A (en) 2022-08-02
CN114845035B true CN114845035B (en) 2024-04-26

Family

ID=82561398

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202210973275.8A Active CN115514882B (en) 2021-01-30 2021-01-30 Distributed shooting method, electronic equipment and medium
CN202110131870.2A Active CN114845035B (en) 2021-01-30 2021-01-30 Distributed shooting method, electronic device and medium

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202210973275.8A Active CN115514882B (en) 2021-01-30 2021-01-30 Distributed shooting method, electronic equipment and medium

Country Status (2)

Country Link
CN (2) CN115514882B (en)
WO (1) WO2022160985A1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116320783B (en) * 2022-09-14 2023-11-14 荣耀终端有限公司 A method and electronic device for capturing images during video recording
CN115714883A (en) * 2022-11-14 2023-02-24 惠州市德赛西威汽车电子股份有限公司 Vehicle-mounted content live broadcast method, system, transparent transmission service module and storage medium
CN115514898A (en) * 2022-11-17 2022-12-23 深圳开鸿数字产业发展有限公司 Photographing method, terminal device and storage medium
WO2024145063A1 (en) * 2022-12-29 2024-07-04 Meta Platforms, Inc. Methods, apparatuses and computer program products for providing virtual cameras for hardware inputs
CN117707242A (en) * 2023-07-11 2024-03-15 荣耀终端有限公司 Temperature control methods and related devices
CN118505574B (en) * 2023-09-13 2025-05-06 荣耀终端股份有限公司 Image processing method and electronic equipment
CN118102074B (en) * 2024-01-06 2025-04-04 珠海视熙科技有限公司 Camera control method, system, medium and terminal device of terminal device

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5796964A (en) * 1996-01-16 1998-08-18 International Business Machines Method for modifying an existing computer bus to enhance system performance
CN101404726A (en) * 2008-10-20 2009-04-08 深圳华为通信技术有限公司 Control method, system and apparatus for far-end camera
CN101446890A (en) * 2007-11-29 2009-06-03 株式会社瑞萨科技 Stream processing apparatus, method for stream processing and data processing system
CN103336677A (en) * 2013-06-25 2013-10-02 北京小米科技有限责任公司 Method, device and system for outputting images to display equipment
CN104284234A (en) * 2014-10-17 2015-01-14 惠州Tcl移动通信有限公司 Method and system for sharing synchronous images among plurality of terminals
CN105493621A (en) * 2014-08-04 2016-04-13 华为技术有限公司 Terminal, server, and terminal control method
WO2017107629A1 (en) * 2015-12-24 2017-06-29 努比亚技术有限公司 Mobile terminal, data transmission system and shooting method of mobile terminal
CN108718383A (en) * 2018-04-24 2018-10-30 天津字节跳动科技有限公司 Cooperate with image pickup method, device, storage medium and terminal device
CN109361869A (en) * 2018-11-28 2019-02-19 维沃移动通信(杭州)有限公司 A shooting method and terminal
CN110224804A (en) * 2018-03-01 2019-09-10 国民技术股份有限公司 Data transfer control method, terminal, base station and computer storage medium
CN110602805A (en) * 2019-09-30 2019-12-20 联想(北京)有限公司 Information processing method, first electronic device and computer system
WO2020057661A1 (en) * 2018-09-21 2020-03-26 华为技术有限公司 Image capturing method, device, and apparatus
CN111083379A (en) * 2019-12-31 2020-04-28 维沃移动通信(杭州)有限公司 Shooting method and electronic equipment
WO2020159154A1 (en) * 2018-02-08 2020-08-06 Samsung Electronics Co., Ltd. Method for encoding images and corresponding terminals
CN111788603A (en) * 2018-02-23 2020-10-16 三星电子株式会社 Electronic device and method for correcting an image corrected with a first image processing scheme in an external electronic device with a second image processing scheme
CN111860530A (en) * 2020-07-31 2020-10-30 Oppo广东移动通信有限公司 Electronic equipment, data processing method and related device
CN111988528A (en) * 2020-08-31 2020-11-24 北京字节跳动网络技术有限公司 Shooting method, shooting device, electronic equipment and computer-readable storage medium
CN112004076A (en) * 2020-08-18 2020-11-27 Oppo广东移动通信有限公司 Data processing method, control terminal, AR terminal, AR system and storage medium
CN112292847A (en) * 2018-06-25 2021-01-29 索尼公司 Image processing apparatus, mobile apparatus, method, and program

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013254432A (en) * 2012-06-08 2013-12-19 Canon Inc Image processing apparatus and image processing method
CN109769087A (en) * 2017-11-09 2019-05-17 中兴通讯股份有限公司 Image pickup method, device and the mobile terminal remotely taken a group photo
KR102482860B1 (en) * 2018-01-02 2022-12-30 삼성전자 주식회사 Method for processing image based on context information and electronic device using the same

Patent Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5796964A (en) * 1996-01-16 1998-08-18 International Business Machines Method for modifying an existing computer bus to enhance system performance
CN101446890A (en) * 2007-11-29 2009-06-03 株式会社瑞萨科技 Stream processing apparatus, method for stream processing and data processing system
CN101404726A (en) * 2008-10-20 2009-04-08 深圳华为通信技术有限公司 Control method, system and apparatus for far-end camera
CN103336677A (en) * 2013-06-25 2013-10-02 北京小米科技有限责任公司 Method, device and system for outputting images to display equipment
CN105493621A (en) * 2014-08-04 2016-04-13 华为技术有限公司 Terminal, server, and terminal control method
CN104284234A (en) * 2014-10-17 2015-01-14 惠州Tcl移动通信有限公司 Method and system for sharing synchronous images among plurality of terminals
WO2017107629A1 (en) * 2015-12-24 2017-06-29 努比亚技术有限公司 Mobile terminal, data transmission system and shooting method of mobile terminal
WO2020159154A1 (en) * 2018-02-08 2020-08-06 Samsung Electronics Co., Ltd. Method for encoding images and corresponding terminals
CN111788603A (en) * 2018-02-23 2020-10-16 三星电子株式会社 Electronic device and method for correcting an image corrected with a first image processing scheme in an external electronic device with a second image processing scheme
CN110224804A (en) * 2018-03-01 2019-09-10 国民技术股份有限公司 Data transfer control method, terminal, base station and computer storage medium
CN108718383A (en) * 2018-04-24 2018-10-30 天津字节跳动科技有限公司 Cooperate with image pickup method, device, storage medium and terminal device
CN112292847A (en) * 2018-06-25 2021-01-29 索尼公司 Image processing apparatus, mobile apparatus, method, and program
WO2020057661A1 (en) * 2018-09-21 2020-03-26 华为技术有限公司 Image capturing method, device, and apparatus
CN110944109A (en) * 2018-09-21 2020-03-31 华为技术有限公司 Photographing method, device and equipment
CN109361869A (en) * 2018-11-28 2019-02-19 维沃移动通信(杭州)有限公司 A shooting method and terminal
CN110602805A (en) * 2019-09-30 2019-12-20 联想(北京)有限公司 Information processing method, first electronic device and computer system
CN111083379A (en) * 2019-12-31 2020-04-28 维沃移动通信(杭州)有限公司 Shooting method and electronic equipment
CN111860530A (en) * 2020-07-31 2020-10-30 Oppo广东移动通信有限公司 Electronic equipment, data processing method and related device
CN112004076A (en) * 2020-08-18 2020-11-27 Oppo广东移动通信有限公司 Data processing method, control terminal, AR terminal, AR system and storage medium
CN111988528A (en) * 2020-08-31 2020-11-24 北京字节跳动网络技术有限公司 Shooting method, shooting device, electronic equipment and computer-readable storage medium

Also Published As

Publication number Publication date
CN115514882B (en) 2024-06-14
CN115514882A (en) 2022-12-23
WO2022160985A1 (en) 2022-08-04
CN114845035A (en) 2022-08-02

Similar Documents

Publication Publication Date Title
CN114845035B (en) Distributed shooting method, electronic device and medium
US11818420B2 (en) Cross-device content projection method and electronic device
CN110109636B (en) Screen projection method, electronic device and system
WO2020244495A1 (en) Screen projection display method and electronic device
CN114697527B (en) Shooting method, system and electronic equipment
WO2021078284A1 (en) Content continuation method and electronic device
CN113542839B (en) Screen projection method of electronic device and electronic device
CN113691842B (en) Cross-device content projection method and electronic device
WO2021185244A1 (en) Device interaction method and electronic device
CN114697732B (en) Shooting method, system and electronic equipment
WO2021249318A1 (en) Screen projection method and terminal
CN114168235B (en) Method for determining function switching entrance and electronic equipment
WO2022156721A1 (en) Photographing method and electronic device
CN115514883B (en) Cross-equipment collaborative shooting method, related device and system
CN115225753B (en) Shooting method, related device and system
CN118312117A (en) Screen projection display method and electronic device
CN114201130A (en) Screen projection method, device and storage medium
CN115460445A (en) Screen projection method of electronic equipment and electronic equipment
CN111131019B (en) Method and terminal for multiplexing multiple HTTP channels
EP4398594A1 (en) Photographing method and related device
WO2022161058A1 (en) Photographing method for panoramic image, and electronic device
CN118363506A (en) Device interaction method and electronic device
CN116055613A (en) Screen projection method and device
CN115309316B (en) Device using method and device
WO2024159925A1 (en) Screen mirroring method, screen mirroring system, and electronic device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant