WO2024046028A1

WO2024046028A1 - Camera calling method, electronic device, readable storage medium and chip

Info

Publication number: WO2024046028A1
Application number: PCT/CN2023/111068
Authority: WO
Inventors: 董斌斌
Original assignee: 华为技术有限公司
Priority date: 2022-09-02
Filing date: 2023-08-03
Publication date: 2024-03-07
Also published as: CN117714854A

Abstract

The present application relates to the technical field of cameras. Provided are a camera calling method, an electronic device, a readable storage medium and a chip. The method is applied to a first device, and an application is installed on the first device. The method comprises: receiving an operation event, wherein the operation event is used for indicating that an application calls a local camera (S601); if the application has a virtual-camera calling permission, in response to there being a multi-device cooperative relationship between a first device and a second device, determining a virtual camera corresponding to the local camera, and supporting the application to call the virtual camera to acquire an image (S602); and if the application does not have the virtual-camera calling permission, supporting the application to directly call the local camera to acquire an image (S603). In the method provided in the present application, when a first device and a second device operate cooperatively, after the first device receives an indication that an application calls a local camera, a virtual camera corresponding to the local camera can be quickly determined and called, thereby achieving a relatively fast calling speed and providing a good user experience.

Description

Camera calling method, electronic device, readable storage medium and chip

This application requests the priority of the Chinese patent application submitted to the State Intellectual Property Office on September 2, 2022, with the application number 202211073229.9 and the application name "Camera calling method, electronic device, readable storage medium and chip", and its entire content incorporated herein by reference.

Technical field

The present application relates to the field of camera technology, and in particular to a camera calling method, electronic equipment, readable storage media and chips.

Background technique

Virtual camera technology is an important component of multi-device interconnection and collaboration technology. Through virtual camera technology, applications on the first device (such as camera applications, video call applications, etc.) can call the camera of the second device to collect images. The camera of the second device is the virtual camera of the first device. However, the process of calling the virtual camera by the first device is complicated, the calling speed is slow, and the user experience is poor.

Contents of the invention

This application provides a camera calling method, electronic device, readable storage medium and chip to solve the problems in the prior art that the virtual camera calling process is complicated, the calling speed is slow, and the user experience is poor.

In order to achieve the above purpose, this application adopts the following technical solutions:

In a first aspect, embodiments of the present application provide a camera calling method, which is applied to a first device. The first device is installed with an application. The method includes: receiving an operation event. The operation event is used to instruct the application to call a local camera. The local camera is the camera of the first device; if the application has virtual camera calling permission, in response to the first device and the second device having a multi-device collaboration relationship, the virtual camera corresponding to the local camera is determined, and the application is supported to call the virtual camera to collect images. The virtual camera It is the camera of the second device. The local camera and the virtual camera have a mapping relationship; if the application does not have the permission to call the virtual camera, the application is supported to directly call the local camera to collect images.

Through the method provided by the embodiment of the present application, when the first device and the second device work together, the first device can quickly determine and call the virtual camera corresponding to the local camera after receiving an instruction from the application to call the local camera. The calling process is simple, has a fast calling speed, and has a good user experience.

In some implementations, the multi-device collaboration relationship includes: the first device sends the first screen content of the first device to the second device, and the second device displays the first screen content, and the first screen content includes relevant content of the application (such as application icons, application interfaces, etc.); based on this, receiving the operation event includes: receiving the operation event sent by the second device, and the operation event is triggered in the first screen content displayed by the second device.

Through the method provided by the embodiments of this application, when the first device projects the screen to the second device, the user can control the application on the first device to call the camera through the second device.

In some implementations, after the application calls the virtual camera to collect images, the method also includes: updating the first screen content according to the image collected by the virtual camera; and sending the updated first screen content to the second device.

In some implementations, if the application has the permission to call the virtual camera, in response to the first device and the second device having a multi-device collaboration relationship and supporting the application to call the virtual camera to collect images, the method also includes: the first device and the second device The device disconnects the multi-device collaboration relationship; switches the virtual camera to a local camera, allowing the application to directly call the local camera to collect images.

Through the method provided by the embodiment of the present application, when the application calls the virtual camera, if the multi-device collaboration relationship between the first device and the second device is disconnected, the first device can switch the camera that collects images from the virtual camera to the local camera. camera to ensure the normal working of the application and improve user experience.

In some implementations, if the application has the permission to call the virtual camera, in response to the first device and the second device having a multi-device collaboration relationship, supporting the application to call the virtual camera to collect images includes: receiving the operation event; supporting the application to directly call the local The camera collects images; the first device establishes a multi-device collaboration relationship with the second device; switches the local camera to a virtual camera, and supports the application to call the virtual camera to collect images.

Through the method provided by the embodiments of this application, during the process of the application calling the local camera, if the first device and the second device establish multiple device synergy, then the application can switch the camera that collects images from the local camera to the virtual camera, thereby releasing the occupation of the local camera to ensure the implementation of some functions of the electronic device that rely on the local camera (such as ensuring the face recognition unlocking function implementation).

In some implementations, the application includes a first application and a second application. The first application has the virtual camera calling permission, and the second application does not have the virtual camera calling permission. The operation event includes instructing the first application to call the first local camera. An operation event and a second operation event that instructs the second application to call the second local camera. The first local camera and the second local camera are the same or different local cameras. The method also includes: establishing a multi-device collaboration relationship between the first device and the second device; receiving a first operation event; determining a first virtual camera corresponding to the first local camera, supporting the first application to call the first virtual camera to collect images; receiving a third Second operation event; supports the second application to directly call the second local camera to collect images.

Through the method provided by the embodiments of this application, during the process of one application calling the virtual camera, another application can call the local camera, thereby ensuring the realization of more local functions of the first device and improving the user experience. For example, when the video chat application of the first device calls the virtual camera of the second device to collect images, the face recognition unlocking application can call the local camera to collect face images and perform face recognition unlocking to ensure the realization of the face recognition unlocking function. .

In some implementations, the application includes a first application and a second application. Both the first application and the second application have permission to call the virtual camera. The operation event includes a first operation event and an instruction that instructs the first application to call the first local camera. The second application calls a second operation event of the second local camera. The method also includes: establishing a multi-device collaboration relationship between the first device and the second device; receiving the first operation event; determining the first virtual camera corresponding to the first local camera, Support the first application to call the first virtual camera to collect images; receive the second operation event; display the first prompt information, the first prompt information is used to prompt that the virtual camera calling function has been occupied; or, stop supporting the first application to call the first virtual camera The camera collects images, determines the second virtual camera corresponding to the second local camera, and supports the second application to call the second virtual camera to collect images.

Wherein, the first local camera and the second local camera are the same local camera, and the first virtual camera and the second virtual camera are the same virtual camera; or, the first local camera and the second local camera are different local cameras, And the first virtual camera and the second virtual camera are different virtual cameras.

Through the method provided by the embodiments of this application, during the process of one application calling the virtual camera, the first device can prompt another application that the virtual camera can no longer be called. For example, in a scenario where the first device and the second device work together, if the video chat application has occupied the virtual camera, the first device prompts that the camera application cannot call the virtual camera for image collection. Alternatively, the first device can switch from the first application to the second application to call the virtual camera to meet the user's current usage needs.

In some implementations, the application includes a first application and a second application. The first application does not have the virtual camera calling permission, and the second application has the virtual camera calling permission. The operation event includes a third application that instructs the first application to call the first local camera. An operation event and a second operation event that instructs the second application to call the second local camera. The first local camera and the second local camera are the same or different local cameras. The method also includes: establishing a multi-device collaboration relationship between the first device and the second device; receiving a first operation event; supporting the first application to directly call the first local camera to collect images; receiving the second operation event; and determining the location corresponding to the second local camera. The second virtual camera supports the second application to call the second virtual camera to collect images.

Through the method provided by the embodiments of this application, while an application of the first device is calling the local camera, another application can call the virtual camera at the same time. For example, while the camera application of the first device calls the local camera to collect images, the video chat application can simultaneously call the virtual camera of the second device to collect images.

In some implementations, the application includes a first application and a second application. Neither the first application nor the second application has permission to call the virtual camera. The operation event includes a first operation event that instructs the first application to call the first local camera and Instruct the second application to call the second operation event of the second local camera, where the first local camera and the second local camera are the same or different local cameras. The method also includes: receiving a first operation event; supporting the first application to directly call the first local camera to collect images; receiving a second operation event; displaying a second prompt information, the second prompt information being used to prompt that the local camera calling function has been occupied ; Or, stop supporting the first application to directly call the first local camera to collect images, and support the second application to directly call the second local camera to collect images.

Through the method provided by the embodiments of this application, during the process of one application calling the local camera, the first device can prompt another application that it can no longer call the virtual camera. For example, in a scenario where the first device and the second device work together, if the camera application has occupied the local camera, the first device can prompt the video chat application that it cannot call the local camera for image collection. Alternatively, the first device may switch the application that calls the local camera from the first application to the second application to meet the user's current usage needs.

In some implementations, determining the virtual camera corresponding to the local camera includes: determining the virtual camera corresponding to the local camera according to the mapping relationship table. Virtual camera, the mapping relationship table includes at least one set of mapping relationships between local cameras and virtual cameras.

In some implementations, the mapping relationship between the at least one set of local cameras and the virtual camera includes: the mapping relationship between the first local camera and the first virtual camera; and the mapping relationship between the second local camera and the second virtual camera. The first local camera and the second local camera are different local cameras, and the first virtual camera and the second virtual camera are different virtual cameras.

In some implementations, the method further includes: establishing a multi-device collaboration relationship between the first device and the second device; receiving the identification information of the virtual camera sent by the second device; and determining the mapping according to the identification information of the local camera and the identification information of the virtual camera. relation.

In some implementations, the first device includes a local camera module for parallel processing and a virtual camera module, the local camera module is used for calling the local camera, and the virtual camera module is used for calling the virtual camera.

In the method provided by the embodiment of the present application, when the first device and the second device establish a multi-device collaboration relationship, a local camera module and a virtual camera module that can process tasks in parallel exist in the first device. Based on this, the first device can use the local camera module to call the local camera, and the virtual camera module to call the virtual camera, so that the first device's calls to the local camera and the virtual camera do not affect each other, ensuring the realization of more functions of the first device. , helping to improve the user experience.

In a second aspect, embodiments of the present application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned first aspect and the above are implemented. The methods shown in each implementation.

In a third aspect, embodiments of the present application provide a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, the method shown in the first aspect is implemented.

In a fourth aspect, embodiments of the present application provide a chip. The chip includes a processor and a memory. A computer program is stored in the memory. When the computer program is executed by the processor, the method shown in the first aspect is implemented.

In a fifth aspect, embodiments of the present application provide a computer program file. The computer program file includes a program. When the program is run by an electronic device, the electronic device implements the method shown in the first aspect.

It can be understood that the beneficial effects of the above-mentioned second aspect to the fifth aspect can be referred to the relevant description in the above-mentioned first aspect, and will not be described again here.

Description of drawings

Figure 1 is a schematic diagram of the connection structure between devices in the multi-device collaboration process provided by the embodiment of the present application;

Figure 2 is a schematic diagram of a multi-device collaboration scenario provided by an embodiment of the present application;

Figure 3 is a schematic diagram of a multi-device collaboration scenario provided by another embodiment of the present application;

Figure 4A is a schematic diagram of a user control scenario in a multi-device collaboration process provided by an embodiment of the present application;

Figure 4B is a schematic diagram of a user control scenario in a multi-device collaboration process provided by another embodiment of the present application;

Figure 4C is a schematic diagram of a video call scene during multi-device collaboration provided by an embodiment of the present application;

Figure 5 is a schematic flow chart of a camera calling method provided by an embodiment of the present application;

Figure 6 is a schematic flow chart of the camera calling method provided by the embodiment of the present application;

Figure 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

Figure 8 is a schematic diagram of the software structure of the first device provided by the embodiment of the present application;

Figure 9 is a schematic flow chart of the first device independently calling the local camera provided by the embodiment of the present application;

Figure 10 is a schematic flow chart of the first device switching a local camera to a virtual camera according to an embodiment of the present application;

Figure 11 is a schematic flow chart of the first device separately calling the virtual camera provided by the embodiment of the present application;

Figure 12 is a schematic flow chart of the first device switching the virtual camera to the local camera provided by the embodiment of the present application;

Figure 13 is a schematic diagram of a display interface of the first device provided by the embodiment of the present application;

Figure 14 is a schematic structural diagram of a chip provided by an embodiment of the present application.

Detailed ways

The technical solutions provided by the embodiments of the present application will be described below with reference to the accompanying drawings.

It should be understood that in the description of the embodiments of this application, unless otherwise stated, "/" means or, for example, A/B can mean A or B; "and/or" in this article is only a descriptive association. The association relationship of objects means that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

In the embodiments of this application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or The latter implicitly indicates the quantity of the technical characteristics indicated. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

Multi-device interconnection and collaboration, referred to as multi-device collaboration, is an important development direction of electronic equipment. Currently, common multi-device collaboration scenarios include mobile phone-tablet collaboration, mobile phone-laptop collaboration, tablet-laptop collaboration, vehicle-machine collaboration, mobile phone-drone collaboration, mobile phone-camera device collaboration, etc. Through multi-device collaboration, different types of electronic devices can break down device and application barriers and achieve cross-platform and cross-system collaboration functions, thereby making full use of the advantages of different devices and improving people's work and life experience.

Figure 1 is a schematic diagram of the connection structure between devices in the multi-device collaboration process provided by the embodiment of the present application. Referring to Figure 1, taking multi-device collaboration between a first device and a second device as an example, the first device and the second device may be connected wirelessly or wired. Among them, wireless connections include wireless fidelity (WiFi) connections, Bluetooth (BT) connections, ultra-wide band (UWB) connections, near field communication (NFC) connections, etc. Wired connections include data line connections, etc., which are not specifically limited in the embodiments of this application.

The first device and the second device can establish a multi-device collaboration relationship based on user operations. For example, when the first device is a mobile phone and the second device is a laptop, with the NFC function of the phone turned on, the user uses the NFC area on the back of the phone to touch the Huawei sharing sensing area of the laptop and confirms the connection to the laptop on the phone. After connecting to the computer, the screen content of the mobile phone can be displayed on the laptop in the form of a floating window (see Figure 2), thereby establishing a multi-device collaboration relationship. Or, when the first device is a mobile phone and the second device is a tablet, with the mobile phone's Bluetooth turned on and the tablet's multi-screen collaboration function turned on, the user can bring the phone close to the tablet and follow the prompts to complete the connection. The screen content is displayed on the tablet in the form of a floating window (see Figure 3), establishing a multi-device collaboration relationship.

Based on this, the user can display the screen content of the first device, which can be called the first screen content, through the floating window on the second device, and control the first device. For example, the second device can feed back the operation events (such as click events, touch sliding events, etc.) triggered by the user in the floating window to the first device, and the first device can update the screen of the first device based on the received operation events. content, and sends the updated screen content to the second device, so that the second device updates the screen content of the first device displayed in the floating window. In this way, some functions of the first device can be operated by the second device, and the first device can simultaneously operate other functions.

Furthermore, virtual camera technology can also be used in the process of multi-device collaboration, that is, a device can not only use a local camera for image collection, which refers to the camera of the device itself, but can also use a virtual camera for image collection. The virtual camera refers to Cameras from other connected devices. In some examples, after the first device (such as a mobile phone) and the second device (such as a laptop) establish a multi-device collaboration relationship, if the first device is the master device and the second device is the collaboration device, then the first device Some applications (Apps) can call the camera of the second device to collect images (including pictures and videos).

In an example, as shown in Figure 3, in the process of collaborative work between a mobile phone and a tablet computer, the mobile phone is the main control device and the tablet computer is the collaborative device. The mobile phone sends the screen content of the mobile phone to the tablet computer, and the tablet computer receives and displays the mobile phone screen content. screen content. As shown in Figure 4A, in response to the user's operation of the Changlian application on the tablet computer, the tablet computer sends an operation event to open the Changlian application to the first device, and the mobile phone updates the screen content of the mobile phone as shown in Figure 4B based on the operation event. The call recording interface appears, and the call recording interface is sent to the tablet for display. The call record interface includes call records between the user and each of his friends. Under some call records, such as the call record of Friend 1, there are voice call controls and video call controls. Referring to FIG. 4B , in response to the user clicking the video call control of friend 1 in the floating window of the second device, the second device sends a first operation event to the first device. In response to the first operation event, the first device calls the camera of the second device to collect images, and generates a call interface based on the collected images. Finally, the first device sends the call interface to the second device for display, thereby enabling the user to use the Changlian application of the first device to make a video call on the second device. During the video call, as shown in Figure 4C, the first device and the second device display the same call interface, including user images, friend images, and other controls (such as "switch camera", "hang up", "update"). "Multiple", etc.), where the user image is captured by the camera of the second device. In addition, it should be noted that when the user controls the first device through the floating window of the second device, the first device can display screen content or turn off the screen.

In some implementations, as shown in Figure 5, the first device usually calls cameras from various sources through the camera session management module (Camera0) of the hardware abstraction layer to implement video stream switching and image collection, where the cameras from each source include the first device Local camera and virtual camera (such as the camera of the second device).

For example, the application of the first device (such as ) The process of calling the local camera is shown in S0-1 to S0-4 in Figure 5.

S0-1, the application sends camera calling instructions to the camera module (Camera Module).

S0-2, in response to the camera calling instruction, the camera module sends a request to the camera session in the camera session management module (Camera0). The unit (Camera Session) sends session establishment instructions.

S0-3, in response to the session establishment instruction, the camera session unit establishes the camera session and sends a driver opening instruction to the camera driver (Camera Driver).

S0-4, in response to the driver opening instruction, the camera driver opens and sends an image acquisition instruction to the local camera.

It should be noted that the images collected by the local camera are returned to the application through the camera driver, camera session unit, and camera module in sequence.

For example, the application of the first device (such as ) The process of calling the virtual camera is shown in S0-1 to S0-9 in Figure 5.

S0-1~S0-4, the application opens the local camera. For details, please refer to the previous description and will not be repeated here.

S0-5: When the first device and the second device have established a multi-device collaboration relationship, the collaboration management application sends a camera switching instruction to the camera session management module through the distributed hardware management module. The camera switching instruction is used to instruct the data stream of the camera session unit to be switched from a local camera to a virtual camera.

S0-6, in response to the camera switching instruction, the data flow switching unit in the camera session management module sends a data interception request to the camera session unit. This data interception request is used to request to intercept images collected by local cameras.

S0-7, in response to the data interception request, the camera session unit sends a driver shutdown instruction to the camera driver to stop calling the local camera. After the first device stops calling the local camera, the first device does not use the local camera to collect images.

S0-8, the camera session unit sends an interception success notification to the data flow switching unit.

S0-9, the data flow switching unit sends a switching success notification to the distributed hardware management module. The switching success notification is used to notify that the data stream of the camera session unit has been switched from the local camera to the virtual camera.

S0-10: The distributed hardware management module sends a camera calling instruction to the second device to collect images through the virtual camera (ie, the camera of the second device).

It should be noted that the images collected by the virtual camera (that is, the camera of the second device) are returned to the application program through the distributed hardware management module, data flow switching unit, camera session unit, and camera module in sequence.

It can be seen that in the process of calling the virtual camera, the first device needs to open the local camera first, and then switch the local camera to the virtual camera according to the instructions of the upper-layer application. There is a redundant operation process for opening the local camera, and the calling process is complicated, resulting in The calling speed of the virtual camera is slow and the user experience is poor.

In addition, since the first device uses the camera session management module to call the virtual camera, the camera session management module will be occupied by the virtual camera, and the local camera cannot be used by other applications. Therefore, the first device cannot simultaneously call the local camera. Camera and virtual camera, thus affecting the implementation of some functions of the first device.

Taking the first device as a mobile phone and the second device as a laptop as an example, after the mobile phone and the laptop establish a multi-device collaboration relationship, the mobile phone can call the virtual camera (that is, the camera of the laptop) to collect images during the video chat. However, when the phone calls the virtual camera, the camera session management module is occupied, so the phone can no longer call the local camera to collect face images. For example, face recognition unlocking will fail, resulting in poor user experience.

To this end, embodiments of the present application provide a camera calling method, which can avoid the redundant process of opening a local camera when an electronic device opens a virtual camera, improve the virtual camera calling speed, and improve user experience. In addition, this method can also reduce the mutual influence when calling the local camera and the virtual camera, ensure the normal realization of more functions of the electronic device, and improve the user experience.

Figure 6 is a schematic flow chart of the camera calling method provided by the embodiment of the present application. The method is applied to the first device and specifically includes the following S601 to S603.

S601: Receive an operation event, which is used to instruct the application to call the local camera.

S602, if the application has the permission to call the virtual camera, in response to the first device and the second device having a multi-device collaboration relationship, determine the virtual camera corresponding to the local camera, and support the application to call the virtual camera to collect images. The virtual camera is the camera of the second device.

S603, if the application does not have the permission to call the virtual camera, the application is supported to call the local camera to collect images.

In the method provided by the embodiment of the present application, when the first device and the second device establish a multi-device collaboration relationship, the first device can quickly determine and call the local camera after receiving an instruction from the application program to call the local camera. The corresponding virtual camera has faster calling speed and better user experience.

FIG. 7 is a schematic structural diagram of an electronic device suitable for providing a camera calling method according to an embodiment of the present application. The electronic device may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, Charging management module 240, power management module 241, battery 242, antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone interface 270D, sensor module 280, Button 290, motor 291, indicator 292, camera 293, display screen 294, and subscriber identification module (subscriber identification module, SIM) card interface 295, etc.

In the embodiment of this application, the electronic device may be a mobile phone, a tablet computer (Pad), a computer with wireless transceiver functions, a smart TV, a projector, a wearable device (such as a smart watch), a vehicle-mounted device, or an augmented reality device. (augmented reality, AR)/virtual reality (VR) equipment, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (personal digital assistant, PDA), etc. The embodiments of this application do not limit the specific type of electronic equipment.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the electronic equipment. In other embodiments of the present application, the electronic device may include more or less components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

As an example, when the electronic device is a mobile phone or a tablet computer, it may include all the components in the illustration, or may only include some of the components in the illustration.

As an example, when the electronic device is a large-screen device, it may include the processor 210 in the figure, the external memory interface 220, the internal memory 221, the universal serial bus (universal serial bus, USB) interface 230, and the charging management module 240. Power management module 241, wireless communication module 260, audio module 270, speaker 270A, receiver 270B, microphone 270C, camera 293, and display screen 294.

The processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) wait. Among them, different processing units can be independent devices or integrated in one or more processors.

Among them, the controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in processor 210 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 210 . If the processor 210 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 210 is reduced, thus improving the efficiency of the system.

The charge management module 240 is used to receive charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 may receive charging input from the wired charger through the USB interface 230 . In some wireless charging embodiments, the charging management module 240 may receive wireless charging input through a wireless charging coil of the electronic device. While charging the battery 242, the charging management module 240 can also provide power to the electronic device through the power management module 241.

The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and/or the charging management module 240 and supplies power to the processor 210, internal memory 221, external memory, display screen 294, camera 293, wireless communication module 260, etc. The power management module 241 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters.

In some other embodiments, the power management module 241 may also be provided in the processor 210 . In other embodiments, the power management module 241 and the charging management module 240 may also be provided in the same device.

The wireless communication function of the electronic device can be realized through the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in an electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.

The mobile communication module 250 can provide wireless communication solutions including 2G/3G/4G/5G applied to electronic devices. The mobile communication module 250 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves from the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation.

In some embodiments, at least part of the functional modules of the mobile communication module 250 may be disposed in the processor 210 . In some embodiments, at least part of the functional modules of the mobile communication module 250 and at least part of the modules of the processor 210 may be provided in the same device.

A modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 270A, receiver 270B, etc.), or displays images or videos through display screen 294. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 210 and may be provided in the same device as the mobile communication module 250 or other functional modules.

The wireless communication module 260 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), and global navigation satellite systems for use in electronic devices. (global navigation satellite system, GNSS), frequency modulation (FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 260 may be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210 . The wireless communication module 260 can also receive the signal to be sent from the processor 210, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module 250, and the antenna 2 is coupled to the wireless communication module 260, so that the electronic device can communicate with the network and other devices through wireless communication technology.

The electronic device implements display functions through the GPU, display screen 294, and application processor. The GPU is an image processing microprocessor and is connected to the display screen 294 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 294 is used to display images, videos, etc. For example, in the teaching video and user action scene video in the embodiment of the present application, the display screen 294 includes a display panel. In some embodiments, the electronic device may include 1 or N display screens 294, where N is a positive integer greater than 1.

The electronic device can realize the shooting function through ISP, camera 293, video codec, GPU, display screen 294 and application processor.

The ISP is used to process the data fed back by the camera 293. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the light signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 293.

Camera 293 is used to capture still images or video. The object passes through the lens to produce an optical image that is projected onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the electronic device may include 1 or N cameras 293, where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

Video codecs are used to compress or decompress digital video. Electronic devices may support one or more video codecs. In this way, electronic devices can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG)1, MPEG2, MPEG3, MPEG4, etc.

NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can continuously learn by itself. Intelligent cognitive applications of electronic devices can be realized through NPU, such as image recognition, face recognition, speech recognition, text understanding, etc.

In this embodiment of the present application, the NPU or other processors may be used to perform operations such as analysis and processing on images in videos stored on electronic devices.

The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 210 through the external memory interface 220 to implement the data storage function. Such as saving music, videos, etc. files in external memory card.

Internal memory 221 may be used to store computer executable program code, which includes instructions. The processor 210 executes instructions stored in the internal memory 221 to execute various functional applications and data processing of the electronic device. The internal memory 221 may include a program storage area and a data storage area. Among them, the stored program area can store the operating system and at least one application program required for a function (such as a sound playback function, an image playback function, etc.). The storage data area can store data created during the use of electronic equipment (such as audio data, phone books, etc.).

In addition, the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

The electronic device can implement audio functions through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone interface 270D, and the application processor.

The audio module 270 is used to convert digital audio signals into analog audio signal outputs, and is also used to convert analog audio inputs into digital audio signals. Audio module 270 may also be used to encode and decode audio signals. In some embodiments, the audio module 270 may be provided in the processor 210 , or some functional modules of the audio module 270 may be provided in the processor 210 .

Speaker 270A, also called "speaker", is used to convert audio electrical signals into sound signals. The electronic device can listen to music through speaker 270A, or listen to hands-free calls.

Receiver 270B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device answers a call or a voice message, the voice can be heard by bringing the receiver 270B close to the human ear.

Microphone 270C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 270C with the human mouth and input the sound signal to the microphone 270C. The electronic device may be provided with at least one microphone 270C.

The headphone interface 270D is used to connect wired headphones. The headphone interface 270D may be a USB interface 230, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

The buttons 290 include a power button, a volume button, etc. Key 290 may be a mechanical key. It can also be a touch button. The electronic device can receive key input and generate key signal input related to user settings and function control of the electronic device.

The motor 291 can generate vibration prompts. The motor 291 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback. For example, touch operations for different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Acting on touch operations in different areas of the display screen 294, the motor 291 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, receiving information, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

The indicator 292 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to indicate messages, missed calls, notifications, etc.

The SIM card interface 295 is used to connect a SIM card. The SIM card can be inserted into the SIM card interface 295 or pulled out from the SIM card interface 295 to realize contact and separation from the electronic device. The electronic device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

Figure 8 is a schematic diagram of the software structure of the first device provided by the embodiment of the present application. Referring to Figure 8, the operating system of the first device is Taking the system as an example, the software system of the first device includes an application layer, an application framework layer (framework, FWK), a hardware abstraction layer (HAL), and a kernel layer (Kernel). The layers are connected through software interfaces. communication.

The application layer includes a series of application packages, which can include Changlian applications, Camera, text messages, videos, emails, face recognition unlocking applications and collaborative management applications, etc. Among these applications, some applications (such as Changlian application, Camera) can call the camera, but some applications (such as text messages, emails) cannot call the camera. Among them, the collaborative management application is used to manage and establish a collaborative relationship between the first device and other devices (such as the second device), which includes controlling and displaying relevant user operation interfaces, and discovering or connecting according to the control operations input by the user in the interface. Other equipment, etc.

It should be noted that, in this embodiment of the present application, applications capable of calling the camera on the first device are divided into first-type applications and second-type applications.

The first type of application has virtual camera calling permission, that is, it can call the virtual camera after the first device and the second device establish a multi-device collaboration relationship. These include: the first device can only call the virtual camera, or the first device can selectively call the virtual camera among the virtual camera and the local camera according to the user operation. For example, the first type of application can be Changlian application, Camera, mirror applications, etc.

In some embodiments, the first device is a mobile phone, the second device is a laptop computer, and the mobile phone and laptop computer establish a multi-device collaboration relationship. After tying, the phone is in use During a video call with a friend, the mobile phone can only call the laptop's camera to collect images, but cannot call the local camera to collect images. If the user wants to use the local camera to collect images, the multi-device collaboration relationship between the mobile phone and laptop needs to be disconnected.

In other embodiments, the first device is a mobile phone and the second device is a laptop. After the mobile phone and the laptop establish a multi-device collaboration relationship, the mobile phone uses During a video call with a friend, the phone gives priority to calling the laptop's camera to collect images. If the user wants to use the local camera of the mobile phone to collect images, he can perform a switching operation on the mobile phone to switch the camera that collects images to the local camera without breaking the multi-device collaboration relationship between the mobile phone and the laptop.

The second type of application does not have the permission to call the virtual camera, that is, after the first device and the second device establish a multi-device collaboration relationship, it cannot call the virtual camera and can only call the local camera. The second type of application may be determined in advance according to the type of application, or may be determined according to the user's configuration. For example, the second type of application may be a face recognition unlocking application, etc.

In some embodiments, assuming that the first device is a mobile phone and the second device is a laptop, after the mobile phone establishes a multi-device collaboration relationship with the laptop, does the application on the mobile phone give priority to calling the camera of the laptop to collect images? The image collected by the local camera can be determined by configuring the type of the application, that is, by configuring whether the application belongs to the first type of application or the second type of application. by mobile phone For example, if the user would Configured as a first-class application, then when the mobile phone and laptop work together, Prioritize calling the virtual camera. If the user will Configured as a second category application, then when the mobile phone and laptop work together, Prioritize calling the local camera.

The application framework layer includes distributed hardware management modules and camera modules. Distributed hardware management module, used to control the hardware of collaborative devices. The camera module includes a camera service module (Camera Service), a camera device module (Camera Device) and a virtual camera mapping management module. Among them, Camera Service is used to provide Camera usage interfaces and services to upper-layer applications. Camera Device is used to provide control capabilities for the corresponding Camera device. The virtual camera mapping management module is used to store the mapping relationship between the local camera of the first device and the camera of the second device (the virtual camera of the first device). When the first device and the second device establish a multi-device collaboration relationship, the virtual camera mapping management module can determine the virtual camera corresponding to the local camera through the mapping relationship when the first type of application requests to call the local camera, and The virtual camera (that is, the camera of the second device) is called through the virtual camera module. For the upper-layer application, there is no need to know the underlying logic of calling different cameras. Correspondingly, the application does not need to know the logic of calling different cameras in the scenario of multi-device collaboration. Compatible programs are individually designed to suit.

Optionally, after the first device and the second device establish a multi-device collaboration relationship, the second device can send its camera information (such as camera ID) to the virtual camera mapping management module of the first device. The virtual camera mapping management module Establish and store the mapping relationship according to preset rules. The embodiment of this application does not specifically limit the preset rules.

In an example, if the local camera of the first device includes front camera Camera1-1 and rear camera Camera1-2, and the camera of the second device includes front camera Camera2-1 and rear camera Camera2-2, then the mapping The relationship can be shown in Table 1. It should be understood that in order to facilitate user operation, the front camera of the first device usually corresponds to the front camera of the second device, and the rear camera of the first device usually corresponds to the rear camera of the second device, that is, Camera1-1 corresponds to Camera2-1 , Camera1-2 corresponds to Camera2-2.

Based on the mapping relationship shown in Table 1, when the user uses the front-facing camera Camera1-1 of the first device to capture an image, the user actually calls the front-facing camera Camera2-1 of the second device to capture the image. Similarly, when the user chooses to use the rear camera Camera1-2 on the first device to capture images, the rear camera Camera2-2 of the second device is actually called to capture images.

Table 1 Mapping relationship table

In another example, if the local camera of the first device includes front camera Camera1-1 and rear camera Camera1-2, and the camera of the second device includes front camera Camera2 and does not include the rear camera, then the mapping relationship It can be shown in Table 2. Based on this mapping relationship, the first device cannot switch between front and rear cameras when calling the camera of the second device through virtual camera technology. Wherein, the inability of the first device to switch the front and rear cameras can be understood as: the first device does not respond to the user's operation of the camera switching control.

Table 2 Mapping relationship table

The hardware abstraction layer is the layer between hardware and software. In the embodiment of this application, the hardware abstraction layer includes a local camera module (which can be called Camera0) and a virtual camera module (Camera X), which are used to establish a corresponding camera session. The camera session can be used to identify a session request of the application, control the request parameters, and pass the request image data, etc.

The local camera module (Camera0) is equipped with a local camera session unit (Camera Session), which is used to establish a local camera session. Through the local camera session unit, the electronic device can call the local camera to collect images.

The virtual camera module (Camera X) is provided with a virtual camera session unit (Virtual Camera Session), which is used to establish a virtual camera session. Through the virtual camera session unit, the first device can call the camera of the second device to collect images through the distributed hardware management module. The virtual camera module may be preset in the hardware abstraction layer of the first device, or may be a module newly registered and added by the first device after establishing a multi-device collaboration relationship with the second device.

The kernel layer (Kernel) includes a series of drivers, such as camera driver (Camera Driver), sensor driver, etc., which are used to drive related hardware of the hardware layer, such as cameras, sensors, etc. In this embodiment of the present application, the camera driver includes a first video module

(/dev/video0) and the second video module (/dev/video1), where the first video module corresponds to the front camera and is used to process images collected by the front camera. The second video module corresponds to the rear camera and is used to process images collected by the rear camera.

Based on the software structure of the first device provided by the above embodiments of the present application, the application program on the first device can call a local camera or a virtual camera (ie, the camera of the second device) to collect images. They are explained below.

(1) The application calls the local camera

In this embodiment of the present application, regardless of whether a multi-device collaboration relationship is established between the first device and the second device, the first type of application and the second type of application on the first device can call the local camera, and the calling process Basically the same, that is, the part of the application responsible for calling the local camera is the same.

For example, as shown in Figure 9, the process of calling a local camera by an application includes the following S1-1 to S1-4.

S1-1. In response to the operation event, the application program of the first device sends a local camera calling instruction to the camera module for calling the local camera of the first device to collect images.

In the embodiment of the present application, the user usually triggers an operation event on the first device, and the operation event is used to instruct the local camera to be called. Taking the first device as a mobile phone as an example, the operation event may be that the user opens the camera application on the mobile phone, or opens a video chat, or picks up the mobile phone to wake up the screen when the screen is turned off and locked, etc.

However, in some implementations, if the first device and the second device establish a multi-device collaboration relationship, and the second device displays the screen content of the first device through a floating window, then the operation event can also be the user's operation on the second device. Triggered within the floating window. For example, the camera application is configured by the user as a second-category application and does not have the permission to call the virtual camera. The user can click the icon of the camera application in the floating window of the second device to trigger an operation event and send the operation event to the mobile phone. So that the mobile phone can call the local camera according to the operation event.

It should be noted that, for the application, no matter whether the first device and the second device have established a multi-device collaboration relationship, the operation events that call the camera are usually the same. Therefore, the first device will first consider that these operation events are all It is to call the local camera. Only when the first device and the second device establish a collaborative relationship, for the first type of application, the first device can further convert the call to the local camera into a call to the virtual camera. For the second type of application, or when the first device and the second device do not establish a collaborative relationship, the first device does not need to convert the camera calling relationship, but directly calls the local camera.

S1-2: The camera module sends a session establishment instruction to the local camera module. The session establishment instruction is used to instruct the local camera module of the first device to establish a local camera session.

S1-3. After establishing the camera session according to the session establishment instruction, the camera session unit in the local camera module sends a driver open instruction to the camera driver. The driver open instruction is used to instruct the camera driver to work.

S1-4: During the working process, the camera driver sends image acquisition instructions to the local camera to drive the local camera to collect images.

It should be noted that the images collected by the local camera are returned to the application through the camera driver, local camera module, and camera module in sequence.

Regardless of whether a multi-device collaboration relationship is established between the first device and the second device, the application on the first device can call Local camera. Therefore, in some embodiments, before the application calls the local camera, a multi-device collaboration relationship is not established between the first device and the second device. Based on this, if the application calls the local camera , the first device and the second device have newly established a multi-device collaboration relationship, then the first device can switch the local camera to the virtual camera.

For example, as shown in Figure 10, the process includes the following steps S2-1 to S2-8.

S2-1～S2-2, in the process of the application of the first device using the local camera to collect images (see S1-1～S1-4), the collaborative management application of the first device controls the distributed hardware module management according to the user operation. Establish a multi-screen collaboration relationship with the second device.

S2-3: The distributed hardware management module configures the mapping relationship table to the virtual camera mapping management module.

Optionally, if the virtual camera module is not preset in the hardware abstraction layer of the first device, the distributed hardware management module also needs to register the virtual camera module in the hardware abstraction layer.

S2-4: After the mapping relationship table is successfully configured, the camera module of the first device sends a session closing notification to the local camera module to notify the local camera module to close the local camera session.

S2-5: After closing the local camera session, the local camera module sends a driver shutdown instruction to the camera driver to instruct the local camera module to close the camera driver.

S2-6~S2-7, the virtual camera management module determines the virtual camera corresponding to the local camera according to the mapping relationship table, and calls the virtual camera through the virtual camera module and the distributed hardware management module.

S2-8, the second device responds to the virtual camera invocation and instructs the virtual camera to collect images.

The images collected by the virtual camera (that is, the camera of the second device) are returned to the application program of the first device through the distributed hardware module, virtual camera module, and camera module of the first device in sequence.

(2) Call virtual camera

In this embodiment of the present application, the application program that calls the virtual camera is a first-type application program. In other words, only the first type of application (such as Changlian application) on the first device can call the virtual camera, while the second type of application (such as face recognition unlocking application) cannot call the virtual camera.

For example, as shown in Figure 11, the process specifically includes the following steps S3-1 to S3-7.

In S3-1 to S3-2, the collaboration management module of the first device sends a multi-device collaboration establishment instruction to the distributed hardware management module according to the user operation, and controls the distributed hardware management module to establish a multi-device collaboration relationship with the second device.

S3-3. The distributed hardware management module configures the mapping relationship table to the virtual camera mapping management module.

S3-4, in response to the operation event, the application program of the first device sends a camera calling instruction to the camera module, which is used to call the local camera of the first device for image collection.

Optionally, the operation event may be triggered by the user on the first device, or may be triggered by the user on the second device and sent by the second device to the first device. For details, please refer to the previous description and will not be repeated here.

S3-5: The camera module determines the virtual camera corresponding to the local camera according to the preconfigured mapping relationship table, and sends a virtual camera calling instruction to the virtual camera module. The virtual camera calling instruction is used to call the virtual camera.

For example, according to the mapping relationship shown in Table 1, if the local camera currently used by the first device is Camera1-1, the virtual camera is Camera2-1. If the local camera currently used by the first device is Camera1-2, the virtual camera is Camera2-2.

S3-6: The virtual camera module sends the virtual camera calling instruction to the second device through the distributed hardware management module.

S3-7, the second device responds to the virtual camera calling instruction and calls the virtual camera to collect images.

After collecting the images, the second device sequentially sends the collected images to the application program of the first device through the distributed hardware module, virtual camera module, and camera module of the first device.

When the application calls the virtual camera, if the multi-device collaboration relationship between the first device and the second device is disconnected, or the user instructs to switch the audio and video during multi-device collaboration to the first device, then the first device can Switch virtual camera to local camera.

For example, as shown in Figure 12, the process includes the following steps S4-1 to S4-5.

S4-1: The first device and the second device disconnect the multi-device collaboration relationship. Alternatively, the first device receives a camera switching instruction input by the user, which is used to switch the camera called by the application program from the virtual camera to the local camera.

In some embodiments, as shown in FIG. 13 , after the multi-device collaboration relationship between the first device and the second device is successfully established, the second device may display a notification message in the notification management box to prompt the user. For example, when the second device is a mobile phone with model "HUAWEI P50Pro" When the phone is powered on, the notification message can include the prompt information "Connected to "HUAWEI P50Pro"", and "You can operate the phone on the tablet and share data with each other", etc. In addition, the notification message also includes "disconnect" controls and "switch audio and video to mobile phone" controls. Among them, the "disconnect" control is used to control the second device to disconnect from the first device (i.e. HUAWEI P50Pro); the "audio and video switching to mobile phone" control is used to control the second device to switch the audio and video of the first device. The collection and playback operations are switched from the current second device to the first device.

In response to the user's operation of the "disconnect" control, the first device and the second device can disconnect the multi-device collaboration relationship. In response to the user's operation of the "switch audio and video to mobile phone" control, the camera of the first device that collects images can be switched from the virtual camera to the local camera.

S4-2, the distributed hardware management module sends a camera switching notification to the camera module, which is used to notify the camera that collects images to switch from a virtual camera to a local camera.

S4-3. The camera module sends a session establishment instruction to the local camera module. The session establishment instruction is used to instruct the local camera module of the first device to establish a local camera session.

S4-4. After establishing the camera session according to the session establishment instruction, the camera session unit in the local camera module sends a driver open instruction to the camera driver. The driver open instruction is used to instruct the camera driver to work.

S4-5: During the working process, the camera driver sends image acquisition instructions to the local camera to drive the local camera to collect images.

It should be noted that the images collected by the local camera of the first device are returned to the application program through the camera driver, the local camera module and the camera module in sequence.

In the embodiment of the present application, when the first device and the second device establish a multi-device collaboration relationship, if the first device receives an instruction from the application program to call the local camera, and the application program has the virtual camera call permission, then the first device The device directly determines the virtual camera corresponding to the local camera based on the mapping relationship table, and calls the virtual camera through the virtual camera module. There is no need to establish a local camera session through the local camera module, and therefore there is no need to open the local camera. Therefore, in the process of opening the virtual camera, redundant operations of opening the local camera are reduced, the opening speed of the virtual camera can be improved, and a better user experience can be achieved.

Since the local camera module is responsible for the first device's call to the local camera, the virtual camera is responsible for the call to the virtual camera, and the two modules can process tasks in parallel. Therefore, when the first device and the second device establish a multi-device collaboration relationship, the application's calls to the local camera and the virtual camera do not affect each other, and two applications can call the local camera and the virtual camera at the same time. Based on the fact that the first device and the second device have established a multi-device collaboration relationship, the process is illustrated below by taking the first application and the second application as examples.

Example 1: Both the first application and the second application are first-type applications.

In other words, both the first application and the second application have the virtual camera calling permission. For example, the first application may be a Changlian application, and the second application may be a camera application.

The process of calling the camera by the first application and the second application at the same time includes the following contents (1) and (2).

(1) The first application calls the first virtual camera to collect images.

Specifically, the user may trigger a first operation event for the first application. The first operation event is used to instruct the first application to call the first local camera of the first device to collect images. Taking the first application as the Changlian application as an example, the first operation event can allow the user to operate the Changlian application to invite friends for a video call. Since the first device has the virtual camera calling authority, the first device determines the first virtual camera corresponding to the first local camera according to the mapping relationship table, and calls the first virtual camera through the virtual camera module to collect images for the first application. . Please refer to the previous description for the specific calling process and will not be repeated here.

(2) The second application fails to call the second virtual camera, or the first application switches to the second application to call the virtual camera.

In some embodiments, during the process of the first application calling the first virtual camera to collect images, the user can trigger the second operation event for the second application. The second operation event is used to instruct the second application to call the second local camera of the first device. Taking the second application as a camera application as an example, the second operation event may be that the user clicks on the application icon of the camera application. At this time, because the virtual camera module that calls the virtual camera has been occupied by the first application, the second application fails to call the virtual camera. To this end, the first device may display first prompt information to prompt that the virtual camera calling function has been occupied.

In other embodiments, when the second application requests to call the second virtual camera, if the virtual camera module is already occupied by the first application, the first device may ask the user whether to switch to the second application to call the virtual camera. Based on this, the first device can continue to control the first application to call the virtual camera according to the user's instruction, or can switch to the second application to call the virtual camera through the virtual camera module.

Alternatively, the first device may automatically switch the application that calls the virtual camera from the first application to the second application. Taking the first application as a Changlian application and the second application as a camera application as an example, during the process of the Changlian application using the first virtual camera of the mobile phone, if the first device detects the user's operation of opening the camera application, it will automatically call the virtual camera. The camera application is switched from Changlian application to Camera application.

It should be noted that the first local camera called by the first operation event controlled by the first application and the second local camera called by the second operation event controlled by the second application may be the same. For example, the first local camera and the second local camera may be the same. Both are front-facing cameras of the first device; they may also be different. For example, one of the first local camera and the second local camera may be a front-facing camera and the other may be a rear-facing camera. If the first local camera and the second local camera are the same, the first virtual camera and the second virtual camera are the same. If the first local camera and the second local camera are different, the first virtual camera and the second virtual camera are different.

Example 2: The first application is a first-type application, and the second application is a second-type application.

In other words, the first application has the permission to call the virtual camera, but the second application does not have the permission to call the virtual camera. For example, the first application is a Changlian application, and the second application is a mirror application. The process of calling the camera by the first application and the second application at the same time includes the following contents (1) and (2).

(1) The first application calls the first virtual camera to collect images.

(2) During the process of the first application calling the first virtual camera to collect images, the second application calls the second local camera to collect images.

Specifically, during the process of the first application calling the first virtual camera to collect images, the user can trigger the second operation event for the second application. The second operation event is used to instruct the second application to call the second local camera of the first device. Taking the second application as a mirror application as an example, the second operation event may be that the user clicks on the application icon of the mirror application. Since the second application does not have the right to call the virtual camera, the first device directly calls the second local camera through the local camera module to collect images for the second application. Please refer to the previous description for the specific calling process and will not be repeated here.

Based on the above description, it can be seen that since the hardware abstraction layer of the first device has both a local camera module and a virtual camera module, and these two camera modules can work in parallel and independently without affecting each other, therefore, the first device is in the process of multi-device collaboration. It is possible to call the local camera and virtual camera at the same time. Taking the first device mobile phone as an example, after the mobile phone and the tablet establish a multi-screen collaboration relationship, if the mobile phone is in the locked screen state and supports facial recognition unlocking, the mobile phone can call the local camera through the local camera module to collect user images, thereby Perform facial recognition unlocking.

Example 3: The first application is a second type application, and the second application is a first type application.

In other words, the first application does not have the virtual camera calling authority, and the second application does have the virtual camera calling authority. For example, the first application is a mirror application, and the second application is a Changlian application. The process of calling the camera by the first application and the second application at the same time includes the following content (1) and content (2).

(1) The first application calls the first local camera to collect images.

Specifically, the user may trigger a first operation event for the first application. The first operation event is used to instruct the first application to call the first local camera of the first device to collect images. Taking the first application as a mirror application as an example, the first operation event may be that the user clicks on the application icon of the mirror application. Since the first device does not have the right to call the virtual camera, the first device directly calls the first local camera through the local camera module to collect images for the first application. Please refer to the previous description for the specific calling process and will not be repeated here.

(2) While the first application calls the first local camera to collect images, the second application calls the second virtual camera to collect images.

Specifically, the user can trigger a second operation event for the second application. The second operation event is used to instruct the second application to call the second local camera of the first device to collect images. Taking the second application as a Changlian application as an example, the second operation event may be the video chat control of Friend 1 in the user's Changlian application. Since the second application has the permission to call the virtual camera, the first device determines the second virtual camera corresponding to the second local camera, and calls the second virtual camera through the virtual camera module to collect images for the second application. Please refer to the previous description for the body calling process and will not go into details here.

Example 4: Both the first application and the second application are second-type applications.

In other words, neither the first application nor the second application has the virtual camera calling permission. For example, the first application is a mirror application, and the second application is a camera application. The process of calling the camera by the first application and the second application at the same time includes the following content (1) and content (2).

(1) The first application calls the first local camera to collect images.

Specifically, the user may trigger a first operation event for the first application. The first operation event is used to instruct the first application to call the first local camera of the first device to collect images. Taking the first application as a mirror application as an example, the first operation event can be that the user clicks on the mirror application. Application icon. Since the first device does not have the right to call the virtual camera, the first device directly calls the first local camera through the local camera module to collect images for the first application. Please refer to the previous description for the specific calling process and will not be repeated here.

(2) During the process of the first application calling the first local camera to collect images, the second application fails to call the second local camera, or the first application switches to the second application to call the local camera.

In some embodiments, during the process of the first application calling the first local camera to collect images, the user can trigger the second operation event for the second application. The second operation event is used to instruct the second application to call the second local camera of the first device. Taking the second application as a camera application as an example, the second operation event may be an application icon of the user's camera application. Since the second application does not have the permission to call the virtual camera, the first device needs to call the second local camera to collect images for the second application. However, the second local camera module responsible for calling the local camera has been occupied by the first application, so the second application fails to call the second local camera.

In other embodiments, when the second application requests to call the second local camera, if the local camera module is already occupied by the first application, the first device may ask the user whether to switch to the second application to call the local camera. The first device may continue to control the first application to call the first local camera according to user instructions, or switch to the second application to call the second local camera.

Alternatively, the first device may switch the application that calls the local camera from the first application to the second application. Take the first application as a mirror application and the second application as a camera application as an example. When the mirror application uses the first local camera, if the first device detects the user's operation of opening the camera application, it will automatically call the local camera application. Switch from the Changlian app to the Camera app.

It should be understood that the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

An embodiment of the present application also provides an electronic device. The electronic device includes a local camera, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program Implement the camera calling method as shown in the above embodiments.

An embodiment of the present application also provides a chip, as shown in Figure 14. The chip includes a processor and a memory. A computer program is stored in the memory. When the computer program is executed by the processor, the camera calling method in the above embodiments is implemented. .

Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the virtual camera calling method provided in the above embodiments is implemented.

An embodiment of the present application also provides a computer program product. The program product includes a computer program. When the computer program is run by an electronic device, it causes the electronic device to implement the camera calling method provided in the above embodiments.

It should be understood that the processor mentioned in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processor, digital signal processor (DSP), or application-specific integrated circuit (ASIC). application specific integrated circuit (ASIC), off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

It should also be understood that the memory mentioned in the embodiments of the present application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Among them, non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically removable memory. Erase electrically programmable read-only memory (EPROM, EEPROM) or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of illustration, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM) ) and direct memory bus random access memory (direct rambus RAM, DR RAM).

In the embodiments provided in this application, the division of each framework or module is only a logical function division. In actual implementation, there may be other division methods. For example, multiple frameworks or modules may be combined or integrated into another system. , or some features can be ignored, or not implemented.

In addition, each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

Reference in this specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise. The terms “including,” “includes,” “having,” and variations thereof all mean “including but not limited to,” unless otherwise specifically emphasized.

The above-described embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still implement the above-mentioned implementations. The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in the embodiments of this application, and should be included in within the protection scope of this application.

Claims

A camera calling method, characterized in that it is applied to a first device, and the first device is installed with an application. The method includes:

Receive an operation event, the operation event is used to instruct the application to call a local camera, and the local camera is the camera of the first device;

If the application has virtual camera calling permission, in response to the first device and the second device having a multi-device collaboration relationship, the virtual camera corresponding to the local camera is determined, and the application is supported to call the virtual camera to collect images, so The virtual camera is the camera of the second device, and the local camera and the virtual camera have a mapping relationship;

If the application does not have the permission to call the virtual camera, the application is supported to directly call the local camera to collect images.
The method of claim 1, wherein the multi-device collaboration relationship includes: the first device sending the first screen content of the first device to the second device, and the first screen content Including relevant content of the application, the second device displays the first screen content;

The receiving the operation event includes: receiving the operation event sent by the second device, the operation event being triggered in the first screen content displayed by the second device.
The method according to claim 2, characterized in that after the supporting application calls the virtual camera to collect images, the method further includes:

Update the first screen content according to the image collected by the virtual camera;

Send the updated first screen content to the second device.
The method according to any one of claims 1 to 3, characterized in that, if the application has virtual camera calling permission, in response to the first device and the second device having a multi-device collaboration relationship, the application calling is supported. After the virtual camera collects images, the method further includes:

The first device and the second device disconnect the multi-device collaboration relationship; switch the virtual camera to the local camera to support the application to directly call the local camera to collect images.
The method according to any one of claims 1 to 3, characterized in that if the application has virtual camera calling permission, in response to the first device and the second device having a multi-device collaboration relationship, the application calling is supported. The virtual camera collecting images includes:

receive the operation event;

Support the application to directly call the local camera to collect images;

The first device establishes a multi-device collaboration relationship with the second device;

The local camera is switched to the virtual camera, and the application is supported to call the virtual camera to collect images.
The method according to any one of claims 1 to 5, characterized in that the application includes a first application and a second application, the first application has virtual camera calling authority, and the second application does not have virtual camera Call permission, the operation event includes a first operation event instructing the first application to call the first local camera and a second operation event instructing the second application to call the second local camera, the first local camera and the The second local camera is the same or a different local camera, and the method further includes:

The first device establishes a multi-device collaboration relationship with the second device;

receiving the first operation event;

Determine the first virtual camera corresponding to the first local camera, and support the first application to call the first virtual camera to collect images;

Receive the second operation event;

The second application is supported to directly call the second local camera to collect images.
The method according to any one of claims 1 to 5, characterized in that the application includes a first application and a second application, both the first application and the second application have virtual camera calling permissions, and the The operation event includes a first operation event that instructs the first application to call the first local camera and a second operation event that instructs the second application to call the second local camera. The method further includes:

The first device establishes a multi-device collaboration relationship with the second device;

receiving the first operation event;

Determine the first virtual camera corresponding to the first local camera, and support the first application to call the first virtual camera to collect images;

Receive the second operation event;

Display the first prompt information, the first prompt information is used to prompt that the virtual camera calling function has been occupied; or,

Stop supporting the first application in calling the first virtual camera to collect images, determine the second virtual camera corresponding to the second local camera, and support the second application in calling the second virtual camera to collect images;

Wherein, the first local camera and the second local camera are the same local camera, and the first virtual camera and the second virtual camera are the same virtual camera; or, the first local camera and The second local camera is a different local camera, and the first virtual camera and the second virtual camera are different virtual cameras.
The method according to any one of claims 1 to 5, characterized in that the application includes a first application and a second application, the first application does not have the virtual camera calling authority, and the second application has the virtual camera Call permission, the operation event includes a first operation event instructing the first application to call the first local camera and a second operation event instructing the second application to call the second local camera, the first local camera and the The second local camera is the same or a different local camera, and the method further includes:

The first device establishes a multi-device collaboration relationship with the second device;

receiving the first operation event;

Support the first application to directly call the first local camera to collect images;

Receive the second operation event;

Determine a second virtual camera corresponding to the second local camera, and support the second application to call the second virtual camera to collect images.
The method according to any one of claims 1 to 5, characterized in that the application includes a first application and a second application, and neither the first application nor the second application has virtual camera calling permission, so The operation event includes a first operation event that instructs the first application to call a first local camera and a second operation event that instructs the second application to call a second local camera. The first local camera and the second local camera The cameras are the same or different local cameras, and the method further includes:

receiving the first operation event;

Support the first application to directly call the first local camera to collect images;

Receive the second operation event;

Display second prompt information, the second prompt information is used to prompt that the local camera calling function has been occupied; or,

Stop supporting the first application to directly call the first local camera to collect images, and support the second application to directly call the second local camera to collect images.
The method according to any one of claims 1 to 9, wherein determining the virtual camera corresponding to the local camera includes:

The virtual camera corresponding to the local camera is determined according to a mapping relationship table, where the mapping relationship table includes at least one set of mapping relationships between local cameras and virtual cameras.
The method according to claim 10, characterized in that the mapping relationship between the at least one set of local cameras and virtual cameras includes: the mapping relationship between the first local camera and the first virtual camera; and, the mapping relationship between the second local camera and the second Mapping relationship of virtual cameras; wherein, the first local camera and the second local camera are different local cameras, and the first virtual camera and the second virtual camera are different virtual cameras.
The method according to any one of claims 1-11, characterized in that the method further includes:

The first device establishes a multi-device collaboration relationship with the second device;

Receive the identification information of the virtual camera sent by the second device;

The mapping relationship is determined based on the identification information of the local camera and the identification information of the virtual camera.
The method according to any one of claims 1 to 11, characterized in that the first device includes a local camera module and a virtual camera module for parallel processing, and the local camera module is used to call the local camera, The virtual camera module is used to call the virtual camera.
An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, it implements claims 1 to The method described in any one of 13.
A computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.
A chip, characterized in that the chip includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the method of any one of claims 1 to 13 is implemented. method.