WO2022105716A1

WO2022105716A1 - Camera control method based on distributed control, and terminal device

Info

Publication number: WO2022105716A1
Application number: PCT/CN2021/130672
Authority: WO
Inventors: 占航
Original assignee: 华为技术有限公司
Priority date: 2020-11-20
Filing date: 2021-11-15
Publication date: 2022-05-27
Also published as: CN115484404B; CN114520867A; CN114520867B; CN115484404A

Abstract

The present application relates to a camera control method based on distributed control, and a terminal device. The method, which is applied to a first device, comprises: displaying cameras to be selected that can be controlled by a first device, wherein the cameras to be selected comprise a local camera of the first device, and a local camera, to which a first virtual camera in the first device is mapped, of a second device; determining a selected camera from the cameras to be selected, and a target task the selected camera needs to execute; and generating a first task command according to a first camera identifier, in the first device, of the selected camera, and the target task, and sending the first task command to the selected camera, such that the selected camera controls the execution of the target task according to the first task command, wherein each first virtual camera of the first device is used for realizing control over the mapped local camera, which has at least one level of mapping relationship, of the second device. By means of the method and apparatus provided in the present application, direct and/or indirect control over local cameras in a plurality of second devices is realized by means of one first device, such that camera control requirements in different application scenarios are met.

Description

Camera control method and terminal device based on distributed control

This application claims the priority of the Chinese patent application with the application number 202011308989.4 and the application title "Camera Control Method and Terminal Equipment Based on Distributed Control", which was filed with the China Patent Office on November 20, 2020, the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the field of terminal technologies, and in particular, to a distributed control-based camera control method and terminal device.

Background technique

With the development of cameras, the types of devices equipped with cameras are increasing, for example, smart TVs with cameras, Bluetooth cameras, home cameras, road surveillance cameras, drones with cameras, and so on. In the related art, a device equipped with a camera can be remotely controlled through a terminal device such as a mobile phone or a system, so as to implement tasks such as image shooting and video shooting. Taking the control terminal of multiple drones for photographing control as an example, in the related art, each drone performs network signal transmission with the control terminal through Ethernet, and the control terminal transmits control commands such as taking pictures to each drone. After analyzing, the man-machine further performs the photographing action, and then transmits the photographic data network back to the control terminal. In order to achieve the above control, each drone needs to include a network module and can be connected to the control terminal to realize the direct control of the drone, and this method causes the control terminal to only control the unmanned aerial vehicle directly connected to the network through the network. For drones that cannot be directly connected to the control terminal through the network, it cannot be controlled. How to realize the indirect control of the camera device on the basis of the direct control of the camera device to meet the needs of different camera device usage scenarios is a technical problem to be solved urgently.

SUMMARY OF THE INVENTION

In view of this, a camera control method and terminal device based on distributed control are proposed.

In a first aspect, an embodiment of the present application provides a distributed control-based camera control method, which is applied to a first device, and the method includes:

Displaying a camera to be selected that can be controlled by the first device, the camera to be selected includes a local camera of the first device and a local camera of the second device mapped by the first virtual camera in the first device;

According to the detected operation of creating a task for the camera to be selected, determine the selected camera and a target task to be performed by the selected camera from the cameras to be selected;

generating a first task command according to the first camera identifier of the selected camera in the first device and the target task;

sending the first task command to the selected camera, so that the selected camera controls the execution of the target task according to the first task command,

The first device includes at least one first virtual camera, each first virtual camera is used to control the local camera of the mapped second device, and each first virtual camera is connected to the mapped second device There is at least one-level mapping relationship between the local cameras of ,

When there is a multi-level mapping relationship between the first virtual camera and the mapped local camera of the second device, the second device and the first device are in different local area networks.

With the method provided in the first aspect, a first device can be used to control local cameras in multiple second devices, and the second devices can directly establish a connection with the first device through the same local area network, or use an intermediate The device establishes an indirect connection with the first device in a different local area network, which realizes the distributed and hierarchical control of the local cameras of different second devices, and can meet the camera control requirements of different application scenarios.

According to the first aspect, in a first possible implementation manner of the method, the method further includes:

When a device connection request is detected, search for a third device that can be connected to the first device and meets a connection condition, where the connection condition includes: the third device is provided with a local camera and/or a third device in the third device at least one second virtual camera is created;

sending a first authorization control request to the third device, and receiving a first authorization indication returned by the third device in response to the first authorization control request;

After determining the authorized first authorized camera according to the first authorization instruction, obtain a second camera identification of the first authorized camera in the third device, where the first authorized camera includes the third camera the local camera of the device and/or the local camera of the fourth device mapped by the second virtual camera;

A first camera identification of the first authorized camera in the first device is determined according to the second camera identification, and a first camera identification for controlling the first authorization is created according to the first camera identification of the first authorized camera Camera's first virtual camera,

Wherein, each second virtual camera is used to control the local camera of the fourth device mapped by the second virtual camera, and there is at least one connection between the second virtual camera and the mapped local camera of the fourth device. level mapping.

Through a first possible implementation, a first virtual camera capable of controlling an authorized camera is created.

According to a first possible implementation manner, in a second possible implementation manner of the method, a first authorization control request is sent to the third device, and a response from the third device to the first authorization is received The first authorization indication returned by the control request includes:

selecting a first requested camera according to a request operation for the local camera of the third device that can be controlled by the third device and/or the local camera of the fourth device mapped by the second virtual camera;

The camera generates the first authorization control request according to the first request, and sends the first authorization control request to the third device, so that the third device can make the third device according to the detected authorization for the first authorization The authorization operation of the control request generates the first authorization indication.

Through the second possible implementation manner, the first request camera can be selected according to the user's needs, so as to meet the authorization control requirements of different users.

According to a first possible implementation manner, in a third possible implementation manner of the method, a first camera identification of the first authorized camera in the first device is determined according to the second camera identification , and create a first virtual camera for controlling the first authorized camera according to the first camera identification of the first authorized camera, including:

When the first authorized camera includes a local camera of the fourth device mapped by the second virtual camera, indicating the mapping between the first authorized camera and the second virtual camera according to the second camera identifier the mapping relationship level of the relationship, determining the first mapping relationship level between the first authorized camera and the first virtual camera that needs to be created to control the first authorized camera;

According to the identification in the second camera identification and the identification of the existing camera in the first device corresponding to the first mapping relationship level, it is determined that the first authorized camera is in the first device 's primary identity;

A first camera identification of the first authorized camera in the first device is determined according to the first mapping relationship level and the first identification identification, and a first camera identification for controlling all cameras is created according to the first camera identification. The first virtual camera of the first authorized camera.

Through the third possible implementation manner, after the first virtual camera is created, the first device can directly determine the relationship between the first device and the first authorized camera according to the first camera identifier of the first virtual camera and the first mapping relationship level The level of the mapping relationship between them is convenient for sending commands and receiving data.

According to a third possible implementation manner, in a fourth possible implementation manner of the method, a first camera identification of the first authorized camera in the first device is determined according to the second camera identification , and create a first virtual camera for controlling the first authorized camera according to the first camera identification of the first authorized camera, including:

When the first authorized camera includes a local camera of the third device, a first-level mapping relationship is determined as a first-level mapping relationship between the first authorized camera and a first virtual camera that needs to be created to control the first authorized camera The first mapping relationship level.

According to a third possible implementation manner, in a fifth possible implementation manner of the method, according to the identity identifier in the second camera identifier, the first device corresponds to the first mapping relationship level The identity identifier of the existing camera is determined, and the first identity identifier of the first authorized camera in the first device is determined, including:

In the identity identifier of the existing camera corresponding to the first mapping relationship level in the first device, when the identity identifier in the second camera identifier already exists, create the identity identifier according to a preset identity identifier creation rule the first identity of the first authorized camera in the first device; or

When the identity identifier in the second camera identifier is different from the identity identifier of the existing camera corresponding to the first mapping relationship level in the first device, the identity identifier in the second camera identifier is changed to It is determined as the first identity identifier of the first authorized camera in the first device.

Through the fifth possible implementation manner, the first identity of the first authorized camera can be guaranteed to be unique among the identities of all cameras corresponding to the first mapping relationship level that can be controlled by the first device, so as to facilitate The distinction between cameras at the same mapping relationship level that can be controlled by the first device.

According to the first aspect, in a sixth possible implementation manner of the method, the selected camera and the selected camera are determined from the candidate cameras according to the detected operation of creating the task for the candidate camera. Select the target tasks to be performed by the camera, including:

determining the selected camera according to the detected selection operation on the camera to be selected;

According to the task setting operation for the selected camera, the target task to be performed by the selected camera is determined,

Wherein, the task parameters of the target task include at least one of task type, execution time information, and camera parameter settings when the selected camera performs the target task, and the task type includes at least one of the following: photographing tasks, image preview tasks.

Through the sixth possible implementation manner, the target task can be precisely set so that the selected camera can perform the target task.

According to the first aspect, in a seventh possible implementation manner of the method, the first task command is sent to the selected camera, so that the selected camera controls execution according to the first task command The target task includes at least one of the following operations:

When the selected camera includes the local camera of the first device, the first task command is sent to the local camera of the first device, so that the local camera of the first device executes the first task the target task indicated by the command;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a first-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, pass The first virtual camera corresponding to the selected camera in the first device forwards the first task command to the local camera of the second device, so that the local camera of the second device executes the first task command. a target task indicated by the task command;

When the selected camera includes the local camera of the second device mapped by the first virtual camera, and there is a multi-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, according to The first mapping relationship level between the selected camera and the corresponding first virtual camera determines at least one intermediate device that completes the forwarding of the first task command, and sequentially passes through each intermediate device corresponding to the selected camera. The virtual camera of the camera forwards the first task command to the local camera of the second device, so that the local camera of the second device executes the target task indicated by the first task command.

Through the seventh possible implementation manner, the first task command can be forwarded to the selected camera by means of at least one intermediate device, so as to realize the control of the local camera of the device in a different local area network from the first device.

According to the first aspect or the seventh possible implementation manner, in an eighth possible implementation manner of the method, the method further includes:

When receiving the target task data obtained by the selected camera performing the target task, performing image and/or video display according to the target task data,

Wherein, receiving the target task data includes at least one of the following ways:

When the selected camera includes the local camera of the first device, directly receiving target task data sent from the local camera of the first device;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a first-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, use The first virtual camera corresponding to the selected camera directly receives the target task data sent by the local camera of the second device;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a multi-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, use The first virtual camera corresponding to the selected camera receives the target task data sequentially sent by the local camera of the second device and forwarded by at least one intermediate device.

Through the eighth possible implementation manner, the first device may receive target task data after the selected camera performs the target task, and perform image and/or video presentation according to the target task data.

According to the first aspect, in a ninth possible implementation manner of the method, the method further includes:

When receiving the second authorization control request from the fifth device, the camera displays an authorization prompt according to the second request in the second authorization control request;

determining an authorized second authorized camera according to the detected authorization operation for the second requested camera;

A second authorization instruction is generated according to the first camera identification of the second authorized camera in the first device, and the second authorization instruction is sent to the fifth device, so that the fifth device The second authorization instruction creates a virtual camera for controlling the second authorized camera.

Through the ninth possible implementation manner, the first device can directly control its local camera and the local camera of the second device mapped by the set first virtual camera according to the command issued by the user, and can also establish control with the fifth device. relationship, controlled by the fifth device.

In a second aspect, an embodiment of the present application provides a terminal device, which can execute the distributed control-based camera of the first aspect or one or more of various possible implementation manners of the first aspect. Control Method.

In a third aspect, embodiments of the present application provide a computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in an electronic When running in the device, the processor in the electronic device executes the first aspect or one or more of the distributed control-based camera control methods in the first aspect or multiple possible implementation manners of the first aspect.

These and other aspects of the present application will be more clearly understood in the following description of the embodiment(s).

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features and aspects of the application and together with the description, serve to explain the principles of the application.

FIG. 1 shows a schematic structural diagram of a terminal device according to an embodiment of the present application.

FIG. 2 shows a block diagram of a software structure of a terminal device according to an embodiment of the present application.

FIG. 3 shows a flowchart of a camera control method based on distributed control according to an embodiment of the present application.

FIG. 4 shows a schematic diagram of an application scenario of a camera control method based on distributed control according to an embodiment of the present application.

FIG. 5 shows a schematic diagram of a process of determining a target task according to an embodiment of the present application.

FIG. 6 shows a flowchart of a camera control method based on distributed control according to an embodiment of the present application.

FIG. 7 shows a schematic diagram of selecting an authorized camera according to an embodiment of the present application.

FIG. 8 shows a schematic diagram of an implementation process of a camera control method based on distributed control according to an embodiment of the present application.

Detailed ways

Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

In addition, in order to better illustrate the present application, numerous specific details are given in the following detailed description. It should be understood by those skilled in the art that the present application may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present application.

In order to solve the above technical problems, the present application provides a camera control method based on distributed control. The distributed control-based camera control method in the embodiment of the present application can realize multi-level indirect control of camera devices, and is suitable for different camera devices. Depending on the usage scenario, the method can be applied to devices such as a terminal device, a control device for implementing camera control, and the like, so as to implement the above method.

The equipment involved in this application (including the first equipment, the second equipment, the third equipment, the fourth equipment, etc.) may refer to the equipment with the wireless connection function, and the wireless connection function refers to the wireless connection methods such as wifi and bluetooth. To connect with other devices, the device of the present application may also have the function of wired connection for communication. The device of the present application can be a touch screen, a non-touch screen, or no screen. The touch screen can control the device by clicking, sliding, etc. on the display screen with a finger, a stylus pen, etc. A device with a touch screen can be connected to input devices such as a mouse, keyboard, and touch panel, and the device can be controlled through the input device. For example, a device without a screen can be a Bluetooth speaker without a screen.

For example, the terminal devices in the devices involved in this application may be smart phones, netbooks, tablet computers, notebook computers, wearable electronic devices (such as smart bracelets, smart watches, etc.), TVs, virtual reality devices, audio, electronic ink, etc.

FIG. 1 shows a schematic structural diagram of a terminal device according to an embodiment of the present application. Taking the terminal device being a mobile phone as an example, FIG. 1 shows a schematic structural diagram of the mobile phone 200 .

The mobile phone 200 may include a processor 210, an external memory interface 220, an internal memory 221, a USB interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 251, a wireless communication module 252, Audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, sensor module 280, buttons 290, motor 291, indicator 292, camera 293, display screen 294, SIM card interface 295, etc. The sensor module 280 may include a gyroscope sensor 280A, an acceleration sensor 280B, a proximity light sensor 280G, a fingerprint sensor 280H, and a touch sensor 280K (of course, the mobile phone 200 may also include other sensors, such as a temperature sensor, a pressure sensor, a distance sensor, and a magnetic sensor. , ambient light sensor, air pressure sensor, bone conduction sensor, etc., not shown in the figure).

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the mobile phone 200 . In other embodiments of the present application, the mobile phone 200 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or Neural-network Processing Unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. The controller may be the nerve center and command center of the mobile phone 200 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in processor 210 is cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 210 . If the processor 210 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided, and the waiting time of the processor 210 is reduced, thereby improving the efficiency of the system.

The processor 210 may execute the distributed control-based camera control method provided by the embodiments of the present application, so as to facilitate multi-level indirect control of the camera device, which is suitable for use scenarios of different camera devices. The processor 210 may include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU may cooperate to execute the distributed control-based camera control method provided in the embodiments of the present application. The CPU is executed, and another part of the algorithm is executed by the GPU to obtain faster processing efficiency.

Display screen 294 is used to display images, videos, and the like. Display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, cell phone 200 may include 1 or N display screens 294, where N is a positive integer greater than 1. The display screen 294 may be used to display information entered by or provided to the user as well as various graphical user interfaces (GUIs). For example, display 294 may display photos, videos, web pages, or documents, and the like. As another example, display 294 may display a graphical user interface. The GUI includes a status bar, a hideable navigation bar, a time and weather widget, and an application icon, such as a browser icon. The status bar includes operator name (eg China Mobile), mobile network (eg 4G), time and remaining battery. The navigation bar includes a back button icon, a home button icon, and a forward button icon. In addition, it can be understood that, in some embodiments, the status bar may further include a Bluetooth icon, a Wi-Fi icon, an external device icon, and the like. It can also be understood that, in other embodiments, the graphical user interface may further include a Dock bar, and the Dock bar may include commonly used application icons and the like. After the processor 210 detects a touch event of the user's finger (or stylus, etc.) on an application icon, in response to the touch event, the user interface of the application corresponding to the application icon is opened and displayed on the display 294 The user interface of the application.

In this embodiment of the present application, the display screen 294 may be an integrated flexible display screen, or a spliced display screen composed of two rigid screens and a flexible screen located between the two rigid screens.

After the processor 210 runs the distributed control-based camera control method provided by the embodiment of the present application, the terminal device as the first device can establish a communication connection with the second device that can be directly connected through the antenna 1 and the antenna 2, and according to the present application The distributed control-based camera control method provided by the embodiment realizes the control of the local camera of the second device by the first device, and the control of the local camera of the second device that cannot directly establish a communication connection with the first device.

Camera 293 (front camera or rear camera, or a camera that can be both a front camera and a rear camera) is used to capture still images or video. Generally, the camera 293 may include a photosensitive element such as a lens group and an image sensor, wherein the lens group includes a plurality of lenses (convex or concave) for collecting the light signal reflected by the object to be photographed, and transmitting the collected light signal to the image sensor . The image sensor generates an original image of the object to be photographed according to the light signal.

Internal memory 221 may be used to store computer executable program code, which includes instructions. The processor 210 executes various functional applications and data processing of the mobile phone 200 by executing the instructions stored in the internal memory 221 . The internal memory 221 may include a storage program area and a storage data area. The storage program area may store operating system, code of application programs (such as camera application, WeChat application, etc.), and the like. The storage data area may store data created during the use of the mobile phone 200 (such as images and videos collected by the camera application) and the like.

The internal memory 221 may also store one or more computer programs 1310 corresponding to the distributed control-based camera control methods provided in the embodiments of the present application. The one or more computer programs 1304 are stored in the aforementioned memory 221 and configured to be executed by the one or more processors 210, and the one or more computer programs 1310 include instructions that may be used to perform the execution of FIG. 3 6, the computer program 1310 may include one or more modules for executing the following steps to realize multi-level indirect control of the camera device, which is suitable for use scenarios of different camera devices.

In addition, the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

Certainly, the codes of the distributed control-based camera control method provided by the embodiments of the present application may also be stored in an external memory. In this case, the processor 210 may execute the codes of the distributed control-based camera control method stored in the external memory through the external memory interface 220 .

The function of the sensor module 280 is described below.

The gyro sensor 280A can be used to determine the movement posture of the mobile phone 200 . In some embodiments, the angular velocity of cell phone 200 about three axes (ie, x, y, and z axes) may be determined by gyro sensor 280A. That is, the gyro sensor 280A can be used to detect the current motion state of the mobile phone 200, such as shaking or still.

When the display screen in the embodiment of the present application is a foldable screen, the gyro sensor 280A can be used to detect a folding or unfolding operation acting on the display screen 294 . The gyroscope sensor 280A may report the detected folding operation or unfolding operation to the processor 210 as an event to determine the folding state or unfolding state of the display screen 294 .

The acceleration sensor 280B can detect the magnitude of the acceleration of the mobile phone 200 in various directions (generally three axes). That is, the gyro sensor 280A can be used to detect the current motion state of the mobile phone 200, such as shaking or still. When the display screen in the embodiment of the present application is a foldable screen, the acceleration sensor 280B can be used to detect a folding or unfolding operation acting on the display screen 294 . The acceleration sensor 280B may report the detected folding operation or unfolding operation to the processor 210 as an event to determine the folding state or unfolding state of the display screen 294 .

Proximity light sensor 280G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The mobile phone emits infrared light outward through light-emitting diodes. Phones use photodiodes to detect reflected infrared light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the phone. When insufficient reflected light is detected, the phone can determine that there are no objects near the phone. When the display screen in the embodiment of the present application is a foldable screen, the proximity light sensor 280G can be arranged on the first screen of the foldable display screen 294, and the proximity light sensor 280G can detect the first screen according to the optical path difference of the infrared signal. The size of the folding or unfolding angle between the screen and the second screen.

The gyroscope sensor 280A (or the acceleration sensor 280B) may send the detected motion state information (such as angular velocity) to the processor 210 . The processor 210 determines, based on the motion state information, whether the current state is the hand-held state or the tripod state (for example, when the angular velocity is not 0, it means that the mobile phone 200 is in the hand-held state).

The fingerprint sensor 280H is used to collect fingerprints. The mobile phone 200 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering incoming calls with fingerprints, and the like.

Touch sensor 280K, also called "touch panel". The touch sensor 280K may be disposed on the display screen 294, and the touch sensor 280K and the display screen 294 form a touch screen, also called a "touch screen". The touch sensor 280K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 294 . In other embodiments, the touch sensor 280K may also be disposed on the surface of the mobile phone 200 , which is different from the location where the display screen 294 is located.

Exemplarily, the display screen 294 of the mobile phone 200 displays a main interface, and the main interface includes icons of multiple applications (such as a camera application, a WeChat application, etc.). The user clicks the icon of the camera application in the main interface through the touch sensor 280K, which triggers the processor 210 to start the camera application and turn on the camera 293 . Display screen 294 displays an interface of a camera application, such as a viewfinder interface.

The wireless communication function of the mobile phone 200 can be realized by the antenna 1, the antenna 2, the mobile communication module 251, the wireless communication module 252, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in handset 200 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 251 can provide a wireless communication solution including 2G/3G/4G/5G, etc. applied on the mobile phone 200 . The mobile communication module 251 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 251 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 251 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 251 may be provided in the processor 210 . In some embodiments, at least part of the functional modules of the mobile communication module 251 may be provided in the same device as at least part of the modules of the processor 210 . In this embodiment of the present application, the mobile communication module 251 may also be used to interact with other terminal devices in information such as first task commands, target task data, and the like.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 270A, the receiver 270B, etc.), or displays images or videos through the display screen 294 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modulation and demodulation processor may be independent of the processor 210, and may be provided in the same device as the mobile communication module 251 or other functional modules.

The wireless communication module 252 can provide applications on the mobile phone 200 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 252 may be one or more devices integrating at least one communication processing module. The wireless communication module 252 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210 . The wireless communication module 252 can also receive the signal to be sent from the processor 210 , perform frequency modulation on the signal, amplify the signal, and then convert it into an electromagnetic wave for radiation through the antenna 2 . In this embodiment of the present application, the wireless communication module 252 is configured to transmit data between other terminal devices under the control of the processor 210. For example, when the processor 210 runs the distributed control-based camera control method provided in the embodiment of the present application , the processor in the first device can control the wireless communication module 252 to send the first task command to the local camera of the second device that establishes a direct communication connection with the first device, and send the first task command to the local camera of the second device that cannot directly communicate with the first device through the mapping of the first virtual camera. The local camera of the second device to which the device establishes a communication connection sends the first task command to realize multi-level indirect control of the camera device, which is suitable for use scenarios of different camera devices.

In addition, the mobile phone 200 can implement audio functions through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, and an application processor. Such as music playback, recording, etc. The cell phone 200 can receive key 290 input and generate key signal input related to user settings and function control of the cell phone 200 . The mobile phone 200 can use the motor 291 to generate vibration alerts (eg, vibration alerts for incoming calls). The indicator 292 in the mobile phone 200 may be an indicator light, which may be used to indicate a charging state, a change in power, and may also be used to indicate a message, a missed call, a notification, and the like. The SIM card interface 295 in the mobile phone 200 is used to connect the SIM card. The SIM card can be contacted and separated from the mobile phone 200 by inserting into the SIM card interface 295 or pulling out from the SIM card interface 295 .

It should be understood that, in practical applications, the mobile phone 200 may include more or less components than those shown in FIG. 1 , which is not limited in this embodiment of the present application. The illustrated handset 200 is merely an example, and the handset 200 may have more or fewer components than those shown, two or more components may be combined, or may have different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The software system of the terminal device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe the software structure of a terminal device.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer.

The application layer can include a series of application packages.

As shown in FIG. 2, the application package may include applications such as phone, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.

Content providers are used to store and retrieve data and make these data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The telephony manager is used to provide the communication function of the terminal device. For example, the management of call status (including connecting, hanging up, etc.).

The resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the terminal device vibrates, and the indicator light flashes.

Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.

The Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

The embodiment of the present application provides a distributed control-based camera control method, which can realize control of local cameras in multiple second devices through a first device, and the second device can directly communicate with the first device through the same local area network Establishing a connection, or establishing an indirect connection with a first device in a different local area network by means of an intermediate device, realizes distributed and hierarchical control of local cameras of different second devices, and can meet camera control requirements in different application scenarios.

FIG. 3 shows a flowchart of a camera control method based on distributed control according to an embodiment of the present application. FIG. 4 shows a schematic diagram of an application scenario of a camera control method based on distributed control according to an embodiment of the present application. As shown in FIG. 3, the method is applied to the first device, and the method includes steps S11 to S14.

In step S11, a camera to be selected that can be controlled by the first device is displayed, and the camera to be selected includes a local camera of the first device and a camera of the second device mapped by the first virtual camera in the first device. local camera. The first device includes at least one first virtual camera, each first virtual camera is used to control the local camera of the mapped second device, and each first virtual camera is connected to the mapped second device There is at least one-level mapping relationship between the local cameras. When there is a multi-level mapping relationship between the first virtual camera and the mapped local camera of the second device, the second device and the first device are in different local area networks.

In this embodiment, the mapping relationship between the first virtual camera and the mapped local camera of the second device may instruct the first virtual camera to realize the local camera of the mapped second device under the control of the first device The number of mappings to be controlled, the data transmission process obtained by the two commands and the task performed by the camera need to be forwarded several times to complete the transmission process. Moreover, the mapping relationship between the first virtual camera and the mapped local camera of the second device can also be considered as the mapping relationship between the first device and the local camera of the second device mapped by the first virtual camera. The lower the level of the mapping relationship, the less the number of times of mapping between the first virtual camera and the mapped local camera of the second device, and the less the number of times of transmission and forwarding. The mapping relationship level includes zero-level mapping relationship, first-level mapping relationship, second-level mapping relationship… level mapping.

In order to further explain the mapping relationship, the following description will be given in conjunction with the application scenario example given in FIG. 4 . As shown in FIG. 4 , the first device A includes a local camera 100 and at least one first virtual camera 200 . The mapping relationship between the local camera 100 of the first device A and the first device A is a zero-level mapping relationship. The second device mapped by each first virtual camera 200 may be a device that can be directly connected to the first device through a network, such as device B1 shown in FIG. 4 , the local camera in device B1 and the corresponding first virtual camera 200 is a first-level mapping relationship, and the device B1 and the first device A may be in the same local area network. The second device mapped by the first virtual camera 200 may also be a device that cannot be directly connected to the first device through the network, and communicates with the first device A indirectly through other devices, such as device B2 and device A as shown in FIG. B3. Among them, the device B2 is mapped to the first virtual camera 202, and the first device A passes the first virtual camera 202 according to the "first virtual camera 202"

The second virtual camera 301

The communication mode of the "local camera of the device B2" realizes the control of the local camera of the device B2, including sending the first task command and receiving the target task data; the device B2 and the first virtual camera 202 are the secondary mapping relationship, and the device B2 and the second A device A is in a different local area network, the device B2 and the device C1 can be in the same local area network, and the device C1 and the first device A can be in the same local area network. The device B3 is mapped to the first virtual camera 203, and the first device A The virtual camera 203 follows the "first virtual camera 203

The communication mode of the second virtual camera 303...the local camera of the device B3" realizes the control of the local camera of the device B3, including sending the first task command and receiving the target task data; the device B3 and the first virtual camera 203 are the x-level mapping. relationship (x=number of virtual cameras), device B3 and first device A are in different local area networks, device C2 and device C3 may be in the same local area network, and device C2 and first device A may be in the same local area network.

The device B1, the device B2, and the device B3 may be the same or different types of devices, for example, the device B1 is a mobile phone, the device B2 is a surveillance camera, and the device B3 is a drone with a camera. The local camera of the device C1, the local camera of the device C2, and the local camera of the device C3 can also be mapped to the first device A to form a virtual camera after being authorized. For the sake of simplicity, the mapping relationship is not shown in FIG. 4 . , those skilled in the art can implement the method according to the method provided in this application according to the implementation example of mapping the device B1, the device B2, and the device B3 to the first device A, which is not repeated here.

In step S12, according to the detected operation of creating a task for the camera to be selected, the selected camera and the target task to be performed by the selected camera are determined from the cameras to be selected.

In a possible implementation manner, step S12 may include: determining the selected camera according to the detected selection operation for the to-be-selected camera; determining the selected camera according to the task setting operation for the selected camera The target task to be performed by the selected camera. The task parameters of the target task may include at least one of task type, execution time information, and camera parameter settings when the selected camera performs the target task, and the task type includes at least one of the following: a photo-taking task, Camera tasks, image preview tasks. In this way, the target task can be precisely set so that the selected camera can perform the target task.

The execution time information and camera parameters corresponding to different task types need to be set accordingly. For a photo-taking task, the execution time information may include the number of photos taken and the time when the photo was taken, and the camera parameters may include the pixels of the photo taken, the exposure duration of the photo, and the photo mode (such as day mode, night mode, panorama mode, etc.) The parameters of the photo shooting. For a photographing task, the execution time information may include the start and end time of the photographing, the photographing duration, etc., and the camera parameters may include the pixels of the photographing, the photographing mode (such as day mode, night mode, etc.), which are used to set the parameters for the camera to shoot video. For the image preview task, the execution time information may include the shooting time of the photo to be previewed (the photo taken by opening the camera viewfinder in real time or the photo taken before the camera), the video to be previewed (the video taken by the camera being opened in real time or the video taken before the camera is turned on) The camera parameters can include the pixels for taking pictures, the exposure time for taking pictures, and the camera mode (such as day mode, night mode, panorama mode, etc.), which are used to set the parameters for the camera to take photos.

In this embodiment, when the first device is provided with a display screen, the camera to be selected may be displayed on the display screen in the form of a combination of pictures and/or text. It is also possible to prompt the user for a camera to be selected for selection by means of a voice, such as a speaker set in the first device. FIG. 5 shows a schematic diagram of a process of determining a target task according to an embodiment of the present application. As shown in FIG. 4 and FIG. 5 , the first device is the first device A shown in FIG. 4 , and the interface T1 displays the camera to be selected by the user “local camera 100 , the local camera mapped to the device B1 ” The first virtual camera 201, the first virtual camera 202 mapped to the local camera of the device B2, and the first virtual camera 203″ mapped to the local camera of the device B3, trigger operations according to the detected click, slide, etc. of the selection control K The selected camera selected by the user "the local camera of the device B1 mapped by the first virtual camera 201" is determined. Then, the interface T1 in the display screen can be further switched to T2, and the task type "photographing, videography, image preview" can be displayed in the interface T2, and then determined according to the detected trigger operations such as clicking, sliding, etc. on the selected control K in the interface T2. The task type selected by the user is "photographing task". After that, the interface T2 in the display screen can be further switched to T3, and the camera parameter setting prompt corresponding to the task type "camera task" can be displayed in the interface T3. Prompt the user to directly input or drop-down selection to determine the shooting duration of the video. "Camera parameters and multiple blank boxes after that" are used to prompt the user to directly input or drop-down selection to determine the camera parameters for photography. Those skilled in the art can set the implementation manner of step S11 and step S12 according to actual needs, which is not limited in this application.

In step S13, a first task command is generated according to the first camera identifier of the selected camera in the first device and the target task.

In this embodiment, the generated first task command includes the target task to be performed by the selected camera and the first camera identifier. The first camera identifier includes a first mapping relationship level and a first identity identifier, and the first mapping relationship level can be used to represent the mapping relationship between the selected camera and its corresponding first virtual camera, or to represent the selected camera and the first virtual camera. A mapping relationship between devices. The first identity identifier may be used to represent the difference between the selected camera and the existing cameras corresponding to the first mapping relationship level in the first device. Those skilled in the art can set the mapping relationship level and the way of identity identification according to actual needs, which is not limited in this application. In this way, different cameras that can be controlled by the first device can be distinguished by the first identity identifier, and the accurate delivery of the first task command is also ensured.

In step S14, the first task command is sent to the selected camera, so that the selected camera is controlled to execute the target task according to the first task command.

In a possible implementation manner, step S14 may include at least one of the following operation 1, operation 2, and operation 3:

Operation 1: when the selected camera includes the local camera of the first device, send the first task command to the local camera of the first device, so that the local camera of the first device executes the The target task indicated by the first task command. For example, as shown in FIG. 4 , the first device A may directly send the first task command to its own local device 100 .

Operation 3: The selected camera includes the local camera of the second device mapped by the first virtual camera, and there is a first-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera when the first task command is forwarded to the local camera of the second device through the first virtual camera corresponding to the selected camera in the first device, so that the local camera of the second device executes The target task indicated by the first task command. For example, as shown in FIG. 4 , the first device A may control the first virtual camera 201 to send the first task command to the “local camera of device B1”.

Operation 3: The selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a multi-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera At least one intermediate device that completes the forwarding of the first task command is determined according to the first mapping relationship level between the selected camera and the corresponding first virtual camera, and sequentially passes through the corresponding intermediate devices in each intermediate device. The virtual camera of the selected camera forwards the first task command to the local camera of the second device, so that the local camera of the second device executes the target task indicated by the first task command. Wherein, in the process of forwarding the first task command and forwarding the target task data and command results described below, the intermediate device may directly forward it in a transparent transmission manner, or may encrypt it according to a preset encryption method. Forwarding is not limited in this application. For example, as shown in FIG. 4 , the first device A can control the first virtual camera 202 to send the first task command to "the second virtual camera 301 of the device C1", and the second virtual camera 301 of the device C1 will then send the first task command to the "second virtual camera 301 of the device C1". The command is forwarded to the "local device of device B2", and the intermediate device is device C1 in this process. The first device A can control the first virtual camera 203 to send the first task command to the "second virtual camera 302 of the device C2", and the second virtual camera 302 of the device C2 forwards the first task command to the "second virtual camera 302 of the device C3". Three virtual cameras 400 ″, the third virtual camera 400 of device C3 . . . until the first task command is sent to the local device of device B2, in this process, the intermediate devices are device C2, device C3 . . . In this way, the first task command can be forwarded to the selected camera by means of at least one intermediate device, so as to realize the control of the local camera of the device in a different local area network from the first device.

In a possible implementation manner, when the selected camera needs to perform different target tasks from different devices that conflict with each other in the same time period, the device with the highest priority can be selected according to the priority of the device sending the target task. target task execution. Alternatively, randomly select one of multiple target tasks for execution. Or, in order to avoid the occurrence of "the selected camera needs to perform conflicting target tasks from different devices at the same time", the device where the selected camera is located can control each camera it can control only when it is authorized. License to only one device to control via virtual camera.

FIG. 6 shows a flowchart of a camera control method based on distributed control according to an embodiment of the present application. In a possible implementation manner, as shown in FIG. 6 , the method may further include steps S15 to S18 of a “virtual camera creation step”. The "virtual camera creation step" may be performed before step S11 (as shown in FIG. 6 ), or may be performed after step S11 (not shown in the figure). When the third device described below is the same device as the second device described above, the "virtual camera creation step" may be performed before step S11. When the following third device is different from the above-mentioned second device, the "virtual camera creation step" may be performed before, after, or simultaneously with step S11 , which is not limited in this application. A first virtual camera capable of controlling an authorized camera is created through the "virtual camera creation step".

In step S15, when a device connection request is detected, a third device that can be connected to the first device and satisfies a connection condition is searched for, and the connection condition may include: the third device is provided with a local camera and/or At least one second virtual camera is created in the third device. Wherein, each second virtual camera is used to control the local camera of the fourth device mapped by the second virtual camera, and there is at least one connection between the second virtual camera and the mapped local camera of the fourth device. level mapping. The setting of the connection conditions can ensure that the existence of the determined third device can be controlled by the first device by creating a virtual camera. Wherein, the third device may be any device such as a mobile phone, a drone, etc., which is not made in this application.

For example, as shown in FIG. 4 , it is assumed that the third devices that satisfy the connection condition determined by the first device A after searching are the device B1 and the device C1 . Then the first device A can access the third device to determine the cameras that can be controlled in each third device. For example, it can be determined through the access that the device B1 can only control its own local camera, the device C1 can control its own local camera, and the local camera of the device B2 mapped by the second virtual camera 301 .

In step S16, a first authorization control request is sent to the third device, and a first authorization indication returned by the third device in response to the first authorization control request is received.

In a possible implementation manner, step S16 may include: according to the local camera of the third device that can be controlled by the third device and/or the local camera of the fourth device mapped by the second virtual camera select the first requesting camera; generate the first authorization control request according to the first requesting camera, and send the first authorization control request to the third device, so that the third The device generates the first authorization indication according to the detected authorization operation for the first authorization control request. In this way, the first request camera can be selected according to the needs of the user, so as to meet the authorization control requirements of different users.

In this implementation manner, information about the third device that the first device can connect to and the camera that the third device can control can be displayed on the display screen of the first device with reference to the display mode of the interface T1 in FIG. 5 . Then, the third device to be connected by the user and the first request camera to be controlled by the third device are determined according to the detected click and other operations (that is, the request operation), and the first authorization control request is generated, and the first authorization control request is generated. The authorization control request is sent to the corresponding third device. After the third device receives the first authorization control request, it generates the first authorization indication according to the user's authorization operation (for this process, refer to the implementation process and manner of the first device responding to the second authorization control request below). The voice issued by the user can also be recognized, and the third device to be connected by the user and the first request camera controlled by the third device to be connected are determined according to the recognition result. The information of the third device and the cameras that can be controlled by the third device can also be played to the user by voice, and then the third device to be connected by the user and the capabilities of the third device that needs to be connected can be determined according to the voice sent by the user's response. The first request to control the camera. Those skilled in the art can set the manner of generating the first authorization control request according to the request operation according to actual needs, which is not limited in this application.

In a possible implementation manner, after the third device is determined, a first authorization control request for each camera controlled by the detected third device may be sent directly to each detected third device, without requiring user-based requests The operation generates a first authorization control request. In this way, the operation of the user can be simplified, the speed of creating the virtual camera can be improved, and the operations required by the user during the creation process can be simplified.

In step S17, after determining the authorized first authorized camera according to the first authorization instruction, obtain the second camera identification of the first authorized camera in the third device, the first authorized camera Including the local camera of the third device and/or the local camera of the fourth device mapped by the second virtual camera.

In this implementation manner, the second camera identification of the first authorized camera in the third device also includes a mapping relationship level and an identification identification, and the meanings indicated by the first authorized camera are referred to above in the first mapping relationship level and the first identification identification. description, which will not be repeated here.

In step S18, a first camera identification of the first authorized camera in the first device is determined according to the second camera identification, and a first camera identification for controlling the first authorized camera is created according to the first camera identification of the first authorized camera A first virtual camera of the first authorized camera.

In a possible implementation manner, step S18 may include: when the first authorized camera includes the local camera of the fourth device mapped by the second virtual camera, according to the indication in the second camera identifier, the first authorized camera A mapping relationship level of the mapping relationship between the authorized camera and the second virtual camera, to determine the first mapping between the first authorized camera and the first virtual camera that needs to be created to control the first authorized camera relationship level; it is determined that the first authorized camera is in the A first identity identifier in a device; determine the first camera identifier of the first authorized camera in the first device according to the first mapping relationship level and the first identity identifier, and determine the first camera identifier of the first authorized camera in the first device according to the first A camera identification creates a first virtual camera for controlling the first authorized camera. In this way, after the first virtual camera is created, the first device can directly determine the mapping relationship level between the first device and the first authorized camera according to the first camera identification of the first virtual camera and the first mapping relationship level, which is convenient for Send commands and receive data.

In a possible implementation manner, the first camera is determined according to the identification in the second camera identification and the identification of the existing camera in the first device corresponding to the first mapping relationship level. Authorizing the first identity of the camera in the first device may include: in the identity of the existing camera corresponding to the first mapping relationship level in the first device, the second camera already exists When identifying the identity in the identity, the first identity of the first authorized camera in the first device is created according to the preset identity creation rule; or the identity of the second camera is the same as the When the identities of the existing cameras corresponding to the first mapping relationship level in the first device are different, determine the identity in the second camera identity as the first authorized camera in the first device. The first identity in . In this way, the uniqueness of the first identification of the first authorized camera among the identifications of all cameras corresponding to the first mapping relationship level that the first device can control can be ensured, so as to facilitate the control of the first device. The distinction between cameras at the same mapping level.

In a possible implementation manner, step S18 may further include:

In this implementation manner, the first mapping relationship level can be obtained by adding one level to "the mapping relationship level in the second camera identifier indicating the mapping relationship between the first authorized camera and the second virtual camera". Different mapping relationship levels can be distinguished by different numbers, letters and other characters. The identity of different cameras at the same mapping relationship level can be distinguished by different numbers, letters and other characters of the camera. identity identification.

For example, referring to the first device A in FIG. 4, it is assumed that the identity of its local camera is "2" in device B1, the identity of its local camera is "2" in device C1, and its local camera is identified as "2" in device B2. The identity of the local camera is "1", and the identity of the local camera in the first device A is "3". The zero-level mapping relationship, the first-level mapping relationship, the second-level mapping relationship... are represented by 0000, 1000, 2000... respectively. but,

In device B1, the camera ID of its local camera may be 0002.

In device B2, the camera ID of its local camera may be 0001.

In the device C1, the camera identifier of its local camera may be 0002, and the camera identifier of the local camera of the device B2 mapped by the second virtual camera 301 may be 1001.

In the first device A, since there is no identity reuse, the first identity of its local camera is 0003, the first identity of the local camera of device C1 can be 1002, and the first identity of the local camera of device B2 is 2001 . However, since the local camera ID of device B1 "2 in 0002" has been occupied by the local camera of device C1, the local camera ID of device B1 can be adjusted. The first identity of the camera is 1004.

In a possible implementation manner, the method may further include: when receiving a command result returned by the selected camera after receiving the first task command, determining whether the selected camera successfully performs the target task according to the command result , whether to receive the first task command and other information. Wherein, receiving the command result may include at least one of the following ways:

When the selected camera includes the local camera of the first device, directly receiving a command result from the local camera of the first device;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a first-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, use The first virtual camera corresponding to the selected camera directly receives the command result sent by the local camera of the second device;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a multi-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, use The first virtual camera corresponding to the selected camera receives the command result sent by the local camera of the second device in sequence and forwarded by at least one intermediate device.

In a possible implementation manner, the method may further include: when receiving target task data obtained by the selected camera performing the target task, performing image and/or video presentation according to the target task data. In addition to displaying images and/or videos, the first device may also store target task data.

In a possible implementation, the method may further include:

In this way, the first device can directly control its local camera and the local camera of the second device mapped by the set first virtual camera according to the command issued by the user, and can also establish a control relationship with the fifth device and be controlled by the fifth device. . Wherein, the fifth device may itself be provided with a local camera, or may not have a local camera.

For example, FIG. 7 shows a schematic diagram of selecting an authorized camera according to an embodiment of the present application. As shown in FIG. 7 , it is assumed that the second requested camera in the second authorization control request is the local camera 100 of the first device A, the local camera of the device B1 to which the first virtual camera 201 is mapped, and the first virtual camera in FIG. 4 The local camera of the device B2 to which the camera 202 is mapped, and the local camera of the device B3 to which the first virtual camera 203 is mapped, the interface T4 may be displayed on the display screen of the first device A, according to each detected second Trigger operations such as clicking, sliding, etc. of the selection control K of the requesting camera determine the second authorized camera authorized by the user, such as the local camera of the device B1 to which the first virtual camera 201 is mapped. Then the first device A can generate a second authorization instruction according to the first camera identifier of the local camera of the device B1 to which the first virtual camera 201 is mapped in the first device A and send it to the fifth device D, so that the fifth device D creates a virtual camera 401 for controlling the second authorized camera according to the second authorization instruction (refer to the implementation process of steps S17 and S18 above).

FIG. 8 shows a schematic diagram of an implementation process of a camera control method based on distributed control according to an embodiment of the present application. As shown in FIG. 8, it shows the process that the first device A controls the local camera of the second device B2 through the intermediate device C1, wherein,

In the process of issuing the first task command:

The "camera application" in the application layer of the first device A detects the task setting operations issued by the user for the preview task, the photographing task and the camera task, and then the "camera service" module in the service layer of the first device A is based on the detected task. The device operation generates camera command 1 (also referred to as command 1 in the following and in the figure), photographing command 2 (also referred to as command 2 in the following and in the figure), and preview command 3 (also referred to as command 3 in the following and in the figure). Task parameters required to execute the task (for the process of determining the command and task parameters, refer to the above, and will not be repeated here). Among them, the generation time of the camera command 1, the camera command 2, and the preview command 3 or the time when the user issues these three commands is the same or different, and FIG. 8 shows the issuing process of the target tasks of three different task types. , all three commands are shown, but in fact, the issuing times of the three commands do not affect each other. The virtual camera device (that is, the second virtual camera 301 in FIG. 4 ) in the virtual camera HAL in the HAL layer (Hardware Abstraction Layer, English Hardware Abstraction Layer) of the first device A converts “command 1, command 2 and/or Command 3" is sent to the "Distributed Device Virtualization Platform Service" in the service layer of the first device A. The "Distributed Device Virtualization Platform Service" of the first device A transparently transmits the "Distributed Device Virtualization Platform Service" through its own transparent transmission pipeline. Command 1, Command 2 and/or Command 3" are sent to the intermediate device C1.

The "multi-device virtualization module" in the application layer of the middle device C1 receives and transmits "command 1, command 2 and/or command 3" to the "distributed device virtualization module" of its own service layer through the transparent transmission pipeline. Platform Services". The "distributed device virtualization platform service" also sends "command 1, command 2 and/or command 3" to the second device B2 in a transparent transmission manner through a transparent transmission channel.

The "multi-device virtualization module" of the application layer of the second device B2 receives "command 1, command 2 and/or command 3" through the transparent transmission pipeline, and then sends "command 1, command 2 and/or command 3" to the service Layer "Camera Service". The "Camera Service" issues specific target tasks to be performed to its camera device (ie, the local camera of B2 shown in Figure 4) according to "Command 1, Command 2 and/or Command 3", so that the camera device can control the camera The sensor of the hardware performs specific operations such as shooting, videography, framing preview, etc., and processes the data collected by the sensor through the image signal processor ISP (Image Signal Processing) of the camera hardware to generate target task data. The target task data may include data corresponding to command 1, data corresponding to command 2, and/or data corresponding to command 3.

During the upload of target task data:

The "camera device" of the second device B2 sends the target task data of different task types to the corresponding buffers of its "camera service" respectively, and then the "multi-device virtualization module" of the second device B2 transfers the target task data to and send the target task data to the "Distributed Device Virtualization Platform Service" of the intermediate device through the transparent transmission pipeline.

After the "Distributed Device Virtualization Platform Service" of the intermediate device C1 receives the target task data through the transparent transmission pipeline, it sends the target task data in a transparent transmission manner through the transparent transmission pipeline of the "Multi-device Virtualization Module" of the intermediate device C1. to the first device A.

The "distributed device virtualization platform service" of the first device A receives the target task data through its own transparent transmission pipeline, and processes the target task data to determine which task type data it corresponds to, and will correspond to different tasks. The type of target task data is sent to the corresponding buffer area of the "camera service" in the service layer of the first device A, so that the "camera application" of the first device A can determine that the corresponding buffer area stores the target task data, and /or upon receiving a user's display instruction, perform preview display, photo display and/or video display.

In the above manner, the control of the local camera of the second device B2 which is no longer in the same local area network is realized by using one first device A.

An embodiment of the present application provides a terminal device, including: a local camera; a first virtual camera; a processor and a memory for storing instructions executable by the processor; wherein, when the processor is configured to execute the instructions The above distributed control-based camera control method is implemented.

Embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the above method.

Embodiments of the present application provide a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), static random access memory (Static Random-Access Memory, SRAM), portable compact disk read-only memory (Compact Disc Read-Only Memory, CD - ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the foregoing .

Computer readable program instructions or code described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

The computer program instructions used to perform the operations of the present application may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or, may be connected to an external computer (eg, use an internet service provider to connect via the internet). In some embodiments, electronic circuits, such as programmable logic circuits, Field-Programmable Gate Arrays (FPGA), or Programmable Logic Arrays (Programmable Logic Arrays), are personalized by utilizing state information of computer-readable program instructions. Logic Array, PLA), the electronic circuit can execute computer readable program instructions to implement various aspects of the present application.

Aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of apparatuses, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in hardware (eg, circuits or ASICs (Application) that perform the corresponding functions or actions. Specific Integrated Circuit, application-specific integrated circuit)), or can be implemented by a combination of hardware and software, such as firmware.

While the invention has been described herein in connection with various embodiments, those skilled in the art will appreciate the understanding and understanding of the accompanying drawings, the disclosure, and the appended claims in practicing the claimed invention. Other variations of the disclosed embodiments are implemented. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

Various embodiments of the present application have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

A camera control method based on distributed control, characterized in that, applied to a first device, the method comprising:

Displaying a camera to be selected that can be controlled by the first device, the camera to be selected includes a local camera of the first device and a local camera of the second device mapped by the first virtual camera in the first device;

According to the detected operation of creating a task for the camera to be selected, determine the selected camera and a target task to be performed by the selected camera from the cameras to be selected;

generating a first task command according to the first camera identifier of the selected camera in the first device and the target task;

sending the first task command to the selected camera, so that the selected camera controls the execution of the target task according to the first task command,

The first device includes at least one first virtual camera, each first virtual camera is used to control the local camera of the mapped second device, and each first virtual camera is connected to the mapped second device There is at least one-level mapping relationship between the local cameras of ,

When there is a multi-level mapping relationship between the first virtual camera and the mapped local camera of the second device, the second device and the first device are in different local area networks.
The method according to claim 1, wherein the method further comprises:

When a device connection request is detected, search for a third device that can be connected to the first device and meets a connection condition, where the connection condition includes: the third device is provided with a local camera and/or a third device in the third device at least one second virtual camera is created;

sending a first authorization control request to the third device, and receiving a first authorization indication returned by the third device in response to the first authorization control request;

After determining the authorized first authorized camera according to the first authorization instruction, obtain a second camera identification of the first authorized camera in the third device, where the first authorized camera includes the third camera the local camera of the device and/or the local camera of the fourth device mapped by the second virtual camera;

A first camera identification of the first authorized camera in the first device is determined according to the second camera identification, and a first camera identification for controlling the first authorization is created according to the first camera identification of the first authorized camera Camera's first virtual camera,

Wherein, each second virtual camera is used to control the local camera of the fourth device mapped by the second virtual camera, and there is at least one connection between the second virtual camera and the mapped local camera of the fourth device. level mapping.
The method according to claim 2, wherein sending a first authorization control request to the third device, and receiving a first authorization indication returned by the third device in response to the first authorization control request, comprising: :

selecting a first requested camera according to a request operation for the local camera of the third device that can be controlled by the third device and/or the local camera of the fourth device mapped by the second virtual camera;

The camera generates the first authorization control request according to the first request, and sends the first authorization control request to the third device, so that the third device can make the third device according to the detected authorization for the first authorization The authorization operation of the control request generates the first authorization indication.
The method according to claim 2, wherein a first camera identification of the first authorized camera in the first device is determined according to the second camera identification, and a first camera identification of the first authorized camera is determined according to the identification of the first authorized camera. The first camera identification creates a first virtual camera for controlling the first authorized camera, including:

When the first authorized camera includes a local camera of the fourth device mapped by the second virtual camera, indicating the mapping between the first authorized camera and the second virtual camera according to the second camera identifier the mapping relationship level of the relationship, determining the first mapping relationship level between the first authorized camera and the first virtual camera that needs to be created to control the first authorized camera;

According to the identification in the second camera identification and the identification of the existing camera in the first device corresponding to the first mapping relationship level, it is determined that the first authorized camera is in the first device 's primary identity;

A first camera identification of the first authorized camera in the first device is determined according to the first mapping relationship level and the first identification identification, and a first camera identification for controlling all cameras is created according to the first camera identification. The first virtual camera of the first authorized camera.
The method according to claim 4, wherein a first camera identification of the first authorized camera in the first device is determined according to the second camera identification, and a first camera identification of the first authorized camera is determined according to the identification of the first authorized camera. The first camera identification creates a first virtual camera for controlling the first authorized camera, including:

When the first authorized camera includes a local camera of the third device, a first-level mapping relationship is determined as a first-level mapping relationship between the first authorized camera and a first virtual camera that needs to be created to control the first authorized camera The first mapping relationship level.
The method according to claim 4, characterized in that, according to the identification in the second camera identification and the identification of the existing camera in the first device corresponding to the first mapping relationship level, determining The first identity of the first authorized camera in the first device, including:

In the identity identifier of the existing camera corresponding to the first mapping relationship level in the first device, when the identity identifier in the second camera identifier already exists, create the identity identifier according to a preset identity identifier creation rule the first identity of the first authorized camera in the first device; or

When the identity identifier in the second camera identifier is different from the identity identifier of the existing camera corresponding to the first mapping relationship level in the first device, the identity identifier in the second camera identifier is changed to It is determined as the first identity identifier of the first authorized camera in the first device.
The method according to claim 1, characterized in that, according to the detected operation of creating a task for the camera to be selected, the selected camera and the tasks to be performed by the selected camera are determined from the cameras to be selected. Objective tasks, including:

determining the selected camera according to the detected selection operation for the camera to be selected;

According to the task setting operation for the selected camera, the target task to be executed by the selected camera is determined, wherein the task parameters of the target task include task type, execution time information, and execution time of the selected camera. At least one of the camera parameter settings during the target task, and the task type includes at least one of the following: a photographing task, a photographing task, and an image previewing task.
The method according to claim 1, wherein sending the first task command to the selected camera, so that the selected camera controls the execution of the target task according to the first task command, comprising: At least one of the following actions:

When the selected camera includes the local camera of the first device, the first task command is sent to the local camera of the first device, so that the local camera of the first device executes the first task the target task indicated by the command;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a first-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, pass The first virtual camera corresponding to the selected camera in the first device forwards the first task command to the local camera of the second device, so that the local camera of the second device executes the first task command. a target task indicated by the task command;

When the selected camera includes the local camera of the second device mapped by the first virtual camera, and there is a multi-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, according to The first mapping relationship level between the selected camera and the corresponding first virtual camera determines at least one intermediate device that completes the forwarding of the first task command, and sequentially passes through each intermediate device corresponding to the selected camera. The virtual camera of the camera forwards the first task command to the local camera of the second device, so that the local camera of the second device executes the target task indicated by the first task command.
The method according to claim 1 or 8, wherein the method further comprises:

When receiving the target task data obtained by the selected camera performing the target task, performing image and/or video display according to the target task data,

Wherein, receiving the target task data includes at least one of the following ways:

When the selected camera includes the local camera of the first device, directly receiving target task data sent from the local camera of the first device;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a first-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, use The first virtual camera corresponding to the selected camera directly receives the target task data sent by the local camera of the second device;

When the selected camera includes a local camera of the second device mapped by the first virtual camera, and there is a multi-level mapping relationship between the selected camera and the first virtual camera corresponding to the selected camera, use The first virtual camera corresponding to the selected camera receives the target task data sequentially sent by the local camera of the second device and forwarded by at least one intermediate device.
The method according to claim 1, wherein the method further comprises:

When receiving a second authorization control request from the fifth device, the camera displays an authorization prompt according to the second request in the second authorization control request;

determining an authorized second authorized camera according to the detected authorization operation for the second requested camera;

A second authorization instruction is generated according to the first camera identification of the second authorized camera in the first device, and the second authorization instruction is sent to the fifth device, so that the fifth device The second authorization instruction creates a virtual camera for controlling the second authorized camera.
A terminal device, characterized in that it includes:

local camera;

a first virtual camera;

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to implement the method of any one of claims 1-10 when executing the instructions.
A non-volatile computer-readable storage medium on which computer program instructions are stored, characterized in that, when the computer program instructions are executed by a processor, the method described in any one of claims 1-10 is implemented.