WO2022105793A1 - 图像处理方法及其设备 - Google Patents

图像处理方法及其设备 Download PDF

Info

Publication number
WO2022105793A1
WO2022105793A1 PCT/CN2021/131204 CN2021131204W WO2022105793A1 WO 2022105793 A1 WO2022105793 A1 WO 2022105793A1 CN 2021131204 W CN2021131204 W CN 2021131204W WO 2022105793 A1 WO2022105793 A1 WO 2022105793A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image processing
camera
sub
auxiliary
Prior art date
Application number
PCT/CN2021/131204
Other languages
English (en)
French (fr)
Inventor
冯可荣
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21893940.3A priority Critical patent/EP4231628A4/en
Publication of WO2022105793A1 publication Critical patent/WO2022105793A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
    • H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
    • H04N23/662—Transmitting camera control signals through networks, e.g. control via the Internet by using master/slave camera arrangements for affecting the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2621—Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Definitions

  • the present application relates to the technical field of image processing, and in particular, to an image processing method and device thereof.
  • a single camera may not be able to meet people's needs.
  • the anchor and the products the anchor is describing cannot be well presented by a single camera, which requires multiple cameras to participate in the shooting at the same time. .
  • the technical solution of using multiple cameras to photograph together is usually to transmit the images photographed by the multiple cameras to the same device for splicing through a communication connection.
  • the effect that this method can present is relatively simple, and cannot meet the individual needs of users. Therefore, the related art requires a technical solution capable of controlling different cameras to participate in shooting at the same time and generating images that meet user requirements.
  • an embodiment of the present application provides an image processing method, which is applied to an electronic terminal having a main camera, the method comprising: determining at least one auxiliary camera that is communicatively connected to the main camera; acquiring at least one sub-camera image, wherein the at least one sub-image refers to an image obtained after performing image processing on at least one auxiliary image captured by the at least one auxiliary camera and a main image captured by the main camera; using the at least one sub-image, Generate composite images.
  • the method can perform image processing on different images obtained by different cameras and use the processed images to generate a composite image, which can not only meet the user's needs for multi-camera shooting, but also can customize the generated composite image according to the user's needs.
  • acquiring at least one sub-image includes: acquiring the at least one auxiliary image captured by the at least one auxiliary camera and the main image captured by the main camera an image; in response to receiving an image processing request, determining from the at least one auxiliary image and the main image a to-be-processed image for performing the image processing and image processing for the to-be-processed image; acquiring the to-be-processed image The at least one sub-image after performing the image processing.
  • various customized image processing can be performed on the image captured by the auxiliary camera in the above manner, so as to achieve the image effect required by the user, so as to better meet the user's personalized needs for the image.
  • an image to be processed for performing the image processing is determined from the at least one auxiliary image and the main image and after the image processing on the image to be processed, the method includes: from candidate execution devices, determining an execution device that performs the image processing on the image to be processed; acquiring the execution device to execute the image processing on the image to be processed. the at least one sub-image after image processing.
  • the execution device includes at least one of the electronic terminal and an auxiliary device where the at least one auxiliary camera is located.
  • these images can be selected to be processed locally or at the remote end, which avoids the long processing time caused by the large difference in equipment, and satisfies the individual needs of people.
  • determining at least one secondary camera that performs a communication connection with the primary camera includes: using a discovery connection service, discover the at least one auxiliary camera in the environment where the electronic terminal is located; establish a communication connection with the auxiliary device where the at least one auxiliary camera is located.
  • the auxiliary cameras around the main camera can be reasonably used, and the auxiliary cameras can be controlled to perform shooting by establishing a communication connection therewith, so that images in different scenarios can be captured.
  • determining an execution device that performs the image processing on the image to be processed includes: acquiring a configuration established with each candidate execution device The communication connection information of the communication connection; according to the communication connection information and/or the device capability information of each candidate execution device, determine the execution device that performs the image processing on the to-be-processed image.
  • the communication connection information and/or the device capability information can be used to select a more suitable execution device, thereby improving the efficiency of image processing and reducing the time delay between the various devices.
  • the execution device includes: using the communication connection information of each candidate execution device to determine a candidate execution device capable of performing communication transmission; among the candidate execution devices capable of performing communication transmission, the execution device is determined according to device capability information.
  • the method can determine the candidate auxiliary device that can perform image processing according to the real-time network state, and then use the device capability information of the auxiliary device to determine the device that performs image processing, so that the appropriate device can be selected under the condition of ensuring normal communication.
  • the processing device solves the low performance of some devices for image processing and shortens the delay caused by cross-device processing.
  • the device capability information includes data processing information and image processing information, wherein the data processing information includes capability information of each auxiliary device to process data, and the image The processing information includes information on the image effects that each auxiliary device can cause the image to produce.
  • both the data processing capability of each device and the image processing capability possessed by each device are considered, so that the execution device can be determined more reasonably.
  • determining the execution device from the candidate auxiliary devices according to the device capability information includes: using data processing information of the candidate execution device and/or Image processing information, sort the candidate execution devices, and determine a priority order for the candidate execution devices to perform image processing; and determine the execution devices according to the priority order.
  • the data processing information and image processing information described above can be used for sorting, and it is preferable to select the one with better data processing capability and image processing capability. equipment, so as to improve the speed of image processing and effectively reduce the data processing burden of some auxiliary equipment.
  • the image processing information further includes executing an image processing algorithm for realizing the image effect.
  • the image processing information also includes a specific image processing algorithm to achieve the image effect.
  • the image processing algorithm is received from the at least one auxiliary device by using the communication connection.
  • each device As an electronic terminal with a main camera, it usually has better performance than each auxiliary device, but each device may have different image processing algorithms and different image effects.
  • the images can be processed as desired by the user, and the image processing algorithm possessed by the auxiliary device can be synchronized to the electronic terminal to ensure that these images can be executed on the electronic terminal.
  • using the at least one sub-image to generate a composite image includes: synthesizing according to a preset image mode, and synthesizing the at least one sub-image to generate the composite image.
  • the composite image can be generated according to a preset image composite mode, so as to meet the user's personalized needs for the composite image.
  • an embodiment of the present application provides an image processing method, the method includes determining a plurality of cameras that are connected to each other for communication; a plurality of sub-images; and using the plurality of sub-images, a composite image is generated.
  • acquiring the multiple sub-images after image processing is performed on the multiple original images captured by the multiple cameras includes: determining to perform a corresponding correspondence on each original image.
  • An image processing execution device acquiring the plurality of sub-images after performing corresponding image processing on each original image by using the execution device.
  • an execution device for performing image processing may be determined, which not only satisfies the user's requirements for each original image, but also reduces the time for image processing.
  • determining an execution device that performs corresponding image processing on each original image includes: determining, according to the device capability information and communication connection information of each execution device, determining An execution device that executes corresponding image processing.
  • the executing device can be determined through the device capability information and the communication connection information, so that the executing device can be determined quantitatively, so that the determined executing device is more suitable for executing image processing.
  • embodiments of the present application provide an image processing device, which can perform the first aspect or one or more of the image processing methods in the multiple possible implementation manners of the first aspect, and The above second aspect or one or more image processing methods in multiple possible implementation manners of the second aspect.
  • embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the above-mentioned first aspect or the first aspect.
  • embodiments of the present application provide a computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in an electronic
  • the processor in the electronic device executes the first aspect or the image processing method of one or more of the multiple possible implementations of the first aspect and the second aspect or the image processing method of the second aspect.
  • One or more image processing methods in a variety of possible implementations.
  • FIG. 1 shows an application scenario diagram for performing image processing according to an embodiment of the present application
  • FIG. 2 shows a flowchart of interaction between devices according to an embodiment of the present application
  • FIG. 3 shows a schematic diagram of a software structure of an electronic device according to an embodiment of the present application
  • FIG. 4 shows a flowchart of steps of an image processing method according to an embodiment of the present application
  • FIG. 5 shows a structural block diagram of an electronic device according to an embodiment of the present application
  • FIG. 6 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the camera mentioned in this application refers to a device that can capture images through the image sensor of the camera
  • the electronic terminal mentioned in this application refers to various devices with cameras, including mobile phones, tablet computers, wearable devices (for example, watches , wristbands, smart helmets, etc.), in-vehicle devices, smart home devices, augmented reality (AR)/virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs) ), netbooks, personal digital assistants (personal digital assistants, PDAs), etc.
  • Some embodiments of the present application take a mobile phone as an example, and it can be understood that these embodiments can also be applied to other types of mobile terminals.
  • the user uses multiple mobile phones to perform shooting.
  • the mobile phone 101 can photograph the natural landscape 10, and the camera 102 can be responsible for photographing the user 11, then the image 1001 photographed by the mobile phone 101 is used as the background image, and the image 1002 photographed by the camera is used as the foreground image, and finally a composite image is generated 1003.
  • the user expects to perform beautification on the user 11 photographed by the camera 102, such as whitening processing, slimming processing, etc., but the camera 102 does not have these capabilities, in this case, the image of the exemplary embodiment of the present application can be used Approach.
  • the present application does not limit the number of cameras, and the cameras mentioned in the present application may include multiple cameras in a single electronic terminal (eg, embedded multiple primary/secondary cameras) and remote devices including cameras.
  • each camera and the electronic terminal with each camera mentioned in this application is not limited, that is to say, the effect captured by each camera and the image processing effect of each electronic terminal may be the same or may be different.
  • the image processing method of the exemplary embodiment of the present application can also be applied to multiple application scenarios.
  • user A may use cell phone A and user B may use cell phone B to perform a video call
  • cell phone A may use images of exemplary embodiments of the present application for image A captured by cell phone A and image B captured by cell phone B
  • the processing method generates a composite image composed of image A and image B with a specific image processing effect.
  • the avatar of user A captured by image A becomes a specific animal image
  • the avatar of user B captured by image B remains unchanged.
  • the image processing method of the exemplary embodiment of the present application can also be used in the scene of live video.
  • the host can use the mobile phone with better performance as the main device to capture the pictures related to the host.
  • multiple auxiliary devices can be used to capture the pictures related to the product, and then the main device and multiple auxiliary devices can be used.
  • Each captured image generates a composite image, in which a picture related to the host is used as the main image, and images captured by the auxiliary device are arranged as sub-images on one side of the main picture.
  • the above only provides examples of application scenarios of the image processing method applicable to the exemplary embodiments of the present application, and is not limiting.
  • the image processing method of the exemplary embodiment of the present application can be used.
  • FIG. 2 shows a flowchart of interaction between devices according to an embodiment of the present application.
  • FIG. 2 includes only two mobile phones 21 and 22, each of which has a built-in camera.
  • a device near the phone can be converted into a virtual component within the phone, so that its capabilities can be used as an extension of the phone.
  • a camera near the phone or a device with a camera can be converted into Use the virtual camera in your phone.
  • the mobile phone 21 can enable the device discovery service, and the mobile phone 21 can learn which devices exist within a certain range near it. This technology of discovering nearby devices is called a short-range discovery technology. Further, with the short-range discovery technology, device-to-device (Device To Device, D2D) communication within a certain distance can also be realized, so these devices with D2D communication capability are called short-range devices.
  • the method for realizing short-range discovery is to set a short-range service server in the network, and the mobile phone 21 registers with the short-range service server to obtain the short-range service.
  • the mobile phone 21 When the mobile phone 21 needs to discover other mobile phones 21 nearby, it sends a discovery request to the short-range service server, and the sending request may have a specific category indication, for example, to discover camera hardware, and the short-range service server controls the measurement between the mobile phones 21 And discovery, or the short-range service server instructs the base station, and the base station controls the measurement and discovery between the mobile phones 21 . Therefore, through the control of the network, the mobile phone 21 can discover devices within a certain distance around it.
  • the short-range service server controls the measurement between the mobile phones 21 And discovery, or the short-range service server instructs the base station, and the base station controls the measurement and discovery between the mobile phones 21 . Therefore, through the control of the network, the mobile phone 21 can discover devices within a certain distance around it.
  • the mobile phone 21 can establish a communication connection with the mobile phone 22 found in the device discovery service, and send an authorization request to the mobile phone 22.
  • a pop-up box prompting the user for authorization can be displayed on the interface of the mobile phone 22, and the user can click the pop-up box. Choose whether to agree to authorize the mobile phone 21 to use the camera inside the mobile phone 22, and preferably, the user can also be prompted to select a different authorization type through a pop-up box.
  • the mobile phone 22 can be authenticated, that is to say, it is determined whether the accounts logged in the mobile phone 21 and the mobile phone 22 are the same. For cross-account devices, a prompt to output verification information is prompted on the interface of the mobile phone 22 to ensure that account authentication is completed.
  • the mobile phone 21 can map the camera in the mobile phone 22 to the mobile phone 21 as a virtual camera in the mobile phone 21.
  • the mobile phone 21 can perform registration on the virtual camera.
  • the registration process includes assigning an identifier to the mobile phone 22 and the virtual camera. information.
  • the mobile phone 21 can also obtain device capability information from the mobile phone 22, wherein the device capability information can include attribute information of the camera, such as the light angle of the camera, optical zoom factor, etc., and also include data processing information, that is, , the capability of the mobile phone 22 to process data, in addition, the device capability information also includes image processing information, for example, image algorithm information that can realize a specific image effect.
  • the cell phone 21 can control to turn on its own camera as well as the camera in the cell phone 22 .
  • the master device/auxiliary device adopts a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • the embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe a software structure of an electronic device.
  • the following description of the software structure of the electronic device only takes the camera application as an example, but in implementation, the electronic device may also include various applications.
  • the electronic device used by the user to generate the composite image is taken as the main device, and the electronic device that assists the main device to generate the composite image is taken as the auxiliary device.
  • the main device includes an application (Application) layer, an application framework (Framework) layer, a hardware abstraction layer (hard abstraction layer, HAL) layer and a kernel (Kernel) layer.
  • application Application
  • Framework application framework
  • HAL hardware abstraction layer
  • Kernel kernel
  • the application layer can include a series of application packages, for example, the application package can include applications such as phone, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS and so on.
  • the application package can include applications such as phone, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS and so on.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the Framework layer provides developers with a full access to the API framework used by the application. Specifically, the Framework layer provides a lot of APIs for developing applications. By calling the corresponding APIs, you can construct an APP that meets relevant business requirements. In the case where the application is a camera, the application framework layer provides many camera-related APIs.
  • the service layer in FIG. 3 indicates various services that the camera can provide.
  • the service layer may include services related to the local camera, including device control commands and device control data.
  • the service layer may also include The virtual camera service including the virtual camera corresponding to the auxiliary device, that is, in the implementation, the virtual camera service that can be invoked to the main device can be provided by adding a virtual camera service corresponding to the virtual camera in the service layer.
  • the HAL layer is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform, provides a virtual hardware platform for the operating system, and makes it hardware independent and can be transplanted on various platforms. From the perspective of software and hardware testing, the testing of software and hardware can be completed based on the hardware abstraction layer, which makes it possible to perform parallel testing of software and hardware.
  • the HAL layer may include camera device information and camera algorithm information, wherein the camera device information refers to parameter information related to the performance of the camera, and the camera algorithm information refers to an image processing algorithm that the camera can process on the image. For example, the user can call the camera algorithm through the HAL layer to blur the captured image.
  • the HAL layer may have a local camera HAL related to the local camera and a virtual camera HAL layer with the same structure as the local camera HAL, that is, the virtual camera HAL also includes camera device information and camera algorithm information.
  • a virtual camera can be added by adding a virtual camera HAL corresponding to the auxiliary camera in the HAL layer, so that the local camera service of the service layer can call the local camera HAL or the virtual camera HAL as required.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display driver, camera driver, audio driver, sensor driver and other drivers.
  • the method described above can be used to convert the remote camera into a virtual camera in the electronic device (eg, mobile phone) where the local camera is located.
  • this application will focus on how to reasonably allocate the local camera (mentioned below The main camera) and the remote camera (the secondary camera mentioned below), so as to improve the efficiency of image processing and meet the needs of users.
  • the image processing method of the exemplary embodiment of the present application will be described in detail below with reference to FIG. 4 .
  • FIG. 4 shows a flowchart of steps of an image processing method according to an embodiment of the present application.
  • the method shown in FIG. 4 can be applied to a main device with a main camera embedded in it. That is to say, when a user uses multiple cameras to perform shooting, the electronic terminal where a certain camera is located is usually used as the main operation terminal.
  • the user's main operation terminal is referred to as the main device, and the camera in the main device is referred to as the main camera.
  • the number of the main cameras is not limited and depends on the number of cameras embedded in the electronic terminal.
  • step S410 at least one secondary camera that performs communication connection with the primary camera is determined.
  • the at least one auxiliary camera in the environment where the electronic terminal is located can be discovered by enabling the discovery connection service in the electronic terminal.
  • Auxiliary cameras within a distance, but in implementation, auxiliary cameras located in different spaces from the electronic terminal may also be discovered through a wireless transmission network.
  • the main camera when using the main camera, the user can discover the auxiliary camera in the same room as the user through Bluetooth.
  • the main camera the user can also use the camera of the user's friend in a different city as the auxiliary camera. camera.
  • a communication connection may be established with the auxiliary device where the at least one auxiliary camera is located.
  • the primary device may acquire the authentication authorization of each secondary device as shown in FIG. 2 , and establish a secondary camera corresponding to the secondary camera in the primary device.
  • the image processing information in each auxiliary device can be sent to the main device, that is, the main device is converted into a device with more image processing information, wherein the image processing information Includes image processing algorithms.
  • the main device usually chooses a device with better performance.
  • various image processing algorithms in the auxiliary device can be synchronized to the main device, so as to ensure that the main device can satisfy the user's requirements for synthesis as much as possible. Requirements for each sub-image included in the image.
  • step S420 at least one sub-image is acquired, wherein the at least one sub-image refers to an image obtained after performing image processing on at least one auxiliary image captured by the at least one auxiliary camera and a main image captured by the main camera.
  • the main device may control the main camera to acquire the main image, and may control at least one auxiliary camera and acquire the at least one auxiliary image captured by the at least one auxiliary camera. Subsequently, in response to an image processing request input by a user, an image to be processed for performing the image processing and an image processing for the to-be-processed image may be determined from the at least one auxiliary image and the main image.
  • the user can select some or all of the auxiliary images.
  • the mobile phone A in the case that the mobile phone A is connected to the camera B and the notebook C in communication , the mobile phone A can control the camera B and the notebook C to shoot auxiliary images, and when the auxiliary images B and C are obtained, the user can select only the auxiliary image B from these auxiliary images.
  • the user can then choose to perform corresponding image processing for these images, for example, after selecting image B, the user selects whitening processing for image B, and after selecting image C, selects blurring processing for image C and slimming for image A deal with.
  • the auxiliary images captured by these auxiliary devices and the available image processing can be displayed on the display page of the main device, and the user can select the desired sub-image and the image processing for the auxiliary image by touching, wherein the Sub-images include sub-images and main images.
  • an execution device for executing the image processing on the image to be processed may be determined first, and the execution device may include a master device and/or a slave device. That is to say, there may be many possibilities regarding the execution device that performs image processing on the images to be processed, including that each image to be processed is processed by the main device, and for example, some images to be processed are processed by the main device, but Some of the images to be processed are processed by corresponding auxiliary devices, wherein the corresponding auxiliary devices may be the auxiliary devices that capture the images to be processed or not. For another example, all the images to be processed are processed by a single auxiliary device. Perform image processing.
  • the preferred embodiment of the present application provides a method for determining the execution device, specifically, acquiring the communication connection information of the communication connection established with each candidate execution device.
  • the communication connection information of each candidate execution device can be acquired in real time or at equal time intervals (the time interval is very short), and the communication state between the master device and the auxiliary device can be determined by using the communication connection information.
  • the execution device determines the execution device that performs the image processing on the to-be-processed image, wherein the device capability information includes data processing information and image processing information Processing information, wherein the data processing information includes information on the capability of each auxiliary device to process data, and the image processing information includes information on the image effects that each auxiliary device can make the image produce. That is, when considering the execution device, a preferred embodiment is to consider the data processing capability and the image processing capability of each auxiliary device at the same time.
  • the candidate secondary device capable of performing the communication transmission can be determined by using the communication connection information of each candidate executing device.
  • a threshold can be set for the communication transmission rate. When the threshold is higher, other execution devices other than the local machine can be selected. When the communication transmission rate is lower than the threshold, the local machine can be selected as the Executive body.
  • the execution device may be determined from the candidate execution devices according to the device capability information.
  • data processing information of the candidate execution devices may be used to sort the candidate execution devices, for example, the candidate execution devices may be sorted according to the CPU performance of the candidate execution devices, so as to determine the priority order for performing image processing.
  • the image processing information of the candidate execution devices may also be used to sort the candidate execution devices, for example, the candidate execution devices may be sorted according to the GPU performance of the candidate execution devices, thereby determining the priority order for performing image processing.
  • the method may first determine the image processing selected by the user for each image to be processed, then determine the devices capable of performing these image processing from these candidate execution devices, and then sort these devices according to the effect of image processing, Thereby, the priority order for performing image processing is determined.
  • the method may also comprehensively consider the data processing information and the image processing information, and sort these candidate execution devices, so as to determine the priority order for executing the image processing.
  • the master device may send the image to be processed to the execution device for processing, and after the execution device performs image processing on the image to be processed, the image processed image is sent to the master device.
  • the captured images including the main image captured by the main camera and the auxiliary images captured by each auxiliary camera
  • it can be determined only according to the device capability information that the image is executed by the shooting subject.
  • Image processing is also performed by the host device.
  • the auxiliary image A captured by the auxiliary device A when the network connection is normal, whether the auxiliary image A is to be processed by the auxiliary device A or the main device can be determined according to the device capability information.
  • step S430 a composite image is generated using the at least one sub-image.
  • these sub-images are synthesized according to a preset image synthesis mode to generate the synthesized image, wherein the preset image synthesis mode may be a mode specified by a user, for example, a certain image may be selected as a background image, Other images are used as foreground images, and for another example, these images can be all stitched together.
  • the preset image synthesis mode may be a mode specified by a user, for example, a certain image may be selected as a background image, Other images are used as foreground images, and for another example, these images can be all stitched together.
  • each camera has image capturing capability and image processing capability, and can be achieved through the above description.
  • each camera can act as a control terminal to determine the execution device of each captured image, and a separate server that is communicatively connected to these cameras can also be used to determine the execution device of each image. Execute the device.
  • a plurality of cameras that are connected to each other for communication can be determined, and then, a plurality of sub-images after image processing is performed on a plurality of original images captured by the plurality of cameras can be obtained.
  • Determining an execution device that executes corresponding image processing on each original image specifically includes: determining an execution device that executes corresponding image processing according to device capability information and communication connection information of each execution device. Acquiring the plurality of sub-images after performing corresponding image processing on each original image by using the executing device. Finally, using the plurality of sub-images, a composite image is generated.
  • the above-mentioned terminal and the like include corresponding hardware structures and/or software modules for executing each function.
  • the embodiments of the present application can be implemented in hardware or a combination of hardware and computer software. Whether a function is performed by hardware or computer software-driven hardware depends on the specific application and design constraints of the technical solution. Experts may use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present application.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. It should be noted that, the division of modules in the embodiments of the present application is schematic, and is only a logical function division, and there may be other division manners in actual implementation.
  • FIG. 5 shows a structural block diagram of the electronic terminal involved in the above embodiment, and the electronic device includes: an auxiliary camera determination unit 510 , a sub-image acquisition unit 520 and The composite image generation unit 530, wherein the secondary camera determination unit 510 is used to determine at least one secondary camera that performs communication connection with the main camera; the sub-image acquisition unit 520 is used to acquire at least one sub-image, wherein the at least one sub-image is The image refers to an image obtained after performing image processing on at least one auxiliary image captured by the at least one auxiliary camera and the main image captured by the main camera; the composite image generation unit 530 is configured to use the at least one sub-image to generate a composite image image.
  • the secondary camera determination unit 510 is used to determine at least one secondary camera that performs communication connection with the main camera
  • the sub-image acquisition unit 520 is used to acquire at least one sub-image, wherein the at least one sub-image is The image refers to an image obtained after performing image processing on at least one auxiliary image captured by the
  • the sub-image acquisition unit 520 includes an image acquisition module, an image processing determination module, and a sub-image acquisition module, wherein the image acquisition module is configured to acquire the at least one auxiliary image captured by the at least one auxiliary camera and the main camera.
  • the image processing determination module is configured to, in response to receiving the image processing request, determine the image to be processed for performing the image processing and the image processing for the image to be processed from the at least one auxiliary image and the main image;
  • the sub-image acquiring module is configured to acquire the at least one sub-image after performing the image processing on the image to be processed.
  • the electronic device further includes an execution device determination module configured to determine an execution device to perform the image processing on the to-be-processed image.
  • the image is the at least one sub-image after the image processing is performed.
  • the execution device includes at least one of the electronic terminal and an auxiliary device where the at least one auxiliary camera is located.
  • the auxiliary camera determination unit 510 includes a discovery module and a communication connection module, wherein the discovery module is used for using discovery connection service to discover the at least one auxiliary camera in the environment where the electronic terminal is located, and the communication connection module is used for connecting with the at least one auxiliary camera.
  • a secondary device where a secondary camera is located establishes a communication connection.
  • the execution device determination module includes a communication connection information determination submodule and an execution device submodule, wherein the communication connection information determination submodule is used to obtain the communication connection information of the communication connection established with each candidate execution device; the execution device submodule is used for The communication connection information and/or the device capability information of each candidate execution device determines the execution device that performs the image processing on the to-be-processed image.
  • the execution device sub-module is specifically configured to use the communication connection information of each candidate execution device to determine a candidate auxiliary device capable of performing communication transmission; and determine the execution device from the candidate auxiliary devices according to the device capability information.
  • the device capability information includes data processing information and image processing information, wherein the data processing information includes the capability information of each auxiliary device to process data, and the image processing information includes the image effect that each auxiliary device can make the image produce. information.
  • the execution device sub-module is specifically configured to use the data processing information and/or image processing information of the candidate execution devices to sort the candidate execution devices, and determine the priority order of the candidate execution devices to perform image processing; The execution devices are determined sequentially.
  • the image processing information further includes executing an image processing algorithm for realizing the image effect.
  • the electronic device further includes an image processing algorithm receiving unit for receiving the image processing algorithm from the at least one auxiliary device using the communication connection.
  • the combined image generating unit 530 is configured to combine the main image and the at least one auxiliary image according to a preset image combining mode to generate the combined image.
  • FIG. 6 shows a schematic structural diagram of a terminal device according to an embodiment of the present application. Taking the terminal device being a mobile phone as an example, FIG. 6 shows a schematic structural diagram of the mobile phone 200 .
  • the mobile phone 200 may include a processor 210, an external memory interface 220, an internal memory 221, a USB interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 251, a wireless communication module 252, Audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, sensor module 280, buttons 290, motor 291, indicator 292, camera 293, display screen 294, SIM card interface 295, etc.
  • a processor 210 an external memory interface 220, an internal memory 221, a USB interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 251, a wireless communication module 252, Audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, sensor module 280, buttons 290, motor 291, indicator 292, camera 293, display screen 294, SIM card interface 295, etc.
  • the sensor module 280 may include a gyroscope sensor 280A, an acceleration sensor 280B, a proximity light sensor 280G, a fingerprint sensor 280H, and a touch sensor 280K (of course, the mobile phone 200 may also include other sensors, such as a temperature sensor, a pressure sensor, a distance sensor, and a magnetic sensor. , ambient light sensor, air pressure sensor, bone conduction sensor, etc., not shown in the figure).
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the mobile phone 200 .
  • the mobile phone 200 may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or Neural-network Processing Unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • the controller may be the nerve center and command center of the mobile phone 200 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 210 for storing instructions and data.
  • the memory in processor 210 is cache memory.
  • the memory may hold instructions or data that have just been used or recycled by the processor 210 . If the processor 210 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided, and the waiting time of the processor 210 is reduced, thereby improving the efficiency of the system.
  • the processor 210 can run the image processing method provided by the embodiment of the present application, which can not only satisfy the user's requirement for shooting with multiple cameras, but also can customize and generate a composite image according to the user's requirement.
  • the processor 210 may include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU may cooperate to execute the image processing method provided by the embodiments of the present application. For example, some algorithms in the image processing method are executed by the CPU, and another part of the algorithms are executed by the GPU. for faster processing efficiency.
  • Display screen 294 is used to display images, videos, and the like.
  • Display screen 294 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
  • LED organic light-emitting diode
  • AMOLED organic light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
  • cell phone 200 may include 1 or N display screens 294, where N is a positive integer greater than 1.
  • the display screen 294 may be used to display information entered by or provided to the user as well as various graphical user interfaces (GUIs).
  • GUIs graphical user interfaces
  • display 294 may display photos, videos, web pages, or documents, and the like.
  • display 294 may display a graphical user interface.
  • the GUI includes a status bar, a hideable navigation bar, a time and weather widget, and an application icon, such as a browser icon.
  • the status bar includes operator name (eg China Mobile), mobile network (eg 4G), time and remaining battery.
  • the navigation bar includes a back button icon, a home button icon, and a forward button icon.
  • the status bar may further include a Bluetooth icon, a Wi-Fi icon, an external device icon, and the like.
  • the graphical user interface may further include a Dock bar, and the Dock bar may include commonly used application icons and the like.
  • the display screen 294 may be an integrated flexible display screen, or a spliced display screen composed of two rigid screens and a flexible screen located between the two rigid screens.
  • the terminal device can establish a connection with other terminal devices through the antenna 1, the antenna 2 or the USB interface, and transmit data and data according to the image processing method provided by the embodiment of the present application.
  • Control display 294 displays a corresponding graphical user interface.
  • Camera 293 front camera or rear camera, or a camera that can be both a front camera and a rear camera is used to capture still images or video.
  • the camera 293 may include a photosensitive element such as a lens group and an image sensor, wherein the lens group includes a plurality of lenses (convex or concave) for collecting the light signal reflected by the object to be photographed, and transmitting the collected light signal to the image sensor .
  • the image sensor generates an original image of the object to be photographed according to the light signal.
  • Internal memory 221 may be used to store computer executable program code, which includes instructions.
  • the processor 210 executes various functional applications and data processing of the mobile phone 200 by executing the instructions stored in the internal memory 221 .
  • the internal memory 221 may include a storage program area and a storage data area.
  • the storage program area may store operating system, code of application programs (such as camera application, WeChat application, etc.), and the like.
  • the storage data area may store data created during the use of the mobile phone 200 (such as images and videos collected by the camera application) and the like.
  • the internal memory 221 may also store one or more computer programs 1310 corresponding to the image execution processing method provided by the embodiment of the present application.
  • the one or more computer programs 1304 are stored in the aforementioned memory 221 and configured to be executed by the one or more processors 210, and the one or more computer programs 1310 include instructions that may be used to perform the execution of FIG. 4
  • the computer program 1310 may include a secondary camera determination module 61 , a sub-image acquisition module 62 , and a composite image generation module 63 .
  • the camera determination module 61 is used to determine at least one secondary camera that is connected to the main camera; the sub-image acquisition module 62 is used to acquire at least one sub-image, wherein the at least one sub-image refers to the At least one auxiliary image captured by the at least one auxiliary camera and an image obtained after performing image processing on the main image captured by the main camera, the composite image generation module 63 is configured to use the at least one sub-image to generate a composite image.
  • the processor 210 may control the display screen to display a composite image.
  • the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • non-volatile memory such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • the code of the image processing method provided by the embodiment of the present application may also be stored in an external memory.
  • the processor 210 may execute the code of the image processing method stored in the external memory through the external memory interface 220 .
  • the function of the sensor module 280 is described below.
  • the gyro sensor 280A can be used to determine the movement posture of the mobile phone 200 .
  • the angular velocity of cell phone 200 about three axes ie, x, y, and z axes
  • the gyro sensor 280A can be used to detect the current motion state of the mobile phone 200, such as shaking or still.
  • the gyro sensor 280A can be used to detect a folding or unfolding operation acting on the display screen 294 .
  • the gyroscope sensor 280A may report the detected folding operation or unfolding operation to the processor 210 as an event to determine the folding state or unfolding state of the display screen 294 .
  • the acceleration sensor 280B can detect the magnitude of the acceleration of the mobile phone 200 in various directions (generally three axes). That is, the gyro sensor 280A can be used to detect the current motion state of the mobile phone 200, such as shaking or still. When the display screen in the embodiment of the present application is a foldable screen, the acceleration sensor 280B can be used to detect a folding or unfolding operation acting on the display screen 294 . The acceleration sensor 280B may report the detected folding operation or unfolding operation to the processor 210 as an event to determine the folding state or unfolding state of the display screen 294 .
  • Proximity light sensor 280G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes.
  • the light emitting diodes may be infrared light emitting diodes.
  • the mobile phone emits infrared light outward through light-emitting diodes.
  • Phones use photodiodes to detect reflected infrared light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the phone. When insufficient reflected light is detected, the phone can determine that there are no objects near the phone.
  • the proximity light sensor 280G can be arranged on the first screen of the foldable display screen 294, and the proximity light sensor 280G can detect the first screen according to the optical path difference of the infrared signal.
  • the gyroscope sensor 280A (or the acceleration sensor 280B) may send the detected motion state information (such as angular velocity) to the processor 210 .
  • the processor 210 determines, based on the motion state information, whether the current state is the hand-held state or the tripod state (for example, when the angular velocity is not 0, it means that the mobile phone 200 is in the hand-held state).
  • the fingerprint sensor 280H is used to collect fingerprints.
  • the mobile phone 200 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering incoming calls with fingerprints, and the like.
  • Touch sensor 280K also called “touch panel”.
  • the touch sensor 280K may be disposed on the display screen 294, and the touch sensor 280K and the display screen 294 form a touch screen, also called a "touch screen”.
  • the touch sensor 280K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output associated with touch operations may be provided via display screen 294.
  • the touch sensor 280K may also be disposed on the surface of the mobile phone 200 , which is different from the location where the display screen 294 is located.
  • the display screen 294 of the mobile phone 200 displays a main interface, and the main interface includes icons of multiple applications (such as a camera application, a WeChat application, etc.).
  • Display screen 294 displays an interface of a camera application, such as a viewfinder interface.
  • the wireless communication function of the mobile phone 200 can be realized by the antenna 1, the antenna 2, the mobile communication module 251, the wireless communication module 252, the modulation and demodulation processor, the baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in handset 200 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 251 can provide a wireless communication solution including 2G/3G/4G/5G, etc. applied on the mobile phone 200 .
  • the mobile communication module 251 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like.
  • the mobile communication module 251 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.
  • the mobile communication module 251 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 .
  • at least part of the functional modules of the mobile communication module 251 may be provided in the processor 210 .
  • the mobile communication module 251 may be provided in the same device as at least part of the modules of the processor 210 .
  • the mobile communication module 251 may also be used for information interaction with other terminal devices. For example, the mobile communication module 251 can send the captured images to other terminal devices, and receive images sent by other terminal devices.
  • the modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the low frequency baseband signal is processed by the baseband processor and passed to the application processor.
  • the application processor outputs sound signals through audio devices (not limited to the speaker 270A, the receiver 270B, etc.), or displays images or videos through the display screen 294 .
  • the modem processor may be a stand-alone device.
  • the modulation and demodulation processor may be independent of the processor 210, and may be provided in the same device as the mobile communication module 251 or other functional modules.
  • the wireless communication module 252 can provide applications on the mobile phone 200 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (bl mobile phone 21tooth, BT), global navigation Satellite system (global navigation satellite system, GNSS), frequency modulation (freq mobile phone 21ncy modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • the wireless communication module 252 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 252 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210 .
  • the wireless communication module 252 can also receive the signal to be sent from the processor 210 , perform frequency modulation on the signal, amplify the signal, and then convert it into an electromagnetic wave for radiation through the antenna 2 .
  • the wireless communication module 252 is used to transmit data between other terminal devices under the control of the processor 210.
  • the processor 210 when the processor 210 executes the image processing method provided by the embodiment of the present application, the processor can control the The wireless communication module 252 sends a captured image request to other terminal devices, and can also receive auxiliary images captured by other terminal devices based on the above captured image request, and uses the auxiliary images to generate a composite image and then controls the display screen 294 to display the composite image, providing users with intuitive visual feedback, and the composite image can be adjusted in real time according to user needs.
  • the mobile phone 200 can implement audio functions through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, and an application processor. Such as music playback, recording, etc.
  • the cell phone 200 can receive key 290 input and generate key signal input related to user settings and function control of the cell phone 200 .
  • the mobile phone 200 can use the motor 291 to generate vibration alerts (eg, vibration alerts for incoming calls).
  • the indicator 292 in the mobile phone 200 may be an indicator light, which may be used to indicate a charging state, a change in power, and may also be used to indicate a message, a missed call, a notification, and the like.
  • the SIM card interface 295 in the mobile phone 200 is used to connect the SIM card. The SIM card can be contacted and separated from the mobile phone 200 by inserting into the SIM card interface 295 or pulling out from the SIM card interface 295 .
  • the mobile phone 200 may include more or less components than those shown in FIG. 5 , which are not limited in this embodiment of the present application.
  • the illustrated handset 200 is merely an example, and the handset 200 may have more or fewer components than those shown, two or more components may be combined, or may have different component configurations.
  • the various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.
  • An embodiment of the present application provides an image processing apparatus, including: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions.
  • Embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the above method.
  • Embodiments of the present application provide a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), static random access memory (Static Random-Access Memory, SRAM), portable compact disk read-only memory (Compact Disc Read-Only Memory, CD - ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the foregoing .
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read-only memory
  • EPROM Errically Programmable Read-Only-Memory
  • SRAM static random access memory
  • portable compact disk read-only memory Compact Disc Read-Only Memory
  • CD - ROM Compact Disc Read-Only Memory
  • DVD Digital Video Disc
  • memory sticks floppy disks
  • Computer readable program instructions or code described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions used to perform the operations of the present application may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or, may be connected to an external computer (eg, use an internet service provider to connect via the internet).
  • electronic circuits such as programmable logic circuits, Field-Programmable Gate Arrays (FPGA), or Programmable Logic Arrays (Programmable Logic Arrays), are personalized by utilizing state information of computer-readable program instructions.
  • Logic Array, PLA the electronic circuit can execute computer readable program instructions to implement various aspects of the present application.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in hardware (eg, circuits or ASICs (Application) that perform the corresponding functions or actions. Specific Integrated Circuit, application-specific integrated circuit)), or can be implemented by a combination of hardware and software, such as firmware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

本申请涉及一种图像处理方法及其设备，所述方法应用于具有主相机的电子终端，所述方法包括：确定与所述主相机执行通信连接的至少一个辅相机；获取至少一个子图像，其中，所述至少一个子图像是指对所述至少一个辅相机拍摄的至少一个辅图像以及所述主相机拍摄的主图像执行图像处理后获取的图像；利用所述至少一个子图像，生成合成图像。采用本申请，不仅可满足用户对于多相机拍摄的需求，还能够根据用户的需要定制生成合成图像。

Description

图像处理方法及其设备
本申请要求于2020年11月20日提交中国专利局、申请号为202011314604.5、申请名称为“图像处理方法及其设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种图像处理方法及其设备。
背景技术
随着科技的进步，人们习惯使用相机记录生活中的点滴，因此，很多电子终端均内嵌有相机。在某些场景下，单个相机可能无法满足人们的需求，例如，在直播场景下，仅依靠单个相机无法将主播以及主播正在描述的产品很好地呈现出来，这就需要多个相机同时参与拍摄。
在相关技术中，采用多个相机共同进行拍摄的技术方案通常是将多个相机拍摄的图像利用通信连接传输到同一设备上进行拼接。但这种方式能够呈现的效果比较单一，无法满足用户的个性化需求。因此，相关技术需要一种能够控制不同相机同时参与拍摄并生成满足用户需求的图像的技术方案。
发明内容
有鉴于此，提出了一种图像处理方法及其设备，至少解决了以上提到的技术问题。
第一方面，本申请的实施例提供了一种图像处理方法，应用于具有主相机的电子终端，所述方法包括：确定与所述主相机执行通信连接的至少一个辅相机；获取至少一个子图像，其中，所述至少一个子图像是指对所述至少一个辅相机拍摄的至少一个辅图像以及所述主相机拍摄的主图像执行图像处理后获取的图像；利用所述至少一个子图像，生成合成图像。
采用该方法能够对不同相机获取的不同图像执行图像处理并利用图像处理后的图像生成合成图像，这样不仅可满足用户对于多相机拍摄的需求，还能够根据用户的需要定制生成合成图像。
根据第一方面，在第一方面的第一种可能的实现方式中，获取至少一个子图像，包括：获取所述至少一个辅相机拍摄的所述至少一个辅图像以及所述主相机拍摄的主图像；响应于接收到图像处理请求，从所述至少一个辅图像和所述主图像中确定执行所述图像处理的待处理图像以及针对所述待处理图像的图像处理；获取所述待处理图像执行所述图像处理后的所述至少一个子图像。
也就是说，对于辅相机拍摄的图像可采用如上方式进行各种定制化的图像处理，达到用户需要的图像效果，从而能够更加满足用户对于图像的个性化需求。
根据第一方面，在第一方面的第二种可能的实现方式中，响应于接收到图像处理请求，从所述至少一个辅图像和所述主图像中确定执行所述图像处理的待处理图像以 及针对所述待处理图像的图像处理之后，包括：从候选执行设备中，确定对所述待处理图像执行所述图像处理的执行设备；获取所述执行设备对所述待处理图像执行所述图像处理后的所述至少一个子图像。
在实施中，可确定执行设备对这些图像执行图像处理，从而能够动态调节各个执行设备所执行的任务，为处理图像选择更合适的执行设备，从而能够提高图像处理的效率，降低时延。
根据第一方面，在第一方面的第三种可能的实现方式中，所述执行设备包括所述电子终端和所述至少一个辅相机所在的辅设备的至少一个。
也就是说，在实施过程中，这些图像可选择在本端处理或者在远端处理，这种方式避免了设备差异过大导致的处理时间过长，并且满足了人们的个性化需求。
根据第一方面或以上第一方面的任意一种实现方式，在第一方面的第四种可能的实现方式中，确定与所述主相机执行通信连接的至少一个辅相机，包括：利用发现连接服务，发现所述电子终端所处环境内的所述至少一个辅相机；与所述至少一个辅相机所在的辅设备建立通信连接。
通过以上方式，可合理利用主相机周围的辅相机，并通过与其建立通信连接的方式，控制辅相机执行拍摄，从而能够拍摄出不同场景下的图像。
根据第一方面，在第一方面的第五种可能的实现方式中，从候选执行设备中，确定对所述待处理图像执行所述图像处理的执行设备包括：获取与每个候选执行设备建立的通信连接的通信连接信息；根据每个候选执行设备的通信连接信息和/或具备的设备能力信息，确定对所述待处理图像执行所述图像处理的执行设备。
在确定执行设备过程中，可利用通信连接信息和/或设备能力信息，选择更合适的执行设备，从而能够提高图像处理的效率，减少各个设备之间的时延。
根据第一方面，在第一方面的第六种可能的实现方式中，根据每个候选执行设备的通信连接信息和/或具备的设备能力信息，确定对所述待处理图像执行所述图像处理的执行设备包括：利用每个候选执行设备的通信连接信息，确定能够执行通信传输的候选执行设备；能够执行通信传输的候选执行设备中按照设备能力信息确定所述执行设备。
也就是说，所述方法可根据实时的网络状态确定能够执行图像处理的候选辅设备后，利用辅设备的设备能力信息确定执行图像处理的设备，从而能够在保证通信正常的情况下选择合适的处理设备，解决了部分设备对于图像处理的性能低以及缩短了跨设备处理导致的时延。
根据第一方面，在第一方面的第七种可能的实现方式中，所述设备能力信息包括数据处理信息和图像处理信息，其中，数据处理信息包括每个辅设备处理数据的能力信息，图像处理信息包括每个辅设备能够使图像产生的图像效果的信息。
也就是说，在确定执行设备的过程中，既考虑到每个设备的数据处理能力又考虑到每个设备所具备的图像处理能力，从而能够更合理地确定执行设备。
根据第一方面，在第一方面的第八种可能的实现方式中，从所述候选辅设备中按照设备能力信息确定所述执行设备包括：利用所述候选执行设备的数据处理信息和/或图像处理信息，对所述候选执行设备进行排序，确定所述候选执行设备执行图像处 理的优先顺序；按照所述优先顺序确定所述执行设备。
可以看出，在确定执行设备的过程中，可在确定通信连接正常的情况下，可利用以上所述的数据处理信息和图像处理信息进行排序，优选选择数据处理能力以及图像处理能力更优的设备，从而能够提高图像处理的速度并有效减轻部分辅设备的数据处理负担。
根据第一方面或第一方面的第八种或第九种可能的实现方式中，所述图像处理信息还包括执行实现所述图像效果的图像处理算法。
也就是说，每个辅设备在处理图像的过程中，存在采用不同图像处理算法却实现相同的图像效果的情况，因此，所述图像处理信息还包括实现图像效果的具体图像处理算法。
根据第一方面或第一方面的第十种可能的实现方式中，利用所述通信连接，从所述至少一个辅设备接收所述图像处理算法。
作为具有主相机的电子终端，通常所具有的性能要优于各个辅设备，但是各个设备具有的图像处理算法可能各有不同，达到的图像效果也存在差异性，为了保证所述电子终端对于所有图像均可执行用户期望的图像处理，可将辅设备所具有的图像处理算法同步给所述电子终端，以保证这些图像可在所述电子终端上执行。
根据第一方面或以上第一方面的任意一种实现方式，在第一方面的第十一种可能的实现方式中，利用所述至少一个子图像，生成合成图像，包括：根据预设图像合成模式，对所述至少一个子图像进行合成，生成所述合成图像。
对于已执行图像处理后的辅图像与主图像，可根据预设图像合成模式生成所述合成图像，从而能够满足用户对于合成图像的个性化需求。
第二方面，本申请的实施例提供了一种图像处理方法，所述方法包括确定彼此之间执行通信连接的多个相机；获取由所述多个相机拍摄的多个原始图像执行图像处理后的多个子图像；利用所述多个子图像，生成合成图像。
也就是说，可利用通信连接的多个相机拍摄图像并获取对这些图像执行图像处理后的子图像，最后利用这些子图像生成合成图像，从而满足了人们对于多个图像拍摄图像期间对各个图像的要求。
根据第二方面，在第二方面的第一种可能的实现方式中，获取由所述多个相机拍摄的多个原始图像执行图像处理后的多个子图像包括：确定对每个原始图像执行对应图像处理的执行设备；获取利用所述执行设备对每个原始图像执行对应图像处理后的所述多个子图像。
在实施中，可在确定对这些原始图像执行图像处理后，确定执行图像处理的执行设备，不仅满足了用户对于各个原始图像的用户需求，并且降低了图像处理的时间。
根据第二方面，在第二方面的第一种可能的实现方式中，确定对每个原始图像执行对应图像处理的执行设备，包括：根据每个执行设备的设备能力信息和通信连接信息，确定执行对应图像处理的执行设备。
在实施中，可通过设备能力信息和通信连接信息来确定执行设备，从而能够量化确定执行设备，使得确定的执行设备更适合执行图像处理。
第三方面，本申请的实施例提供了一种图像处理设备，该图像处理设备可以执行 上述第一方面或者第一方面的多种可能的实现方式中的一种或几种的图像处理方法以及上述第二方面或者第二方面的多种可能的实现方式中的一种或几种的图像处理方法。
第四方面，本申请的实施例提供一种非易失性计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述第一方面或者第一方面的多种可能的实现方式中的一种或几种的图像处理方法以及上述第二方面或者第二方面的多种可能的实现方式中的一种或几种的图像处理方法。
第五方面，本申请的实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行上述第一方面或者第一方面的多种可能的实现方式中的一种或几种的图像处理方法以及上述第二方面或者第二方面的多种可能的实现方式中的一种或几种的图像处理方法。
本申请的这些和其他方面在以下(多个)实施例的描述中会更加简明易懂。
附图说明
包含在说明书中并且构成说明书的一部分的附图与说明书一起示出了本申请的示例性实施例、特征和方面，并且用于解释本申请的原理。
图1示出根据本申请一实施例的执行图像处理的应用场景图；
图2示出根据本申请一实施例的设备间的交互流程图；
图3示出根据本申请一实施例的电子设备的软件结构示意图；
图4示出根据本申请一实施例的图像处理方法的步骤流程图；
图5示出根据本申请一实施例的电子设备的结构框图；
图6示出根据本申请一实施例的电子设备的结构示意图。
具体实施方式
以下将参考附图详细说明本申请的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
另外，为了更好的说明本申请，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本申请同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本申请的主旨。
本申请中提及的相机是指可通过相机的图像传感器捕捉图像的装置，本申请所述的电子终端是指具有相机的各种设备，包括诸如手机、平板电脑、可穿戴设备(例如，手表、手环、智能头盔等)、车载设备、智能家居设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等。本申请一些实施例以手机为例，可以理解的是，这些实施例也可 应用于其他类型的移动终端。
为了更好地描述本申请，以下将结合图1具体描述根据本申请的示例性实施例的应用场景。
如图1所示的场景中，用户采用多个手机执行拍摄。在拍摄期间，手机101可将自然景观10拍摄下来，而相机102可负责拍摄用户11，随后将手机101拍摄的图像1001作为背景图像，将相机拍摄的图像1002作为前景图像，并最终生成合成图像1003。在此场景下，用户期望对相机102拍摄的用户11执行美化，比如，美白处理、瘦身处理等，但是相机102不具备这些能力，在此情况下，可采用本申请的示例性实施例的图像处理方法。
虽然图1中仅示出了两个相机和/或包含相机的设备，但本申请不限定相机的数量，在本申请中提及的相机可包括在单个电子终端内的多个相机(例如，内嵌的多个主/辅相机)以及包括相机的远端设备。
在本申请中提及的各个相机以及具有各个相机的电子终端的性能并不做限定，也就是说，每个相机拍摄出的效果以及各个电子终端对于图像的处理效果可以是相同的也可以是不同的。
除了图1示出的场景，本申请的示例性实施例的图像处理方法还可应用于多个应用场景下。例如，用户A可采用手机A并且用户B可采用手机B执行视频通话，随后，手机A可对于由手机A捕捉的图像A与手机B捕捉的图像B，采用本申请的示例性实施例的图像处理方法生成由图像A与图像B合成的具有特定图像处理效果的合成图像。举例来说，在该合成图像中，图像A捕捉的用户A的头像变成的特定动物图像，图像B捕捉的用户B的头像不变。
此外，本申请的示例性实施例的图像处理方法还可用于视频直播的场景下。在直播场景下，主播可利用性能较好的手机作为主设备，捕捉与主播相关的画面，同时，可采用多个辅设备捕捉与商品相关的画面，随后，可利用主设备和多个辅设备捕捉的各个图像生成合成图像，在合成图像中，将主播相关的画面作为主要图像，而将辅设备捕捉的图像作为各个子图布置在主要画面的一侧。
以上仅给出了可应用于本申请的示例性实施例的图像处理方法的应用场景的示例，并非限定性的。在实施过程中，只要所生成的最终图像是采用本地设备捕捉的图像以及远程设备捕捉的图像而生成的图像的情况，则可采用本申请的示例性实施例的图像处理方法。
图2示出根据本申请一实施例的设备间的交互流程图。为了便于描述，图2中仅包括两个手机21和22，每个手机均有内嵌的相机。针对相机应用，为了实现多个电子设备之间的控制。
总的来说，可通过将手机附近的设备转换为手机内的虚拟组件，从而将其能力作为手机的扩展能力来使用，在本申请中，可将手机附近的相机或具有相机的设备转换为手机内的虚拟相机来使用。
为此，如图2所示，手机21可开启设备发现服务，在手机21可获知其附近一定范围内都存在哪些设备，这种发现附近的设备的技术被称为近距离发现技术。进一步的，借助近距离发现技术，还可以实现一定距离内的设备到设备(Device To Device， D2D)通信，故而称这些具备D2D通信能力的设备为近距离设备。在相关技术中，实现近距离发现的方法是在网络中设置一个近距离业务服务器，手机21注册到近距离业务服务器中以获得近距离服务。当手机21需要发现附近的其他手机21时，向近距离业务服务器发送发现请求，该发送请求中可具有具体的类别指示，例如，发现相机类硬件，近距离业务服务器控制手机21之间的测量和发现，或者近距离业务服务器指示基站，由基站控制手机21之间的测量和发现。因此，手机21通过网络的控制，可以发现周围一定距离内的设备。
随后，手机21可与设备发现服务中发现的手机22建立通信连接，并向手机22发送授权请求，在实施中，手机22的界面上可显示提示用户授权的弹框，用户可在弹框中选择是否同意授权手机21使用手机22内部的相机，优选地，还可通过弹框提示用户选择不用的授权类型。
在已获取到手机22的授权后，可对手机22进行认证，也就是说，确定手机21与手机22所登录的账号是否相同，若相同，则认证为同账号认证，若不同，则确定为跨账号设备，则在手机22的界面上提示输出验证信息，确保完成账号认证。
随后，手机21可将手机22内的相机映射到手机21内作为手机21内的虚拟相机，在此过程中，手机21可对虚拟相机执行注册，该注册过程包括为手机22以及虚拟相机分配标识信息。此外，手机21还可从手机22获取到设备能力信息，其中，所述设备能力信息可包括相机的属性信息，例如，相机的光角、光学变焦倍数等，还包括数据处理信息，也就是说，手机22处理数据的能力，此外，所述设备能力信息还包括图像处理信息，例如，能够实现特定图像效果的图像算法信息。最后，手机21可控制打开其自有相机以及手机22内的相机。
为了更好地描述本申请，以下将参照图3对电子终端内部的架构进行描述。在本申请实施例中，主设备/辅设备看采用分层架构、事件驱动架构、微核架构、微服务架构或云架构。本申请实施例以分层架构的Android系统为例，示例性地描述电子设备的软件结构。
为了更好地说明本申请，以下针对电子设备的软件结构的描述仅以相机应用为例，但在实施中，电子设备还可包括各种应用。此外，为了方便对电子设备进行描述，将用户用于生成合成图像的电子设备作为主设备，辅助主设备生成合成图像的电子设备作为辅设备。
如图3所示，主设备包括应用程序(Application)层、应用程序框架(Framework)层、硬件抽象(hard abstraction layer，HAL)层以及内核(Kernel)层。
应用程序层可以包括一系列应用程序包，例如，应用程序包可以包括电话、相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。Framework层为开发人员提供了一个可以完全访问应用程序所使用的API框架。具体的，Framework层为开发应用程序提供了非常多的API，通过调用相应的API可以构造满足相关业务需求的APP。在应用程序为相机的情况下，应用程序框架层可提供很多与相机相关的API。
图3中服务层指示相机可提供的各种服务，在实施中，所述服务层可包括本地相机的相关服务，包括设备控制命令以及设备控制数据，在本申请中，所述服务层还可包括与辅设备对应的虚拟相机的虚拟相机服务，也就是说，在实施中，可通过在服务层增加与虚拟相机对应的虚拟相机服务的方式向主设备提供可供调用的虚拟相机服务。
HAL层是位于操作系统内核与硬件电路之间的接口层，其目的在于将硬件抽象化。它隐藏了特定平台的硬件接口细节,为操作系统提供虚拟硬件平台,使其具有硬件无关性,可在多种平台上进行移植。从软硬件测试的角度来看，软硬件的测试工作都可分别基于硬件抽象层来完成，使得软硬件测试工作的并行进行成为可能。
在HAL层中可包括相机设备信息以及相机算法信息，其中，相机设备信息是指与相机的性能相关的参数信息，而相机算法信息是指相机可对图像执行处理的图像处理算法。例如，用户可通过HAL层调用相机算法对拍摄到的图像进行虚化处理。如图3所示，HAL层可具有与本地相机相关的本地相机HAL以及与本地相机HAL结构相同的虚拟相机HAL层，也就是说，虚拟相机HAL同样包括相机设备信息以及相机算法信息。在实施中，可通过在HAL层增加与辅相机对应的虚拟相机HAL的方式，增加虚拟相机，这样服务层的本地相机服务可根据需求调用本地相机HAL或者虚拟相机HAL。
最后，内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动等驱动。
在实施中，可采用以上描述的方式将远程相机转换为本地相机所在的电子设备(例如，手机)内的虚拟相机，在此基础上，本申请将重点描述如何合理分配本地相机(以下提及的主相机)以及远程相机(以下提及的辅相机)，从而提高图像处理效率，满足用户的需求。以下将结合图4对本申请的示例性实施例的图像处理方法进行详细描述。图4示出了根据本申请一实施例的图像处理方法的步骤流程图。
图4所示的方法可应用于内嵌有主相机的主设备中，也就是说，用户在采用多相机执行拍摄时，通常将某一相机所在的电子终端作为主要操作终端，基于此，可将用户的主要操作终端称作主设备，该主设备内的相机称作主相机。应注意，所述主相机的数量不限，取决于该电子终端内嵌的相机数量。
在步骤S410，确定与所述主相机执行通信连接的至少一个辅相机。如以上所述，可通过开启电子终端内的发现连接服务，发现所述电子终端所处环境内的所述至少一个辅相机，优选地，可利用不同进场通信技术发现所述电子终端的近距离内的辅相机，但在实施中，也可通过无线传输网络发现与电子终端处于不同空间内的辅相机。举例来说，用户在使用主相机时，可通过蓝牙发现与该用户处于同一房间内的辅相机，此外，用户在使用主相机时，还可将与用户处于不同城市的用户朋友的相机作为辅相机。
随后，可与所述至少一个辅相机所在的辅设备建立通信连接。在此期间，主设备可如图2所示获取每个辅设备的认证授权，并在主设备内建立与所述辅相机对应的辅相机。应注意，在与辅设备建立通信连接后，可将各个辅设备内的图像处理信息发送给主设备，也就是说，将主设备转换为具有更多图像处理信息的设备，其中，图像处理信息包括图像处理算法。
在实施中，主设备通常是选择性能较好的设备，在这种情况下，可将辅设备内的各种图像处理算法均同步给主设备，这样可保证主设备尽可能地满足用户对于合成图 像所包括的各个子图像的要求。
在步骤S420，获取至少一个子图像，其中，所述至少一个子图像是指对所述至少一个辅相机拍摄的至少一个辅图像以及所述主相机拍摄的主图像执行图像处理后获取的图像。
在实施中，主设备可控制主相机获取主图像，并可控制至少一个辅相机并获取所述至少一个辅相机拍摄的所述至少一个辅图像。随后，可响应于用户输入的图像处理请求，从所述至少一个辅图像和主图像中确定执行所述图像处理的待处理图像以及针对所述待处理图像的图像处理。
具体来说，针对至少一个辅相机拍摄的多个辅图像，用户可从这些辅图像中选择部分图像或者全部图像，举例来说，在手机A与相机B以及笔记本C均进行通信连接的情况下，手机A可控制相机B以及笔记本C拍摄辅图像，在获取到辅图像B和辅图像C的情况下，用户可从这些辅图像只选择辅图像B。
随后，用户可选择为这些图像执行相应的图像处理，例如，用户在选择图像B后，为图像B选择美白处理，并在选择图像C后，为图像C选择虚化处理并为图像A选择瘦身处理。在具体实施中，可在主设备的显示页面上显示这些辅设备拍摄的辅图像以及可提供的图像处理，用户可通过触摸选择需要的子图像以及针对该辅图像的图像处理，其中，所述子图像包括辅图像和主图像。
最后，获取所述待处理图像执行所述图像处理后的所述至少一个子图像。具体来说，可先确定对所述待处理图像执行所述图像处理的执行设备，所述执行设备可包括主设备和/或辅设备。也就是说，关于对待处理图像执行图像处理的执行设备可能存在多种可能性，包括每个待处理图像均由主设备执行图像处理，又例如，部分待处理图像由主设备执行图像处理，但是部分待处理图像由相应的辅设备执行图像处理，其中，相应的辅设备可以是捕捉待处理图像的辅设备也可以不是捕捉待处理图像的设备，又例如，所有待处理图像均由单个辅设备执行图像处理。
基于以上各种可能性，本申请的优选实施例提供了一种确定执行设备的方式，具体来说，获取与每个候选执行设备建立的通信连接的通信连接信息，在实施中，为了保证通信有效性，可实时或者等时间间隔(时间间隔很短)地获取每个候选执行设备的通信连接信息，利用所述通信连接信息可确定主设备与辅设备的通信状态。
随后，根据每个候选执行设备的通信连接信息和/或具备的设备能力信息，确定对所述待处理图像执行所述图像处理的执行设备，其中，所述设备能力信息包括数据处理信息和图像处理信息，其中，数据处理信息包括每个辅设备处理数据的能力信息，图像处理信息包括每个辅设备能够使图像产生的图像效果的信息。也就是说，在考虑执行设备时，优选地实施例是同时考虑每个辅设备的数据处理能力和图像处理能力。
具体来说，可利用每个候选执行设备的通信连接信息，确定能够执行通信传输的候选辅设备。在实施过程中，可为通信传输速率设置阈值，在高于该阈值的情况下，可选择除了本机之外的其他执行设备，而在通信传输速率低于阈值的情况下，选择本机作为执行主体。
随后，可从所述候选执行设备中按照设备能力信息确定所述执行设备。简言之，利用所述候选执行设备的数据处理信息和/或图像处理信息，对所述候选执行设备进行 排序，确定所述候选执行设备执行图像处理的优先顺序，随后按照所述优先顺序确定所述执行设备。
作为示例，可利用候选执行设备的数据处理信息对这些候选执行设备进行排序，举例来说，可根据候选执行设备的CPU性能对这些候选执行设备进行排序，从而确定执行图像处理的优先顺序。
作为示例，还可利用候选执行设备的图像处理信息对这些候选执行设备进行排序，举例来说，可根据候选执行设备的GPU性能对这些候选执行设备进行排序，从而确定执行图像处理的优先顺序。
此外，所述方法还可先确定用户为各个待处理图像选择的图像处理，然后，从这些候选执行设备中确定能够执行这些图像处理的设备，随后，将这些设备按照图像处理的效果进行排序，从而确定执行图像处理的优先顺序。在实施中，所述方法还可综合考虑数据处理信息和图像处理信息，对这些候选执行设备进行排序，从而确定执行图像处理的优先顺序。
在确定了执行设备后，主设备可将待处理图像发送到执行设备执行处理，执行设备在对待处理图像执行图像处理后，将图像处理后的图像发送给主设备。
在实施中，为了提高各个设备之间的安全性，针对拍摄的图像(包括主相机拍摄的主图像以及各个辅相机拍摄的辅图像)，可仅根据设备能力信息确定该图像是由拍摄主体执行图像处理还是由主设备执行图像处理。举例来说，辅设备A拍摄的辅图像A，可在网络连接正常的情况下，根据设备能力信息确定辅图像A是由辅设备A执行相应的图像处理还是由主设备执行相应的图像处理。
在步骤S430，利用所述至少一个子图像，生成合成图像。
在实施中，根据预设图像合成模式，对这些子图像进行合成，生成所述合成图像，其中，预设图像合成模式可以是由用户指定的模式，例如，可选择某一图像作为背景图像，其他图像作为前景图像，又例如，可将这些图像全部拼接出来。
以上是以用户所操作的用户终端作为主视角进行的描述，在实施中，可能存在一种场景是存在多个相机，每个相机均具有图像捕捉能力以及图像处理能力，并且可通过以上描述的方式彼此之间进行通信，在此情况下，每个相机均可作为控制终端来确定捕捉的各个图像的执行设备，此外，还可采用与这些相机执行通信连接的单独的服务器来确定各个图像的执行设备。
在此场景下，首先，可确定彼此之间执行通信连接的多个相机，随后，可获取由所述多个相机拍摄的多个原始图像执行图像处理后的多个子图像，具体来说，可确定对每个原始图像执行对应图像处理的执行设备，具体包括：根据每个执行设备的设备能力信息和通信连接信息，确定执行对应图像处理的执行设备。获取利用所述执行设备对每个原始图像执行对应图像处理后的所述多个子图像。最后，利用所述多个子图像，生成合成图像。
可以理解的是，上述终端等为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请实施例能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决 于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请实施例的范围。
本申请实施例可以根据上述方法示例对上述终端等进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图5示出了上述实施例中所涉及的电子终端的一种结构框图，该电子装置包括：辅相机确定单元510、子图像获取单元520以及合成图像生成单元530，其中，辅相机确定单元510用于确定与所述主相机执行通信连接的至少一个辅相机；子图像获取单元520用于获取至少一个子图像，其中，所述至少一个子图像是指对所述至少一个辅相机拍摄的至少一个辅图像以及所述主相机拍摄的主图像执行图像处理后获取的图像；合成图像生成单元530用于利用所述至少一个子图像，生成合成图像。
可选地，子图像获取单元520包括图像获取模块、图像处理确定模块以及子图像获取模块，其中，图像获取模块用于获取所述至少一个辅相机拍摄的所述至少一个辅图像以及主相机拍摄的主图像；图像处理确定模块用于响应于接收到图像处理请求，从所述至少一个辅图像和主图像中确定执行所述图像处理的待处理图像以及针对所述待处理图像的图像处理；子图像获取模块用于获取所述待处理图像执行所述图像处理后的所述至少一个子图像。
所述电子设备还包括执行设备确定模块用于确定对所述待处理图像执行所述图像处理的执行设备，在此基础上，子图像获取模块具体用于获取所述执行设备对所述待处理图像执行所述图像处理后的所述至少一个子图像。
可选地，所述执行设备包括所述电子终端和所述至少一个辅相机所在的辅设备的至少一个。
辅相机确定单元510包括发现模块和通信连接模块，其中，发现模块用于利用发现连接服务，发现所述电子终端所处环境内的所述至少一个辅相机，通信连接模块用于与所述至少一个辅相机所在的辅设备建立通信连接。
执行设备确定模块包括通信连接信息确定子模块以及执行设备子模块，其中，通信连接信息确定子模块用于获取与每个候选执行设备建立的通信连接的通信连接信息；执行设备子模块用于根据每个候选执行设备的通信连接信息和/或具备的设备能力信息，确定对所述待处理图像执行所述图像处理的执行设备。
可选地，执行设备子模块具体用于利用每个候选执行设备的通信连接信息，确定能够执行通信传输的候选辅设备；从所述候选辅设备中按照设备能力信息确定所述执行设备。
可选地，所述设备能力信息包括数据处理信息和图像处理信息，其中，数据处理信息包括每个辅设备处理数据的能力信息，图像处理信息包括每个辅设备能够使图像产生的图像效果的信息。
执行设备子模块具体用于利用所述候选执行设备的数据处理信息和/或图像处理 信息，对所述候选执行设备进行排序，确定所述候选执行设备执行图像处理的优先顺序；按照所述优先顺序确定所述执行设备。
可选地，所述图像处理信息还包括执行实现所述图像效果的图像处理算法。
所述电子设备还包括图像处理算法接收单元，用于利用所述通信连接，从所述至少一个辅设备接收所述图像处理算法。
可选地，合成图像生成单元530用于根据预设图像合成模式，对所述主图像与所述至少一个辅图像进行合成，生成所述合成图像。
图6示出根据本申请一实施例的终端设备的结构示意图。以终端设备是手机为例，图6示出了手机200的结构示意图。
手机200可以包括处理器210，外部存储器接口220，内部存储器221，USB接口230，充电管理模块240，电源管理模块241，电池242，天线1，天线2，移动通信模块251，无线通信模块252，音频模块270，扬声器270A，受话器270B，麦克风270C，耳机接口270D，传感器模块280，按键290，马达291，指示器292，摄像头293，显示屏294，以及SIM卡接口295等。其中传感器模块280可以包括陀螺仪传感器280A，加速度传感器280B，接近光传感器280G、指纹传感器280H，触摸传感器280K(当然，手机200还可以包括其它传感器，比如温度传感器，压力传感器、距离传感器、磁传感器、环境光传感器、气压传感器、骨传导传感器等，图中未示出)。
可以理解的是，本申请实施例示意的结构并不构成对手机200的具体限定。在本申请另一些实施例中，手机200可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器210可以包括一个或多个处理单元，例如：处理器210可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor,ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor,DSP)，基带处理器，和/或神经网络处理器(Neural-network Processing Unit,NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。其中，控制器可以是手机200的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器210中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器210中的存储器为高速缓冲存储器。该存储器可以保存处理器210刚用过或循环使用的指令或数据。如果处理器210需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器210的等待时间，因而提高了系统的效率。
处理器210可以运行本申请实施例提供的图像处理方法，不仅可满足用户对于多相机拍摄的需求，还能够根据用户的需要定制生成合成图像。处理器210可以包括不同的器件，比如集成CPU和GPU时，CPU和GPU可以配合执行本申请实施例提供的图像处理方法，比如图像处理方法中部分算法由CPU执行，另一部分算法由GPU执行，以得到较快的处理效率。
显示屏294用于显示图像，视频等。显示屏294包括显示面板。显示面板可以采 用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，手机200可以包括1个或N个显示屏294，N为大于1的正整数。显示屏294可用于显示由用户输入的信息或提供给用户的信息以及各种图形用户界面(graphical user interface，GUI)。例如，显示器294可以显示照片、视频、网页、或者文件等。再例如，显示器294可以显示图形用户界面。其中，图形用户界面上包括状态栏、可隐藏的导航栏、时间和天气小组件(widget)、以及应用的图标，例如浏览器图标等。状态栏中包括运营商名称(例如中国移动)、移动网络(例如4G)、时间和剩余电量。导航栏中包括后退(back)键图标、主屏幕(home)键图标和前进键图标。此外，可以理解的是，在一些实施例中，状态栏中还可以包括蓝牙图标、Wi-Fi图标、外接设备图标等。还可以理解的是，在另一些实施例中，图形用户界面中还可以包括Dock栏，Dock栏中可以包括常用的应用图标等。当处理器210检测到用户的手指(或触控笔等)针对某一应用图标的触摸事件后，响应于该触摸事件，打开与该应用图标对应的应用的用户界面，并在显示器294上显示该应用的用户界面。
在本申请实施例中，显示屏294可以是一个一体的柔性显示屏，也可以采用两个刚性屏以及位于两个刚性屏之间的一个柔性屏组成的拼接显示屏。
当处理器210运行本申请实施例提供的图像处理方法后，终端设备可以通过天线1、天线2或者USB接口与其他的终端设备建立连接，并根据本申请实施例提供的图像处理方法传输数据以及控制显示屏294显示相应的图形用户界面。
摄像头293(前置摄像头或者后置摄像头，或者一个摄像头既可作为前置摄像头，也可作为后置摄像头)用于捕获静态图像或视频。通常，摄像头293可以包括感光元件比如镜头组和图像传感器，其中，镜头组包括多个透镜(凸透镜或凹透镜)，用于采集待拍摄物体反射的光信号，并将采集的光信号传递给图像传感器。图像传感器根据所述光信号生成待拍摄物体的原始图像。
内部存储器221可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器210通过运行存储在内部存储器221的指令，从而执行手机200的各种功能应用以及数据处理。内部存储器221可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，应用程序(比如相机应用，微信应用等)的代码等。存储数据区可存储手机200使用过程中所创建的数据(比如相机应用采集的图像、视频等)等。
内部存储器221还可以存储本申请实施例提供的图像执行处理方法对应的一个或多个计算机程序1310。该一个或多个计算机程序1304被存储在上述存储器221中并被配置为被该一个或多个处理器210执行，该一个或多个计算机程序1310包括指令，上述指令可以用于执行如图4相应实施例中的各个步骤，该计算机程序1310可以包括辅相机确定模块61、子图像获取模块62、以及合成图像生成模块63。其中，相机确定模块61，用于确定与所述主相机执行通信连接的至少一个辅相机；子图像获取模块 62，用于获取至少一个子图像，其中，所述至少一个子图像是指对所述至少一个辅相机拍摄的至少一个辅图像以及所述主相机拍摄的主图像执行图像处理后获取的图像，合成图像生成模块63，用于利用所述至少一个子图像，生成合成图像。当内部存储器221中存储的数据的传输方法的代码被处理器210运行时，处理器210可以控制显示屏显示合成图像。
此外，内部存储器221可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。
当然，本申请实施例提供的图像处理方法的代码还可以存储在外部存储器中。这种情况下，处理器210可以通过外部存储器接口220运行存储在外部存储器中的图像处理方法的代码。
下面介绍传感器模块280的功能。
陀螺仪传感器280A，可以用于确定手机200的运动姿态。在一些实施例中，可以通过陀螺仪传感器280A确定手机200围绕三个轴(即，x，y和z轴)的角速度。即陀螺仪传感器280A可以用于检测手机200当前的运动状态，比如抖动还是静止。
当本申请实施例中的显示屏为可折叠屏时，陀螺仪传感器280A可用于检测作用于显示屏294上的折叠或者展开操作。陀螺仪传感器280A可以将检测到的折叠操作或者展开操作作为事件上报给处理器210，以确定显示屏294的折叠状态或展开状态。
加速度传感器280B可检测手机200在各个方向上(一般为三轴)加速度的大小。即陀螺仪传感器280A可以用于检测手机200当前的运动状态，比如抖动还是静止。当本申请实施例中的显示屏为可折叠屏时，加速度传感器280B可用于检测作用于显示屏294上的折叠或者展开操作。加速度传感器280B可以将检测到的折叠操作或者展开操作作为事件上报给处理器210，以确定显示屏294的折叠状态或展开状态。
接近光传感器280G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。发光二极管可以是红外发光二极管。手机通过发光二极管向外发射红外光。手机使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时，可以确定手机附近有物体。当检测到不充分的反射光时，手机可以确定手机附近没有物体。当本申请实施例中的显示屏为可折叠屏时，接近光传感器280G可以设置在可折叠的显示屏294的第一屏上，接近光传感器280G可根据红外信号的光程差来检测第一屏与第二屏的折叠角度或者展开角度的大小。
陀螺仪传感器280A(或加速度传感器280B)可以将检测到的运动状态信息(比如角速度)发送给处理器210。处理器210基于运动状态信息确定当前是手持状态还是脚架状态(比如，角速度不为0时，说明手机200处于手持状态)。
指纹传感器280H用于采集指纹。手机200可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。
触摸传感器280K，也称“触控面板”。触摸传感器280K可以设置于显示屏294，由触摸传感器280K与显示屏294组成触摸屏，也称“触控屏”。触摸传感器280K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏294提供与触摸操作相关的视觉输 出。在另一些实施例中，触摸传感器280K也可以设置于手机200的表面，与显示屏294所处的位置不同。
示例性的，手机200的显示屏294显示主界面，主界面中包括多个应用(比如相机应用、微信应用等)的图标。用户通过触摸传感器280K点击主界面中相机应用的图标，触发处理器210启动相机应用，打开摄像头293。显示屏294显示相机应用的界面，例如取景界面。
手机200的无线通信功能可以通过天线1，天线2，移动通信模块251，无线通信模块252，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。手机200中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块251可以提供应用在手机200上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块251可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块251可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块251还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块251的至少部分功能模块可以被设置于处理器210中。在一些实施例中，移动通信模块251的至少部分功能模块可以与处理器210的至少部分模块被设置在同一个器件中。在本申请实施例中，移动通信模块251还可以用于与其它终端设备进行信息交互。例如，移动通信模块251可将拍摄的图像发送到其它终端设备，并收到其它终端设备发送的图像。
调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器270A，受话器270B等)输出声音信号，或通过显示屏294显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器210，与移动通信模块251或其他功能模块设置在同一个器件中。
无线通信模块252可以提供应用在手机200上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bl手机21tooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(freq手机21ncy modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块252可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块252经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器210。无线通信模块252还可以从处理器210接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。本申请实施例中，无线通信模块252，用于在处理器210的控制下与其他终端设备之间传输数据，比如，处理器210运行本申请实施例提 供的图像处理方法时，处理器可以控制无线通信模块252向其他终端设备发送拍摄图像请求，还可以接收其他终端设备基于上述拍摄图像请求所拍摄的辅图像，并利用辅图像生成合成图像后控制显示屏294显示合成图像，为用户提供直观的视觉反馈，并可根据用户需求实时调整合成图像。
另外，手机200可以通过音频模块270，扬声器270A，受话器270B，麦克风270C，耳机接口270D，以及应用处理器等实现音频功能。例如音乐播放，录音等。手机200可以接收按键290输入，产生与手机200的用户设置以及功能控制有关的键信号输入。手机200可以利用马达291产生振动提示(比如来电振动提示)。手机200中的指示器292可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。手机200中的SIM卡接口295用于连接SIM卡。SIM卡可以通过插入SIM卡接口295，或从SIM卡接口295拔出，实现和手机200的接触和分离。
应理解，在实际应用中，手机200可以包括比图5所示的更多或更少的部件，本申请实施例不作限定。图示手机200仅是一个范例，并且手机200可以具有比图中所示出的更多的或者更少的部件，可以组合两个或更多的部件，或者可以具有不同的部件配置。图中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
本申请的实施例提供了一种图像处理装置，包括：处理器以及用于存储处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令时实现上述方法。
本申请的实施例提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。
本申请的实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行上述方法。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(Random Access Memory，RAM)、只读存储器(Read Only Memory，ROM)、可擦式可编程只读存储器(Electrically Programmable Read-Only-Memory，EPROM或闪存)、静态随机存取存储器(Static Random-Access Memory，SRAM)、便携式压缩盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、数字多功能盘(Digital Video Disc，DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。
这里所描述的计算机可读程序指令或代码可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本申请操作的计算机程序指令可以是汇编指令、指令集架构(Instruction Set Architecture，ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或可编程逻辑阵列(Programmable Logic Array，PLA)，该电子电路可以执行计算机可读程序指令，从而实现本申请的各个方面。
这里参照根据本申请实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本申请的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本申请的多个实施例的装置、系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。
也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行相应的功能或动作的硬件(例如电路或ASIC(Application Specific Integrated Circuit，专用集成电路))来实现，或者可以用硬件和软件的组合，如固件等来实现。
尽管在此结合各实施例对本发明进行了描述，然而，在实施所要求保护的本发明过程中，本领域技术人员通过查看所述附图、公开内容、以及所附权利要求书，可理解并实现所述公开实施例的其它变化。在权利要求中，“包括”(comprising)一词不排除其他组成部分或步骤，“一”或“一个”不排除多个的情况。单个处理器或其它单元可以实现权利要求中列举的若干项功能。相互不同的从属权利要求中记载了某些措施，但这并不表示这些措施不能组合起来产生良好的效果。
以上已经描述了本申请的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (17)

  1. 一种图像处理方法，所述方法应用于具有主相机的电子终端，其特征在于，包括：
    确定与所述主相机执行通信连接的至少一个辅相机；
    获取至少一个子图像，其中，所述至少一个子图像是指对所述至少一个辅相机拍摄的至少一个辅图像以及所述主相机拍摄的主图像执行图像处理后获取的图像；
    利用所述至少一个子图像，生成合成图像。
  2. 如权利要求1所述的方法，其特征在于，获取至少一个子图像，包括：
    获取所述至少一个辅相机拍摄的所述至少一个辅图像以及所述主相机拍摄的所述主图像；
    响应于接收到图像处理请求，从所述至少一个辅图像以及所述主图像中确定执行所述图像处理的待处理图像以及针对所述待处理图像的图像处理；
    获取所述待处理图像执行所述图像处理后的所述至少一个子图像。
  3. 如权利要求2所述的方法，其特征在于，响应于接收到图像处理请求，从所述至少一个图像以及所述主图像中确定执行所述图像处理的待处理图像以及针对所述待处理图像的图像处理之后，包括：
    从候选执行设备中，确定对所述待处理图像执行所述图像处理的执行设备；
    获取所述执行设备对所述待处理图像执行所述图像处理后的所述至少一个子图像。
  4. 如权利要求3所述的方法，其特征在于，所述候选执行设备包括所述电子终端和所述至少一个辅相机所在的辅设备的至少一个。
  5. 如权利要求4所述的方法，其特征在于，确定与所述主相机执行通信连接的至少一个辅相机，包括：
    利用发现连接服务，发现所述电子终端所处环境内的所述至少一个辅相机；
    与所述至少一个辅相机所在的辅设备建立通信连接。
  6. 如权利要求5所述的方法，其特征在于，从候选执行设备中，确定对所述待处理图像执行所述图像处理的执行设备包括：
    获取与每个候选执行设备建立的通信连接的通信连接信息；
    根据每个候选执行设备的通信连接信息和/或具备的设备能力信息，确定对所述待处理图像执行所述图像处理的执行设备。
  7. 如权利要求6所述的方法，其特征在于，根据每个候选执行设备的通信连接信息和/或具备的设备能力信息，确定对所述待处理图像执行所述图像处理的执行设备包括：
    利用每个候选执行设备的通信连接信息，确定能够执行通信传输的候选执行设备；
    从能够执行通信传输的候选执行设备中，按照设备能力信息确定所述执行设备。
  8. 如权利要求7所述的方法，其特征在于，所述设备能力信息包括数据处理信息和图像处理信息，其中，数据处理信息包括每个辅设备处理数据的能力信息，图像处理信息包括每个辅设备能够使图像产生的图像效果的信息。
  9. 如权利要求8所述的方法，其特征在于，从所述候选辅设备中按照设备能力信息确定所述执行设备,包括：
    利用所述候选执行设备的数据处理信息和/或图像处理信息，对所述候选执行设备进行排序，确定所述候选执行设备执行图像处理的优先顺序；
    按照所述优先顺序确定所述执行设备。
  10. 如权利要求8或9所述的方法，其特征在于，所述图像处理信息还包括执行实现所述图像效果的图像处理算法。
  11. 如权利要求10所述的方法，其特征在于，还包括：
    利用所述通信连接，从所述至少一个辅设备接收所述图像处理算法。
  12. 如权利要求1至11中的任一权利要求所述的方法，其特征在于，利用所述至少一个子图像，生成合成图像，包括：
    根据预设图像合成模式，对所述至少一个子图像进行合成处理，生成所述合成图像。
  13. 一种图像处理方法，其特征在于，包括：
    确定彼此之间执行通信连接的多个相机；
    获取由所述多个相机拍摄的多个原始图像执行图像处理后的多个子图像；
    利用所述多个子图像，生成合成图像。
  14. 如权利要求13所述的方法，其特征在于，获取由所述多个相机拍摄的多个原 始图像执行图像处理后的多个子图像包括：
    确定对每个原始图像执行对应图像处理的执行设备；
    获取利用所述执行设备对每个原始图像执行对应图像处理后的所述多个子图像。
  15. 如权利要求14所述的方法，其特征在于，确定对每个原始图像执行对应图像处理的执行设备，包括：
    根据每个执行设备的设备能力信息和通信连接信息，确定执行对应图像处理的执行设备。
  16. 一种图像处理设备，其特征在于，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为执行所述指令时实现权利要求1-12任意一项所述的方法，或者实现权利要求13-15任意一项所述的方法。
  17. 一种非易失性计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现权利要求1-12中任意一项所述的方法，或者，实现权利要求13-15任意一项所述的方法。
PCT/CN2021/131204 2020-11-20 2021-11-17 图像处理方法及其设备 WO2022105793A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP21893940.3A EP4231628A4 (en) 2020-11-20 2021-11-17 IMAGE PROCESSING METHOD AND DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011314604.5 2020-11-20
CN202011314604.5A CN114520871B (zh) 2020-11-20 2020-11-20 图像处理方法及其设备

Publications (1)

Publication Number Publication Date
WO2022105793A1 true WO2022105793A1 (zh) 2022-05-27

Family

ID=81595503

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/131204 WO2022105793A1 (zh) 2020-11-20 2021-11-17 图像处理方法及其设备

Country Status (3)

Country Link
EP (1) EP4231628A4 (zh)
CN (1) CN114520871B (zh)
WO (1) WO2022105793A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116017128A (zh) * 2022-12-02 2023-04-25 北京拙河科技有限公司 一种边缘相机辅助图像构建方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106454149A (zh) * 2016-11-29 2017-02-22 广东欧珀移动通信有限公司 图像拍摄方法、装置及终端设备
CN107005675A (zh) * 2014-09-05 2017-08-01 富士胶片株式会社 动态图像编辑装置、动态图像编辑方法及动态图像编辑程序
CN107105315A (zh) * 2017-05-11 2017-08-29 广州华多网络科技有限公司 直播方法、主播客户端的直播方法、主播客户端及设备
KR101887380B1 (ko) * 2017-03-30 2018-08-10 에스피테크놀러지 주식회사 복수 개의 카메라를 이용한 촬영 이미지의 전송 및 처리를 위한 장치 및 방법
CN109803867A (zh) * 2016-10-11 2019-05-24 三星电子株式会社 用于将视线确保图像提供给车辆的方法及用于其的电子设备和计算机可读记录介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012004739A (ja) * 2010-06-15 2012-01-05 Sony Corp 情報処理装置、情報処理方法、及びプログラム
US9117483B2 (en) * 2011-06-03 2015-08-25 Michael Edward Zaletel Method and apparatus for dynamically recording, editing and combining multiple live video clips and still photographs into a finished composition
CN103078924A (zh) * 2012-12-28 2013-05-01 华为技术有限公司 视野共享方法及设备
EP2887352A1 (en) * 2013-12-19 2015-06-24 Nokia Corporation Video editing
CN106257960B (zh) * 2015-06-18 2021-03-30 中兴通讯股份有限公司 多设备协同操作的方法及设备
CN106412432A (zh) * 2016-09-30 2017-02-15 维沃移动通信有限公司 一种拍照方法及移动终端
CN106331529A (zh) * 2016-10-27 2017-01-11 广东小天才科技有限公司 一种图像拍摄方法及装置
CN107197144A (zh) * 2017-05-24 2017-09-22 珠海市魅族科技有限公司 拍摄控制方法及装置、计算机装置和可读存储介质
CN109240823B (zh) * 2018-08-10 2019-08-02 北京小蓦机器人技术有限公司 多端联动处理任务的方法、装置和可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107005675A (zh) * 2014-09-05 2017-08-01 富士胶片株式会社 动态图像编辑装置、动态图像编辑方法及动态图像编辑程序
CN109803867A (zh) * 2016-10-11 2019-05-24 三星电子株式会社 用于将视线确保图像提供给车辆的方法及用于其的电子设备和计算机可读记录介质
CN106454149A (zh) * 2016-11-29 2017-02-22 广东欧珀移动通信有限公司 图像拍摄方法、装置及终端设备
KR101887380B1 (ko) * 2017-03-30 2018-08-10 에스피테크놀러지 주식회사 복수 개의 카메라를 이용한 촬영 이미지의 전송 및 처리를 위한 장치 및 방법
CN107105315A (zh) * 2017-05-11 2017-08-29 广州华多网络科技有限公司 直播方法、主播客户端的直播方法、主播客户端及设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4231628A4

Also Published As

Publication number Publication date
CN114520871B (zh) 2023-09-01
EP4231628A1 (en) 2023-08-23
CN114520871A (zh) 2022-05-20
EP4231628A4 (en) 2024-03-27

Similar Documents

Publication Publication Date Title
WO2020143408A1 (zh) 显示方法及相关装置
WO2022100239A1 (zh) 设备协作方法、装置、系统、电子设备和存储介质
CN114520868B (zh) 视频处理方法、装置及存储介质
CN110554816A (zh) 一种界面生成方法及设备
CN115039378A (zh) 一种音频输出方法及终端设备
WO2022057742A1 (zh) 一种跨设备调用应用的方法及电子设备
EP4184905A1 (en) Device recognition method and related apparatus
WO2021135618A1 (zh) 界面显示方法及相关装置
CN114356195B (zh) 一种文件传输的方法及相关设备
WO2022062796A1 (zh) 一种网络控制方法、装置及电子设备
EP4407421A1 (en) Device collaboration method and related apparatus
CN111897465B (zh) 弹窗显示方法、装置、设备及存储介质
CN114371963A (zh) 一种故障检测方法及电子终端
US20220215050A1 (en) Picture Search Method and Device
WO2022134691A1 (zh) 一种终端设备中啸叫处理方法及装置、终端
WO2022095712A1 (zh) 数据分享方法、装置、系统及电子设备
WO2022105793A1 (zh) 图像处理方法及其设备
CN114466131B (zh) 一种跨设备的拍摄方法及相关设备
WO2022105716A1 (zh) 基于分布式控制的相机控制方法及终端设备
WO2022194005A1 (zh) 一种跨设备同步显示的控制方法及系统
WO2022121751A1 (zh) 相机控制方法、装置和存储介质
CN114513479B (zh) 消息收发方法、装置、终端、服务器及存储介质
WO2023169237A1 (zh) 一种截屏方法、电子设备及系统
WO2022166614A1 (zh) 针对控件操作的执行方法、装置、存储介质和控件
WO2023280077A1 (zh) 图像校正方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21893940

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021893940

Country of ref document: EP

Effective date: 20230518

NENP Non-entry into the national phase

Ref country code: DE