WO2022042587A1 - 用于移动终端的图像处理方法及装置、存储介质、终端 - Google Patents

用于移动终端的图像处理方法及装置、存储介质、终端 Download PDF

Info

Publication number
WO2022042587A1
WO2022042587A1 PCT/CN2021/114485 CN2021114485W WO2022042587A1 WO 2022042587 A1 WO2022042587 A1 WO 2022042587A1 CN 2021114485 W CN2021114485 W CN 2021114485W WO 2022042587 A1 WO2022042587 A1 WO 2022042587A1
Authority
WO
WIPO (PCT)
Prior art keywords
processing
image
image processing
platforms
subtasks
Prior art date
Application number
PCT/CN2021/114485
Other languages
English (en)
French (fr)
Inventor
刘晓伟
Original Assignee
展讯通信(天津)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 展讯通信(天津)有限公司 filed Critical 展讯通信(天津)有限公司
Priority to US18/023,223 priority Critical patent/US20230325963A1/en
Publication of WO2022042587A1 publication Critical patent/WO2022042587A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/725—Cordless telephones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5017—Task decomposition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/509—Offload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/52—Details of telephonic subscriber devices including functional features of a camera
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present invention relates to the technical field of image algorithm processing, and in particular, to an image processing method and device for a mobile terminal, a storage medium and a terminal.
  • an existing mobile terminal algorithm acceleration solution for image processing usually adopts a specific algorithm acceleration solution.
  • the speed of the algorithm can be increased several times to dozens of times, it still cannot meet the increasing demand for image processing speed.
  • the processing time of some advanced and complex image processing algorithms is still long, resulting in poor experience of end products, and even cannot be applied to low-end products.
  • the technical problem solved by the present invention is how to further reduce the time-consuming of algorithm processing.
  • an embodiment of the present invention provides an image processing method for a mobile terminal, including: receiving an image captured by the mobile terminal, and generating a first image processing task for the image;
  • the minimum data processing unit of the first processing platform determines a first division granularity, and divides the first image processing task into a plurality of subtasks according to the first division granularity; and assigns the plurality of subtasks to the plurality of first divisions.
  • a processing platform receiving the processing results fed back by the first processing platforms and merging them to obtain image processing results.
  • the assigning the multiple subtasks to the multiple first processing platforms includes: allocating the multiple subtasks to the multiple first processing platforms according to the respective computing power and real-time load of the multiple first processing platforms.
  • the plurality of first processing platforms are described.
  • the task amount of subtasks allocated by at least a part of the plurality of first processing platforms is different from the task amount of subtasks allocated by other first processing platforms.
  • the multiple first processing platforms accelerate the processing of the assigned subtasks according to their respective running algorithms, and each first processing platform runs the same algorithm or multiple algorithms that can be implemented in parallel.
  • the determining the first division granularity according to the minimum data processing units of the plurality of first processing platforms includes: for the respective minimum data processing units of the plurality of first processing platforms, according to the least common multiple of the minimum data processing units.
  • the first partition granularity is determined.
  • the dividing the first image processing task into multiple subtasks according to the first division granularity includes: dividing the image according to the first division granularity to obtain multiple image regions, the multiple The image areas are in one-to-one correspondence with the plurality of subtasks.
  • the image regions corresponding to at least a part of the subtasks of the plurality of subtasks have image overlap during segmentation.
  • the image processing method further includes: generating a second image processing task according to the image processing result; The granularity divides the second image processing task into multiple sub-tasks; assigns the multiple sub-tasks to the multiple second processing platforms; receives the processing results fed back by the second processing platforms and fuses them to obtain the processed Image processing results.
  • the algorithm run by the second processing platform and the algorithm run by the first processing platform are implemented in series.
  • the first division granularity is different from the second division granularity.
  • the first processing platform and the second processing platform are at least selected from: a CPU processing platform, a GPU processing platform and a DSP processing platform.
  • an embodiment of the present invention further provides an image processing apparatus for a mobile terminal, including: a first receiving module, configured to receive an image captured by the mobile terminal, and generate a first image processing device for the image.
  • An image processing task ; a division module, configured to determine a first division granularity according to the minimum data processing units of a plurality of first processing platforms, and divide the first image processing task into a plurality of subtasks according to the first division granularity; an allocation module for allocating the plurality of subtasks to the plurality of first processing platforms; and a second receiving module for receiving processing results fed back by each of the first processing platforms and merging them to obtain image processing results.
  • an embodiment of the present invention further provides a storage medium on which a computer program is stored, and the computer program executes the steps of the above method when the computer program is run by a processor.
  • an embodiment of the present invention further provides a terminal, including a memory and a processor, where the memory stores a computer program that can run on the processor, and when the processor runs the computer program, Perform the steps of the above method.
  • An embodiment of the present invention provides an image processing method for a mobile terminal, including: receiving an image captured by the mobile terminal, and generating a first image processing task for the image;
  • the data processing unit determines a first division granularity, and divides the first image processing task into multiple subtasks according to the first division granularity; assigns the multiple subtasks to the multiple first processing platforms; receives each The processing results fed back by the first processing platform are merged to obtain image processing results.
  • this embodiment can realize heterogeneous fusion acceleration of multiple hardware, integrate existing resources and algorithm acceleration solutions on mobile terminals, and further reduce algorithm processing time.
  • the image processing task is allocated to multiple processing platforms, so that under the existing computing hardware resources of the mobile terminal, combined with the actual hardware load, the algorithm processing speed is further accelerated, and the response speed and experience of the mobile terminal are improved. , which improves the adaptability of advanced and complex algorithms.
  • the image processing method further includes: generating a second image processing task according to the image processing result; determining a second division granularity according to the smallest data processing unit of a plurality of second processing platforms, and dividing the The second image processing task is divided into multiple sub-tasks; the multiple sub-tasks are allocated to the multiple second processing platforms; the processing results fed back by the second processing platforms are received and merged to obtain the processed image processing result.
  • the first re-splitting can be performed according to the algorithm used for processing the image.
  • the first image processing task is generated, according to the first processing platform that implements the algorithm first. Determine the partition granularity to partition image processing tasks. Further, after the first processing platform completes the processing of the allocated images, the processing results of the first processing platforms are fused, and then the division granularity is re-determined according to the second processing platform that implements the algorithm after running, and tasks are reassigned to the second processing platform. platform to get the final image processing result.
  • FIG. 1 is a flowchart of an image processing method for a mobile terminal according to an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of an image processing apparatus for a mobile terminal according to an embodiment of the present invention.
  • the first type is the algorithm processing acceleration based on hardware application specific integrated circuit (ASIC), such as image signal processing (ISP) chip.
  • ASIC application specific integrated circuit
  • ISP image signal processing
  • the ISP chip can process image algorithms such as black level correction, automatic exposure, automatic white balance, and noise reduction for the image output by the image sensor.
  • the second category is the software implementation of algorithm processing acceleration based on general-purpose computing hardware.
  • General computing hardware includes but is not limited to central processing unit (Central Processing Unit, referred to as CPU), graphics processing unit (Graphics Processing Unit, referred to as GPU), digital signal processing (Digital Signal Processing, referred to as DSP), etc., generally have parallel acceleration technology .
  • Neon registers can be thought of as a set of vector elements of the same data type, and Neon instructions can operate on multiple elements in a vector at the same time.
  • SIMT Single Instruction Multiple Threads
  • SIMT Single Instruction Multiple Data
  • SIMT Single Instruction Multiple Threads
  • SIMT Single Instruction Multiple Data
  • DSP uses DSP to accelerate algorithm processing.
  • the advanced variable instruction length, super-long instruction word, and processor architecture that supports hardware multi-threading mechanism adopted by a specific DSP platform can be used to accelerate algorithm processing.
  • the algorithm acceleration scheme adopted by the mobile terminal to process images usually only adopts one of the above algorithm processing acceleration schemes, which cannot meet the increasing demands. Even if some advanced and complex image processing algorithms are accelerated by the above scheme, it still takes a long time. This time-consuming problem will lead to poor user experience of the mobile terminal, and even cannot be applied to low-end products.
  • an embodiment of the present invention provides an image processing method for a mobile terminal, including: receiving an image captured by the mobile terminal, and generating a first image processing task for the image;
  • the minimum data processing unit of the first processing platform determines a first division granularity, and divides the first image processing task into a plurality of subtasks according to the first division granularity; and assigns the plurality of subtasks to the plurality of first divisions.
  • a processing platform receiving the processing results fed back by the first processing platforms and merging them to obtain image processing results.
  • This embodiment can realize heterogeneous fusion acceleration of multiple hardwares, integrate existing resources and algorithm acceleration solutions on the mobile terminal, and further reduce the time-consuming of algorithm processing.
  • the image processing task is allocated to multiple processing platforms, so that under the existing computing hardware resources of the mobile terminal, combined with the actual hardware load, the algorithm processing speed is further accelerated, and the response speed and experience of the mobile terminal are improved. , which improves the adaptability of advanced and complex algorithms.
  • FIG. 1 is a flowchart of an image processing method for a mobile terminal according to an embodiment of the present invention.
  • the mobile terminal described in this embodiment may be configured with multiple processing platforms, including but not limited to a CPU processing platform, a GPU processing platform, and a DSP processing platform.
  • the computing resources of multiple existing processing platforms of the mobile terminal can be fully utilized, the time-consuming of algorithm processing can be further reduced by the heterogeneous fusion acceleration of multiple hardware, the image processing speed of the mobile terminal can be improved, and the user experience can be optimized.
  • This embodiment can be applied to low-end mobile terminals, so that the low-end mobile terminals can run advanced and complex algorithms to process images by means of heterogeneous fusion acceleration.
  • the image processing method for a mobile terminal provided by the following steps S101 to S104 may be executed by a chip with an image processing function in the user equipment, or by a baseband chip, a GPU chip, CPU chips, etc. execute.
  • the user equipment may include the mobile terminal.
  • the image processing method for a mobile terminal described in this embodiment may include the following steps:
  • Step S101 receiving an image captured by the mobile terminal, and generating a first image processing task for the image
  • Step S102 determining a first division granularity according to the minimum data processing units of a plurality of first processing platforms, and dividing the first image processing task into a plurality of subtasks according to the first division granularity;
  • Step S103 assigning the multiple subtasks to the multiple first processing platforms
  • Step S104 receiving the processing results fed back by the first processing platforms and merging them to obtain the image processing results.
  • the image may be acquired by an image sensor configured on the mobile terminal.
  • the first image processing task may be used to perform specific algorithm processing on the image. Such as white balance, black level correction, automatic exposure and other algorithm processing.
  • Image computing tasks generally have pixel similarity and are suitable for parallel operations. Therefore, in order to perform fusion calculation in this embodiment, the calculation task is firstly divided into several subtasks.
  • an appropriate division granularity may be selected according to actual algorithm conditions.
  • the step S102 may include the step of: for the respective minimum data processing units of the plurality of first processing platforms, determining the first division granularity according to the least common multiple of the respective minimum data processing units.
  • an integer multiple of the least common multiple may be used as the first division granularity.
  • the step S102 may further include the step of: dividing the image according to the first division granularity to obtain multiple image regions, where the multiple image regions correspond to the multiple subtasks one-to-one.
  • the division by line can be divided according to the granularity of 8 lines of image behavior
  • the division by block can be divided according to the pixel block of 32 ⁇ 32 size.
  • the image regions corresponding to at least a part of the subtasks of the plurality of subtasks have image overlap during segmentation. That is, when the image is divided according to the first division granularity, there may be overlapping areas between adjacent image areas.
  • step S104 when the image processing results of the first processing platforms are fused in step S104, special processing may be performed on the image processing results of the overlapping regions by different first processing platforms to ensure the fusion accuracy. For example, perform memory consistency processing, etc.
  • the developer may select multiple first processing platforms to which the first image processing task needs to be assigned, and perform software development and algorithm performance simulation for the selected processing platform, and obtain performance data.
  • the performance data includes the computing power of the processing platform.
  • the step S103 may include the step of: allocating the multiple subtasks to the multiple first processing platforms according to the respective computing power and real-time load of the multiple first processing platforms.
  • the subtasks are divided reasonably, mainly considering the time balance, and minimizing the waiting time during synchronization.
  • the task amount of the subtasks allocated by at least a part of the first processing platforms in the plurality of first processing platforms is different from the task amount of the subtasks allocated by other first processing platforms.
  • the simulation determines that the computing power of the CPU processing platform is 1.0 and the computing power of the DSP processing platform is 2.0, the current load of the two first processing platforms is similar. Then, when step S103 is performed, the first image processing task can be divided into 1/3 to the CPU processing platform and 2/3 to the DSP processing platform, and the two tasks can be allocated almost simultaneously.
  • the plurality of first processing platforms may accelerate the processing of the assigned subtasks according to the respective running algorithms, and each first processing platform runs the same algorithm.
  • both the GPU processing platform and the DSP processing platform accelerate the processing of the black level correction algorithm for the allocated image area.
  • each first processing platform runs a plurality of algorithms that can be implemented in parallel.
  • the CPU processing platform can be assigned to perform noise reduction algorithm processing on the first part of the image area Acceleration, assigning the GPU processing platform and the DSP processing platform to respectively perform high dynamic processing acceleration on a part of the second part of the image area.
  • step S104 when each subtask is fused, the image processing results of each subtask may be fused according to the adjacent relationship between the image regions corresponding to each subtask.
  • the image processing method further includes: generating a second image processing task according to the image processing result; determining a second division according to the minimum data processing units of multiple second processing platforms granularity, and divide the second image processing task into multiple subtasks according to the second division granularity; assign the multiple subtasks to the multiple second processing platforms; receive processing feedback from each second processing platform The results are fused to obtain the processed image processing results.
  • the first re-splitting can be performed according to the algorithm used for processing the image.
  • the first image processing task is generated, according to the first processing platform that implements the algorithm first. Determine the partition granularity to partition image processing tasks. Further, after the first processing platform completes the processing of the allocated images, the processing results of the first processing platforms are fused, and then the division granularity is re-determined according to the second processing platform that implements the algorithm after running, and tasks are reassigned to the second processing platform. platform to get the final image processing result.
  • the algorithm run by the second processing platform is implemented in series with the algorithm run by the first processing platform.
  • step S103 the CPU processing platform and the DSP processing platform can be assigned to respectively perform noise reduction algorithm processing on a part of the image to accelerate the reduction of the image. After the noise processing results are fused, they are dispatched to the GPU processing platform and the DSP processing platform to accelerate the automatic exposure processing of a part of the image respectively.
  • the assignment of image processing tasks in noise reduction processing is carried out according to the computing power and real-time load of CPU processing platform and DSP processing platform, and the allocation of image processing tasks in automatic exposure processing is carried out according to GPU processing platform and DSP processing platform. computing power and real-time load.
  • the first division granularity is different from the second division granularity.
  • the partition granularity is adjustable during the entire image processing.
  • this embodiment can realize heterogeneous fusion acceleration of multiple hardware, and integrate the existing resources and algorithm acceleration solutions on the mobile terminal, and further reduce the time consumption of algorithm processing.
  • the image processing task is allocated to multiple processing platforms, so that under the existing computing hardware resources of the mobile terminal, combined with the actual hardware load, the algorithm processing speed is further accelerated, and the response speed and experience of the mobile terminal are improved. , which improves the adaptability of advanced and complex algorithms.
  • this embodiment is adaptive when splitting subtasks, does not limit the execution ratio of the processing platform, but distributes in combination with the actual processing platform load and algorithm performance, so as to achieve the best acceleration effect.
  • FIG. 2 is a schematic structural diagram of an image processing apparatus for a mobile terminal according to an embodiment of the present invention.
  • the image processing apparatus 2 for a mobile terminal described in this embodiment can be used to implement the method and technical solution described in the embodiment shown in FIG. 1 above.
  • the image processing apparatus 2 for a mobile terminal may include: a first receiving module 21, configured to receive an image captured by the mobile terminal, and generate a first receiving module 21 for the image.
  • An image processing task ; the division module 22 is configured to determine a first division granularity according to the minimum data processing units of a plurality of first processing platforms, and divide the first image processing task into a plurality of subtasks according to the first division granularity
  • the distribution module 23 is used to distribute the multiple subtasks to the multiple first processing platforms; the second receiving module 24 is used to receive the processing results fed back by the first processing platforms and fuse them to obtain the image processing results .
  • the above-mentioned image processing apparatus 2 for a mobile terminal may correspond to a chip with an image processing function in a mobile terminal, or a chip with a data processing function, such as a system-on-a-chip (System-On-a-Chip) , referred to as SOC), GPU chip, CPU chip, etc.; or corresponding to a chip module including a chip with an image processing function in a mobile terminal; or corresponding to a chip module with a data processing function chip, or corresponding to a mobile terminal.
  • a chip with an image processing function in a mobile terminal or a chip with a data processing function, such as a system-on-a-chip (System-On-a-Chip) , referred to as SOC), GPU chip, CPU chip, etc.
  • SOC system-on-a-chip
  • GPU chip GPU chip
  • CPU chip etc.
  • a chip module including a chip with an image processing function in a mobile terminal or corresponding to a chip module with a data processing function chip,
  • each module/unit included in each device and product described in the above embodiments it may be a software module/unit, a hardware module/unit, or a part of a software module/unit, a part of which is a software module/unit. is a hardware module/unit.
  • each module/unit included therein may be implemented by hardware such as circuits, or at least some of the modules/units may be implemented by a software program.
  • Running on the processor integrated inside the chip the remaining (if any) part of the modules/units can be implemented by hardware such as circuits; for each device and product applied to or integrated in the chip module, the modules/units contained therein can be They are all implemented by hardware such as circuits, and different modules/units can be located in the same component of the chip module (such as chips, circuit modules, etc.) or in different components, or at least some of the modules/units can be implemented by software programs.
  • the software program runs on the processor integrated inside the chip module, and the remaining (if any) part of the modules/units can be implemented by hardware such as circuits; for each device and product applied to or integrated in the terminal, each module contained in it
  • the units/units may all be implemented in hardware such as circuits, and different modules/units may be located in the same component (eg, chip, circuit module, etc.) or in different components in the terminal, or at least some of the modules/units may be implemented in the form of software programs Realization, the software program runs on the processor integrated inside the terminal, and the remaining (if any) part of the modules/units can be implemented in hardware such as circuits.
  • an embodiment of the present invention further discloses a storage medium, on which a computer program is stored, and when the computer program is run by a processor, the method and technical solution described in the embodiment shown in FIG. 1 is executed.
  • the storage medium may include a computer-readable storage medium such as a non-volatile memory or a non-transitory memory.
  • the storage medium may include ROM, RAM, magnetic or optical disks, and the like.
  • an embodiment of the present invention further discloses a terminal, including a memory and a processor, the memory stores a computer program that can be run on the processor, and the processor executes the above diagram when running the computer program.
  • the terminal may be a mobile terminal such as a mobile phone and an IPAD.
  • the mobile terminal may be configured with an image sensor.
  • the terminal may include the image processing apparatus 2 shown in FIG. 2 for a mobile terminal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Image Processing (AREA)

Abstract

一种用于移动终端的图像处理方法及装置、存储介质、终端，所述方法包括：接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；将所述多个子任务分配至所述多个第一处理平台；接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。通过本发明方案能够实现多硬件的异构融合加速，利于进一步减少算法处理耗时。

Description

用于移动终端的图像处理方法及装置、存储介质、终端
本申请要求2020年8月26日提交中国专利局、申请号为202010873807.1、发明名称为“用于移动终端的图像处理方法及装置、存储介质、终端”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及图像算法处理技术领域，具体地涉及一种用于移动终端的图像处理方法及装置、存储介质、终端。
背景技术
随着移动终端市场和图像技术的快速发展，人们对移动终端上拍照效果的要求越来越高，但往往更好的拍照效果意味着更高的算法复杂度和更长的处理耗时。因此，算法性能成为算法产品化的瓶颈。各终端厂商或芯片厂商都提供了基于各自产品特点的软、硬件方案来加速算法处理。但是，在有限的条件或资源下，单一的算法加速方案可能仍然无法满足移动终端图像处理的性能需求。
具体而言，现有移动终端针对图像处理的算法加速方案通常采用某一种特定的算法加速方案。虽然可以将算法速度提升几倍至几十倍，但是仍然无法满足日益增长的图像处理速度需求。尤其一些先进、复杂的图像处理算法的处理耗时仍然较长，导致终端产品的体验变差，甚至无法在低端产品上应用。
发明内容
本发明解决的技术问题是如何进一步减少算法处理耗时。
为解决上述技术问题，本发明实施例提供一种用于移动终端的图像处理方法，包括：接收所述移动终端拍摄得到的图像，并生成针对 所述图像的第一图像处理任务；根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；将所述多个子任务分配至所述多个第一处理平台；接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
可选的，所述将所述多个子任务分配至所述多个第一处理平台包括：根据所述多个第一处理平台各自的算力和实时负载，将所述多个子任务分配至所述多个第一处理平台。
可选的，所述多个第一处理平台中至少一部分第一处理平台分配的子任务的任务量，不同于其他第一处理平台分配的子任务的任务量。
可选的，所述多个第一处理平台按各自运行的算法加速处理分配的子任务，且各第一处理平台运行的是相同算法或可并行实施的多个算法。
可选的，所述根据多个第一处理平台的最小数据处理单元确定第一划分粒度包括：对于所述多个第一处理平台各自的最小数据处理单元，根据各最小数据处理单元的最小公倍数确定所述第一划分粒度。
可选的，所述按所述第一划分粒度将所述第一图像处理任务划分为多个子任务包括：按所述第一划分粒度分割所述图像以得到多个图像区域，所述多个图像区域与所述多个子任务一一对应。
可选的，所述多个子任务中至少一部分子任务对应的图像区域在分割时存在图像重叠。
可选的，所述图像处理方法还包括：根据所述图像处理结果生成第二图像处理任务；根据多个第二处理平台的最小数据处理单元确定第二划分粒度，并按所述第二划分粒度将所述第二图像处理任务划分为多个子任务；将所述多个子任务分配至所述多个第二处理平台；接收各第二处理平台反馈的处理结果并融合，以得到处理后的图像处理 结果。
可选的，所述第二处理平台运行的算法与所述第一处理平台运行的算法是串行实施的。
可选的，所述第一划分粒度不同于所述第二划分粒度。
可选的，所述第一处理平台和第二处理平台至少选自：CPU处理平台、GPU处理平台以及DSP处理平台。
为解决上述技术问题，本发明实施例还提供一种用于移动终端的图像处理装置，包括：第一接收模块，用于接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；划分模块，用于根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；分配模块，用于将所述多个子任务分配至所述多个第一处理平台；第二接收模块，用于接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
为解决上述技术问题，本发明实施例还提供一种存储介质，其上存储有计算机程序，所述计算机程序被处理器运行时执行上述方法的步骤。
为解决上述技术问题，本发明实施例还提供一种终端，包括存储器和处理器，所述存储器上存储有能够在所述处理器上运行的计算机程序，所述处理器运行所述计算机程序时执行上述方法的步骤。
与现有技术相比，本发明实施例的技术方案具有以下有益效果：
本发明实施例提供一种用于移动终端的图像处理方法，包括：接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；将所述多个子任务分配至所述多个第一处理平台；接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
较之现有移动终端基于单一算法加速处理图像的技术方案，本实施方案能够实现多硬件的异构融合加速，融合移动终端上已有的资源和算法加速方案，进一步减少算法处理耗时。具体而言，将图像处理任务分摊到多个处理平台，从而在移动终端已有计算硬件资源下，结合实际的硬件负载情况，进一步对算法处理速度进行加速，提升了移动终端的响应速度和体验，提升了先进复杂算法的适应性。
进一步，所述图像处理方法还包括：根据所述图像处理结果生成第二图像处理任务；根据多个第二处理平台的最小数据处理单元确定第二划分粒度，并按所述第二划分粒度将所述第二图像处理任务划分为多个子任务；将所述多个子任务分配至所述多个第二处理平台；接收各第二处理平台反馈的处理结果并融合，以得到处理后的图像处理结果。
由此，对移动终端采集到的图像进行处理期间，可以根据处理图像所用算法进行第一重拆分，对于需要先实施的算法生成第一图像处理任务，按运行先实施算法的第一处理平台确定划分粒度划分图像处理任务。进一步，在第一处理平台对分配到的图像完成处理后，融合各第一处理平台的处理结果，然后再按运行后实施算法的第二处理平台重新确定划分粒度，重新分派任务到第二处理平台，得到最后的图像处理结果。
附图说明
图1是本发明实施例一种用于移动终端的图像处理方法的流程图；
图2是本发明实施例一种用于移动终端的图像处理装置的结构示意图。
具体实施方式
如背景技术所言，在有限的条件或资源下，单一的算法加速方案可能仍然无法满足移动终端图像处理的性能需求。
具体而言，现有移动终端通常采用的算法加速方案主要有两类。
第一类是基于硬件特殊应用集成电路(Application Specific Integrated Circuit，简称ASIC)实现的算法处理加速，如图像信号处理(Image Signal Processing，简称ISP)芯片。ISP芯片可以对图像传感器输出的图像进行黑电平矫正、自动曝光、自动白平衡、降噪等图像算法的处理。
但是，基于硬件实现的算法处理加速方案不具有普适性和扩展性，无法对其他算法处理加速。
第二类是基于通用计算硬件的软件实现算法处理加速。通用计算硬件包括但不限于中央处理器(Central Processing Unit，简称CPU)、图形处理器(Graphics Processing Unit，简称GPU)、数字信号处理(Digital Signal Processing，简称DSP)等，一般都具有并行加速技术。
利用CPU多核、并行指令进行算法处理加速。比如，多线程、ARM架构处理器扩展结构(Arm Neon)技术。Neon寄存器可以认为是一组相同数据类型的向量元素，Neon指令可以同时操作向量中的多个元素。
利用GPU进行算法处理加速。其中，GPU采用的是单指令多线程架构(Single Instruction Multiple Threads，简称SIMT)。有别于单指令多数据流(Single Instruction Multiple Data，简称SIMD)，SIMT的好处就是无需开发者费力把数据凑成合适的向量长度，并且SIMT允许每个线程有不同的分支。纯粹使用SIMD不能并行的执行有条件跳转的函数，很显然条件跳转会根据输入数据不同在不同的线程中有不同表现，这个只有利用SIMT才能做到。
利用DSP进行算法处理加速。比如，可以利用特定DSP平台所采用的高级的可变指令长度、超长指令字、支持硬件多线程机制的处理器架构进行算法处理加速。
但是，移动终端在处理图像是采用的算法加速方案通常只会采用上述算法处理加速方案中的一种，无法满足日益增长的需求。一些先进复杂的图像处理算法即使采用上述方案加速，耗时仍然很长。这一耗时问题会导致移动终端的使用体验变差，甚至无法在低端产品上应用。
为解决上述技术问题，本发明实施例提供一种用于移动终端的图像处理方法，包括：接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；将所述多个子任务分配至所述多个第一处理平台；接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
本实施方案能够实现多硬件的异构融合加速，融合移动终端上已有的资源和算法加速方案，进一步减少算法处理耗时。具体而言，将图像处理任务分摊到多个处理平台，从而在移动终端已有计算硬件资源下，结合实际的硬件负载情况，进一步对算法处理速度进行加速，提升了移动终端的响应速度和体验，提升了先进复杂算法的适应性。
为使本发明的上述目的、特征和有益效果能够更为明显易懂，下面结合附图对本发明的具体实施例做详细的说明。
图1是本发明实施例一种用于移动终端的图像处理方法的流程图。
本实施方案所述移动终端可以配置有多个处理平台，包括但不限于CPU处理平台、GPU处理平台、DSP处理平台。
采用本实施方案，可以充分利用移动终端已有的多个处理平台的计算资源，通过多硬件的异构融合加速进一步减少算法处理耗时，提高移动终端的图像处理速度，优化用户体验。
本实施方案可以应用于低端移动终端，以通过异构融合加速的方 式使得低端移动终端能够运行先进复杂算法处理图像。
在具体实施中，下述步骤S101～步骤S104所提供的用于移动终端的图像处理方法可以由用户设备中的具有图像处理功能的芯片执行，也可以由用户设备中的基带芯片、GPU芯片、CPU芯片等执行。例如，所述用户设备可以包括所述移动终端。
具体地，参考图1，本实施例所述用于移动终端的图像处理方法可以包括如下步骤：
步骤S101，接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；
步骤S102，根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；
步骤S103，将所述多个子任务分配至所述多个第一处理平台；
步骤S104，接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
在一个具体实施中，所述图像可以是所述移动终端配置的图像传感器采集得到的。
进一步，所述第一图像处理任务可以用于对所述图像进行特定算法处理。如白平衡、黑电平矫正、自动曝光等算法处理。
图像计算任务一般都具有像素相似性，适合并行运算。因此，本实施方案为进行融合计算首先要将计算任务拆分成若干个子任务。
在一个具体实施中，可以根据实际算法情况选择合适的划分粒度。具体地，所述步骤S102可以包括步骤：对于所述多个第一处理平台各自的最小数据处理单元，根据各最小数据处理单元的最小公倍数确定所述第一划分粒度。
例如，可以将最小公倍数的整倍数作为所述第一划分粒度。
进一步，所述步骤S102还可以包括步骤：按所述第一划分粒度分割所述图像以得到多个图像区域，所述多个图像区域与所述多个子任务一一对应。
例如，可以按行划分或者按块划分的方法进行拆分。比如，按行划分可按8行图像行为粒度进行划分，按块划分可按32×32大小的像素块进行划分。
进一步，按第一划分粒度分割图像时，相邻图像区域之间可以是没有重叠的。
在一个变化例中，所述多个子任务中至少一部分子任务对应的图像区域在分割时存在图像重叠。也即，按第一划分粒度分割图像时，相邻图像区域之间可以是存在重叠区域的。
相应的，在步骤S104中融合各第一处理平台的图像处理结果时，可以对不同第一处理平台针对重叠区域的图像处理结果进行特殊处理，以确保融合准确性。例如，进行内存一致性处理等。
在一个具体实施中，需要分派第一图像处理任务的多个第一处理平台可以由开发者选择，并针对选用的处理平台进行软件开发和算法性能仿真，并获取性能数据。
例如，所述性能数据包括处理平台的算力。
在一个具体实施中，所述步骤S103可以包括步骤：根据所述多个第一处理平台各自的算力和实时负载，将所述多个子任务分配至所述多个第一处理平台。
具体地，在任务派发时，结合前期仿真得到的各第一处理平台的算法性能和负载情况，合理的划分子任务，主要考虑时间上的均衡，尽量减少同步时的等待时间。
相应的，所述多个第一处理平台中至少一部分第一处理平台分配 的子任务的任务量，不同于其他第一处理平台分配的子任务的任务量。
以多个第一处理平台为CPU处理平台和DSP处理平台为例，假设仿真确定CPU处理平台的算力为1.0，DSP处理平台的算力为2.0，当前两个第一处理平台的负载差不多。则在执行步骤S103时，所述第一图像处理任务可以分1/3给CPU处理平台，分2/3给DSP处理平台，这样的任务分派两者几乎可以同时完成。
在一个具体实施中，所述多个第一处理平台可以按各自运行的算法加速处理分配的子任务，且各第一处理平台运行的是相同算法。
例如，GPU处理平台和DSP处理平台均对分配到的图像区域进行黑电平矫正算法处理加速。
在一个变化例中，各第一处理平台运行的是可并行实施的多个算法。
假设步骤S101获取的图像仅第一部分图像区域需要进行降噪处理，仅第二部分图像区域需要进行高动态处理，则在步骤S103中，可以分派CPU处理平台对第一部分图像区域进行降噪算法处理加速，分派GPU处理平台和DSP处理平台分别对第二部分图像区域的一部分进行高动态处理加速。
在一个具体实施中，在步骤S104中，融合各子任务时，可以按照各子任务对应图像区域之间的相邻关系融合各子任务的图像处理结果。
在一个具体实施中，在所述步骤S104后，所述图像处理方法还包括：根据所述图像处理结果生成第二图像处理任务；根据多个第二处理平台的最小数据处理单元确定第二划分粒度，并按所述第二划分粒度将所述第二图像处理任务划分为多个子任务；将所述多个子任务分配至所述多个第二处理平台；接收各第二处理平台反馈的处理结果并融合，以得到处理后的图像处理结果。
由此，对移动终端采集到的图像进行处理期间，可以根据处理图像所用算法进行第一重拆分，对于需要先实施的算法生成第一图像处理任务，按运行先实施算法的第一处理平台确定划分粒度划分图像处理任务。进一步，在第一处理平台对分配到的图像完成处理后，融合各第一处理平台的处理结果，然后再按运行后实施算法的第二处理平台重新确定划分粒度，重新分派任务到第二处理平台，得到最后的图像处理结果。
进一步，所述第二处理平台运行的算法与所述第一处理平台运行的算法是串行实施的。
例如，假设步骤S101获取的图像需要先进行降噪处理，再进行自动曝光处理，则在步骤S103中，可以分派CPU处理平台和DSP处理平台分别对图像的一部分进行降噪算法处理加速，将降噪处理结果融合后再分派给GPU处理平台和DSP处理平台分别对图像的一部分进行自动曝光处理加速。
其中，降噪处理时对图像处理任务的分派是按照CPU处理平台和DSP处理平台的算力和实时负载进行的，自动曝光处理时对图像处理任务的分派则是按照GPU处理平台和DSP处理平台的算力和实时负载进行的。
也就是说，所述第一划分粒度不同于所述第二划分粒度。换言之，划分粒度在整个图像处理期间是可调的。
由上，本实施方案能够实现多硬件的异构融合加速，融合移动终端上已有的资源和算法加速方案，进一步减少算法处理耗时。具体而言，将图像处理任务分摊到多个处理平台，从而在移动终端已有计算硬件资源下，结合实际的硬件负载情况，进一步对算法处理速度进行加速，提升了移动终端的响应速度和体验，提升了先进复杂算法的适应性。
进一步，本实施方案在拆分子任务时具有自适应性，不限制处理 平台的执行比例，而是结合实际的处理平台负载和算法性能进行派发，以达到最佳的加速效果。
图2是本发明实施例一种用于移动终端的图像处理装置的结构示意图。本领域技术人员理解，本实施例所述用于移动终端的图像处理装置2可以用于实施上述图1所述实施例中所述的方法技术方案。
具体地，参考图2，本实施例所述用于移动终端的图像处理装置2可以包括：第一接收模块21，用于接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；划分模块22，用于根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；分配模块23，用于将所述多个子任务分配至所述多个第一处理平台；第二接收模块24，用于接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
关于所述用于移动终端的图像处理装置2的工作原理、工作方式的更多内容，可以参照上述图1中的相关描述，这里不再赘述。
在具体实施中，上述的用于移动终端的图像处理装置2可以对应于移动终端中具有图像处理功能的芯片，或者对应于具有数据处理功能的芯片，例如片上系统(System-On-a-Chip，简称SOC)、GPU芯片、CPU芯片等；或者对应于移动终端中包括具有图像处理功能芯片的芯片模组；或者对应于具有数据处理功能芯片的芯片模组，或者对应于移动终端。
在具体实施中，关于上述实施例中描述的各个装置、产品包含的各个模块/单元，其可以是软件模块/单元，也可以是硬件模块/单元，或者也可以部分是软件模块/单元，部分是硬件模块/单元。
例如，对于应用于或集成于芯片的各个装置、产品，其包含的各个模块/单元可以都采用电路等硬件的方式实现，或者，至少部分模块/单元可以采用软件程序的方式实现，该软件程序运行于芯片内部 集成的处理器，剩余的(如果有)部分模块/单元可以采用电路等硬件方式实现；对于应用于或集成于芯片模组的各个装置、产品，其包含的各个模块/单元可以都采用电路等硬件的方式实现，不同的模块/单元可以位于芯片模组的同一组件(例如芯片、电路模块等)或者不同组件中，或者，至少部分模块/单元可以采用软件程序的方式实现，该软件程序运行于芯片模组内部集成的处理器，剩余的(如果有)部分模块/单元可以采用电路等硬件方式实现；对于应用于或集成于终端的各个装置、产品，其包含的各个模块/单元可以都采用电路等硬件的方式实现，不同的模块/单元可以位于终端内同一组件(例如，芯片、电路模块等)或者不同组件中，或者，至少部分模块/单元可以采用软件程序的方式实现，该软件程序运行于终端内部集成的处理器，剩余的(如果有)部分模块/单元可以采用电路等硬件方式实现。
进一步地，本发明实施例还公开一种存储介质，其上存储有计算机程序，所述计算机程序被处理器运行时执行上述图1所示实施例中所述的方法技术方案。优选地，所述存储介质可以包括诸如非挥发性(non-volatile)存储器或者非瞬态(non-transitory)存储器等计算机可读存储介质。所述存储介质可以包括ROM、RAM、磁盘或光盘等。
进一步地，本发明实施例还公开一种终端，包括存储器和处理器，所述存储器上存储有能够在所述处理器上运行的计算机程序，所述处理器运行所述计算机程序时执行上述图1所示实施例中所述的方法技术方案。例如，所述终端可以为手机、IPAD等移动终端。所述移动终端可以配置有图像传感器。或者，所述终端可以包括上述图2所示的用于移动终端的图像处理装置2。
虽然本发明披露如上，但本发明并非限定于此。任何本领域技术人员，在不脱离本发明的精神和范围内，均可作各种更动与修改，因此本发明的保护范围应当以权利要求所限定的范围为准。

Claims (14)

  1. 一种用于移动终端的图像处理方法，其特征在于，包括：
    接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；
    根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；
    将所述多个子任务分配至所述多个第一处理平台；
    接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
  2. 根据权利要求1所述的图像处理方法，其特征在于，所述将所述多个子任务分配至所述多个第一处理平台包括：
    根据所述多个第一处理平台各自的算力和实时负载，将所述多个子任务分配至所述多个第一处理平台。
  3. 根据权利要求2所述的图像处理方法，其特征在于，所述多个第一处理平台中至少一部分第一处理平台分配的子任务的任务量，不同于其他第一处理平台分配的子任务的任务量。
  4. 根据权利要求1所述的图像处理方法，其特征在于，所述多个第一处理平台按各自运行的算法加速处理分配的子任务，且各第一处理平台运行的是相同算法或可并行实施的多个算法。
  5. 根据权利要求1所述的图像处理方法，其特征在于，所述根据多个第一处理平台的最小数据处理单元确定第一划分粒度包括：
    对于所述多个第一处理平台各自的最小数据处理单元，根据各最小数据处理单元的最小公倍数确定所述第一划分粒度。
  6. 根据权利要求1所述的图像处理方法，其特征在于，所述按所述 第一划分粒度将所述第一图像处理任务划分为多个子任务包括：
    按所述第一划分粒度分割所述图像以得到多个图像区域，所述多个图像区域与所述多个子任务一一对应。
  7. 根据权利要求6所述的图像处理方法，其特征在于，所述多个子任务中至少一部分子任务对应的图像区域在分割时存在图像重叠。
  8. 根据权利要求1所述的图像处理方法，其特征在于，还包括：
    根据所述图像处理结果生成第二图像处理任务；
    根据多个第二处理平台的最小数据处理单元确定第二划分粒度，并按所述第二划分粒度将所述第二图像处理任务划分为多个子任务；
    将所述多个子任务分配至所述多个第二处理平台；
    接收各第二处理平台反馈的处理结果并融合，以得到处理后的图像处理结果。
  9. 根据权利要求8所述的图像处理方法，其特征在于，所述第二处理平台运行的算法与所述第一处理平台运行的算法是串行实施的。
  10. 根据权利要求8所述的图像处理方法，其特征在于，所述第一划分粒度不同于所述第二划分粒度。
  11. 根据权利要求1至10任一项所述的图像处理方法，其特征在于，所述第一处理平台和第二处理平台至少选自：CPU处理平台、GPU处理平台以及DSP处理平台。
  12. 一种用于移动终端的图像处理装置，其特征在于，包括：
    第一接收模块，用于接收所述移动终端拍摄得到的图像，并生成针对所述图像的第一图像处理任务；
    划分模块，用于根据多个第一处理平台的最小数据处理单元确定第一划分粒度，并按所述第一划分粒度将所述第一图像处理任务划分为多个子任务；
    分配模块，用于将所述多个子任务分配至所述多个第一处理平台；
    第二接收模块，用于接收各第一处理平台反馈的处理结果并融合，以得到图像处理结果。
  13. 一种存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器运行时执行权利要求1至11任一项所述方法的步骤。
  14. 一种终端，包括权利要求12所述的用于移动终端的图像处理装置，或者，包括存储器和处理器，所述存储器上存储有能够在所述处理器上运行的计算机程序，其特征在于，所述处理器运行所述计算机程序时执行权利要求1至11任一项所述方法的步骤。
PCT/CN2021/114485 2020-08-26 2021-08-25 用于移动终端的图像处理方法及装置、存储介质、终端 WO2022042587A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/023,223 US20230325963A1 (en) 2020-08-26 2021-08-25 Image processing method and apparatus for mobile terminal, and storage medium and terminal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010873807.1 2020-08-26
CN202010873807.1A CN111984417B (zh) 2020-08-26 2020-08-26 用于移动终端的图像处理方法及装置、存储介质、终端

Publications (1)

Publication Number Publication Date
WO2022042587A1 true WO2022042587A1 (zh) 2022-03-03

Family

ID=73441505

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/114485 WO2022042587A1 (zh) 2020-08-26 2021-08-25 用于移动终端的图像处理方法及装置、存储介质、终端

Country Status (3)

Country Link
US (1) US20230325963A1 (zh)
CN (1) CN111984417B (zh)
WO (1) WO2022042587A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111984417B (zh) * 2020-08-26 2023-01-24 展讯通信(天津)有限公司 用于移动终端的图像处理方法及装置、存储介质、终端
CN113362219B (zh) * 2021-07-02 2023-08-11 展讯通信(天津)有限公司 一种图像数据处理方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103344648A (zh) * 2013-07-10 2013-10-09 鞍钢股份有限公司 一种基于gpu的钢板表面质量检测方法及系统
CN106951322A (zh) * 2017-02-28 2017-07-14 中国科学院深圳先进技术研究院 一种cpu/gpu异构环境的图像协同处理程序获取方法及系统
CN107146193A (zh) * 2017-04-28 2017-09-08 南京觅踪电子科技有限公司 一种应用于图像处理的基于双显卡的gpu并行计算方法
CN107945098A (zh) * 2017-11-24 2018-04-20 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机设备和存储介质
CN109547708A (zh) * 2018-12-04 2019-03-29 中国航空工业集团公司西安航空计算技术研究所 一种合成视景图像处理系统
US10372428B1 (en) * 2015-06-16 2019-08-06 Architecture Technology Corporation Dynamic computational acceleration using a heterogeneous hardware infrastructure
CN111984417A (zh) * 2020-08-26 2020-11-24 展讯通信(天津)有限公司 用于移动终端的图像处理方法及装置、存储介质、终端

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4907488B2 (ja) * 2007-10-24 2012-03-28 株式会社リコー 画像処理装置、画像処理方法及び該方法を実行させるためのプログラムを格納したコンピュータ読み取り可能な記録媒体
JP6766598B2 (ja) * 2016-10-31 2020-10-14 セイコーエプソン株式会社 画像処理装置、画像処理方法および制御プログラム
CN110751676A (zh) * 2019-10-21 2020-02-04 中国科学院空间应用工程与技术中心 一种基于目标检测的异构计算系统、方法和可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103344648A (zh) * 2013-07-10 2013-10-09 鞍钢股份有限公司 一种基于gpu的钢板表面质量检测方法及系统
US10372428B1 (en) * 2015-06-16 2019-08-06 Architecture Technology Corporation Dynamic computational acceleration using a heterogeneous hardware infrastructure
CN106951322A (zh) * 2017-02-28 2017-07-14 中国科学院深圳先进技术研究院 一种cpu/gpu异构环境的图像协同处理程序获取方法及系统
CN107146193A (zh) * 2017-04-28 2017-09-08 南京觅踪电子科技有限公司 一种应用于图像处理的基于双显卡的gpu并行计算方法
CN107945098A (zh) * 2017-11-24 2018-04-20 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机设备和存储介质
CN109547708A (zh) * 2018-12-04 2019-03-29 中国航空工业集团公司西安航空计算技术研究所 一种合成视景图像处理系统
CN111984417A (zh) * 2020-08-26 2020-11-24 展讯通信(天津)有限公司 用于移动终端的图像处理方法及装置、存储介质、终端

Also Published As

Publication number Publication date
CN111984417A (zh) 2020-11-24
US20230325963A1 (en) 2023-10-12
CN111984417B (zh) 2023-01-24

Similar Documents

Publication Publication Date Title
WO2022042587A1 (zh) 用于移动终端的图像处理方法及装置、存储介质、终端
US11593644B2 (en) Method and apparatus for determining memory requirement in a network
US11151442B2 (en) Convolutional neural network processing method and device
US9311157B2 (en) Method and apparatus for dynamic resource allocation of processing units on a resource allocation plane having a time axis and a processing unit axis
US8453156B2 (en) Method and system to perform load balancing of a task-based multi-threaded application
CN107688495B (zh) 调度处理器的方法及设备
WO2022134948A1 (zh) 神经网络的计算部署方法及装置、存储介质、计算机设备
WO2021136137A1 (zh) 一种资源调度方法、装置及相关设备
KR102113662B1 (ko) 모바일 에지 컴퓨팅 환경에서 태스크를 분할하여 대리 노드들에 할당하는 방법
CN111352727B (zh) 一种应用于图像混合集群处理系统的图像处理方法
US11630983B2 (en) Graph conversion method
CN112181657A (zh) 视频处理方法、装置、电子设备及存储介质
CN112463383A (zh) 一种gpu分配方法、系统、存储介质及设备
Karasawa et al. A parallel skeleton library for multi-core clusters
CN107634978B (zh) 一种资源调度方法及装置
CN117271136A (zh) 数据处理方法、装置、设备和存储介质
CN115775199B (zh) 数据处理方法和装置、电子设备和计算机可读存储介质
CN114327856A (zh) 数据处理方法、装置、电子设备及存储介质
KR101558807B1 (ko) 호스트 프로세서와 협업 프로세서 간에 협업 처리를 위한 프로세서 스케줄링 방법 및 그 방법을 수행하는 호스트 프로세서
JP2001331332A (ja) アプリケーションシステムのリソース予約方法、予約装置、リソース量推定装置およびコンピュータシステム
CN113778656A (zh) 线程池的线程配置方法、装置、服务器和存储介质
CN112346852A (zh) 矩阵求和运算的分布式物理处理
KR101989033B1 (ko) 플랫폼 관리 장치 및 방법
CN116468597B (zh) 基于多gpu的图像渲染方法、装置及可读存储介质
CN114896076B (zh) 一种图形处理器集群的资源分配控制方法、系统和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21860422

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 030723)

122 Ep: pct application non-entry in european phase

Ref document number: 21860422

Country of ref document: EP

Kind code of ref document: A1