WO2019071618A1 - 一种图像处理方法及设备 - Google Patents

一种图像处理方法及设备 Download PDF

Info

Publication number
WO2019071618A1
WO2019071618A1 PCT/CN2017/106197 CN2017106197W WO2019071618A1 WO 2019071618 A1 WO2019071618 A1 WO 2019071618A1 CN 2017106197 W CN2017106197 W CN 2017106197W WO 2019071618 A1 WO2019071618 A1 WO 2019071618A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
displayed
image processing
application
resolution
Prior art date
Application number
PCT/CN2017/106197
Other languages
English (en)
French (fr)
Inventor
庄光庭
金鑫
沈慧海
林于超
孙骏雄
刘晓东
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to US16/754,567 priority Critical patent/US11132766B2/en
Priority to CN201780003404.8A priority patent/CN108496198B/zh
Priority to EP17928530.9A priority patent/EP3671627A4/en
Priority to AU2017435234A priority patent/AU2017435234B2/en
Publication of WO2019071618A1 publication Critical patent/WO2019071618A1/zh
Priority to US17/460,726 priority patent/US20220058772A1/en
Priority to AU2021229220A priority patent/AU2021229220B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G06T5/70—
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/22—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of characters or indicia using display control signals derived from coded signals representing the characters or indicia, e.g. with a character-code memory
    • G09G5/222—Control of the character-code memory
    • G09G5/227—Resolution modifying circuits, e.g. variable screen formats, resolution change between memory contents and display screen
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37—Details of the operation on graphic patterns
    • G09G5/373—Details of the operation on graphic patterns for modifying the size of the graphic pattern
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/02—Handling of images in compressed format, e.g. JPEG, MPEG
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas

Definitions

  • the present application relates to the field of terminal technologies, and in particular, to an image processing method and device.
  • Intelligent mobile terminals such as smart phones
  • An intelligent mobile terminal with an Android system the operating system from the upper layer to the lower layer usually includes an application layer, a framework layer, a runtime, a core class library, a hardware abstraction layer, and a Linux kernel layer.
  • the core function implementation in the Android system is based on this application development.
  • the application layer of the Android system is composed of all the applications running on the Android device. It includes not only system applications such as calls, short messages, and contacts (pre-installed on the smart mobile terminal along with the Android system), but also other subsequent installations.
  • Third-party applications in the device Third-party applications are developed based on the software development kit (SDK) provided by Android and are subject to the SDK interface.
  • SDK software development kit
  • the system application pre-installed in the device can call the interfaces and modules of the entire framework layer.
  • the existing intelligent mobile terminal operating system after the third-party application is installed and authorized by the system, the control interface can be directly retrieved from the Android framework layer.
  • an application usually calls the framework layer of the Android system when displaying an image.
  • the Image View control draws the image, but because the image is displayed to save traffic, many image details are compressed based on bandwidth considerations, resulting in poor image clarity.
  • the present application provides an image processing method and device for solving the problem that the image displayed by the application in the existing terminal device has poor definition.
  • an embodiment of the present application provides an image processing method, where the method is applicable to an image processing device having an operating system, and the image processing module of the operating system receives an image processing module of the first application calling the operating system.
  • the instruction because the instruction carries an image to be displayed; therefore, the image processing module performs image optimization processing on the image to be displayed, and displays an image after the image optimization process.
  • the image processing module of the operating system of the image processing device is improved, and the image optimization function is added to optimize the processing of the image to be displayed in different applications in the application layer. That is to say, when the multimedia file in the application calls the interface of the image processing module of the operating system to display the image, the image optimization process of the image processing module is first performed, and the finally displayed image is an optimized image, such as an optimized image.
  • the resolution is improved and the resolution is better.
  • the operating system is an Android operating system
  • the image processing module is an image view class in a framework layer of the Android operating system, so that the image view class can obtain the instruction from the instruction.
  • a width and height of an image to be displayed and then the image view class determines that a width and a height of the image to be displayed satisfy a setting condition, the setting condition being a height of a display screen of the image processing device and the image to be displayed.
  • the difference between the heights is less than the second threshold, and/or the difference between the width of the display screen of the image processing apparatus and the width of the image to be displayed is less than the third threshold. That is to say, the image view class can optimize the image only for images of sufficient size, which helps to improve the efficiency of image optimization.
  • the operating system is an Android operating system
  • the image processing module is a bitmap factory class in a framework layer of the Android operating system, such that the image processing module is to be displayed
  • the image to be displayed is decoded before the image is subjected to image optimization processing.
  • This method mainly compensates for the image optimization in the scene where the application itself has the image view class, because when the application itself in the application layer already has the image view class, the ImageView in the frame layer is no longer called, but it is still called.
  • the bitmap layer factory class (BitmapFactory) decodes the image to be displayed to obtain a decoded image. So you can extend the image optimization function on the BitmapFactory so that when the application in the application layer calls this interface, it will trigger image optimization.
  • the image processing module determines that the resolution of the image to be displayed is lower than the first threshold, the image to be displayed is subjected to super-resolution processing, and the super-resolution processing can adjust the original The resolution of the image, so the image sharpness after image optimization becomes higher.
  • the image processing module may further determine, according to the identifier of the first application in the instruction, that the first application has the permission of super-resolution processing, and obviously, the image may be improved by doing so. Optimized efficiency helps focus on multimedia files with image elements.
  • the image processing module determines whether the identifier of the first application exists in a preset whitelist; if yes, the image processing module determines that the first application has image optimized permissions Otherwise, the image of this application is not optimized for images, because the whitelist can be updated by the user at any time, so it is more convenient to control.
  • the method that the image processing module performs super-resolution processing on the image to be displayed may be: the image processing module adds the image to be displayed as a task object to a task queue;
  • the image processing module determines an image optimization algorithm corresponding to the image to be displayed according to the resolution of the image to be displayed; the image processing module uses the corresponding image optimization algorithm to display the image to be displayed in the task queue The corresponding task object is super-resolution processed.
  • the image processing module determines that the task object corresponding to the image to be displayed completes the super-resolution processing, and releases the memory space corresponding to the task object, so as to perform memory management.
  • the first task in the task queue is assigned to the first processor to perform super-resolution processing
  • the second task in the task queue is assigned to the second processor to perform super-resolution processing, so that different Structure acceleration.
  • an embodiment of the present invention provides an image processing apparatus having a function of implementing an image processing module behavior of an operating system in the above method example.
  • the function can execute the corresponding software through hardware achieve.
  • the hardware or software includes one or more modules corresponding to the functions described above.
  • the image processing apparatus includes a receiving unit and a processing unit, wherein:
  • a receiving unit configured to receive an instruction that the first application invokes an image processing module of the operating system, where the instruction carries an image to be displayed;
  • a processing unit configured to perform image optimization processing on the image to be displayed, and display an image after the image optimization processing.
  • the processing unit is specifically configured to: perform super-resolution processing on the image to be displayed when the resolution of the image to be displayed is lower than a first threshold.
  • the image processing apparatus further includes a determining unit, configured to determine, according to the identifier of the first application, that the first application has the permission of super-resolution processing.
  • the determining unit is specifically configured to: determine whether the identifier of the first application exists in a preset whitelist; if yes, determine that the first application has an image optimized permission.
  • the processing unit is specifically configured to:
  • the task object corresponding to the image to be displayed in the task queue is subjected to super-resolution processing by using a corresponding image optimization algorithm.
  • processing unit is further configured to: after determining that the task object corresponding to the image to be displayed completes the super-resolution processing, release the memory space corresponding to the task object.
  • the first task in the task queue is assigned to the first processor to perform super-resolution processing
  • the second task in the task queue is assigned to the second processor to execute super Resolution processing to speed up image processing
  • the image processing device is an image view class of an operating system
  • the instruction further includes a width and height of the image to be displayed
  • the processing unit is further configured to: determine the image to be displayed The width and height satisfy a setting condition that the difference between the height of the display screen of the terminal device and the height of the image to be displayed is less than a second threshold, and/or the display of the terminal device The difference between the width of the screen and the width of the image to be displayed is less than a third threshold.
  • the image processing apparatus is a bitmap factory class in a framework layer of the Android operating system, and then the image processing apparatus decodes the to-be-displayed image before performing image optimization processing on the image to be displayed. Display the image.
  • an embodiment of the present invention provides an image processing device, where the image processing device includes a memory, a display, and a processor.
  • the processor may be a central processing unit (CPU) or digitally processed. Unit and so on.
  • the processor performs an image optimization function according to an instruction of the first application to invoke an image processing module of the operating system.
  • a memory an instruction for storing the first application, a program instruction storing an operating system, and a program executed by the processor.
  • the display is configured to display the image after the processor image optimization process on the human-computer interaction interface of the first application.
  • the processor is configured to perform super-resolution processing on the image to be displayed when the resolution of the image to be displayed is lower than a first threshold.
  • the instructions of the first application further include an identifier of the first application
  • the processor is further configured to: determine the first according to an identifier of the first application The application has super-resolution processing permissions.
  • the processor is specifically configured to: determine whether the identifier of the first application exists in a preset whitelist; if yes, determine that the first application has image optimization permission.
  • the processor is specifically configured to: add the image to be displayed as a task object to a task queue; determine an image corresponding to the image to be displayed according to a resolution of the image to be displayed optimization;
  • the task object corresponding to the image to be displayed in the task queue is subjected to super-resolution processing by using a corresponding image optimization algorithm.
  • the operating system is an Android operating system
  • the image processing module is an image view class or a bitmap factory class in a framework layer of the Android operating system
  • the processor may adopt the following two Ways to optimize the image:
  • the processor determines that the width and height of the image to be displayed meets a set condition, and performs super-resolution processing on the image that satisfies the condition, wherein the setting condition is a height and a display of the display of the image processing device.
  • the difference between the heights of the image to be displayed is less than the second threshold, and/or the difference between the width of the display screen of the image processing apparatus and the width of the image to be displayed is less than the third threshold.
  • the processor decodes the image to be displayed, and then satisfies the conditional image for super-resolution processing.
  • the processor is further configured to: after determining that the task object corresponding to the image to be displayed completes the super-resolution processing, releasing the memory space corresponding to the task object
  • a first task in the task queue is assigned to a first processor to perform super-resolution processing
  • a second task in the task queue is assigned to a second processor to perform super-resolution Processing
  • the embodiment of the present application further provides a computer storage medium, where the software program stores a software program, and the software program can implement any one of the first aspects when being read and executed by one or more processors.
  • the method provided by the design is not limited to:
  • a computer program product is provided in the embodiment of the present application.
  • the computer program product is executed by a computer, the computer is caused to perform the method provided by any one of the first aspects.
  • the improved image processing module in the operating system does not need to separately develop each application, so it has strong reusability, and the multimedia files from different applications are in the Image information is automatically image optimized when displayed, improving the user experience.
  • FIG. 1 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure
  • FIG. 2 is a schematic structural diagram of a mobile phone according to an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of a method for image processing according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of a method for judging a screen according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an Android operating system framework with image optimization function according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an image view class of an integrated image optimization function according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a bitmap factory class of an integrated image optimization function according to an embodiment of the present application.
  • FIG. 8 is a schematic flowchart 1 of an image optimization method for an image view class according to an embodiment of the present application.
  • FIG. 8 is a schematic flowchart 2 of an image optimization method for an image view class according to an embodiment of the present application.
  • FIG. 9 is a schematic flowchart of an image optimization method of a bitmap factory class according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a method for controlling a policy of an image optimization queue according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of an image processing apparatus provided by the present application.
  • FIG. 12 is a schematic structural diagram of another image processing apparatus provided by the present application.
  • the image processing device of the present invention is applicable to the image processing device shown in FIG. 1 , the image processing device includes an application layer 110 and a frame layer 120, wherein the application layer 110 includes a first application. 111, and a plurality of other applications, the frame layer 120 includes an image processing module 122.
  • the image processing module 122 receives the image transmitted by the first application 111, and the image processing module 122 uses the image to perform image optimization processing, and transmits the image processed by the image processing module 122 to the first application 111 to make the first application.
  • the program 111 displays the processed image.
  • the framework layer 120 further includes a storage module for buffering the image to be processed and the processed image as an example of the following image processing procedure, and illustrates the workflow of the image processing module:
  • the image processing module 122 After receiving the image to be displayed, the image processing module 122 first acquires the resolution of the image to be displayed, determines whether the resolution is lower than a threshold, and if so, performs super-resolution processing on the image to be displayed, and displays Super-resolution processed image.
  • the so-called super-resolution processing refers to the improvement of the resolution of the original image by hardware or software.
  • the process of obtaining a high-resolution image through a series of low-resolution images is super-resolution reconstruction.
  • the image processing module 122 returns the super-resolution processed image to the first application 111 for interface display.
  • An image processing method provided by an example of the present invention is also applicable to a mobile phone as shown in FIG. 2, and the specific structural composition of the mobile phone will be briefly described below.
  • the handset 200 includes a display device 210, a processor 220, and a memory 230.
  • the memory 230 can be used to store software programs and data, and the processor 220 executes various functional applications and data processing of the mobile phone 200 by running software programs and data stored in the memory 230.
  • the memory 230 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as an image collection function, etc.), and the like; and the storage data area may be stored according to the use of the mobile phone 200.
  • memory 230 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • the processor 220 is a control center of the mobile phone 200, and connects various parts of the entire mobile phone using various interfaces and lines, and executes various functions and processing data of the mobile phone 200 by running or executing software programs and/or data stored in the memory 230. In order to monitor the mobile phone as a whole.
  • the processor 220 may include one or more general-purpose processors, and may also include one or more DSPs (digital signal processors), and may also include one or more ISPs (image signal processors). For performing related operations, the technical solutions provided by the embodiments of the present application are implemented.
  • the camera 260 can be a normal camera or a focus camera.
  • the handset 200 can also include an input device 240 for receiving input digital information, character information, or contact touch/contactless gestures, as well as generating signal inputs related to user settings and function control of the handset 200.
  • an input device 240 for receiving input digital information, character information, or contact touch/contactless gestures, as well as generating signal inputs related to user settings and function control of the handset 200.
  • the display device 210 includes a display panel 211 for displaying information input by the user or information provided to the user, and various menu interfaces of the mobile phone 200, etc., which are mainly used to display the camera or the sensor in the mobile phone 100 in the embodiment of the present application.
  • the image to be detected is acquired.
  • the display panel can be configured by using a liquid crystal display (LCD) or an OLED (organic light-emitting diode).
  • the handset 200 can also include a power source 250 for powering other modules.
  • the handset 200 may also include one or more sensors 270, such as image sensors, infrared sensors, laser sensors, and the like.
  • Mobile phone 200 also The radio frequency (RF) circuit 280 may be included for network communication with the wireless network device, and may further include a WiFi module 290 for performing WiFi communication with other devices, acquiring images or data transmitted by other devices, and the like.
  • RF radio frequency
  • the embodiment of the present application provides an image processing method and an image processing device for solving the problem that the image display of the multimedia file in the application is unclear.
  • the method and the image processing device of the present application are based on the same inventive concept. Since the method and the image processing device solve the problem similarly, the implementation of the image processing device and the method can be mutually referred to, and the repetition is not Let me repeat.
  • the image optimization module of the operating system of the image processing device is improved, and the image optimization function is added to optimize the processing of the image to be displayed in different applications in the application layer. That is to say, the embodiment of the present application improves the layer (such as the framework layer) in the operating system except the application layer, adds an image optimization program, and calls related hardware to implement optimization processing, when the multimedia file in the application calls the operating system.
  • the image processing module displays the image, it will first pass through the image optimization processing process of the image processing module, and the final displayed image is an optimized image, and the resolution has been improved, so the definition is also better, because the embodiment of the present application improves.
  • the image processing module in the operating system does not need to separately develop each application. For the user who operates the application in the application layer, the user of the entire image optimization process is non-perceived, so the method is said to have Strong reusability is also more automated.
  • the application program according to the embodiment of the present application refers to a software having a visual user interface and capable of interacting with a user, for example, a short message application, a multimedia message application, various email applications, a microblog, a WeChat, Tencent chat software (QQ), even my (Line), photo sharing (instagram), nails, headlines, browsers and so on.
  • a short message application for example, a short message application, a multimedia message application, various email applications, a microblog, a WeChat, Tencent chat software (QQ), even my (Line), photo sharing (instagram), nails, headlines, browsers and so on.
  • QQ Tencent chat software
  • instagram even my (Line)
  • nails headlines, browsers and so on.
  • the image processing device may be referred to as a user equipment (User Equipment, UE) as a device that can install various communication applications or have a communication function.
  • UE User Equipment
  • the image related to the embodiment of the present application is derived from a multimedia file, wherein the multimedia file is an image, a collection of images, or a video file composed of a plurality of frames of images.
  • the specific process of the image processing method may include:
  • Step 201a The first application sends an instruction to the image processing module of the operating system to call the image processing module to perform image optimization.
  • the first application sends an instruction to the image processing module upon detecting that the user views the image. For example, when the user uses WeChat, the user sends an image sent by the friend, and the WeChat dialog box displays the image as a thumbnail. When the user clicks the thumbnail to view a large image of the image, the WeChat calls the image processing module.
  • Step 202a When the image processing module determines that the resolution of the image to be displayed is lower than the first threshold, the image to be displayed is subjected to super-resolution processing, and the super-resolution processed image is displayed.
  • the image processing module may first obtain the resolution of the image to be displayed, and then determine the resolution, if the judgment is lower than the first Threshold, then start super-resolution processing, otherwise, no super-resolution processing.
  • the image optimization processing is described by taking super-resolution processing as an example.
  • image optimization methods may be used to optimize the image, such as adjusting the brightness, saturation, color, or beautifying the face of the image.
  • image optimization methods can also be used together, or different image optimization methods can be triggered according to different detection conditions, such as identifying the current nighttime reading mode, reducing the brightness of the image; or recognizing the face in the image, performing facial beauty Yan's image optimization method and so on.
  • the significance of image optimization processing is to improve the quality of images, this has different effects and meanings on users in different applications. For example, Weibo users usually browse short videos or photos, so Weibo applications require higher image resolution. For example, in email applications, emails are mostly text messages. Usually only a small number of images are inserted in emails. Image information, so this type of application requires less image resolution. If the conventional method is based on bandwidth considerations, losing part of the pixels and reducing the image resolution will seriously affect the user experience. Therefore, in a possible design, the image processing module will further according to the first application carried in the instruction. Identification, determining whether the first application has image optimization authority, and performing image optimization on an application having image optimization authority.
  • the image processing module pre-stores a whitelist with image optimization authority, and the whitelist includes an identifier of the application layer application, if the image processing module determines that the identifier of the application acquired in the instruction is in the whitelist. Obviously, it has the right to optimize the image, which can further trigger the super-resolution processing. Otherwise, the super-resolution processing is not performed.
  • the image processing module authenticates the identifier of the application. If the authentication passes, it proves that the application has the right to optimize the image, and the super-resolution processing may be further triggered. Otherwise, the super-resolution processing is not performed.
  • other conditions may be used to further determine whether an image to be displayed requires image optimization before performing super-resolution processing. For example, after the image processing module acquires the image, it is determined whether the image has been subjected to super-resolution processing, and if it has been processed, the image is not subjected to super-resolution processing.
  • the image processing module When considering that there are more images to be displayed in the application, the image processing module will exert a lot of pressure on performance and memory, so that some images can be filtered at this time, for example, the image whose size meets the preset condition is processed. For example, the image processing module determines whether the image size of the image to be displayed is close to the full screen, and super-resolution processing is performed only when the image size is close to the screen size. As shown in FIG. 4, the width of the image to be displayed is consistent with the screen width, and the image to be displayed satisfies the condition for performing super-resolution processing.
  • the image width of the partial application may be slightly smaller than the screen width, when the difference between the height of the display to be displayed and the height of the image to be displayed is less than the second threshold, or the image processing apparatus
  • the difference between the width of the display screen and the width of the image to be displayed is less than a third threshold.
  • the second threshold and the third threshold may be the same, for example, the width of the image to be displayed satisfies [0.95, 1.0] times of the screen width, or [0.95, 1.0] times the height of the image to be displayed satisfies the height , you can continue to perform super-resolution processing.
  • the operating system of the image processing device is an Android operating system.
  • the embodiment of the present application further describes the image processing method in combination with the system architecture of the Android operating system.
  • the Android operating system usually includes the application layer, the framework layer, the runtime, the core class library, the hardware abstraction layer, and the Linux kernel layer from the upper layer to the lower layer.
  • the image processing module of the embodiment of the present application belongs to a function module in the framework layer.
  • the Android operating system architecture with image optimization function includes an application layer 301, a framework layer 302, a hardware abstraction layer 303, and a kernel chip 304.
  • the scene recognition module 305 in the image processing module in the framework layer is configured.
  • image optimization is required, for example, whether the first application calling the image processing module is in the white list of image optimization, whether the image to be displayed is to be displayed, such as the width of the image to be displayed. Whether the height is close to the width or height of the screen; whether the resolution of the image to be displayed satisfies the resolution threshold, whether there is a human face in the image to be displayed, and the like.
  • the image view class (ImageView) 306 and the bitmap factory class (BitmapFactory) 307 are used to perform image optimization on the image to be displayed. Specifically, the image optimization task queue and the memory call can be managed, and the image is executed by calling the image optimization algorithm in the HiAI service. Optimized processing.
  • the HiAI service platform 308 in the framework layer includes various image optimization algorithms, such as a DNN hard algorithm and a Raisr soft algorithm, wherein the DNN hard algorithm is a super-resolution image hard algorithm, and interacts with a neural network processor (IPU). The performance and effect are improved; the Raisr-like soft algorithm is a super-image soft algorithm, which is mainly aimed at low-end and low-end image processing devices without IPU hardware.
  • the storage module 309 is also included in the framework layer, and the function has been described in FIG. 1 and will not be described herein.
  • the image optimization function can be expanded on the image view class, so that the application in the application layer can be called.
  • the image view class triggers the above image processing method.
  • different images in the application layer 401 respectively create an ImageView class, such as ImageView of the first application, ImageView of the second application, and ImageView3 of the third application, thereby triggering the framework layer.
  • the image processing module 404 in 402 calls the heterogeneous processor 403 to perform image optimization, and the image view class 405 in the image processing module 404 calls an image optimization algorithm to perform image optimization processing, and the image processing module passes through the image optimization process.
  • the processor performs heterogeneous acceleration to speed up the process of image optimization.
  • the image to be displayed is displayed.
  • Decoding is performed to obtain a decoded image. So you can extend the image optimization function on the BitmapFactory so that the application in the application layer will trigger the above image processing method once the bitmap factory class is called.
  • the first application in the application layer calls the BitmapFactory in the framework layer of the operating system to decode the multimedia file during the image display process; the image after the decoding in step 502 passes through the scene recognition module.
  • a series of conditional judgments, such as recognition of the resolution size, etc., the image satisfying the condition is added to the image optimization queue; in step 503, considering part of the image, for example, the information of the first few frames or the last few frames usually does not matter, and may be directly The image optimization is abandoned for it; in step 504, after filtering, the remaining images are sequentially subjected to super-resolution processing.
  • the most commonly used image display module is the ImageView (Image View Module).
  • ImageView Image View Module
  • the embodiment of the present application adds a condition judging process related to the image optimization authority in the interface to determine whether the current application has the right to optimize the image. The following takes super-resolution processing as an example, as shown in FIG. 8 :
  • step 601 the application creates an ImageView and calls the initImageView() interface to initialize the data.
  • initImageView() is provided with a plurality of conditions for determining whether the current application has the capability of image optimization capability, and the conditions are as follows:
  • step 603 the subsequent judgment logic is executed, that is, step 603 is performed; otherwise, the image optimization processing is not continued, that is, step 605 is executed to draw an image for display.
  • the meaning of image optimization processing is to improve the quality of the image
  • the impact and meaning of the image in different applications are different, and need to be distinguished according to the importance and value of the function, so the current image optimization process only Open for specific parts of the app.
  • the package name and the application are usually one-to-one correspondence
  • the package name of the current application can be obtained by calling getPackageName(). If the package name of the application is in the white list, it is determined that ImageView allows image optimization and continues the scene recognition logic; if not, the subsequent behavior of ImageView is consistent with the image optimization feature.
  • Step 603 obtaining an image size.
  • the Drawable's getIntrinsicWidth() and getIntrinsicHeight() methods in the Android View drawing system can obtain the "width" of an image, so the image to be displayed can be measured by these methods.
  • the width and height for example, in the case of BitmapDrawable, actually returns the width and height of the held Bitmap.
  • Step 604 when the difference between the height of the to-be-displayed screen and the height of the image to be displayed is less than a first threshold, or between the width of the display screen of the image processing apparatus and the width of the image to be displayed If the difference is less than the second threshold, if any one of the two conditions is satisfied, the super-resolution processing is performed on the image to be displayed, that is, step 605 is performed, otherwise step 609 is performed.
  • the width of the image to be displayed satisfies [0.95, 1.0] times the width of the screen, or when the height of the image to be displayed satisfies [0.95, 1.0] times the height of the image, it can be determined that the size of the image to be displayed is sufficiently large, and thus It is necessary to optimize only the image for this image.
  • Step 605 ImageView draws the set Drawable data in the standard method onDraw() of the View, and draws the content through the standard method draw() of Drawable.
  • BitmapDrawable draws the content of a Bitmap onto the ImageView
  • ColorDrawable is Color is applied to ImageView and so on.
  • Step 606 determining whether the state of the device of the super-resolution optimization algorithm is normal, because when the image is displayed in batches, the task optimization queue of the resolution optimization algorithm is congested, so it is necessary to determine that the current super-resolution optimization algorithm is in a normal state, and if so, execute Step 507, otherwise step 509 is performed.
  • step 607 for performance, memory, and effect considerations, the image optimization process is performed only for images of a certain range of resolution.
  • the resolution size setting rules are different for different applications. For example, Weibo requires higher resolution for images than other applications.
  • the image optimization is started only when the resolution of the application is lower than the corresponding size setting rule. Therefore, it is necessary to continue to judge whether the resolution satisfies the setting condition. If yes, step 608 is performed, otherwise step 609 is performed.
  • Step 608 performing super-resolution processing on the image to be displayed that satisfies the above condition, wherein the super-resolution algorithm executed by the super-resolution corresponds to the image resolution size setting of the application. Then, the process returns to step 605 to re-draw and display the optimized image. It should be noted that after the step 605 is executed, the image optimization process is ended.
  • step 609 the imageview does not perform optimization on the image. It also needs to draw the set Drawable data in the standard method onDraw() of the View, and draw the content through the standard method draw() of Drawable, as shown in step 505.
  • the embodiment of the present application first determines whether the switch is turned on, and then measures whether the size of the image is large enough, and then determines whether the resolution is lower than the set standard. .
  • Figure 8a shows the framework and flow for image optimization using the ImageView class, including applications in the operating system.
  • Layer A10 frame layer A20, HAL layer/hardware A30.
  • the application layer A10 has an application A11 using Imageview.
  • the trigger image optimization module in ImageViewA21 in the frame layer A20 is used to judge the timing of image optimization; the scene recognition A211 is used to determine which images need to be optimized, such as whether the image size meets the preset condition, and whether the image resolution is corrected.
  • Threshold value whether there is a human face in the image, etc., specifically may include the above steps 502-504, 506-507, and actual conditions may be set according to the purpose of image optimization to determine which images need to be image optimized; optimization task allocation and content management module A212 is used to manage the multi-threaded task and content of the image optimization process, and can be managed according to the manner of FIG. 10; the optimization algorithm A22 provides a plurality of image optimization models A221, that is, an image optimization algorithm; the heterogeneous optimization processing module A222 is used to call different Processor A32 accelerates image optimization processing.
  • the HAL layer/hardware is used to implement the display, the hardware driver of the processor.
  • the image is sent to ImageViewA21 by using A11.
  • the trigger image optimization module in ImageViewA21 confirms the timing of image optimization processing.
  • the scene recognition module A211 determines whether the image needs to be optimized, and puts the image to be optimized into the queue of the optimization task.
  • the optimization task allocation and memory management module A212 manages the optimization task, and the optimization algorithm A22 is used to optimize the image in the optimization task team.
  • the heterogeneous optimization processing module A222 calls different processors A32 to run the optimization algorithm.
  • the updated image is updated using the optimized image and displayed on the display A31.
  • the image optimization processing in the BitmapFactory provides a prediction model, and the image in the prediction display interface needs to be subjected to super-resolution processing, and the image outside the display interface is filtered out, so as to ensure that the image that meets the condition can be optimized.
  • the specific steps are shown in Figure 9.
  • Step 701 The image of the application decodes the image to be displayed by calling BitmapFactory.decodeFile() of BitmapFactory, and outputs a source bitmap object, for example, decoding the image in JPEG format to be displayed into a bitmap.
  • Step 702 The BitmapFactory calls the BitmapFactory class of the framework layer and substitutes the source bitmap object.
  • step 703 the image processing module internally creates/manages a Destination Bitmap, uses the source bitmap and the Destination Bitmap as parameters, and calls the HisidDK Native API to perform data processing.
  • Step 704 The HisiDDK internally transmits the sourceBitmap and the Destination Bitmap to the HIAI service process through the Binder interface, inter-process communication (IPC), and invokes the IPU to perform super-resolution processing.
  • IPC inter-process communication
  • Step 705 The super-image processed by the HIAI service process is stored in the Destination Bitmap, and the Binder process of the library process is asynchronously transferred back through the Binder interface.
  • step 706 after the Binder receives the processing of the image processing module, the Binder transmits the Destination Bitmap to the view display interface for re-rendering.
  • step 707 the redrawn image is displayed on the display interface of the application.
  • Bitmap is the most likely cause of memory anomalies. Bitmap follows the Java GC mechanism. When there is no strong reference to the Bitmap object, the Bitmap object will be released. If there is an unreasonable pointer holding the Bitmap, it will cause memory usage or leak. Therefore, the embodiment of the present application clearly gives the life cycle of the Bitmap in the image optimization process, so as to facilitate memory management. At the same time, there are many image optimization algorithms and high computational requirements. It is possible to further manage different heterogeneous processors such as GPU and FPGA, and assign tasks to accelerate the optimization of image processing performance.
  • the first application is WeChat
  • user A sends a self-timer picture to user B through WeChat
  • the thumbnail of the self-timer picture is displayed in the WeChat chat box of user B
  • the image processing module's ImageView or BitmapFactory will optimize the image of the self-timer image, and then the user B views the self-timer image.
  • the resolution will be higher and the resolution will be better.
  • the above image processing method for third-party applications does not require repeated development of image optimization functions, and the super-resolution processing can be triggered by directly calling an existing interface on the operating system. .
  • the embodiment of the present application further adds a management and control policy to the image optimization queue of the image processing module.
  • the control strategy is mainly that the task queue receives the super-resolution task request initiated by the image processing module, and This task is added to the queue; in step 802, if the cancel super-resolution task request initiated by the image processing module is received, the task is deleted from the queue; in step 803, the task queue detects the super-resolution state machine ready state. At this time, the task is taken out from the task queue and sent to DDK processing.
  • Bitmap is the most likely cause of memory anomalies. Bitmap follows the Java GC mechanism. When there is no strong reference to the Bitmap object, the Bitmap object will be released. If there is an unreasonable pointer holding the Bitmap, it will cause memory usage/leakage. Therefore, the embodiment of the present application clarifies the life cycle of the Bitmap in the image optimization process, so as to manage the memory.
  • the embodiment of the present application provides an image processing apparatus 900, which belongs to the framework layer of the operating system of the image processing apparatus, and is specifically used to implement the description of the embodiment described in FIG.
  • the method of the device is as shown in FIG. 11, and includes a receiving unit 901 and a processing unit 902, wherein:
  • the receiving unit 901 is configured to receive, by the first application, an instruction of an image processing module of the operating system, where the instruction carries an image to be displayed;
  • the processing unit 902 is configured to perform image optimization processing on the image to be displayed, and display an image after the image optimization processing.
  • the processing unit 902 is specifically configured to perform super-resolution processing on the image to be displayed when the resolution of the image to be displayed is lower than the first threshold.
  • the image processing apparatus further includes a determining unit 903, configured to determine, according to the identifier of the first application, that the first application has the permission of super-resolution processing.
  • the determining unit 903 is specifically configured to: determine whether the identifier of the first application exists in a preset whitelist; if yes, determine that the first application has an image optimized permission.
  • processing unit 902 is specifically configured to:
  • the task object corresponding to the image to be displayed in the task queue is subjected to super-resolution processing by using a corresponding image optimization algorithm.
  • processing unit 902 is further configured to: after determining that the task object corresponding to the image to be displayed completes the super-resolution processing, release the memory space corresponding to the task object.
  • the first task in the task queue is assigned to the first processor to perform super-resolution processing
  • the second task in the task queue is assigned to the second processor to execute super Resolution processing to speed up image processing
  • the image processing device is assumed to be an image view class of the operating system, and the instruction further includes a width and height of the image to be displayed; the processing unit 902 is further configured to: determine the to-be-displayed The width and height of the image satisfy a setting condition that the difference between the height of the display screen of the terminal device and the height of the image to be displayed is less than a second threshold, and/or the terminal device The difference between the width of the display screen and the width of the image to be displayed is less than a third threshold.
  • the image processing apparatus is a bitmap factory class in a framework layer of the Android operating system, and then the image processing apparatus decodes the to-be-displayed image before performing image optimization processing on the image to be displayed. Display the image.
  • the embodiment of the present application further provides an image processing device, where the image processing device is used to implement the method described in the embodiment described in FIG. 3, as shown in FIG. 12, the device includes a processor. 1001, a memory 1002, and a display 1003.
  • the processor 1001 can be a central processing unit (CPU), or can optimize functions for a digital processing unit or the like.
  • the memory 1002 is configured to store an instruction of the first application and a program instruction that stores an operating system.
  • the display 1003 is configured to display an image after the image optimization process by the processor 1001 on the human-computer interaction interface of the first application.
  • connection medium between the processor 1001 and the memory 1002 is not limited in the embodiment of the present application.
  • the memory 1002, the processor 1001, and the display 1003 are connected by a bus 1004 in FIG. 12, and the bus is indicated by a thick line in FIG. 12, and the connection manner between other components is only schematically illustrated. Not limited to limits.
  • the bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in Figure 12, but it does not mean that there is only one bus or one type of bus.
  • the memory 1002 may be a volatile memory, such as a random-access memory (RAM); the memory 1002 may also be a non-volatile memory, such as a read-only memory, flashing A flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or a memory 1002 is a program code that can be used to carry or store a desired program or data structure and can be Any other medium accessed by the computer, but is not limited to this.
  • the memory 1003 may be a combination of the above memories.
  • the processor 1001 executes program instructions in the memory 1002 for implementing the image processing method as shown in FIG. 3, including: the first application sends an instruction to the image processing module in the operating system to call the image processing module to perform image optimization.
  • the image processing module performs image optimization processing on the image to be displayed and returns it to the first application for display.
  • the image optimization processing method adopted by the processor 1001 may be performed by performing super-resolution processing on the image to be displayed when the resolution of the image to be displayed is lower than the first threshold.
  • the instruction of the first application further includes an identifier of the first application
  • the processor 1001 is further configured to: determine, according to the identifier of the first application, the The first application has the permission to handle super-resolution.
  • the embodiment of the present application further provides a computer readable storage medium, which is stored as computer software instructions for executing the above-mentioned processor, and includes a program for executing the above-mentioned processor.
  • the embodiment of the present application further provides a computer program product, when the computer program product is executed by a computer, causing the computer to perform the image processing method as described above.
  • the image processing module of the operating system of the image processing device is improved, and the image optimization function is added to optimize the image to be displayed in different applications in the application layer. deal with. That is to say, when the multimedia file in the application calls the interface of the image processing module of the operating system to display an image, the image optimization process of the image processing module is first performed, and the final displayed image is an optimized image, and the resolution has been Improved, and therefore sharper, because the embodiment of the present application improves the image processing module in the operating system, and does not need to separately develop each application, for the user who operates the application in the application layer, the entire image The optimization process user is non-perceived, so the method is more automated while being more reusable.
  • the various illustrative logic blocks, modules and circuits described in the embodiments of the present application may be implemented by a general purpose processing unit, a digital signal processing unit, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic. Devices, discrete gate or transistor logic, discrete hardware components, or any combination of the above are designed to implement or operate the functions described.
  • the general purpose processing unit may be a micro processing unit.
  • the general purpose processing unit may be any conventional processing unit, controller, microcontroller or state machine.
  • the processing unit may also be implemented by a combination of computing devices, such as a digital signal processing unit and a microprocessing unit, a plurality of microprocessing units, one or more microprocessing units in conjunction with a digital signal processing unit core, or any other similar configuration. achieve.
  • the steps of the method or algorithm described in the embodiments of the present application may be directly embedded in hardware, a software module executed by a processing unit, or a combination of the two.
  • the software modules can be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art.
  • the storage medium can be coupled to the processing unit such that the processing unit can read information from the storage medium and can write information to the storage medium.
  • the storage medium can also be integrated into the processing unit.
  • the processing unit and the storage medium may be configured in an ASIC, and the ASIC may be configured in the user terminal. Alternatively, the processing unit and the storage medium may also be configured in different components in the user terminal.
  • the above-described functions described in the embodiments of the present application may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, these functions may be stored on a computer readable medium or transmitted as one or more instructions or code to a computer readable medium.
  • Computer readable media includes computer storage media and communication media that facilitates the transfer of computer programs from one place to another.
  • the storage medium can be any available media that any general purpose or special computer can access.
  • Such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other device or data structure that can be used for carrying or storing Other media that can be read by a general purpose or special computer, or a general or special processing unit.
  • any connection can be appropriately defined as a computer readable medium, for example, if the software is from a website site, server or other remote source through a coaxial cable, fiber optic computer, twisted pair, digital subscriber line (DSL) Or wirelessly transmitted in, for example, infrared, wireless, and microwave, is also included in the defined computer readable medium.
  • DSL digital subscriber line
  • the disks and discs include compact disks, laser disks, optical disks, DVDs, floppy disks, and Blu-ray disks. Disks typically replicate data magnetically, while disks typically optically replicate data with a laser. Combinations of the above may also be included in a computer readable medium.

Abstract

一种图像处理方法及设备，其特征在于，所述方法适用于具备操作系统的图像处理设备，包括：操作系统的图像处理模块接收第一应用程序调用操作系统的图像处理模块的指令，所述指令中携带有待显示图像，以及所述待显示图像的分辨率；在所述待显示图像的分辨率低于第一阈值时，所述图像处理模块对所述待显示图像进行超分辨率处理，并显示超分辨率处理后的图像，用以解决传统显示方法图像清晰度不佳的问题。

Description

一种图像处理方法及设备
本申请要求在2017年10月9日提交中华人民共和国知识产权局、申请号为201710931459.7、发明名称为“一种图像处理方法及终端设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，尤其涉及一种图像处理方法及设备。
背景技术
近年来，随着电子产业和通信技术的飞速发展，以数据、话音、视频为基础的新业务发展迅猛。微电子技术、计算机软硬件技术的快速发展，为图像处理设备处理越来越复杂的工作打下了基础，为图像处理设备个性化提供了实现可能，使得终端从某种程度上摆脱了网络的制约，可以具备越来越强大的功能。此外，用户本身也对终端有迫切的需求，希望终端功能更强大、更灵活、更简捷。信息技术的发展，使终端技术走向智能化、移动化、多功能化。
随着移动终端的普及，尤其是智能手机的迅猛发展，不仅使人们的生活更方便，也使人们享受到高科技带来的成果。智能移动终端如智能手机，因为其有强大的操作系统，大容量的存储空间，可以方便地安装各种软件等诸多优点而越来越被人们接受。较之于传统的移动终端，智能移动终端能够安装更多的第三方应用。具有安卓(Android)系统的智能移动终端，操作系统从高层到低层通常依次包括：应用层、框架层、运行时、核心类库、硬件抽象层、Linux内核层。通常，Android系统中核心的功能实现，包括框架层、核心类库等，每个Android应用的开发者，都是在此基础上进行应用开发的。Android系统的应用层由运行在Android设备上的所有应用共同构成，它不仅包括通话、短消息、联系人等系统应用(随Android系统一起预装在智能移动终端上)，还包括其他后续安装到设备中的第三方应用。第三方应用都是基于Android提供的软件开发工具包(software development kit，SDK)进行开发的，并受到SDK接口的约束。而预装在设备中的系统应用，则可以调用整个框架层的接口和模块。现有智能移动终端操作系统，第三方应用在安装完毕并获得系统授权后，即可直接从Android框架层调取控件接口，例如，一个应用程序在显示图像时通常会调用Android系统的框架层的Image View控件来绘制图像、但是因为图像显示时为了节省流量，所以很多图像细节基于频宽考量都被压缩掉了，导致图像清晰度不佳。
发明内容
本申请提供一种图像处理方法及设备，用以解决现有终端设备中的应用程序显示图像信息存在清晰度不佳的问题。
第一方面，本申请实施例提供了一种图像处理方法，所述方法适用于具备操作系统的图像处理设备，包括：操作系统的图像处理模块接收第一应用程序调用操作系统的图像处理模块的指令，因为，所述指令中携带有待显示图像；所以所述图像处理模块对所述待显示图像进行图像优化处理，并显示图像优化处理后的图像。
通过上述方法，完善了图像处理设备的操作系统的图像处理模块，增加图像优化的功能，以对应用层中不同应用程序中的待显示图像进图像行优化处理。也就是说当应用程序中多媒体文件调用操作系统的图像处理模块的接口显示图像时，会先经过图像处理模块的图像优化处理过程，最终显示的图像是经过优化的图像，比如说优化后的图像的分辨率得到提高，因此清晰度也更佳。
在一种可能的设计中，所述操作系统为安卓操作系统，所述图像处理模块为所述安卓操作系统的框架层中的图像视图类，这样，图像视图类就可以从指令中获取所述待显示图像的宽高，然后所述图像视图类确定所述待显示图像的宽高满足设定条件，所述设定条件为所述图像处理设备的显示屏的高与所述待显示图像的高之间的差值小于第二阈值，和/或，所述图像处理设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值。也就是说，图像视图类可以仅对足够大尺寸的图像进行图像优化，这样有助于提高图像优化的效率。
在一种可能的设计中，所述操作系统为安卓操作系统，所述图像处理模块为所述安卓操作系统的框架层中位图工厂类，这样的话，所述图像处理模块对所述待显示图像进行图像优化处理之前，解码所述待显示图像。这一方法主要弥补在应用程序自身具备图像视图类的场景下的图像优化，因为当应用层中的应用程序自身已具有图像视图类时，就不再调用框架层中的ImageView，但是仍然会调用框架层的位图工厂类(BitmapFactory)，对待显示图像进行解码，得到解码后的图像。所以可以在BitmapFactory上扩展图像优化功能，这样，应用层中的应用程序一旦调用这一接口，就会触发图像优化。
在一种可能的设计中，所述图像处理模块确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理，通过超分辨率处理可以调整原有图像的分辨率，因此经过图像优化后的图像清晰度变高。
在一种可能的设计中，所述图像处理模块还可以根据指令中的所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限，显然，这样做可以提高图像优化的效率，有助于集中处理具有图像元素的多媒体文件。
可选地，所述图像处理模块判断所述第一应用程序的标识是否存在于预设的白名单列表中；若存在，则所述图像处理模块确定所述第一应用程序具备图像优化的权限，否则的话，就不对这一应用程序的图像进行图像优化，因为白名单可以随时被用户更新，所以较为方便管控。
具体地，所述图像处理模块对所述待显示图像进行超分辨率处理的方法可以是：所述图像处理模块将所述待显示图像作为任务对象加入到任务队列中；
所述图像处理模块根据所述待显示图像的分辨率，确定所述待显示图像对应的图像优化算法；所述图像处理模块利用对应的图像优化算法，对所述任务队列中所述待显示图像对应的任务对象进行超分辨率处理。
另外，所述图像处理模块确定所述待显示图像对应的任务对象完成超分辨率处理后，释放所述任务对象对应的内存空间，以便于进行内存管理。
另外，所述任务队列中的第一任务被分配至第一处理器执行超分辨率处理，所述任务队列中的第二任务被分配至第二处理器执行超分辨率处理，这样可以实现异构加速。
第二方面，本发明实施例提供了一种图像处理装置，所述图像处理装置具有实现上述方法实例中操作系统的图像处理模块行为的功能。所述功能可以通过硬件执行相应的软件 实现。所述硬件或软件包括一个或多个与上述功能相对应的模块。
在一种可能的设计中，所述图像处理装置包括接收单元和处理单元，其中：
接收单元，用于接收第一应用程序调用操作系统的图像处理模块的指令，所述指令中携带有待显示图像；
处理单元，用于对所述待显示图像进行图像优化处理，并显示图像优化处理后的图像。
在一种可能的设计中，所述处理单元具体用于：确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理。
在另一种可能的设计中，该图像处理装置还包括确定单元，用于根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限。
进一步地，所述确定单元具体用于：判断所述第一应用程序的标识是否存在于预设的白名单列表中；若存在，则确定所述第一应用程序具备图像优化的权限。
在一种可能的设计中，所述处理单元具体用于：
将所述待显示图像作为任务对象加入到任务队列中；
根据所述待显示图像的分辨率，确定所述待显示图像对应的图像优化算法；
利用对应的图像优化算法，对所述任务队列中所述待显示图像对应的任务对象进行超分辨率处理。
进一步地，所述处理单元还用于：确定所述待显示图像对应的任务对象完成超分辨率处理后，释放所述任务对象对应的内存空间。
另外，在一种可能的设计中，所述任务队列中的第一任务被分配至第一处理器执行超分辨率处理，所述任务队列中的第二任务被分配至第二处理器执行超分辨率处理，以便于加速进行图像处理。
在一种可能的设计中，该图像处理装置为操作系统的图像视图类，所述指令中还包括所述待显示图像的宽高；所述处理单元还用于：确定所述待显示图像的宽高满足设定条件，所述设定条件为所述终端设备的显示屏的高与所述待显示图像的高之间的差值小于第二阈值，和/或，所述终端设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值。
在一种可能的设计中，所述图像处理装置为所述安卓操作系统的框架层中的位图工厂类，那么述图像处理装置对所述待显示图像进行图像优化处理之前，解码所述待显示图像。
第三方面，本发明实施例提供了一种图像处理设备，所述图像处理设备包括存储器、显示器以及处理器；处理器，可以是一个中央处理单元(central processing unit，CPU)，或者为数字处理单元等等。处理器根据第一应用程序调用操作系统的图像处理模块的指令，执行图像优化功能。存储器，用于存储所述第一应用程序的指令、存储了操作系统的程序指令，以及处理器所执行的程序。
显示器，用于在第一应用程序的人机交互界面显示经过处理器图像优化处理后的图像。
具体的，所述处理器用于确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理。
在一种可能的设计中，所述第一应用程序的指令还包括所述第一应用程序的标识，所述处理器还用于：根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限。
在一种可能的设计中，所述处理器具体用于：判断所述第一应用程序的标识是否存在于预设的白名单列表中；若存在，则确定所述第一应用程序具备图像优化的权限。
在一种可能的设计中，所述处理器具体用于：将所述待显示图像作为任务对象加入到任务队列中；根据所述待显示图像的分辨率，确定所述待显示图像对应的图像优化算法；
利用对应的图像优化算法，对所述任务队列中所述待显示图像对应的任务对象进行超分辨率处理。
在一种可能的设计中，所述操作系统为安卓操作系统，所述图像处理模块为所述安卓操作系统的框架层中的图像视图类或者位图工厂类，所述处理器可以采用如下两种方式优化图像：
方式一，处理器确定所述待显示图像的宽高满足设定条件，对满足条件的图像进行超分辨率处理，其中，所述设定条件为所述图像处理设备的显示屏的高与所述待显示图像的高之间的差值小于第二阈值，和/或，所述图像处理设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值。
方式二，处理器解码所述待显示图像，再满足条件的图像进行超分辨率处理。
在一种可能的设计中，所述处理器还用于确定所述待显示图像对应的任务对象完成超分辨率处理后，释放所述任务对象对应的内存空间
在一种可能的设计中，所述任务队列中的第一任务被分配至第一处理器执行超分辨率处理，所述任务队列中的第二任务被分配至第二处理器执行超分辨率处理，这样可以实现异构加速。
第四方面，本申请实施例中还提供一种计算机存储介质，该存储介质中存储软件程序，该软件程序在被一个或多个处理器读取并执行时可实现第一方面的任意一种设计提供的方法。
第五方面，本申请实施例中还提供一种计算机程序产品，当所述计算机程序产品被计算机执行时，使所述计算机执如第一方面的任意一种设计提供的方法。
本申请实施例提供的方案中，改进的是操作系统中的图像处理模块，不需要对各个应用程序进行分别开发，所以具有具有较强的复用性，另外来自不同应用程序中多媒体文件在的图像信息在显示时实现自动图像优化，提高了用户体验。
附图说明
图1为本申请实施例提供的一种图像处理设备结构示意图；
图2为本申请实施例提供的一种手机的结构示意图；
图3为本申请实施例提供的一种图像处理的方法流程示意图；
图4为本申请实施例提供的一种撑屏判断方法示意图；
图5为本申请实施例提供的一种具有图像优化功能的安卓操作系统框架示意图；
图6为本申请实施例提供的一种集成图像优化功能的图像视图类的示意图；
图7为本申请实施例提供的一种集成图像优化功能的位图工厂类的示意图；
图8为本申请实施例提供的一种图像视图类的图像优化方法流程示意图一；
图8a为本申请实施例提供的一种图像视图类的图像优化方法流程示意图二；
图9为本申请实施例提供的一种位图工厂类的图像优化方法流程示意图；
图10为本申请实施例提供的一种图像优化队列的管控策略方法示意图；
图11为本申请提供的一种图像处理装置结构示意图；
图12为本申请提供的另一种图像处理设备结构示意图。
具体实施方式
下面将结合附图对本申请实施例作进一步地详细描述。
本发明实例提供的一种图像处理的方法，可以适用于如图1所示的图像处理设备，该图像处理设备包括应用层110、框架层120，其中，所述应用层110包括第一应用程序111，以及其它多个应用程序，所述框架层120中包含有图像处理模块122。其中，图像处理模块122接收第一应用程序111传输的图像，图像处理模块122用于图像进行图像优化处理，将图像处理模块122处理后的图像传输给第一应用程序111，以使第一应用程序111显示所述处理后的图像。可以理解地，框架层120中还包括存储模块，用于缓存待处理的图像以及处理后的图像以下述图像处理过程为例，说明图像处理模块的工作流程：
图像处理模块122接收到待显示图像后，首先获取所述待显示图像的分辨率，判断该分辨率是否低于低于阈值，若是，则对所述待显示图像进行超分辨率处理，并显示超分辨率处理后的图像。所谓超分辨率处理，指的是通过硬件或软件的方法提高原有图像的分辨率，通过一系列低分辨率的图像来得到一幅高分辨率的图像过程就是超分辨率重建。图像处理模块122将超分辨率处理后的图像返回至第一应用程序111进行界面显示。
本发明实例提供的一种图像处理的方法，也同样适用于如图2所示的手机，下述先简单介绍手机的具体结构组成。
参考图2所示，为本申请实施例应用的手机的硬件结构示意图。如图2所示，手机200包括显示设备210、处理器220以及存储器230。存储器230可用于存储软件程序以及数据，处理器220通过运行存储在存储器230的软件程序以及数据，从而执行手机200的各种功能应用以及数据处理。存储器230可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如图像采集功能等)等；存储数据区可存储根据手机200的使用所创建的数据(比如音频数据、电话本、图像等)等。此外，存储器230可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。处理器220是手机200的控制中心，利用各种接口和线路连接整个手机的各个部分，通过运行或执行存储在存储器230内的软件程序和/或数据，执行手机200的各种功能和处理数据，从而对手机进行整体监控。处理器220可以包括一个或多个通用处理器，还可包括一个或多个DSP(digital signal processor，数字信号处理器)，也可以包括一个或者多个ISP(image signal processor，图像信号处理器)，用于执行相关操作，以实现本申请实施例所提供的技术方案。
手机200中还包括用于拍摄图像或视频的摄像头260。摄像头260可以是普通摄像头，也可以是对焦摄像头。
手机200还可以包括输入设备240，用于接收输入的数字信息、字符信息或接触式触摸操作/非接触式手势，以及产生与手机200的用户设置以及功能控制有关的信号输入等。
显示设备210，包括的显示面板211，用于显示由用户输入的信息或提供给用户的信息以及手机200的各种菜单界面等，在本申请实施例中主要用于显示手机100中摄像头或者传感器获取的待检测图像。可选的，显示面板可以采用液晶显示器(liquid crystal display，LCD)或OLED(organic light-emitting diode，有机发光二极管)等形式来配置显示面板211。
除以上之外，手机200还可以包括用于给其他模块供电的电源250。手机200还可以包括一个或多个传感器270，例如图像传感器、红外传感器、激光传感器等。手机200还 可以包括无线射频(radio frequency，RF)电路280，用于与无线网络设备进行网络通信，还可以包括WiFi模块290，用于与其他设备进行WiFi通信，获取其他设备传输的图像或者数据等。
基于上述介绍，本申请实施例提供了一种图像处理的方法及图像处理设备，用以解决应用程序中的多媒体文件的图像显示不清晰的问题。其中，本申请所述方法和图像处理设备基于同一发明构思，由于所述方法和所述图像处理设备解决问题的原理相似，因此所述图像处理设备与方法的实施可以相互参见，重复之处不再赘述。
在本申请实施例中，主要是通过完善图像处理设备的操作系统的图像处理模块，增加图像优化的功能，以对应用层中不同应用程序中的待显示图像进图像行优化处理。也就是说，本申请实施例改进了操作系统中除了应用层之外的层(例如框架层)，增加图像优化程序并调用相关的硬件来实现优化处理，当应用程序中多媒体文件调用操作系统的图像处理模块显示图像时，会先经过图像处理模块的图像优化处理过程，最终显示的图像是经过优化的图像，这时分辨率已经被提高，因此清晰度也更佳，因为本申请实施例改进的是操作系统中的图像处理模块，不需要对各个应用程序进行分别开发，对于操作应用层中的应用程序的用户来说，整个图像优化过程用户是无感知的，所以说该方法在具有较强复用性的同时也更为自动化。
以下，对本申请中的部分用语进行解释说明，以便于本领域技术人员理解。
1)、本申请实施例涉及的应用程序，指的是具有可视的用户界面，可以和用户进行人机交互的软件，例如，短信应用、彩信应用、各种邮箱应用、微博、微信、腾讯聊天软件(QQ)、连我(Line)、照片分享(instagram)、钉钉、头条新闻、浏览器等。用户通过应用，可以将文字、语音、图像、视频文件以及其他各种文件等信息分享给其他联系人，或者获取到上述信息。
3)、本申请实施例涉及的图像处理设备，又可以称为用户设备(User Equipment，UE)为可以安装各种通信应用，或具有通信功能的设备。例如，智能手机、平板电脑、各类可穿戴设备、车载设备、计算机等。
4)、本申请实施例涉及的图像来源于多媒体文件，其中，多媒体文件为图像、图像集合、或由多帧图像组成的视频文件。
结合图1所述的图像处理设备的操作系统架构，在本申请如下实施例中，对图像处理方法的具体过程进行详细阐述，参阅图3所示，该方法的具体流程可以包括：
步骤201a：第一应用程序向操作系统的图像处理模块发送指令以调用该图像处理模块执行图像优化。
具体地，在检测到用户查看图像时，第一应用程序向该图像处理模块发送指令。比如，用户使用微信时收到好友发的图像，微信的对话界面上以缩略图进行显示收到该图像，用户点击该缩略图以查看该图像的大图时，微信调用该图像处理模块。
步骤202a：图像处理模块判断所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理，并显示超分辨率处理后的图像。
在步骤202a中，因为指令中携带有待显示图像，以及所述待显示图像的分辨率，所以图像处理模块可以先获取待显示图像的分辨率，然后对分辨率进行判断，若判断低于第一阈值，再启动超分辨率处理，否则，则不进行超分辨率处理。此处图像优化处理以超分辨率处理为例进行说明。
需要说明的是，除了超分辨处理这种图像优化方法，还可以采用其它的图像优化方法对图像进行优化，例如调节图像的亮度、饱和度、颜色，或者美化人脸等等。这些图像优化方法也可以一起使用，也可以根据不同检测条件来触发不同的图像优化方法，例如识别当前处于夜间阅读模式，则降低图像的亮度；或者识别出图像中有人脸，则执行人脸美颜的图像优化方法等等。
考虑到图像优化处理的意义在于使图像的质量得到改善，这在不同的应用中对用户的影响和意义是不同的。比如说，微博用户通常浏览短视频或者照片，所以微博这一类应用程序对图像分辨率要求较高，再比如说邮箱应用，邮件多为文字信息，邮件中通常仅插入少量图像或者没有图像信息，所以这一类应用程序对图像分辨率要求较低。如果按照传统的方法基于频宽考量，丢失部分像素，降低图像分辨率，就会严重影响用户体验，因此在一种可能的设计中，图像处理模块会进一步根据指令中携带的第一应用程序的标识，确定第一应用程序是否具有图像优化的权限，对具有图像优化权限的应用程序执行图像优化。
也就是说，图像处理模块中预先保存有具有图像优化权限的白名单，在该白名单中包含有应用层应用程序的标识，如果图像处理模块判断指令中获取的应用程序的标识在白名单中，显然就具有图像优化的权限，可以进一步触发超分辨率处理，否则，则不进行超分辨率处理。或者是，图像处理模块对应用程序的标识进行认证，若认证通过，则证明该应用程序具有图像优化的权限，可以进一步触发超分辨率处理，否则，则不进行超分辨率处理。
在另一种可能的设计中，为了提高图像处理的效率，在执行超分辨率处理之前，还可以利用其它条件进一步判断待显示的图像是否需要图像优化。比如说，图像处理模块获取到图像之后，判断该图像是否已经经过超分辨率处理，若已经处理过，则不再对该图像进行超分辨率处理。
考虑应用程序中有较多待显示的图像时，图像处理模块在性能和内存上会出现较大压力，因此这时可以过滤部分图像，比如对尺寸符合预设条件的图像进行处理。例如，图像处理模块判断待显示图像的图像尺寸是否接近全屏，只有在图像尺寸接近屏幕尺寸的情况下才会做超分辨率处理。如图4所示，待显示图像的宽与屏幕宽相一致，这一待显示图像满足执行超分辨率处理的条件。因为在实际操作中，部分应用的图像宽度会略小于屏宽，所以当待显示屏的高与所述待显示图像的高之间的差值小于第二阈值，或者，所述图像处理设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值，这两个条件任意一个满足时，则对该待显示图像执行超分辨率处理。所述第二阈值和所述第三阈值可以相同比如说，所述待显示图像的宽度满足屏宽的[0.95，1.0]倍，或者是待显示图像高度满足高度的[0.95，1.0]倍时，就可以继续执行超分辨率处理。
目前图像处理设备较为常用的操作系统为安卓操作系统，本申请实施例进一步结合安卓操作系统的系统架构，对上述图像处理方法进行说明。安卓操作系统从高层到低层通常依次包括：应用层、框架层、运行时、核心类库、硬件抽象层、Linux内核层。本申请实施例的图像处理模块属于框架层中功能模块。
如图5所示，具有图像优化功能的安卓操作系统架构包括应用层301、框架层302、硬件抽象层303、内核芯片304；其中，框架层中的图像处理模块中场景识别模块305，其中设置有各种条件，用于是否需要进行图像优化，例如判断调用图像处理模块的第一应用程序是否在图像优化的白名单中，待显示的图像是否要撑屏显示，如待显示的图像的宽度 或高度是否接近屏幕的宽度或高度；待显示的图像的分辨率是否满足分辨率阈值，待显示图像中是否有人脸等等。图像视图类(ImageView)306和位图工厂类(BitmapFactory)307用于对待显示图像执行图像优化，具体可以管理图像优化的任务队列、内存的调用，通过调用HiAI服务中的图像优化算法对图像执行优化处理。框架层中的HiAI服务平台308中包含有各种图像优化算法，例如DNN硬算法和Raisr软算法等，其中DNN硬算法为超分辨率图像硬算法，透过与神经网络处理器(IPU)交互，性能与效果得到提升；类Raisr软算法为超分图像软算法，主要针对的是没有IPU硬件的中低端图像处理设备。除此之外，框架层中还包括存储模块309，作用在图1中已经说明，在此不再赘述。
因为应用层的应用程序不同，部分应用程序会调用图像视图类(ImageView)306绘制图像，但是部分应用不会调用ImageView，而是调用位图工厂类(BitmapFactory)307，所以图像处理模块在图像优化的具体实现主要有以下两种方式。
方式一，因为应用层中的大部分应用程序会调用框架层中的图像视图类306去显示一张图像，所以可以在图像视图类上扩容图像优化功能，这样，应用层中的应用程序可以调用图像视图类，触发上述图像处理方法。如图6所示，应用层401中的不同的应用程序显示图像时会分别创建ImageView类，例如第一应用程序的ImageView1、第二应用程序的ImageView2、第三应用程序的ImageView3，进而触发框架层402中的图像处理模块404调用异构处理器403执行图像优化，图像处理模块404中的图像视图类405调用图像优化算法以执行图像优化处理，图像处理模块在执行图像优化处理过程中会通过多个处理器进行异构加速，从而加快图像优化的进程。
方式二，当应用层401中的应用程序自身已具有图像视图类405时，就不再调用框架层402中的ImageView，但是仍然会调用框架层的位图工厂类(BitmapFactory)406，对待显示图像进行解码，得到解码后图像。所以可以在BitmapFactory上扩展图像优化功能，这样，应用层中的应用程序一旦调用位图工厂类，就会触发上述图像处理方法。如图7所示，步骤501，应用层中的第一应用程序在进行图像显示过程中会调用操作系统的框架层中的BitmapFactory对多媒体文件进行解码；步骤502解码之后的图像经过场景识别模块的一系列条件判断，例如分辨率大小的识别等，满足条件的图像加入到图像优化队列中；步骤503，考虑部分图像，例如前几帧图像或者最后几帧图像通常包含的信息不重要，可以直接对其放弃图像优化；步骤504，过滤之后，剩余的图像则依次进行超分辨处理。
对于方式一，具体来说，在Android操作系统中，最为常用的图像显示模块就是ImageView(图像视图模块)。通过分析Android ImageView源码，不论各个应用程序的ImageView以何种方式创建，最后均会统一走到initImageView()(初始化图像视图接口)中，执行一般的数据初始化。因此本申请实施例在此接口中增加与图像优化权限相关的条件判断过程，以判断当前应用程序是否具有图像优化的权限，以下以超分辨率处理为例进行说明，如图8所示：
步骤601，应用程序创建ImageView，调用initImageView()接口进行数据初始化。
步骤602，initImageView()中设置有多个条件，用于判断当前应用程序是否具有图像优化能力的权限，条件如下：
(1)、预设的图像优化能力开关是否开启；(2)、手机型号是否支持图像优化；(3)、应用程序的包名是否在白名单中。
其中，若满足上述三个条件，则可以确定当前该应用程序创建的ImageView具有图像 优化权限，可以继续后面的判断逻辑，即执行步骤603；否则的话就不再继续图像优化处理，即执行步骤605绘制图像进行显示。
其中，因为图像优化处理的意义在于使图像较差的质量得到改善，这在不同的应用中对用户的影响和意义是不同的，需要根据重要性和功能的价值区分，所以当前图像优化处理只针对特定的部分应用开启。另外，因为在Android应用中，package(包)名和应用程序通常是一一对应的，所以通过调用getPackageName()可以获取当前应用程序的包名。若应用程序的包名在白名单中，则确定ImageView允许进行图像优化，继续后面的场景识别逻辑；如果不满足则ImageView后续行为和无图像优化特性时一致。
步骤603，获取图像尺寸。为了继续根据图像尺寸判断图像是否需要图像优化，Android View绘制系统的中的Drawable的getIntrinsicWidth()和getIntrinsicHeight()方法可以获取一种图像的“宽高”，所以可以通过这些方法测量待显示图像的宽高，例如，以BitmapDrawable为例，实际上返回的就是所持有的Bitmap的宽与高。
步骤604，当待显示屏的高与所述待显示图像的高之间的差值小于第一阈值，或者，所述图像处理设备的显示屏的宽与所述待显示图像的宽之间的差值小于第二阈值，这两个条件任意一个满足时，则对该待显示图像执行超分辨率处理，即执行步骤605，否则的话执行步骤609。比如说，所述待显示图像的宽度满足屏宽的[0.95，1.0]倍，或者是待显示图像高度满足高度的[0.95，1.0]倍时，就可以确定待显示图像尺寸足够大，因此有必要对该图像仅图像优化。
步骤605，ImageView在View的标准方法onDraw()中绘制被设置好的Drawable数据，通过Drawable的标准方法draw()绘制内容，例如BitmapDrawable就是将一张Bitmap的内容绘制到ImageView上，而ColorDrawable就是将颜色涂至ImageView等。
步骤606，判断超分辨率优化算法器件状态是否正常，因为当图像批量显示时，会导致分辨率优化算法器件任务队列拥塞，所以需要确定当前超分辨率优化算法器件状态正常，是的话，则执行步骤507，否则执行步骤509。
步骤607，出于性能、内存及效果的考虑，图像优化处理只针对一定范围分辨率的图像才进行。分辨率大小设置规则针对不同应用程序的要求不同，例如微博对图像的分辨率要求高于其它应用。只有在该应用程序的分辨率低于对应的大小设置规则时，才出发图像优化，因此需要继续判断分辨率是否满足设定条件，是的话，则执行步骤608，否则执行步骤609。
步骤608，对满足上述条件的待显示图像执行超分辨率处理，其中超分辨率所执行的超分辨率算法与该应用程序的图像分辨率大小设置相对应。然后再返回执行步骤605，对优化后图像进行重新绘制和显示，需要说明的是，这时执行完步骤605之后就结束了图像优化流程。
步骤609，Imageview不对图像执行优化，也需要按照在View的标准方法onDraw()中绘制被设置好的Drawable数据，通过Drawable的标准方法draw()绘制内容，即如步骤505所示。
需要说明的是，上述判断顺序，并不存在严格的先后顺序，一般地，本申请实施例先判断一下开关是否开启，再测量图像的尺寸是否足够大，进而判断分辨率是否低于设定标准。
如图8a示出了使用ImageView类实现图像优化的框架和流程，操作系统中包括应用 层A10，框架层A20,HAL层/硬件A30。应用层A10中具有使用Imageview的应用程序A11。框架层A20中的ImageViewA21中的触发图像优化模块用于判断图像优化的时机；场景识别A211用于判断哪些图像需要进行图像优化处理，比如图像尺寸是否符合预设条件，图像分辨率是否复核分辨率阈值，图像中是否有人脸等等，具体地可以包括上述步骤502-504,506-507，实际可根据图像优化的目的设置不同的条件来判断哪些图像需要进行图像优化处理；优化任务分配和内容管理模块A212用于管理图像优化处理的多线程任务以及内容，具体可以参照图10的方式进行管理；优化算法A22提供多种图像优化模型A221，即图像优化算法；异构优化处理模块A222用于调用不同的处理器A32加速图像优化处理。HAL层/硬件用于实现显示，处理器的硬件驱动。应用A11将图像发给ImageViewA21,ImageViewA21中的触发图像优化模块确认出图像优化处理的时机，场景识别模块A211判断所述图像是否需要进行优化，对需要优化的图像放入优化任务的队列中，由优化任务分配及内存管理模块A212管理优化任务，调用优化算法A22对优化任务队里中的图像进行优化处理，实现优化处理时，通过异构优化处理模块A222调用不同的处理器A32来运行优化算法，使用优化后的图像进行更新画面，在显示器A31上显示。
对于方式二，具体来说，在BitmapFactory的图像优化处理提供了预测模型，预测显示界面内图像需要进行超分辨率处理，过滤掉显示界面外的图像，这样确保符合条件的图像可以进行图像优化处理，满足性能端到端的要求，具体步骤如图9所示。
步骤701，应用程序的图像通过调用BitmapFactory的BitmapFactory.decodeFile()去解码待显示图像，输出source bitmap(源位图)对象，例如将待显示的JPEG格式的图像解码为位图。
步骤702、BitmapFactory调用框架层的BitmapFactory类，代入source bitmap对象。
步骤703，图像处理模块内部创建/管理Destination Bitmap(目标位图)，把图像的source bitmap和Destination Bitmap当做参数，调用HisiDDK的Native API执行数据处理。
步骤704，HisiDDK内部透过Binder接口，进程间通信(inter-process communication，IPC)传输sourceBitmap和Destination Bitmap到HIAI服务进程，调用IPU做超分辨率算法处理。
步骤705，HIAI服务进程处理完的超分图像存储在Destination Bitmap，在透过Binder接口异步调回图库进程的Binder进程。
步骤706，Binder收到图像处理模块处理完成后，再把Destination Bitmap传送到视图显示接口进行重新绘制。
步骤707，经过重新绘制后的图像在应用程序的显示界面进行显示。
因为上述图像优化处理过程中，涉及到Bitmap的创建，在多数应用场景下，Bitmap是最有可能导致内存异常的因素。Bitmap遵循Java GC机制，当没有强引用指向Bitmap对象时，Bitmap对象就会被释放，若是存在不合理的指针持有Bitmap，就会导致内存占用或者泄露。因此本申请实施例明确给出了图像优化处理中Bitmap的生命周期，以便于进行内存管理。同时，目前图像优化算法众多，运行计算要求高，可以进一步管理GPU、FPGA等不同的异构处理器，对其进行任务分配，以加速图像优化的处理性能。
举例来说，假设第一应用程序是微信，用户A通过微信向用户B发了一个自拍图，用户B的微信聊天框中会显示出该自拍图的缩略图，当用户B点击查看该自拍图时，用户B手机中微信这一应用程序会触发操作系统的图像处理模块进行图像显示流程，所以图像处理模块的ImageView或者BitmapFactory会对该自拍图进行图像优化，进而用户B查看到的自拍图的分辨率会变高，清晰度更佳。
综上，上述图像处理方法对于第三方应用而言，各个第三方应用并不需要重复开发图像优化功能，通过直接调用操作系统上的已有接口，就可以触发超分辨率处理，该方法较为通用。
另外，本申请实施例进一步对图像处理模块的图像优化队列增加管控策略，如图10所示，步骤801，管控策略主要是任务队列接收到图像处理模块发起的超分辨率任务请求，就会将这一任务加入到队列；步骤802，若接收到图像处理模块发起的取消超分辨率任务请求，则将这一任务从队列中删除；步骤803，另外任务队列检测到超分辨率状态机就绪状态时，就会从任务队列中取出任务发送至DDK处理。
同时，考虑到在完整的图像优化处理过程中，涉及到Bitmap的创建，在多数应用场景下，Bitmap是最有可能导致内存异常的因素。Bitmap遵循Java GC机制，当没有强引用指向Bitmap对象时，Bitmap对象就会被释放，若是存在不合理的指针持有Bitmap，就会导致内存占用/泄露。因此本申请实施例明确了图像优化处理中Bitmap的生命周期，以便于管理内存。
基于与方法实施例的同一发明构思，本申请实施例提供一种图像处理装置900，所述图像处理装置属于图像处理设备的操作系统的框架层，具体用于实现图3所述的实施例描述的方法，该装置的结构如图11所示，包括接收单元901和处理单元902，其中：
接收单元901，用于接收第一应用程序调用操作系统的图像处理模块的指令，所述指令中携带有待显示图像；
处理单元902，用于对所述待显示图像进行图像优化处理，并显示图像优化处理后的图像。
在一种可能的设计中，所述处理单元902具体用于：确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理。
在另一种可能的设计中，该图像处理装置还包括确定单元903，用于根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限。
进一步地，所述确定单元903具体用于：判断所述第一应用程序的标识是否存在于预设的白名单列表中；若存在，则确定所述第一应用程序具备图像优化的权限。
在一种可能的设计中，所述处理单元902具体用于：
将所述待显示图像作为任务对象加入到任务队列中；
根据所述待显示图像的分辨率，确定所述待显示图像对应的图像优化算法；
利用对应的图像优化算法，对所述任务队列中所述待显示图像对应的任务对象进行超分辨率处理。
进一步地，所述处理单元902还用于：确定所述待显示图像对应的任务对象完成超分辨率处理后，释放所述任务对象对应的内存空间。
另外，在一种可能的设计中，所述任务队列中的第一任务被分配至第一处理器执行超分辨率处理，所述任务队列中的第二任务被分配至第二处理器执行超分辨率处理，以便于加速进行图像处理。
在一种可能的设计中，假设该图像处理装置为操作系统的图像视图类，所述指令中还包括所述待显示图像的宽高；所述处理单元902还用于：确定所述待显示图像的宽高满足设定条件，所述设定条件为所述终端设备的显示屏的高与所述待显示图像的高之间的差值小于第二阈值，和/或，所述终端设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值。
在一种可能的设计中，所述图像处理装置为所述安卓操作系统的框架层中的位图工厂类，那么述图像处理装置对所述待显示图像进行图像优化处理之前，解码所述待显示图像。
基于以上实施例，本申请实施例还提供了一种图像处理设备，所述图像处理设备用于实现图3所述的实施例描述的方法，参阅如图12所示，所述设备包括处理器1001、存储器1002以及显示器1003。
处理器1001，可以是一个中央处理单元(central processing unit，CPU)，或者为数字处理单元等等优化功能。
存储器1002，用于存储所述第一应用程序的指令、存储了操作系统的程序指令。
显示器1003，用于在第一应用程序的人机交互界面显示经过处理器1001图像优化处理后的图像。
本申请实施例中不限定上述处理器1001以及存储器1002之间的具体连接介质。本申请实施例在图12中以存储器1002、处理器1001以及显示器1003之间通过总线1004连接，总线在图12中以粗线表示，其它部件之间的连接方式，仅是进行示意性说明，并不引以为限。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图12中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
存储器1002可以是易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储器1002也可以是非易失性存储器(non-volatile memory)，例如只读存储器，快闪存储器(flash memory)，硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)、或者存储器1002是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器1003可以是上述存储器的组合。
处理器1001执行存储器1002中的程序指令，用于实现如图3所示的图像处理的方法，包括：第一应用程序向操作系统中的图像处理模块发送指令以调用该图像处理模块执行图像优化算法，所述图像处理模块对所述待显示图像进行图像优化处理返回给第一应用程序进行显示。
所述处理器1001采用的图像优化处理方法可通过如下方式：确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理。
在一种可能的实现方式中，所述第一应用程序的指令还包括所述第一应用程序的标识，所述处理器1001还用于：根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限。
本申请实施例还提供了一种计算机可读存储介质，存储为执行上述处理器所需执行的计算机软件指令，其包含用于执行上述处理器所需执行的程序。
本申请实施例还提供了一种计算机程序产品，当所述计算机程序产品被计算机执行时，使所述计算机执如上述图像处理方法。
综上所述，在本申请实施例中，主要是通过完善图像处理设备的操作系统的图像处理模块，增加图像优化的功能，以对应用层中不同应用程序中的待显示图像进图像行优化处理。也就是说当应用程序中多媒体文件调用操作系统的图像处理模块的接口显示图像时，会先经过图像处理模块的图像优化处理过程，最终显示的图像是经过优化的图像，这时分辨率已经被提高，因此清晰度也更佳，因为本申请实施例改进的是操作系统中的图像处理模块，不需要对各个应用程序进行分别开发，对于操作应用层中的应用程序的用户来说，整个图像优化过程用户是无感知的，所以说该方法在具有较强复用性的同时也更为自动化。
本所属领域的技术人员可以清楚地了解到，本发明提供的各实施例的描述可以相互参照，为描述的方便和简洁，关于本申请实施例提供的各装置、设备的功能以及执行的步骤可以参照本发明方法实施例的相关描述，在此不做赘述。
本领域技术人员还可以了解到本申请实施例列出的各种说明性逻辑块(illustrative logical block)和步骤(step)可以通过电子硬件、电脑软件，或两者的结合进行实现。为清楚展示硬件和软件的可替换性(interchangeability)，上述的各种说明性部件(illustrative components)和步骤已经通用地描述了它们的功能。这样的功能是通过硬件还是软件来实现取决于特定的应用和整个系统的设计要求。本领域技术人员可以对于每种特定的应用，可以使用各种方法实现所述的功能，但这种实现不应被理解为超出本申请实施例保护的范围。
本申请实施例中所描述的各种说明性的逻辑块，模块和电路可以通过通用处理单元，数字信号处理单元，专用集成电路(ASIC)，现场可编程门阵列(FPGA)或其它可编程逻辑装置，离散门或晶体管逻辑，离散硬件部件，或上述任何组合的设计来实现或操作所描述的功能。通用处理单元可以为微处理单元，可选地，该通用处理单元也可以为任何传统的处理单元、控制器、微控制器或状态机。处理单元也可以通过计算装置的组合来实现，例如数字信号处理单元和微处理单元，多个微处理单元，一个或多个微处理单元联合一个数字信号处理单元核，或任何其它类似的配置来实现。
本申请实施例中所描述的方法或算法的步骤可以直接嵌入硬件、处理单元执行的软件模块、或者这两者的结合。软件模块可以存储于RAM存储器、闪存、ROM存储器、EPROM存储器、EEPROM存储器、寄存器、硬盘、可移动磁盘、CD-ROM或本领域中其它任意形式的存储媒介中。示例性地，存储媒介可以与处理单元连接，以使得处理单元可以从存储媒介中读取信息，并可以向存储媒介存写信息。可选地，存储媒介还可以集成到处理单元中。处理单元和存储媒介可以配置于ASIC中，ASIC可以配置于用户终端中。可选地，处理单元和存储媒介也可以配置于用户终端中的不同的部件中。
在一个或多个示例性的设计中，本申请实施例所描述的上述功能可以在硬件、软件、固件或这三者的任意组合来实现。如果在软件中实现，这些功能可以存储与电脑可读的媒介上，或以一个或多个指令或代码形式传输于电脑可读的媒介上。电脑可读媒介包括电脑存储媒介和便于使得让电脑程序从一个地方转移到其它地方的通信媒介。存储媒介可以是任何通用或特殊电脑可以接入访问的可用媒体。例如，这样的电脑可读媒体可以包括但不限于RAM、ROM、EEPROM、CD-ROM或其它光盘存储、磁盘存储或其它磁性存储装置，或其它任何可以用于承载或存储以指令或数据结构和其它可被通用或特殊电脑、或通用或特殊处理单元读取形式的程序代码的媒介。此外，任何连接都可以被适当地定义为电脑可读媒介，例如，如果软件是从一个网站站点、服务器或其它远程资源通过一个同轴电缆、光纤电脑、双绞线、数字用户线(DSL)或以例如红外、无线和微波等无线方式传输的也被包含在所定义的电脑可读媒介中。所述的碟片(disk)和磁盘(disc)包括压缩磁盘、镭射盘、光盘、DVD、软盘和蓝光光盘，磁盘通常以磁性复制数据，而碟片通常以激光进行光学复制数据。上述的组合也可以包含在电脑可读媒介中。
本申请的上述描述可以使得本领域技术任何可以利用或实现本申请的内容，任何基于所公开内容的修改都应该被认为是本领域显而易见的，本申请所描述的基本原则可以应用到其它变形中而不偏离本申请的发明本质和范围。因此，本申请所公开的内容不仅仅局限 于所描述的实施例和设计，还可以扩展到与本申请原则和所公开的新特征一致的最大范围。

Claims (29)

  1. 一种图像处理方法，其特征在于，所述方法适用于具备操作系统的图像处理设备，包括：
    操作系统的图像处理模块接收第一应用程序调用操作系统的图像处理模块的指令，所述指令中携带有待显示图像；
    所述图像处理模块对所述待显示图像进行图像优化处理，并显示图像优化处理后的图像。
  2. 如权利要求1所述的图像处理方法，其特征在于，所述图像处理模块对所述待显示图像进行图像优化处理，包括：
    所述图像处理模块确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理。
  3. 如权利要求2所述的图像处理方法，其特征在于，所述指令还包括所述第一应用程序的标识，所述图像处理模块对所述待显示图像进行超分辨率处理之前，还包括：
    所述图像处理模块根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限。
  4. 如权利要求3所述的图像处理方法，其特征在于，所述图像显示模块根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限，包括：
    所述图像处理模块判断所述第一应用程序的标识是否存在于预设的白名单列表中；
    若存在，则所述图像处理模块确定所述第一应用程序具备图像优化的权限。
  5. 如权利要求2-4任一项所述的图像处理方法，其特征在于，所述图像处理模块对所述待显示图像进行超分辨率处理，包括：
    所述图像处理模块将所述待显示图像作为任务对象加入到任务队列中；
    所述图像处理模块根据所述待显示图像的分辨率，确定所述待显示图像对应的图像优化算法；
    所述图像处理模块利用对应的图像优化算法，对所述任务队列中所述待显示图像对应的任务对象进行超分辨率处理。
  6. 如权利要求5所述的图像处理方法，其特征在于，还包括：
    所述图像处理模块确定所述待显示图像对应的任务对象完成超分辨率处理后，释放所述任务对象对应的内存空间。
  7. 如权利要求5所述的图像处理方法，其特征在于，所述任务队列中的第一任务被分配至第一处理器执行超分辨率处理，所述任务队列中的第二任务被分配至第二处理器执行超分辨率处理。
  8. 如权利要求1至7任一项所述的图像处理方法，其特征在于，所述操作系统为安卓操作系统，所述图像处理模块为所述安卓操作系统的框架层中的图像视图类，所述指令中还包括所述待显示图像的宽高；
    所述图像显示模块接收第一应用程序调用操作系统的图像处理模块的指令之后，所述图像处理模块对所述待显示图像进行图像优化处理之前，还包括：
    所述图像视图类确定所述待显示图像的宽高满足设定条件，所述设定条件为所述图像处理设备的显示屏的高与所述待显示图像的高之间的差值小于第二阈值，和/或，所述图像 处理设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值。
  9. 如权利要求1至7任一项所述的图像处理方法，其特征在于，所述操作系统为安卓操作系统，所述图像处理模块为所述安卓操作系统的框架层中的位图工厂类；
    所述图像处理模块接收第一应用程序调用操作系统的图像处理模块的指令之后，所述图像处理模块对所述待显示图像进行图像优化处理之前，还包括：
    所述位图工厂类解码所述待显示图像。
  10. 一种图像处理设备，其特征在于，所述图像处理设备包括存储器、处理器以及显示器；
    所述存储器，用于存储所述第一应用程序的指令、所述操作系统的程序指令以及处理器执行的程序；
    所述处理器，用于根据第一应用程序调用操作系统的图像处理模块的指令，对所述待显示图像进行图像优化处理；
    所述显示器，用于显示图像优化处理后的图像。
  11. 如权利要求10所述的图像处理设备，其特征在于，所述处理器具体用于：
    确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理。
  12. 如权利要求11所述的图像处理设备，其特征在于，所述第一应用程序的指令还包括所述第一应用程序的标识，所述处理器还用于：
    根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限。
  13. 如权利要求12所述的图像处理设备，其特征在于，所述处理器具体用于：
    判断所述第一应用程序的标识是否存在于预设的白名单列表中；
    若存在，则确定所述第一应用程序具备图像优化的权限。
  14. 如权利要求10所述的图像处理设备，其特征在于，所述处理器具体用于：
    将所述待显示图像作为任务对象加入到任务队列中；
    根据所述待显示图像的分辨率，确定所述待显示图像对应的图像优化算法；
    利用对应的图像优化算法，对所述任务队列中所述待显示图像对应的任务对象进行超分辨率处理。
  15. 如权利要求14所述的图像处理设备，其特征在于，所述处理器还用于：
    确定所述待显示图像对应的任务对象完成超分辨率处理后，释放所述任务对象对应的内存空间。
  16. 如权利要求14所述的图像处理设备，其特征在于，所述任务队列中的第一任务被分配至第一处理器执行超分辨率处理，所述任务队列中的第二任务被分配至第二处理器执行超分辨率处理。
  17. 如权利要求10至16任一项所述的图像处理设备，其特征在于，所述操作系统为安卓操作系统，所述图像处理模块为所述安卓操作系统的框架层中的图像视图类，所述第一应用程序的指令中还包括所述待显示图像的宽高；
    所述处理器还用于：确定所述待显示图像的宽高满足设定条件，所述设定条件为所述图像处理设备的显示屏的高与所述待显示图像的高之间的差值小于第二阈值，和/或，所述图像处理设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值。
  18. 如权利要求10至16任一项所述的图像处理设备，其特征在于，所述操作系统为 安卓操作系统，所述图像处理模块为所述安卓操作系统的框架层中的位图工厂类；
    所述处理器还用于：解码所述待显示图像。
  19. 一种图像处理装置，其特征在于，所述图像处理装置属于图像处理设备的操作系统的框架层，所述操作系统的应用层中包括第一应用程序，所述图像处理装置包括：
    接收单元，用于接收第一应用程序调用操作系统的图像处理模块的指令，所述指令中携带有待显示图像；
    处理单元，用于对所述待显示图像进行图像优化处理，并显示图像优化处理后的图像。
  20. 如权利要求19所述的图像处理装置，其特征在于，所述处理单元具体用于：确定所述待显示图像的分辨率低于第一阈值时，对所述待显示图像进行超分辨率处理。
  21. 如权利要求20所述的图像处理装置，其特征在于，还包括：
    确定单元，用于根据所述第一应用程序的标识，确定所述第一应用程序具有超分辨率处理的权限。
  22. 如权利要求21所述的图像处理装置，其特征在于，所述确定单元具体用于：
    判断所述第一应用程序的标识是否存在于预设的白名单列表中；若存在，则确定所述第一应用程序具备图像优化的权限。
  23. 如权利要求20所述的图像处理装置，其特征在于，所述处理单元具体用于：
    将所述待显示图像作为任务对象加入到任务队列中；
    根据所述待显示图像的分辨率，确定所述待显示图像对应的图像优化算法；
    利用对应的图像优化算法，对所述任务队列中所述待显示图像对应的任务对象进行超分辨率处理。
  24. 如权利要求23所述的图像处理装置，其特征在于，所述处理单元还用于：
    确定所述待显示图像对应的任务对象完成超分辨率处理后，释放所述任务对象对应的内存空间。
  25. 如权利要求23所述的图像处理装置，其特征在于，所述任务队列中的第一任务被分配至第一处理器执行超分辨率处理，所述任务队列中的第二任务被分配至第二处理器执行超分辨率处理。
  26. 如权利要求19至25任一项所述的图像处理装置，其特征在于，所述指令中还包括所述待显示图像的宽高；
    所述处理单元还用于：确定所述待显示图像的宽高满足设定条件，所述设定条件为所述终端设备的显示屏的高与所述待显示图像的高之间的差值小于第二阈值，和/或，所述终端设备的显示屏的宽与所述待显示图像的宽之间的差值小于第三阈值。
  27. 如权利要求19至25任一项所述的图像处理装置，其特征在于，所述处理单元还用于：解码所述待显示图像。
  28. 一种计算机存储介质，其特征在于，所述计算机存储介质存储有计算机可执行指令，所述计算机可执行指令用于使计算机执行如权利要求1～9任一项所述的方法。
  29. 一种计算机程序产品，其特征在于，当所述计算机程序产品被计算机执行时，使所述计算机执如行权利要求1～9任一项所述的方法。
PCT/CN2017/106197 2017-10-09 2017-10-13 一种图像处理方法及设备 WO2019071618A1 (zh)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US16/754,567 US11132766B2 (en) 2017-10-09 2017-10-13 Image processing method and device using multiple layers of an operating system
CN201780003404.8A CN108496198B (zh) 2017-10-09 2017-10-13 一种图像处理方法及设备
EP17928530.9A EP3671627A4 (en) 2017-10-09 2017-10-13 IMAGE PROCESSING METHOD AND DEVICE
AU2017435234A AU2017435234B2 (en) 2017-10-09 2017-10-13 Image processing method and device
US17/460,726 US20220058772A1 (en) 2017-10-09 2021-08-30 Image Processing Method and Device
AU2021229220A AU2021229220B2 (en) 2017-10-09 2021-09-09 Image processing method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710931459.7 2017-10-09
CN201710931459 2017-10-09

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US16/754,567 A-371-Of-International US11132766B2 (en) 2017-10-09 2017-10-13 Image processing method and device using multiple layers of an operating system
US17/460,726 Continuation US20220058772A1 (en) 2017-10-09 2021-08-30 Image Processing Method and Device

Publications (1)

Publication Number Publication Date
WO2019071618A1 true WO2019071618A1 (zh) 2019-04-18

Family

ID=66101214

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/106197 WO2019071618A1 (zh) 2017-10-09 2017-10-13 一种图像处理方法及设备

Country Status (5)

Country Link
US (2) US11132766B2 (zh)
EP (1) EP3671627A4 (zh)
CN (2) CN113837939A (zh)
AU (2) AU2017435234B2 (zh)
WO (1) WO2019071618A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113837939A (zh) * 2017-10-09 2021-12-24 华为技术有限公司 一种图像处理方法及设备
CN109522254B (zh) * 2017-10-30 2022-04-12 上海寒武纪信息科技有限公司 运算装置及方法
CN107895345B (zh) * 2017-11-29 2020-05-26 浙江大华技术股份有限公司 一种提高人脸图像分辨率的方法和装置
US20210056220A1 (en) * 2019-08-22 2021-02-25 Mediatek Inc. Method for improving confidentiality protection of neural network model
CN115482143B (zh) * 2021-06-15 2023-12-19 荣耀终端有限公司 应用的图像数据调用方法、系统、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102722875A (zh) * 2012-05-29 2012-10-10 杭州电子科技大学 一种基于视觉注意和质量可变的图像超分辨率重建方法
CN104702846A (zh) * 2015-03-20 2015-06-10 惠州Tcl移动通信有限公司 移动终端相机预览图像处理方法及系统
CN106603885A (zh) * 2015-10-20 2017-04-26 腾讯科技（深圳）有限公司 视频图像处理方法和装置
CN106919401A (zh) * 2017-03-10 2017-07-04 广东欧珀移动通信有限公司 一种动态调节分辨率的方法、装置及移动终端

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8581906B2 (en) * 2008-08-04 2013-11-12 Kabushiki Kaisha Toshiba Image processing apparatus and image processing method
JP5206218B2 (ja) * 2008-08-20 2013-06-12 富士通株式会社 指紋画像取得装置、指紋認証装置、指紋画像取得方法及び指紋認証方法
US8369653B1 (en) * 2009-02-27 2013-02-05 Adobe Systems Incorporated System and method for image upsampling using natural image statistics of first and second derivatives
US8698845B2 (en) 2010-01-06 2014-04-15 Apple Inc. Device, method, and graphical user interface with interactive popup views
CN103916550A (zh) 2012-12-30 2014-07-09 北京新媒传信科技有限公司 一种安卓平台中的图片处理方法和装置
CN103902318A (zh) 2012-12-30 2014-07-02 青岛海尔软件有限公司 基于Android平台的图片加载方法
CN103761471A (zh) 2014-02-21 2014-04-30 北京奇虎科技有限公司 基于智能终端设备安装应用程序的方法与装置
JP6567511B2 (ja) * 2014-05-27 2019-08-28 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 空気調和装置が実行するセンサの制御方法
CN110888521B (zh) 2015-07-16 2023-07-18 华为技术有限公司 一种移动终端及其运行应用程序的方法
CN105447820B (zh) * 2015-12-22 2018-10-23 厦门美图移动科技有限公司 一种图片处理方法、装置和移动终端
CN105808356A (zh) 2016-03-11 2016-07-27 广州市久邦数码科技有限公司 一种基于安卓系统的Bitmap重复利用方法及其系统
CN107615770B (zh) 2016-03-31 2020-02-21 华为技术有限公司 应用程序的处理方法及终端设备
CN106408632B (zh) * 2016-09-08 2019-10-25 广州阿里巴巴文学信息技术有限公司 一种基于属性动画的色调变化动画实现方法及装置
CN107025629B (zh) 2017-04-27 2021-03-26 维沃移动通信有限公司 一种图像处理方法及移动终端
CN113837939A (zh) * 2017-10-09 2021-12-24 华为技术有限公司 一种图像处理方法及设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102722875A (zh) * 2012-05-29 2012-10-10 杭州电子科技大学 一种基于视觉注意和质量可变的图像超分辨率重建方法
CN104702846A (zh) * 2015-03-20 2015-06-10 惠州Tcl移动通信有限公司 移动终端相机预览图像处理方法及系统
CN106603885A (zh) * 2015-10-20 2017-04-26 腾讯科技（深圳）有限公司 视频图像处理方法和装置
CN106919401A (zh) * 2017-03-10 2017-07-04 广东欧珀移动通信有限公司 一种动态调节分辨率的方法、装置及移动终端

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3671627A4

Also Published As

Publication number Publication date
US20200258195A1 (en) 2020-08-13
AU2017435234A1 (en) 2020-04-09
EP3671627A1 (en) 2020-06-24
AU2021229220B2 (en) 2023-05-18
AU2017435234B2 (en) 2021-06-10
AU2021229220A1 (en) 2021-10-07
EP3671627A4 (en) 2020-10-21
CN113837939A (zh) 2021-12-24
CN113808016A (zh) 2021-12-17
US11132766B2 (en) 2021-09-28
US20220058772A1 (en) 2022-02-24

Similar Documents

Publication Publication Date Title
CN108496198B (zh) 一种图像处理方法及设备
CN108491275B (zh) 程序优化方法、装置、终端及存储介质
WO2019071618A1 (zh) 一种图像处理方法及设备
EP2763034B1 (en) Method and device for image-capturing application screen for use in mobile terminal
WO2018103484A1 (zh) 直播页面的数据处理方法、装置和系统
US10181203B2 (en) Method for processing image data and apparatus for the same
US20170237901A1 (en) Apparatus and method for providing dynamic panorama function
WO2021143182A1 (zh) 游戏的处理方法、装置、电子设备及计算机可读存储介质
EP4060475A1 (en) Multi-screen cooperation method and system, and electronic device
WO2019091411A1 (zh) 拍摄图像的方法、装置、终端和存储介质
Haris et al. Evolution of android operating system: a review
RU2619089C2 (ru) Способ и устройство для воспроизведения множества видео
CN104133711A (zh) 基于Android系统的摄像头安全切换方法
US20240086231A1 (en) Task migration system and method
CN111367456A (zh) 通信终端及多窗口模式下的显示方法
WO2015110060A1 (en) Filter realization method and apparatus of camera application
WO2019091487A1 (zh) 拍摄图像的方法、装置、终端和存储介质
US11194598B2 (en) Information display method, terminal and storage medium
CN116700601B (zh) 内存优化方法、设备及存储介质
WO2014067368A1 (zh) 资源预览图的获取方法、装置以及设备
CN113641431A (zh) 二维码的增强显示的方法和终端设备
CN111142648B (zh) 一种数据处理方法和智能终端
WO2022247664A1 (zh) 图形界面显示方法、电子设备、介质以及程序产品
CN116048349B (zh) 一种图片显示方法、装置及终端设备
CN115175002B (zh) 一种视频播放方法及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17928530

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017928530

Country of ref document: EP

Effective date: 20200316

ENP Entry into the national phase

Ref document number: 2017435234

Country of ref document: AU

Date of ref document: 20171013

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE