WO2020051599A1 - Non-real-time data processing method for image or video data - Google Patents

Non-real-time data processing method for image or video data Download PDF

Info

Publication number
WO2020051599A1
WO2020051599A1 PCT/VN2019/000014 VN2019000014W WO2020051599A1 WO 2020051599 A1 WO2020051599 A1 WO 2020051599A1 VN 2019000014 W VN2019000014 W VN 2019000014W WO 2020051599 A1 WO2020051599 A1 WO 2020051599A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
data processing
processing
image
rechargeable battery
Prior art date
Application number
PCT/VN2019/000014
Other languages
French (fr)
Inventor
Dinh Nam NGUYEN
Original Assignee
Dinh Nam NGUYEN
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dinh Nam NGUYEN filed Critical Dinh Nam NGUYEN
Publication of WO2020051599A1 publication Critical patent/WO2020051599A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/127—Prioritisation of hardware or computational resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/20—Cooling means
    • G06F1/203—Cooling means for portable computers, e.g. for laptops
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/20—Cooling means
    • G06F1/206—Cooling means comprising thermal management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
    • G06F1/3209—Monitoring remote activity, e.g. over telephone lines or network connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
    • G06F1/3212—Monitoring battery levels, e.g. power saving mode being initiated when battery voltage goes below a certain level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
    • G06F1/3215—Monitoring of peripheral devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
    • G06F1/3215—Monitoring of peripheral devices
    • G06F1/3218—Monitoring of peripheral devices of display devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G06F1/324—Power saving characterised by the action undertaken by lowering clock frequency
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G06F1/3287—Power saving characterised by the action undertaken by switching off individual functional units in the computer system
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present disclosure relates to compressing or processing image or video data using devices having rechargeable battery. More particularly, the disclosure relates to method for saving battery and extending the battery life, as well as the service life of the battery when the device using rechargeable battery needs to compress or process image or video data.
  • the electronic device will consume a large amount of energy and generate an amount of heat larger than the amount of heat that can be discharged to the environment, and thus causing the device using rechargeable battery to stop operating to wait for the temperature to decrease or to face an explosion risk.
  • the existing devices using rechargeable battery are usually provided with compression technologies using IC but excluding the best compression technologies that are battery-consuming. This causes a large data size of video being stored in the electronic device, the video is usually 2 to 3 times larger than if the video is compressed using the best available compression technology.
  • Battery life is the time period that the device can operate, since the charging stops until the battery runs out. Assuming that the energy consumed by the device remains stably unchanged during the entire process.
  • the unit of the battery life may be a time unit such as second, hour, or percentage, which is the actual battery life divided by the maximum battery life of the device.
  • Acceptable temperature threshold is a temperature threshold predefined by the device or set up by the user.
  • One option to set up this temperature threshold is to select the value which guarantees that the device is safe and does not explode due to overheat; another option is to select the value which guarantees not to cause an uncomfortable feeling or an explosion concern as the user holds the device while the device is performing the data processing method; another option is to select the value which guarantees not to harm the service life of the battery because the service life usually reduces significantly when the battery is overheated.
  • Limiting data processing speed is a method that causes the processing operations to run slower than the capacity of the device such that the system meets certain requirements. Limiting data processing speed can be achieved using different implementations, for example, using functionalities supported by the operating system, or hibernating the entire device for a time period and then resuming the operations of the device.
  • a video after being captured by a sensor is compressed using a popular compression standard such as H.264/AVC, H.265/HEVC, etc.
  • Fundamental concept of these compression standards is to divide the video into groups of frames. Each frame is then divided into macroblocks (e.g., 16x16 macroblock in H.264/AVC). The process of compressing each macroblock is performed by prediction, residual quantization, and entropy compression.
  • H.264 which refers to MPEG-4 Part 10 AVC (Advance Video Coding) and is usually abbreviated as MPEG-4 AVC/H.264 is a potential compression format that can improve the image compression efficiency higher than other common compression formats such as MPEG-4 or MPEG-2.
  • H.264 standard has become one of the most popular video compression standards in recording, compressing, and transmitting images with high display quality and reduced bandwidth.
  • H.264 standard is now popularly used in applications such as Blu-ray disc.
  • Camera recorder, video recorder, set-top-box, etc. are all capable of decoding H.264 format. This standard is also used in video streaming services (e.g., Youtube, Vimeo, iTunes Store, etc.), web software (e.g., Adobe Flash, Microsoft Silverlight, HTML5, etc.).
  • video streaming services e.g., Youtube, Vimeo, iTunes Store, etc.
  • web software e.g., Adobe Flash, Microsoft Silverlight, HTML5, etc.
  • H.265 standard is a potential alternative compression standard for the H.264 standard which is very popular today.
  • H.265 standard is capable of compressing the data 2 times more than other existing compression standards while maintaining the image quality. This standard can support the resolution up to 8192 x 4320, including 8k Ultra HD quality.
  • U.S. Patent No. US7911509B2 discloses a solution for selecting adaptive video encode scaling in which the processing logic circuit selects the video encode scaling based on a condition of the memory, a condition of the power supply (battery), or processing logic.
  • U.S. Patent Publication No. US20080084491A1 discloses a solution for selecting video processing scaling based on a power savings factor, in particular, adjusting complexity of video encoding based on the power savings factor.
  • U.S. Patent Publication No. US20060039469A1 discloses a solution for scalable video compression based on remaining battery capacity.
  • U.S. Patent Publication No. US20060215755A1 discloses video encoding method and system for battery-powered apparatus in which picture type, picture size, and picture rate are determined based on the power level of the battery.
  • U.S. Patent Publication No. US20120083258A1 discloses a solution to manage the processing of the mobile device based on available power resources, in which a battery threshold calculator provides a notification of a battery charge threshold sufficient for processing a scheduled program that has a certain processing complexity.
  • U.S. Patent Publication No. US20150181208A1 discloses a method for managing thermal and power of a device for video encoding, which includes the step of adjusting encoding parameters based on operating characteristics of the components of the electronic device and based on video quality.
  • None of these documents discloses a solution for processing data using the best technologies although they may have higher processing complexity in order to maximally reduce the data size and completely avoid reducing the battery life.
  • the purpose of the invention is to provide a technology for devices using rechargeable battery to apply data processing method that has highest efficiency while still guarantee that the battery life is comparable to existing processing methods.
  • the invention proposes a non-real-time data processing method for image or video data on a device using rechargeable battery, the method includes the following steps:
  • the data processing method is a data compressing method
  • data processing is data compression
  • compressed data is an image or a video that was previously compressed but is currently decompressed and then recompressed using a method that is more optimal in terms of data size.
  • the method further includes a step of processing the data only when a screen is off.
  • the method further includes a step of processing the data only when a user does not input data into the device during a particular time period.
  • the data input method may be typing on a key, touching sensors or buttons.
  • Example time period can be any value from 1 minute to 1 hour.
  • the method further includes a step of limiting the data processing speed such that the processing operation does not impact other operations of the device, e.g., synchronizing data from the device to the Internet.
  • the method further includes a step of processing the data only when the processing operation does not occupy Random Access Memory (RAM) of the running applications.
  • RAM Random Access Memory
  • the method further includes a step of not processing the data when the device is charged by Portable Power Banks.
  • the method further includes a step of limiting the data processing speed such that the energy required for processing is sufficiently provided by the external source, therefore the data processing operation does not reduce battery life.
  • the method further includes a step of limiting the data processing speed such that the battery life is always above a value predefined by the device, this value may be redefined by the user. For example, this value may be 90% or 100%.
  • the data processing method is an image recognition method to extract useful information from the data, wherein data processing is recognizing image to extract useful information from the data, or classifying the data to enable a user to mine the data more efficiently.
  • the data processing method is a data analyzing method, wherein analyzing data is to suggest the user to make a data processing decision such as backup or delete redundant data.
  • the invention proposes a device using rechargeable battery (1) including at least the following components: a rechargeable battery (11), an image processing IC or chip (12), a screen (13), a microcontroller unit (MCU) (14), a camera (15), a microphone (16), a speaker (17), a temperature sensor (18), an input component (19), other sensors (110), and memory (111).
  • the MCU (14) of the device using rechargeable battery (1) is configured to control the components of the device using rechargeable battery (1) to perform the non-real-time data processing method of the invention according to one of the embodiments and aspects discussed above.
  • Examples of the device using rechargeable battery (1) may be mobile phone, smart phone, tablet, even laptop using rechargeable battery, etc.
  • the invention also proposes a computer program product or a computer readable medium including a computer program, the computer program includes instructions that cause suitable hardware components to perform the non-real-time data processing method according to the invention.
  • the devices using rechargeable battery are capable of compressing image or video using the best technologies that maximally reduce the data size and completely avoid reducing the battery life as compared to existing methods.
  • Figure 1 is a flowchart of a non-real-time data processing method for image or video data on a device using rechargeable battery according to one embodiment of the invention.
  • Figure 2 is a flowchart of a non-real-time data processing method for image or video data on a device using rechargeable battery according to a particular embodiment of the invention, in which data processing method is data compression.
  • Figure 3 is a block diagram of a device using rechargeable battery that uses a non- real-time data processing method according to one embodiment of the invention.
  • a non-real-time data processing method for image or video data on a device using rechargeable battery includes the steps illustrated in Fig. 1.
  • the device using rechargeable battery 1 is usually mobile device such as mobile phone, smart phone, tablet (e.g., iPad) that has integrated camera and is provided with specialized IC for processing image and video data.
  • mobile device such as mobile phone, smart phone, tablet (e.g., iPad) that has integrated camera and is provided with specialized IC for processing image and video data.
  • battery life of these devices is generally not suitable to run high efficiency compression algorithms for image and video data in long term, for example, H.265/HEVC (High Efficiency Video Coding) compression standard. Even if the device runs the algorithms to compress video, image with these compression standards, the device will have the battery life reduced and will be heated quickly.
  • the device using rechargeable battery 1 is installed a control software to perform the non-real-time data processing method according to the invention.
  • step Sl 1 the device using rechargeable battery 1 starts the non-real time data processing method.
  • step S12 the device using rechargeable battery 1 receives image or video data captured by an integrated camera.
  • step S13 the device (using rechargeable battery) 1 determines whether the device is currently charged by an external source. If it is true, the device using rechargeable battery 1 performs data processing in step S14. If it is false, after a time period customized by the designer or the user, the device 1 returns to step S13.
  • step S15 in which the device 1 determines whether the data processing is completed. If it is true, the device 1 stores the processed data in step S16 and ends the method in step S17. If it is false, the device 1 returns to step S13.
  • Fig. 2 illustrates another embodiments of the invention, in which the data processing method is a data compressing method.
  • the method starts in step S21, the device 1 then receives the image or video data from its integrated camera in step S22.
  • step S23 image or video data is preliminarily compressed to be easily stored in the memory. This data is marked to be re-compressed at least one more time using a compression standard that has higher efficiency.
  • step S24 the device (using rechargeable battery) 1 determines whether the device is currently charged by an external source. If it is true, the device using rechargeable battery 1 re-compresses image or video data using high efficiency compression algorithm in step S241. If it is false, the device 1 temporarily stores the image or video data that was preliminarily compressed in step S242. Until the method is ended, after each time period customized by the designer or the user, the device 1 returns to step S23.
  • step S25 in which the device 1 determines whether the data compression is completed. If it is true, the device 1 stores the compressed data in step S26 and ends the method in step S27. If it is false, the device 1 returns to step S24.
  • image compression standard such as WebP or BPG may be used, these are advanced efficient image compression method having high image compression ratio.
  • high efficiency video compression standard such as HEYC may be used.
  • Preliminary compression of image or video data in real-time is to guarantee that the captured image and video data is not interrupted, and the user can immediately view, query, and use the data.
  • the non-real-time image or video data processing method according to the invention is different in that, the processor of the device using rechargeable battery is configured to:
  • the compression process may be temporarily stopped and resumed when continue charging the device using rechargeable battery with the external source.
  • Limiting the data processing speed of the IC is to guarantee that the temperature of the device using rechargeable battery in general and the temperature of the battery in particular do not increase too high that may result in explosion risk or cause discomfort to the user.
  • the method further includes a step of processing the data only when a screen is off.
  • the method further includes a step of processing the data only when a user does not input data into the device during a particular time period.
  • the data input method may be typing on a key, touching sensors or buttons.
  • Example time period can be any value from 1 minute to 1 hour.
  • the method further includes a step of limiting the data processing speed such that the data processing operation does not impact other operations of the device, e.g., synchronizing data from the device to the Internet.
  • the method further includes a step of processing the data only when the data processing does not occupy Random Access Memory (RAM) of the running applications.
  • RAM Random Access Memory
  • the method further includes a step of not processing the data when the device is charged by Portable Power Banks.
  • the method further includes a step of limiting the data processing speed such that the energy required for processing is sufficiently provided by the external source, therefore the processing operation does not reduce battery life.
  • the method further includes a step of limiting the data processing speed such that the battery life is always above a value predefined by the device, this value may be redefined by the user. For example, this value may be from 90% to 100%.
  • the invention can also be applied to analyze, identify, or other form of data processing.
  • the data processing method is an image recognition method to extract useful information from the data, wherein data processing is recognizing image to extract useful information from the data, or classifying the data to enable a user to mine the data more efficiently.
  • the data processing method is a data analyzing method, wherein analyzing data is to suggest the user to make a data processing decision such as backup or delete redundant data.
  • Fig. 3 illustrates a block diagram of a device using rechargeable battery 1 to perform the non-real-time data processing method according to another embodiment of the invention.
  • the device using rechargeable battery 1 includes at least the following components: a rechargeable battery 11, an image processing IC or chip 12, a screen 13, a microcontroller unit (MCU) 14, a camera 15, a microphone 16, a speaker 17, a temperature sensor 18, an input component 19, other sensors 110, and memory 111.
  • the MCU 14 of the device using rechargeable battery 1 is configured to control the components of the device using rechargeable battery 1 to perform the non-real -time data processing method of the invention according to one of the embodiments and aspects discussed above.
  • Examples of the device using rechargeable batery 1 may be mobile phone, smart phone, tablet, even laptop using rechargeable batery, etc.
  • the data processing method according to the invention may be implemented in the form of a computer program stored in a computer program product, a computer readable medium, the computer program includes instructions that cause suitable hardware components to perform the non-real-time data processing method according to the invention.
  • the present technology according to the invention relates to using external power source to process complicated computation operations, as well as limiting the processing speed of IC/chip to not make the temperature of the device using rechargeable batery exceed the safe temperature.
  • Example computing devices or processors can be multi-functional processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, etc.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field Programmable Gate Array
  • Various embodiments of the invention can also be implemented or illustrated by a combination of these devices.
  • the method set forth herein allows the device using rechargeable battery to apply data processing method that has highest efficiency while still guarantees that the battery life is comparable to other existing processing methods. For example, compressing image or video using the best technologies allows for maximally reducing the data size while completely avoid reducing the battery life as compared to existing methods. In addition to compression, the method is also applicable to analyzing, identifying, or other form of data processing.

Abstract

The invention relates to a non-real-time data processing method for image or video data on a device using rechargeable battery, the method comprises: receiving data that can be compressed by the device using rechargeable battery; processing the received data only during a time period in which the device is charged by an external source; and limiting a data processing speed such that the temperature of the device using rechargeable battery does not exceed an acceptable temperature threshold. The method set forth herein enables devices using rechargeable battery to apply data processing method having highest efficiency, while still guarantees the battery life like the processing methods that save battery the most. For example, compressing image or video using the best technologies allows for maximally reducing the data size while completely avoid reducing the battery life as compared to available compression method. In addition to compression, the method is also applicable to analyzing, identifying, or other form of data processing.

Description

NON-REAL-TIME DATA PROCESSING METHOD
FOR IMAGE OR VIDEO DATA
TECHNICAL FIELD
[0001] The present disclosure relates to compressing or processing image or video data using devices having rechargeable battery. More particularly, the disclosure relates to method for saving battery and extending the battery life, as well as the service life of the battery when the device using rechargeable battery needs to compress or process image or video data.
BACKGROUND OF THE INVENTION
[0002] In 2018, there are 4 billion smart phones in the world. In 2020, there will be 6.1 billion people using smart phones. It is predicted that in the fourth quarter of 2018, each smart phone will store approximately 60 GB of data on average, most of which is image and video data (https://www.counterpointresearch.com/average-smartphone-nand-storage-capacitv- will-ton- 60gb-bv-20l8/I.
[0003] Devices using rechargeable battery such as smart phone, camera, video recorder, laptop usually collect and store a large amount of external data in the devices, specially image and video data. Images and videos always need to be stored and transmitted in the form of lossy compression. Even if being executed by a specialized integrated circuit (IC) or executed by a software, data compression is a process that consumes a large amount of processing capacity and electricity to reduce the data size, thereby reducing the storage and transmission costs. The problem is that the devices using rechargeable battery usually have a battery life shorter than the usage demand, therefore the hardware and software designer usually selects the compression method that is optimal for saving battery instead of optimal for reducing the data size. On the other hand, if the compression method is too complicated, the electronic device will consume a large amount of energy and generate an amount of heat larger than the amount of heat that can be discharged to the environment, and thus causing the device using rechargeable battery to stop operating to wait for the temperature to decrease or to face an explosion risk. Because of these problems regarding battery and temperature, the existing devices using rechargeable battery are usually provided with compression technologies using IC but excluding the best compression technologies that are battery-consuming. This causes a large data size of video being stored in the electronic device, the video is usually 2 to 3 times larger than if the video is compressed using the best available compression technology.
[0004] Before discussing the existing solutions to save battery and extend the battery life as well as the service life of the battery when compressing or processing image or video data, some related technical definitions are defined in details as follows.
[0005] Battery life is the time period that the device can operate, since the charging stops until the battery runs out. Assuming that the energy consumed by the device remains stably unchanged during the entire process. The unit of the battery life may be a time unit such as second, hour, or percentage, which is the actual battery life divided by the maximum battery life of the device.
[0006] Acceptable temperature threshold is a temperature threshold predefined by the device or set up by the user. One option to set up this temperature threshold is to select the value which guarantees that the device is safe and does not explode due to overheat; another option is to select the value which guarantees not to cause an uncomfortable feeling or an explosion concern as the user holds the device while the device is performing the data processing method; another option is to select the value which guarantees not to harm the service life of the battery because the service life usually reduces significantly when the battery is overheated. [0007] Limiting data processing speed is a method that causes the processing operations to run slower than the capacity of the device such that the system meets certain requirements. Limiting data processing speed can be achieved using different implementations, for example, using functionalities supported by the operating system, or hibernating the entire device for a time period and then resuming the operations of the device.
[0008] Typically, a video after being captured by a sensor is compressed using a popular compression standard such as H.264/AVC, H.265/HEVC, etc. Fundamental concept of these compression standards is to divide the video into groups of frames. Each frame is then divided into macroblocks (e.g., 16x16 macroblock in H.264/AVC). The process of compressing each macroblock is performed by prediction, residual quantization, and entropy compression.
[0009] H.264, which refers to MPEG-4 Part 10 AVC (Advance Video Coding) and is usually abbreviated as MPEG-4 AVC/H.264 is a potential compression format that can improve the image compression efficiency higher than other common compression formats such as MPEG-4 or MPEG-2.
[0010] Being first introduced in 2004, H.264 standard has become one of the most popular video compression standards in recording, compressing, and transmitting images with high display quality and reduced bandwidth.
[0011] H.264 standard is now popularly used in applications such as Blu-ray disc.
Camera recorder, video recorder, set-top-box, etc., are all capable of decoding H.264 format. This standard is also used in video streaming services (e.g., Youtube, Vimeo, iTunes Store, etc.), web software (e.g., Adobe Flash, Microsoft Silverlight, HTML5, etc.).
[0012] H.265 standard is a potential alternative compression standard for the H.264 standard which is very popular today. H.265 standard is capable of compressing the data 2 times more than other existing compression standards while maintaining the image quality. This standard can support the resolution up to 8192 x 4320, including 8k Ultra HD quality.
[0013] Existing solutions include:
[0014] U.S. Patent No. US7911509B2 discloses a solution for selecting adaptive video encode scaling in which the processing logic circuit selects the video encode scaling based on a condition of the memory, a condition of the power supply (battery), or processing logic.
[0015] U.S. Patent Publication No. US20080084491A1 discloses a solution for selecting video processing scaling based on a power savings factor, in particular, adjusting complexity of video encoding based on the power savings factor.
[0016] U.S. Patent Publication No. US20060039469A1 discloses a solution for scalable video compression based on remaining battery capacity.
[0017] U.S. Patent Publication No. US20060215755A1 discloses video encoding method and system for battery-powered apparatus in which picture type, picture size, and picture rate are determined based on the power level of the battery.
[0018] U.S. Patent Publication No. US20120083258A1 discloses a solution to manage the processing of the mobile device based on available power resources, in which a battery threshold calculator provides a notification of a battery charge threshold sufficient for processing a scheduled program that has a certain processing complexity.
[0019] U.S. Patent Publication No. US20150181208A1 discloses a method for managing thermal and power of a device for video encoding, which includes the step of adjusting encoding parameters based on operating characteristics of the components of the electronic device and based on video quality. [0020] None of these documents discloses a solution for processing data using the best technologies although they may have higher processing complexity in order to maximally reduce the data size and completely avoid reducing the battery life.
DISCLOSURE OF THE INVENTION
[0021] The purpose of the invention is to provide a technology for devices using rechargeable battery to apply data processing method that has highest efficiency while still guarantee that the battery life is comparable to existing processing methods.
[0022] According to one embodiment, the invention proposes a non-real-time data processing method for image or video data on a device using rechargeable battery, the method includes the following steps:
[0023] receiving data that can be compressed by the device using rechargeable battery;
[0024] processing the received data only during a time period in which the device is charged by an external source; and
[0025] limiting a data processing speed such that a temperature of the device using rechargeable battery does not exceed an acceptable temperature threshold.
[0026] According to one particular embodiment of the invention, the data processing method is a data compressing method, and data processing is data compression, compressed data is an image or a video that was previously compressed but is currently decompressed and then recompressed using a method that is more optimal in terms of data size.
[0027] According to one particular embodiment of the invention, the method further includes a step of processing the data only when a screen is off.
[0028] According to one particular embodiment of the invention, the method further includes a step of processing the data only when a user does not input data into the device during a particular time period. The data input method may be typing on a key, touching sensors or buttons. Example time period can be any value from 1 minute to 1 hour.
[0029] According to one particular embodiment of the invention, the method further includes a step of limiting the data processing speed such that the processing operation does not impact other operations of the device, e.g., synchronizing data from the device to the Internet.
[0030] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when the processing operation does not occupy Random Access Memory (RAM) of the running applications.
[0031] According to one particular embodiment of the invention, the method further includes a step of not processing the data when the device is charged by Portable Power Banks.
[0032] According to one particular embodiment of the invention, the method further includes a step of limiting the data processing speed such that the energy required for processing is sufficiently provided by the external source, therefore the data processing operation does not reduce battery life.
[0033] According to one particular embodiment of the invention, the method further includes a step of limiting the data processing speed such that the battery life is always above a value predefined by the device, this value may be redefined by the user. For example, this value may be 90% or 100%.
[0034] According to another embodiment of the invention, the data processing method is an image recognition method to extract useful information from the data, wherein data processing is recognizing image to extract useful information from the data, or classifying the data to enable a user to mine the data more efficiently. [0035] According to another embodiment of the invention, the data processing method is a data analyzing method, wherein analyzing data is to suggest the user to make a data processing decision such as backup or delete redundant data.
[0036] Additionally, according to another embodiment, the invention proposes a device using rechargeable battery (1) including at least the following components: a rechargeable battery (11), an image processing IC or chip (12), a screen (13), a microcontroller unit (MCU) (14), a camera (15), a microphone (16), a speaker (17), a temperature sensor (18), an input component (19), other sensors (110), and memory (111). The MCU (14) of the device using rechargeable battery (1) is configured to control the components of the device using rechargeable battery (1) to perform the non-real-time data processing method of the invention according to one of the embodiments and aspects discussed above. Examples of the device using rechargeable battery (1) may be mobile phone, smart phone, tablet, even laptop using rechargeable battery, etc.
[0037] Additionally, according to another embodiment, the invention also proposes a computer program product or a computer readable medium including a computer program, the computer program includes instructions that cause suitable hardware components to perform the non-real-time data processing method according to the invention.
[0038] By applying the solution according to the invention, when compressing data, the devices using rechargeable battery are capable of compressing image or video using the best technologies that maximally reduce the data size and completely avoid reducing the battery life as compared to existing methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Aspects and advantages of the invention are described and reference to the following drawings:
[0040] Figure 1 is a flowchart of a non-real-time data processing method for image or video data on a device using rechargeable battery according to one embodiment of the invention.
[0041] Figure 2 is a flowchart of a non-real-time data processing method for image or video data on a device using rechargeable battery according to a particular embodiment of the invention, in which data processing method is data compression.
[0042] Figure 3 is a block diagram of a device using rechargeable battery that uses a non- real-time data processing method according to one embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
[0043] The invention is understood more clearly below and described according to various particular embodiments with reference to the accompanying drawings. However, various embodiments are merely possible configurations in which specific features, as described above, is performed individually or omitted. Similar elements illustrated in the drawings are referred to by the same reference number. The description related to similar elements in the drawings may be omitted.
[0044] In actual implementation, according to one preferred embodiment of the invention, a non-real-time data processing method for image or video data on a device using rechargeable battery includes the steps illustrated in Fig. 1.
[0045] First, it should be understood that the device using rechargeable battery 1 is usually mobile device such as mobile phone, smart phone, tablet (e.g., iPad) that has integrated camera and is provided with specialized IC for processing image and video data. Usually, due to limitation in design related to battery size and increased data processing demand, although the devices is provided with specialized IC for processing image, video data, battery life of these devices is generally not suitable to run high efficiency compression algorithms for image and video data in long term, for example, H.265/HEVC (High Efficiency Video Coding) compression standard. Even if the device runs the algorithms to compress video, image with these compression standards, the device will have the battery life reduced and will be heated quickly. The device using rechargeable battery 1 is installed a control software to perform the non-real-time data processing method according to the invention.
[0046] In Fig. 1, in step Sl 1, the device using rechargeable battery 1 starts the non-real time data processing method.
[0047] Next, in step S12, the device using rechargeable battery 1 receives image or video data captured by an integrated camera.
[0048] Next, in step S13, the device (using rechargeable battery) 1 determines whether the device is currently charged by an external source. If it is true, the device using rechargeable battery 1 performs data processing in step S14. If it is false, after a time period customized by the designer or the user, the device 1 returns to step S13.
[0049] Following step S14 is step S15, in which the device 1 determines whether the data processing is completed. If it is true, the device 1 stores the processed data in step S16 and ends the method in step S17. If it is false, the device 1 returns to step S13.
[0050] Fig. 2 illustrates another embodiments of the invention, in which the data processing method is a data compressing method. [0051] The method starts in step S21, the device 1 then receives the image or video data from its integrated camera in step S22.
[0052] Next, in step S23, image or video data is preliminarily compressed to be easily stored in the memory. This data is marked to be re-compressed at least one more time using a compression standard that has higher efficiency.
[0053] Next, in step S24, the device (using rechargeable battery) 1 determines whether the device is currently charged by an external source. If it is true, the device using rechargeable battery 1 re-compresses image or video data using high efficiency compression algorithm in step S241. If it is false, the device 1 temporarily stores the image or video data that was preliminarily compressed in step S242. Until the method is ended, after each time period customized by the designer or the user, the device 1 returns to step S23.
[0054] Following step S24 is step S25, in which the device 1 determines whether the data compression is completed. If it is true, the device 1 stores the compressed data in step S26 and ends the method in step S27. If it is false, the device 1 returns to step S24.
[0055] According to this data processing method, when the user does not use the device 1 and this device is charged by the external source (plugin battery charging), or when the electronic device needs to be charged by the external source, as soon as being charged by the external source (not Portable Power Bank), the step of re-compressing the image or video data will be triggered.
[0056] According to an example embodiment, for image compression, image compression standard such as WebP or BPG may be used, these are advanced efficient image compression method having high image compression ratio. For digital video compression, high efficiency video compression standard such as HEYC may be used. [0057] Preliminary compression of image or video data in real-time is to guarantee that the captured image and video data is not interrupted, and the user can immediately view, query, and use the data.
[0058] In addition, in order to explain more clearly, the non-real-time image or video data processing method according to the invention is different in that, the processor of the device using rechargeable battery is configured to:
[0059] limit the data processing speed, i.e., still maintaining the high computation complexity to achieve high data compression efficiency, but limit the processing speed of the IC of the device using rechargeable battery to guarantee that the temperature of the device does not exceed the safety temperature threshold of this device; and
[0060] the compression process may be temporarily stopped and resumed when continue charging the device using rechargeable battery with the external source.
[0061] Limiting the data processing speed of the IC is to guarantee that the temperature of the device using rechargeable battery in general and the temperature of the battery in particular do not increase too high that may result in explosion risk or cause discomfort to the user.
[0062] At the same time, maintaining high efficiency compression algorithm, for example, compression algorithm according to WebP/BPG standard for image compression, and compression algorithm according to HEVC standard provides optimal effect of saving memory without impacting the battery life.
[0063] The non-real-time image or video data processing method discussed above is merely an example embodiment of the invention. Scope and principle of the invention are not limited to this method. [0064] Other aspects of the invention are further described below (not shown in the drawings).
[0065] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when a screen is off.
[0066] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when a user does not input data into the device during a particular time period. The data input method may be typing on a key, touching sensors or buttons. Example time period can be any value from 1 minute to 1 hour.
[0067] According to one particular aspect of this embodiment, the method further includes a step of limiting the data processing speed such that the data processing operation does not impact other operations of the device, e.g., synchronizing data from the device to the Internet.
[0068] According to one particular aspect of this embodiment, the method further includes a step of processing the data only when the data processing does not occupy Random Access Memory (RAM) of the running applications.
[0069] According to one particular aspect of this embodiment, the method further includes a step of not processing the data when the device is charged by Portable Power Banks.
[0070] According to one particular aspect of this embodiment, the method further includes a step of limiting the data processing speed such that the energy required for processing is sufficiently provided by the external source, therefore the processing operation does not reduce battery life.
[0071] According to one particular aspect of this embodiment, the method further includes a step of limiting the data processing speed such that the battery life is always above a value predefined by the device, this value may be redefined by the user. For example, this value may be from 90% to 100%.
[0072] In addition to applying the non-real-time data processing method according to the invention to compress data, the invention can also be applied to analyze, identify, or other form of data processing.
[0073] According to another aspect of this embodiment, the data processing method is an image recognition method to extract useful information from the data, wherein data processing is recognizing image to extract useful information from the data, or classifying the data to enable a user to mine the data more efficiently.
[0074] According to another aspect of this embodiment, the data processing method is a data analyzing method, wherein analyzing data is to suggest the user to make a data processing decision such as backup or delete redundant data.
[0075] Additionally, according to another embodiment, Fig. 3 illustrates a block diagram of a device using rechargeable battery 1 to perform the non-real-time data processing method according to another embodiment of the invention.
[0076] As depicted in Fig. 3, the device using rechargeable battery 1 includes at least the following components: a rechargeable battery 11, an image processing IC or chip 12, a screen 13, a microcontroller unit (MCU) 14, a camera 15, a microphone 16, a speaker 17, a temperature sensor 18, an input component 19, other sensors 110, and memory 111. The MCU 14 of the device using rechargeable battery 1 is configured to control the components of the device using rechargeable battery 1 to perform the non-real -time data processing method of the invention according to one of the embodiments and aspects discussed above. Examples of the device using rechargeable batery 1 may be mobile phone, smart phone, tablet, even laptop using rechargeable batery, etc.
[0077] Additionally, the data processing method according to the invention may be implemented in the form of a computer program stored in a computer program product, a computer readable medium, the computer program includes instructions that cause suitable hardware components to perform the non-real-time data processing method according to the invention.
[0078] In general, the present technology according to the invention relates to using external power source to process complicated computation operations, as well as limiting the processing speed of IC/chip to not make the temperature of the device using rechargeable batery exceed the safe temperature.
[0079] It should be understood that different embodiments of the invention can be installed or implemented using computing devices (processors). Example computing devices or processors can be multi-functional processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, etc. Various embodiments of the invention can also be implemented or illustrated by a combination of these devices.
[0080] In addition, different embodiments of the inventions can also be implemented by software modules, these modules are executed by processors or executed directly in hardware. In addition, it is also possible to combine software modules and hardware. The software modules can be stored in any form of computer-readable storage media, e.g., RAM, EPROM, EEPROM, flash memory, register, hard disk, CD-ROM, DVD, etc. [0081] It should be noted that, the specific features of different embodiments may belong to a specific element or may be optionally integrated in other elements of the invention.
[0082] Person skilled in the art should understand that there are various alternatives and modifications that can be made with the implementations of the invention without departing the principle and scope of the invention. Therefore, these embodiments are considered in all aspects and not to limit the invention.
TECHNICAL EFFECTS OF THE INVENTION
[0083] The method set forth herein allows the device using rechargeable battery to apply data processing method that has highest efficiency while still guarantees that the battery life is comparable to other existing processing methods. For example, compressing image or video using the best technologies allows for maximally reducing the data size while completely avoid reducing the battery life as compared to existing methods. In addition to compression, the method is also applicable to analyzing, identifying, or other form of data processing.

Claims

WHAT IS CLAIMED IS:
1. A non-real-time data processing method for image or video data on a device using rechargeable battery, the method comprises:
processing received data only during a time period in which the device is charged by an external source; and
limiting a data processing speed such that a temperature of the device using rechargeable battery does not exceed an acceptable temperature threshold.
2. The method according to claim 1, wherein a data processing method is a data compressing method, and data processing is data compression, compressed data is an image or a video that was previously compressed but is currently decompressed and then recompressed using a method that is more optimal in terms of data size.
3. The method according to claim 1 or 2, wherein the method further includes processing the data only when a screen is off.
4. The method according to any one of claims 1-3, wherein the method further includes: processing the data only when a user does not input data into the device during a particular time period, and a data input method may be typing on a key, touching sensors or buttons, the particular time period can be any value from 1 minute to 1 hour.
5. The method according to any one of claims 1-4, wherein the method further includes limiting the data processing speed such that data processing operation does not impact on synchronizing data from the device to the Internet.
6. The method according to any one of claims 1-5, wherein the method further includes processing the data only when the data processing does not occupy Random Access Memory (RAM) of running applications.
7. The method according to any one of claims 1-6, wherein the method further includes not processing the data when the device is charged by portable power banks.
8. The method according to any one of claims 1-7, wherein the method further includes limiting the data processing speed such that energy required for processing is sufficiently provided by the external source, therefore the data processing operation does not reduce a battery life.
9. The method according to any one of claims 1-8, wherein the method further includes limiting the data processing speed such that the battery life is always above a value predefined by the device, the value can be redefined by the user, the value is from 90% to 100%.
10. The method according to any one of claims 1, 3-9, wherein the data processing method is an image recognition method to extract useful information from the data, wherein data processing is recognizing image to extract the useful information from the data, or classifying the data to enable a user to mine the data more efficiently.
11. The method according to any one of claims 1, 3-9, wherein the data processing method is a data analyzing method, wherein analyzing data is to suggest the user to make a data processing decision such as backup or delete redundant data.
PCT/VN2019/000014 2018-09-06 2019-09-05 Non-real-time data processing method for image or video data WO2020051599A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
VN201803929 2018-09-06
VN1-2018-03929 2018-09-06

Publications (1)

Publication Number Publication Date
WO2020051599A1 true WO2020051599A1 (en) 2020-03-12

Family

ID=68345099

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/VN2019/000014 WO2020051599A1 (en) 2018-09-06 2019-09-05 Non-real-time data processing method for image or video data

Country Status (1)

Country Link
WO (1) WO2020051599A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114156495A (en) * 2022-02-10 2022-03-08 一道新能源科技(衢州)有限公司 Laminated battery assembly processing method and system based on big data

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060039469A1 (en) 2002-11-21 2006-02-23 Koninklijke Philips Electronics N.V. Scalable video compression based on remaining battery capacity
US20060215755A1 (en) 2005-03-24 2006-09-28 Mediatek Incorporation Video encoding methods and systems for battery-powered apparatus
US20070188600A1 (en) * 2004-03-05 2007-08-16 Cooper Jeffrey A Multi-stage media compression technique for power and storage efficiency
US20080084491A1 (en) 2006-10-06 2008-04-10 Freescale Semiconductor Inc. Scaling video processing complexity based on power savings factor
JP2009055560A (en) * 2007-08-29 2009-03-12 Sony Ericsson Mobilecommunications Japan Inc Mobile device, and method for recording imaged image data
US7911509B2 (en) 2008-03-12 2011-03-22 Sony Ericsson Mobile Communications Ab Adaptive video encode scaling
US20120083258A1 (en) 2010-09-30 2012-04-05 Qualcomm Incorporated System and method to manage processes of a mobile device based on available power resources
US20150181208A1 (en) 2013-12-20 2015-06-25 Qualcomm Incorporated Thermal and power management with video coding
WO2015160590A1 (en) * 2014-04-15 2015-10-22 Qualcomm Incorporated System and method for deferring power consumption by post-processing sensor data

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060039469A1 (en) 2002-11-21 2006-02-23 Koninklijke Philips Electronics N.V. Scalable video compression based on remaining battery capacity
US20070188600A1 (en) * 2004-03-05 2007-08-16 Cooper Jeffrey A Multi-stage media compression technique for power and storage efficiency
US20060215755A1 (en) 2005-03-24 2006-09-28 Mediatek Incorporation Video encoding methods and systems for battery-powered apparatus
US20080084491A1 (en) 2006-10-06 2008-04-10 Freescale Semiconductor Inc. Scaling video processing complexity based on power savings factor
JP2009055560A (en) * 2007-08-29 2009-03-12 Sony Ericsson Mobilecommunications Japan Inc Mobile device, and method for recording imaged image data
US7911509B2 (en) 2008-03-12 2011-03-22 Sony Ericsson Mobile Communications Ab Adaptive video encode scaling
US20120083258A1 (en) 2010-09-30 2012-04-05 Qualcomm Incorporated System and method to manage processes of a mobile device based on available power resources
US20150181208A1 (en) 2013-12-20 2015-06-25 Qualcomm Incorporated Thermal and power management with video coding
WO2015160590A1 (en) * 2014-04-15 2015-10-22 Qualcomm Incorporated System and method for deferring power consumption by post-processing sensor data

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
EFRAIM ROTEM ET AL: "Temperature measurement in the Intel Core TM Duo Processor", PROCEEDINGS OF 12TH INTERNATIONAL WORKSHOP ON THERMAL INVESTIGATIONS OF ICS, THERMINIC 2006, 12 September 2007 (2007-09-12), pages 23 - 27, XP055598825, Retrieved from the Internet <URL:https://arxiv.org/ftp/arxiv/papers/0709/0709.1861.pdf> [retrieved on 20190624] *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114156495A (en) * 2022-02-10 2022-03-08 一道新能源科技(衢州)有限公司 Laminated battery assembly processing method and system based on big data
CN114156495B (en) * 2022-02-10 2022-04-26 一道新能源科技(衢州)有限公司 Laminated battery assembly processing method and system based on big data

Similar Documents

Publication Publication Date Title
EP3085071B1 (en) Thermal and power management
CN101939992B (en) Reference selection for video interpolation or extrapolation
CN106713915B (en) Method for encoding video data
US11575938B2 (en) Cascaded prediction-transform approach for mixed machine-human targeted video coding
US8126058B2 (en) Power aware method and apparatus of video decoder on a multi-core platform
US10509588B2 (en) System and method for controlling memory frequency using feed-forward compression statistics
JP2017504255A (en) Temperature and power management using video coding
KR100937616B1 (en) Computationally constrained video encoding
CN110248192B (en) Encoder switching method, decoder switching method, screen sharing method and screen sharing system
US20150326887A1 (en) Method for dynamically adapting the encoding of an audio and/or video stream transmitted to a device
CN115016885B (en) Virtual machine garbage recycling operation method and electronic equipment
CN104661059A (en) Picture playing method and device as well as set-top box
US20240040127A1 (en) Video encoding method and apparatus and electronic device
WO2020051599A1 (en) Non-real-time data processing method for image or video data
Yang et al. Cost effective IP camera for video surveillance
CN104244004A (en) Low-power coding method and low-power coding device
CN116437088A (en) Image buffer area updating method and device, server and storage medium
Chen et al. Worst case driven display frame compression for energy-efficient ultra-HD display processing
Yamauchi et al. An 81 MHz, 1280/spl times/720pixels/spl times/30frames/s MPEG-4 video/audio CODEC processor
US8897368B2 (en) Image coding device, image coding method, image coding integrated circuit and image coding program
US20150110475A1 (en) Video processing apparatus and method of controlling video processing apparatus
CN103414902A (en) AVC parallel coding method used for low power consumption applications
US20240013046A1 (en) Apparatus, method and computer program product for learned video coding for machine
US11405617B1 (en) Method and system to enhance compression efficiency in encoded video by using dual pass entropy coding
Zheyuan et al. Performance analysis for DCT-based coded image communication in wireless multimedia sensor networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19794858

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24.06.21)

122 Ep: pct application non-entry in european phase

Ref document number: 19794858

Country of ref document: EP

Kind code of ref document: A1