WO2022042671A1 - 可穿戴设备及其图像信号处理装置 - Google Patents

可穿戴设备及其图像信号处理装置 Download PDF

Info

Publication number
WO2022042671A1
WO2022042671A1 PCT/CN2021/114896 CN2021114896W WO2022042671A1 WO 2022042671 A1 WO2022042671 A1 WO 2022042671A1 CN 2021114896 W CN2021114896 W CN 2021114896W WO 2022042671 A1 WO2022042671 A1 WO 2022042671A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
image
module
wearable device
data processing
Prior art date
Application number
PCT/CN2021/114896
Other languages
English (en)
French (fr)
Inventor
饶国明
张慧敏
肖正飞
陈波
Original Assignee
展讯通信（上海）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 展讯通信（上海）有限公司 filed Critical 展讯通信（上海）有限公司
Priority to US18/023,437 priority Critical patent/US20240031672A1/en
Publication of WO2022042671A1 publication Critical patent/WO2022042671A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/65—Control of camera operation in relation to power supply
    • H04N23/651—Control of camera operation in relation to power supply for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/65—Control of camera operation in relation to power supply
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/665—Control of cameras or camera modules involving internal camera communication with the image sensor, e.g. synchronising or multiplexing SSIS control signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof

Definitions

  • the present invention relates to the technical field of smart devices, in particular to a wearable device and an image signal processing device thereof.
  • smart wearable devices have increasingly become a hot spot in the market, among which smart watches (such as children's smart watches, adult fashion watches, elderly smart watches, etc.), smart glasses (such as adult fashion glasses, etc.) are the most widely used types of devices.
  • smart watch and smart glasses products it can support both Camera (camera) and various applications, such as supporting video calls, remote photography, parental remote monitoring, eye tracking and other functions, providing a very colorful application possibility. Satisfy users with a higher experience.
  • the technical problem to be solved by the present invention is to overcome the defects in the prior art that the wearable device consumes too much power or supports too low pixels, so that the user's needs cannot be well met, and provides a wearable device that can Wearable device and its image signal processing device.
  • the present invention provides an image signal processing device of a wearable device, the image signal processing device comprising an image acquisition device, an image data processing module and an application processor;
  • the image data processing module is respectively connected in communication with the image acquisition device and the application processor;
  • the application processor is configured to control the image data processing module to switch from a sleep state or a power-down state to a working state when it is detected that the wearable device is in a shooting preview mode;
  • the image acquisition device is used to acquire original image data
  • the image data processing module is used for acquiring the original image data and processing the original image data.
  • the image data processing module includes a first image data processing unit and a second image data processing unit;
  • the image acquisition device is connected in communication with the first image data processing unit, and the second image data processing unit is in communication connection with the image acquisition device and the first image data processing unit, respectively;
  • the first image data processing unit is configured to obtain the original image data, and preprocess the original image data to obtain the first image data, and send it to the second image data processing unit;
  • the second image data processing unit is configured to use a target algorithm to process the first image data to obtain a processing result, and control the image acquisition device and/or the first image data processing unit according to the processing result ;
  • the running power consumption of the second image data processing unit is less than the running power consumption of the application processor.
  • the second image data processing unit includes an MCU (Micro Control Unit).
  • MCU Micro Control Unit
  • the first image data processing unit includes an ISP (Image Signal Processor).
  • ISP Image Signal Processor
  • the target algorithm includes 3A algorithms, namely AF (Auto Focus Algorithm), AE (Auto Exposure) and AWB (Auto White Balance Algorithm).
  • 3A algorithms namely AF (Auto Focus Algorithm), AE (Auto Exposure) and AWB (Auto White Balance Algorithm).
  • the application processor is configured to control the image data processing module to switch from a working state to a sleep state or a power-down state when it is detected that the wearable device has turned off the shooting preview mode;
  • the application processor automatically switches to a sleep state.
  • the image signal processing device further comprises an image encoding module, a video encoding module and a data storage module;
  • the image encoding module and the video encoding module are both connected in communication with the second image data processing unit and the application processor, and the data storage module is in communication connection with the application processor;
  • the image encoding module is configured to perform image encoding processing based on the image parameters in the processing result to obtain the image encoding result and send it to the application processor;
  • the video encoding module is configured to perform video encoding processing based on the image parameters in the processing result to obtain the video encoding result and send it to the application processor;
  • the application processor is configured to send the image encoding result and the video encoding result to the data storage module for storage.
  • the image signal processing device further includes a display control module and a display module;
  • the display processing module is respectively connected in communication with the second image data processing unit and the display module;
  • the display processing module is configured to perform display processing on the processing result to obtain a display processing result and send it to the display module for display; and/or,
  • the image signal processing device further includes a data transmission interface
  • the data transmission interface is respectively connected in communication with the image acquisition device and the first image data storage unit;
  • the first image data storage unit acquires the original image data collected by the image collection device through the data transmission interface.
  • the image data processing module, the application processor, the data transmission interface, the image coding module, the video coding module, the data storage module and the display control module are all built-in in the SOC chip (system-on-chip) of the wearable device; and/or,
  • the data transmission interface includes MIPI CSI interface (a camera serial interface).
  • the present invention also provides a wearable device, the wearable device includes the above-mentioned image signal processing apparatus of the wearable device.
  • the ISP architecture of a new type of wearable device is designed, so that when the device is in the shooting state, the MCU is used to perform algorithm processing on the image data alone, that is, the AP (application processor) is no longer used in the algorithm calculation process, which effectively reduces the power consumption; at the same time, the AP automatically switches to the sleep state in the shooting state to further reduce power consumption; in addition, the processing results processed by the MCU are used to feedback control the image acquisition device to select more appropriate parameters to collect the original image data, and feedback control
  • the ISP selects more suitable parameters for image data processing, which effectively improves the image processing capability of the image signal processing device, improves the photographing quality of the wearable device, better improves the user experience, and satisfies the user's higher requirements. Usage requirements.
  • FIG. 1 is a schematic structural diagram of an image signal processing apparatus of a wearable device according to Embodiment 1 of the present invention
  • FIG. 2 is a schematic structural diagram of an image signal processing apparatus of a wearable device according to Embodiment 1 of the present invention
  • FIG. 3 is a schematic structural diagram of an image signal processing apparatus of a wearable device according to Embodiment 1 of the present invention.
  • the image signal processing apparatus of the wearable device in this embodiment includes an image acquisition device 1 , an image data processing module 2 and an application processor 3 .
  • wearable devices include but are not limited to smart watches and smart glasses.
  • the image data processing module 2 is connected in communication with the image acquisition device 1 and the application processor 3 respectively.
  • the wearable device will turn off the shooting preview function.
  • the image data processing module 2 is in a dormant state or a power-down state, and only needs to run the AP application processor; only when the shooting preview function is turned on, the image data processing module 2 Only then will it enter the working state, that is, the image data processing module 2 does not need to be kept in the working state all the time, which effectively saves the running power consumption.
  • the application processor 3 is configured to generate a first control instruction to control the image data processing module 2 to switch from the sleep state or the power-down state to the working state when it is detected that the wearable device is in the shooting preview mode.
  • the image acquisition device 1 is used to acquire original image data
  • the image data processing module 2 is used for acquiring original image data and processing the original image data.
  • the application processor 3 is configured to generate a second control instruction to control the image data processing module 2 to switch from the working state to the sleep state or the power-down state when it is detected that the wearable device has turned off the shooting preview mode; that is, when the shooting preview is not started
  • the entire image data processing module 2 does not generate or basically generate no power consumption, that is, the effect of reducing the running power consumption is achieved.
  • the application processor 3 When the image data processing module 2 is in the working state, the application processor 3 automatically switches to the sleep state, that is, the application processor 3 with high power consumption is not required to participate in the image processing process, which further achieves the effect of reducing power consumption.
  • the image data processing module 2 includes a first image data processing unit 4 and a second image data processing unit 5, the image acquisition device 1 is connected in communication with the first image data processing unit 4, and the second image data The processing unit 5 is connected in communication with the image acquisition device 1 and the first image data processing unit 4 respectively.
  • the first image data processing unit 4 is used to obtain the original image data, and preprocess the original image data to obtain the first image data, and send it to the second image data processing unit 5;
  • the second image data processing unit 5 is configured to use the target algorithm to process the first image data to obtain a processing result, and control the image acquisition device 1 and/or the first image data processing unit 4 according to the processing result;
  • the running power consumption of the second image data processing unit 5 is smaller than the running power consumption of the application processor 3 .
  • the first image data processing unit 4 is an ISP
  • the second image data processing unit 5 is an MCU.
  • the running power consumption of the MCU is far lower than the AP chip (ie, the application processor) of the Cortex-A (a kind of processor) architecture.
  • the ISP preprocesses the collected raw image data through the ISP Hardware Pipeline (hardware pipeline), such as processing the raw image data with functions such as AEC (Automatic Exposure Control), AWB, Anti-Flicker (Remove Water Ripple Flash), etc.
  • the processed image data is sent to the MCU for subsequent processing.
  • the target algorithm includes but is not limited to the 3A algorithm.
  • the 3A algorithm is an algorithm library running on the ISP firmware, including algorithms such as AE/AF/AWB/AFL. Running the 3A algorithm effectively reduces the power consumption of the overall image processing process and effectively improves the endurance of the wearable device.
  • MCU runs on RTOS operating system (real-time operating system, a lightweight microkernel operating system, different from Android, Windows, IOS and other intelligent operating systems, can be applied to MCU controller, can provide microsecond-level operating system Response speed, ultra-low power consumption operating system), at the same time, when the part of the application processor 3 that performs algorithm processing is removed, the application processor 3 can also adopt the MCU architecture, which further reduces the power generated by the program running. It is suitable for application scenarios of low-power wearable devices.
  • RTOS operating system real-time operating system, a lightweight microkernel operating system, different from Android, Windows, IOS and other intelligent operating systems, can be applied to MCU controller, can provide microsecond-level operating system Response speed, ultra-low power consumption operating system
  • MCU includes but is not limited to ARM-M series, RISC-V, MIPS, MCS-51 (ARM-M series, RISC-V, MIPS, MCS-51 are all microcontroller units).
  • the image signal processing apparatus further includes an image encoding module 6 , a video encoding module 7 , a data storage module 8 , a data transmission interface 9 , a display control module 10 and a display module 11 .
  • the image encoding module 6 and the video encoding module 7 are all connected in communication with the second image data processing unit 5 and the application processor 3, and the data storage module 8 is connected in communication with the application processor 3;
  • the image encoding module 6 is configured to perform image encoding processing based on the image parameters in the processing result to obtain the image encoding result and send it to the application processor 3;
  • the video encoding module 7 is used to perform video encoding processing based on the image parameters in the processing results to obtain the video encoding results and send them to the application processor 3; wherein, the image parameters in the processing results are YUV format (a kind of image format) parameter data .
  • the application processor 3 is configured to send the image encoding result and the video encoding result to the data storage module 8 for storage.
  • the data transmission interface 9 is respectively connected in communication with the image acquisition device 1 and the first image data storage unit;
  • the first image data storage unit acquires the original image data collected by the image collection device 1 through the data transmission interface 9 .
  • the data transmission interface 9 includes but is not limited to MIPI CSI interface.
  • the display processing module is connected in communication with the second image data processing unit 5 and the display module 11 respectively;
  • the display processing module is configured to perform display processing on the processing result to obtain the display processing result and send it to the display module 11 for display.
  • the display module 11 displays images in RGB format (an image format).
  • the image data processing module 2, the application processor 3, the data transmission interface 9, the image coding module 6, the video coding module 7, the data storage module 8 and the display control module 10 are all built in the SOC chip of the wearable device.
  • the image acquisition device 1 is a Camera that supports the RAW Sensor (RAW sensor) type. Specifically, the image acquisition device 1 includes a camera and an image sensor, and the image sensor obtains raw image data through the camera and transmits it to the first image data processing through the data transmission interface 9. Unit 4; data communication is performed between the image sensor and the second image data processing unit through the I2C bus.
  • the image sensor includes, but is not limited to, a RAW sensor, which supports outputting raw image data in Bayer format.
  • the processing result of the second image data processing unit 5 includes a first control parameter and a second control parameter.
  • the second image data processing unit 5 is configured to send the first control parameter to the first image data processing unit 4, and the first image data processing unit
  • the unit 4 is configured to select and process the original image data more reasonably according to the first control parameter, so as to optimize the image data processing manner.
  • the second image data processing unit 5 is also used for sending the second control parameter to the image sensor in the image acquisition device 1, and the image sensor is used for collecting the original image data more reasonably and effectively according to the second control parameter to optimize the image data collection method, Therefore, the image processing capability of the image signal processing device is effectively improved, the output image quality is guaranteed, the purpose of automatically adjusting the image quality is achieved, and the use performance of the wearable device is improved, so that the high-quality shooting effect experience can be satisfied and the user is satisfied. pixel requirements.
  • Lens ie camera
  • RAW Sensor ie RAW sensor
  • Lens and RAW Sensor constitute an image acquisition device for collecting raw image data
  • AP ie application processor
  • ISP ISP
  • MCU ISP firmware, i.e. the second image data processing unit
  • MIPI CSI interface i.e. data transmission interface
  • Display Unit i.e. display processing module
  • JPEG Encode i.e. image Encoding module
  • Video Encode ie video encoding module
  • LCD ie display module
  • the wearable device is a smart device such as a smart watch or smart glasses
  • the auto-focus function in this embodiment can be cancelled, and it only needs to support the FF (fixed focus) camera module, so that the image data processing module 2 is not required.
  • the feedback control of the camera reduces the computing power requirements of the MCU, thereby further reducing the power consumption generated by the operation of the wearable device.
  • the ISP architecture of a new type of wearable device is designed, so that when the device is in the shooting state, the MCU is used to perform algorithm processing on the image data alone, that is, the AP is no longer used in the algorithm calculation process, which effectively reduces the power consumption; , in the shooting state, the AP automatically switches to the sleep state, which further reduces the power consumption; in addition, the processing results processed by the MCU are used to feedback control the image acquisition device to select more appropriate parameters to collect the original image data, and feedback to control the ISP to select more appropriate parameters It can effectively improve the image processing capability of the image signal processing device, improve the photographing quality of the wearable device, achieve the purpose of automatically adjusting the image quality, better improve the user experience, and satisfy the meet the higher demands of users.
  • the wearable device of this embodiment includes the image signal processing apparatus of the wearable device of Embodiment 1.
  • wearable devices include but are not limited to smart watches or smart glasses.
  • the wearable device in this embodiment adopts the above-mentioned ISP architecture, so that when the device is in the shooting state, the MCU is used to perform algorithm processing on the image data alone, that is, the AP is no longer used in the algorithm calculation process, which effectively reduces the power consumption; When shooting, the AP automatically switches to the sleep state, which further reduces power consumption, thereby effectively saving power consumption; in addition, it can automatically adjust the image quality, improve the user experience, and meet the user's higher needs. .

Abstract

本发明公开了一种可穿戴设备及其图像信号处理装置，所述图像信号处理装置包括图像采集设备、图像数据处理模块和应用处理器；所述应用处理器用于在检测到所述可穿戴设备开启拍摄预览模式时，控制所述图像数据处理模块从休眠状态或掉电状态切换至工作状态；所述图像采集设备用于采集原始图像数据；所述图像数据处理模块用于获取所述原始图像数据，并对所述原始图像数据进行处理。本发明能够大大地降低设备在运行过程中产生的功耗，有效提升可穿戴设备的续航能力；且能够自动调节图像处理质量，提高可穿戴设备的拍照质量，从而提升了用户的使用体验，满足了用户更高的使用需求。

Description

可穿戴设备及其图像信号处理装置
本申请要求2020年8月31日提交中国专利局、申请号为202010900334.X、发明名称为“可穿戴设备及其图像信号处理装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及智能设备技术领域，特别涉及一种可穿戴设备及其图像信号处理装置。
背景技术
目前，智能可穿戴设备已经日益成为市场的热点，其中智能手表(如儿童智能手表、成人时尚手表、老人智能手表等)、智能眼镜(如成人时尚眼镜等)等是应用最为广泛的设备类型。对于智能手表和智能眼镜的产品，其既能支持Camera(摄像头)又支持丰富多彩的应用，例如支持视频通话、远程拍照、家长远程监控、眼球跟踪等功能，提供了非常丰富多彩的应用可能，满足用户更高的使用体验。
由于可穿戴设备具有外部物理尺寸狭小、可使用电池容量低等特点，在此类设备中需要支持Camera时就具有一定的限制。现有虽然也出现一些方式来改善上述问题，但是仍存在要么耗电太大，要么支持像素太低等问题，从而仍然不能很好地满足用户的使用需求。
发明内容
本发明要解决的技术问题是为了克服现有技术中可穿戴设备存在要么耗电太大，要么支持的像素太低等问题，从而不能很好地满足用户的使用需求等缺陷，提供一种可穿戴设备及其图像信号处理装置。
本发明是通过下述技术方案来解决上述技术问题：
本发明提供一种可穿戴设备的图像信号处理装置，所述图像信号处理装置包括图像采集设备、图像数据处理模块和应用处理器；
所述图像数据处理模块分别与所述图像采集设备和所述应用处理器通信连接；
所述应用处理器用于在检测到所述可穿戴设备开启拍摄预览模式时，控制所述图像数据处理模块从休眠状态或掉电状态切换至工作状态；
所述图像采集设备用于采集原始图像数据；
所述图像数据处理模块用于获取所述原始图像数据，并对所述原始图像数据进行处理。
较佳地，所述图像数据处理模块包括第一图像数据处理单元和第二图像数据处理单元；
所述图像采集设备与所述第一图像数据处理单元通信连接，所述第二图像数据处理单元分别与所述图像采集设备和所述第一图像数据处理单元通信连接；
所述第一图像数据处理单元用于获取所述原始图像数据，并对所述原始图像数据进行预处理以获取第一图像数据，并发送至所述第二图像数据处理单元；
所述第二图像数据处理单元用于采用目标算法对所述第一图像数据进行处理以获取处理结果，并根据所述处理结果控制所述图像采集设备和/或所述第一图像数据处理单元；
其中，所述第二图像数据处理单元的运行功耗小于所述应用处理器的运行功耗。
较佳地，所述第二图像数据处理单元包括MCU(微控制单元)。
较佳地，所述第一图像数据处理单元包括ISP(图像信号处理器)。
较佳地，所述目标算法包括3A算法，即AF(自动对焦算法)、AE(自动曝光)和AWB(自动白平衡算法)。
较佳地，所述应用处理器用于在检测到所述可穿戴设备关闭拍摄预览模式时，控制所述图像数据处理模块从工作状态切换至休眠状态或掉电状态；
其中，在所述图像数据处理模块处于工作状态时，所述应用处理器自动切换至休眠状态。
较佳地，所述图像信号处理装置还包括图像编码模块、视频编码模块和数据存储模块；
所述图像编码模块、所述视频编码模块均与所述第二图像数据处理单元和所述应用处理器通信连接，所述数据存储模块与所述应用处理器通信连接；
所述图像编码模块用于基于所述处理结果中的图像参数进行图像编码处理以获取图像编码结果并发送至所述应用处理器；
所述视频编码模块用于基于所述处理结果中的图像参数进行视频编码处理以获取视频编码结果并发送至所述应用处理器；
所述应用处理器用于将所述图像编码结果和所述视频编码结果发送至所述数据存储模块进行存储。
较佳地，所述图像信号处理装置还包括显示控制模块和显示模块；
所述显示处理模块分别与所述第二图像数据处理单元和所述显示模块通信连接；
所述显示处理模块用于对所述处理结果进行显示处理以获取显示处理结果并发送至所述显示模块进行显示；和/或，
所述图像信号处理装置还包括数据传输接口；
所述数据传输接口分别与所述图像采集设备和所述第一图像数据存储单元通信连接；
所述第一图像数据存储单元通过所述数据传输接口获取所述图像采集设备采集的所述原始图像数据。
较佳地，所述图像数据处理模块、所述应用处理器、所述数据传输接口、所述图像编码模块、所述视频编码模块、所述数据存储模块和所述显示控制模块均内置在所述可穿戴设备的SOC芯片(系统级芯片)中；和/或，
所述数据传输接口包括MIPI CSI接口(一种相机串行接口)。
本发明还提供一种可穿戴设备，所述可穿戴设备包括上述的可穿戴设备的图像信号处理装置。
在符合本领域常识的基础上，上述各优选条件，可任意组合，即得本发明各较佳实例。
本发明的积极进步效果在于：
本发明中，设计新型的可穿戴设备的ISP架构，实现在设备处于拍摄状态时，采用MCU单独对图像数据进行算法处理，即算法计算过程不再采用AP(应用处理器)，有效地降低了功耗；同时，在拍摄状态时AP自动切换至休眠状态，进一步地降低功耗；另外，采用MCU处理后的处理结果反馈控制图像采集设备选取更合适的参数以采集原始图像数据，并反馈控制ISP选取更合适的参数进行图像数据处理，从而有效地提升了图像信号处理装置的图像处理能力，提高了可穿戴设备的拍照质量，更好地提升了用户的使用体验，满足了用户更高的使用需求。
附图说明
图1为本发明实施例1的可穿戴设备的图像信号处理装置的结构示意图；
图2为本发明实施例1的可穿戴设备的图像信号处理装置的结构示意图；
图3为本发明实施例1的可穿戴设备的图像信号处理装置的结构示意图。
具体实施方式
下面通过实施例的方式进一步说明本发明，但并不因此将本发明限制在所述的实施例范围之中。
实施例1
如图1所示，本实施例的可穿戴设备的图像信号处理装置包括图像采集设备1、图像数据处理模块2和应用处理器3。其中，可穿戴设备包括但不限于智能手表、智能眼镜。
图像数据处理模块2分别与图像采集设备1和应用处理器3通信连接。
正常状态下，可穿戴设备会关闭拍摄预览功能，此时图像数据处理模块2处于休眠状态或掉电状态，只需要运行AP应用处理器即可；只有在开启拍摄预览功能时图像数据处理模块2才会进入工作状态，即图像数据处理模块2无需一直保持在工作状态，有效地节省了运行功耗。
应用处理器3用于在检测到可穿戴设备开启拍摄预览模式时，生成第一控制指令以控制图像数据处理模块2从休眠状态或掉电状态切换至工作状态。
图像采集设备1用于采集原始图像数据；
图像数据处理模块2用于获取原始图像数据，并对原始图像数据进行处理。
另外，应用处理器3用于在检测到可穿戴设备关闭拍摄预览模式 时，生成第二控制指令以控制图像数据处理模块2从工作状态切换至休眠状态或掉电状态；即在未启动拍摄预览功能时，整个图像数据处理模块2不产生或基本不产生功耗，即达到降低运行功耗的效果。
在图像数据处理模块2处于工作状态时，应用处理器3自动切换至休眠状态，即图像处理过程中无需高耗电的应用处理器3参与，进一步地达到降低的功耗效果。
通过将预览模式下图像处理功能的执行操作剥离出来，独立分配至图像数据处理模块2来完成，无需高能耗应用处理器3直接参与处理，从而大幅度地降低程序运行产生的功耗，有效地节省了可穿戴设备在运行过程中的功耗，增强了现有的可穿戴设备的续航能力。
具体地，如图2所示，图像数据处理模块2包括第一图像数据处理单元4和第二图像数据处理单元5，图像采集设备1与第一图像数据处理单元4通信连接，第二图像数据处理单元5分别与图像采集设备1和第一图像数据处理单元4通信连接。
第一图像数据处理单元4用于获取原始图像数据，并对原始图像数据进行预处理以获取第一图像数据，并发送至第二图像数据处理单元5；
第二图像数据处理单元5用于采用目标算法对第一图像数据进行处理以获取处理结果，并根据处理结果控制图像采集设备1和/或第一图像数据处理单元4；
其中，第二图像数据处理单元5的运行功耗小于应用处理器3的运行功耗。
在一个可选实施的方案中，第一图像数据处理单元4为ISP，第二图像数据处理单元5为MCU。其中，MCU的运行功耗远远低于Cortex-A(一种处理器)架构的AP芯片(即应用处理器)。
ISP通过ISP Hardware Pipeline(硬件流水线)对采集的原始图像数据进行预处理，例如对原始图像数据进行AEC(自动曝光控制)、 AWB，Anti-Flicker(去水波纹闪光)等功能的处理，并将处理后的图像数据发送至MCU进行后续处理。
目标算法包括但不限于3A算法，3A算法为在ISP firmware上运行的算法库，包括AE/AF/AWB/AFL等算法，即本实施例中采用功耗较低的MCU作为ISP firmware运行平台来运行3A算法，有效地降低了整体的图像处理过程产生的功耗，有效地提升了可穿戴设备的续航能力。
其中，MCU运行在RTOS操作系统(实时操作系统，一种轻量级的微内核操作系统，不同于Android，Windows，IOS等智能操作系统，可应用于MCU控制器上，能够提供微秒级的响应速度，超低功耗的操作系统)上，同时当将对应用处理器3中进行算法处理的部分移除时，应用处理器3也可采用MCU架构，进一步地降低了程序运行产生的功耗，适用于低功耗的可穿戴设备的应用场景。
MCU包括但不限于ARM-M系列、RISC-V、MIPS、MCS-51(ARM-M系列、RISC-V、MIPS、MCS-51均为微控制单元)。
另外，图像信号处理装置还包括图像编码模块6、视频编码模块7、数据存储模块8、数据传输接口9、显示控制模块10和显示模块11。
图像编码模块6、视频编码模块7均与第二图像数据处理单元5和应用处理器3通信连接，数据存储模块8与应用处理器3通信连接；
图像编码模块6用于基于处理结果中的图像参数进行图像编码处理以获取图像编码结果并发送至应用处理器3；
视频编码模块7用于基于处理结果中的图像参数进行视频编码处理以获取视频编码结果并发送至应用处理器3；其中，处理结果中的图像参数为YUV格式(一种图像格式)的参数数据。
应用处理器3用于将图像编码结果和视频编码结果发送至数据存储模块8进行存储。
数据传输接口9分别与图像采集设备1和第一图像数据存储单元通信连接；
第一图像数据存储单元通过数据传输接口9获取图像采集设备1采集的原始图像数据。
其中，数据传输接口9包括但不限于MIPI CSI接口。
显示处理模块分别与第二图像数据处理单元5和显示模块11通信连接；
显示处理模块用于对处理结果进行显示处理以获取显示处理结果并发送至显示模块11进行显示。其中，显示模块11显示的是RGB格式(一种图像格式)的图像。
其中，图像数据处理模块2、应用处理器3、数据传输接口9、图像编码模块6、视频编码模块7、数据存储模块8和显示控制模块10均内置在可穿戴设备的SOC芯片中。
图像采集设备1为支持RAW Sensor(RAW传感器)类型的Camera，具体地，图像采集设备1包括摄像头和图像传感器，图像传感器通过摄像头获取原始图像数据并通过数据传输接口9传输至第一图像数据处理单元4；图像传感器与第二图像数据处理单元之间通过I2C总线进行数据通信。图像传感器包括但不限于RAW传感器，即支持输出Bayer(拜耳)格式的原始图像数据。
第二图像数据处理单元5的处理结果包括第一控制参数和第二控制参数，第二图像数据处理单元5用于将第一控制参数发送至第一图像数据处理单元4，第一图像数据处理单元4用于根据第一控制参数选择对原始图像数据进行更合理处理以优化图像数据处理方式。
第二图像数据处理单元5还用于将第二控制参数发送至图像采集设备1中的图像传感器，图像传感器用于根据第二控制参数更合理有效地采集原始图像数据以优化图像数据采集方式，从而有效地提升图像信号处理装置的图像处理能力，保证输出的图像质量，达到了自 动调节图像质量的目的，提升了可穿戴设备的使用性能，使得能够满足高质量的拍摄效果体验，满足了用户的像素要求。
还可以通过图3所示的结构示意图获知本实施例的图像信号处理装置的各个组成部分以及连接关系。具体地，Lens(即摄像头)，RAW Sensor(即RAW传感器)，Lens和RAW Sensor构成图像采集设备，用于采集原始图像数据；内置在SOC芯片中的AP(即应用处理器)、ISP(ISP Hardware Pipeline，即第一图像数据处理单元)、MCU(ISP firmware，即第二图像数据处理单元)、MIPI CSI接口(即数据传输接口)、Display Unit(即显示处理模块)、JPEG Encode(即图像编码模块)和Video Encode(即视频编码模块)；LCD(即显示模块)。
另外，当可穿戴设备为智能手表或智能眼镜等智能设备时，本实施例中的可以取消自动对焦功能，仅需支持FF(定焦)摄像头模组即可，这样就无需图像数据处理模块2反馈控制摄像头，减少了对MCU算力要求，从而进一步地降低了可穿戴设备运行产生的功耗。
本实施例中，设计新型的可穿戴设备的ISP架构，实现在设备处于拍摄状态时，采用MCU单独对图像数据进行算法处理，即算法计算过程不再采用AP，有效地降低了功耗；同时，在拍摄状态时AP自动切换至休眠状态，进一步地降低功耗；另外，采用MCU处理后的处理结果反馈控制图像采集设备选取更合适的参数以采集原始图像数据，并反馈控制ISP选取更合适的参数进行图像数据处理，从而有效地提升了图像信号处理装置的图像处理能力，提高了可穿戴设备的拍照质量，达到了自动调节图像质量的目的，更好地提升了用户的使用体验，满足了用户更高的使用需求。
实施例2
本实施例的可穿戴设备包括实施例1的可穿戴设备的图像信号处理装置。其中，可穿戴设备包括但不限于智能手表或智能眼镜。
本实施例的可穿戴设备采用上述的ISP架构，实现在设备处于拍 摄状态时，采用MCU单独对图像数据进行算法处理，即算法计算过程不再采用AP，有效地降低了功耗；同时，在拍摄状态时AP自动切换至休眠状态，进一步地降低功耗，从而有效地节省了功耗；另外，还能够自动调节图像质量，提升了用户的使用体验，满足了用户更高的使用需求的效果。
虽然以上描述了本发明的具体实施方式，但是本领域的技术人员应当理解，这仅是举例说明，本发明的保护范围是由所附权利要求书限定的。本领域的技术人员在不背离本发明的原理和实质的前提下，可以对这些实施方式做出多种变更或修改，但这些变更和修改均落入本发明的保护范围。

Claims (10)

  1. 一种可穿戴设备的图像信号处理装置，其特征在于，所述图像信号处理装置包括图像采集设备、图像数据处理模块和应用处理器；
    所述图像数据处理模块分别与所述图像采集设备和所述应用处理器通信连接；
    所述应用处理器用于在检测到所述可穿戴设备开启拍摄预览模式时，控制所述图像数据处理模块从休眠状态或掉电状态切换至工作状态；
    所述图像采集设备用于采集原始图像数据；
    所述图像数据处理模块用于获取所述原始图像数据，并对所述原始图像数据进行处理。
  2. 如权利要求1所述的可穿戴设备的图像信号处理装置，其特征在于，所述图像数据处理模块包括第一图像数据处理单元和第二图像数据处理单元；
    所述图像采集设备与所述第一图像数据处理单元通信连接，所述第二图像数据处理单元分别与所述图像采集设备和所述第一图像数据处理单元通信连接；
    所述第一图像数据处理单元用于获取所述原始图像数据，并对所述原始图像数据进行预处理以获取第一图像数据，并发送至所述第二图像数据处理单元；
    所述第二图像数据处理单元用于采用目标算法对所述第一图像数据进行处理以获取处理结果，并根据所述处理结果控制所述图像采集设备和/或所述第一图像数据处理单元；
    其中，所述第二图像数据处理单元的运行功耗小于所述应用处理器的运行功耗。
  3. 如权利要求2所述的可穿戴设备的图像信号处理装置，其特征在 于，所述第二图像数据处理单元包括MCU。
  4. 如权利要求2或3所述的可穿戴设备的图像信号处理装置，其特征在于，所述第一图像数据处理单元包括ISP。
  5. 如权利要求4所述的可穿戴设备的图像信号处理装置，其特征在于，所述目标算法包括3A算法。
  6. 如权利要求1所述的可穿戴设备的图像信号处理装置，其特征在于，所述应用处理器用于在检测到所述可穿戴设备关闭拍摄预览模式时，控制所述图像数据处理模块从工作状态切换至休眠状态或掉电状态；
    其中，在所述图像数据处理模块处于工作状态时，所述应用处理器自动切换至休眠状态。
  7. 如权利要求2所述的可穿戴设备的图像信号处理装置，其特征在于，所述图像信号处理装置还包括图像编码模块、视频编码模块和数据存储模块；
    所述图像编码模块、所述视频编码模块均与所述第二图像数据处理单元和所述应用处理器通信连接，所述数据存储模块与所述应用处理器通信连接；
    所述图像编码模块用于基于所述处理结果中的图像参数进行图像编码处理以获取图像编码结果并发送至所述应用处理器；
    所述视频编码模块用于基于所述处理结果中的图像参数进行视频编码处理以获取视频编码结果并发送至所述应用处理器；
    所述应用处理器用于将所述图像编码结果和所述视频编码结果发送至所述数据存储模块进行存储。
  8. 如权利要求7所述的可穿戴设备的图像信号处理装置，其特征在于，所述图像信号处理装置还包括显示控制模块和显示模块；
    所述显示处理模块分别与所述第二图像数据处理单元和所述显示模块通信连接；
    所述显示处理模块用于对所述处理结果进行显示处理以获取显示处理结果并发送至所述显示模块进行显示；和/或，
    所述图像信号处理装置还包括数据传输接口；
    所述数据传输接口分别与所述图像采集设备和所述第一图像数据存储单元通信连接；
    所述第一图像数据存储单元通过所述数据传输接口获取所述图像采集设备采集的所述原始图像数据。
  9. 如权利要求8所述的可穿戴设备的图像信号处理装置，其特征在于，所述图像数据处理模块、所述应用处理器、所述数据传输接口、所述图像编码模块、所述视频编码模块、所述数据存储模块和所述显示控制模块均内置在所述可穿戴设备的SOC芯片中；和/或，
    所述数据传输接口包括MIPI CSI接口。
  10. 一种可穿戴设备，其特征在于，所述可穿戴设备包括权利要求1-9中任一项所述的可穿戴设备的图像信号处理装置。
PCT/CN2021/114896 2020-08-31 2021-08-27 可穿戴设备及其图像信号处理装置 WO2022042671A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/023,437 US20240031672A1 (en) 2020-08-31 2021-08-27 Wearable device and image signal processing apparatus thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010900334.XA CN111988511B (zh) 2020-08-31 2020-08-31 可穿戴设备及其图像信号处理装置
CN202010900334.X 2020-08-31

Publications (1)

Publication Number Publication Date
WO2022042671A1 true WO2022042671A1 (zh) 2022-03-03

Family

ID=73448138

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/114896 WO2022042671A1 (zh) 2020-08-31 2021-08-27 可穿戴设备及其图像信号处理装置

Country Status (3)

Country Link
US (1) US20240031672A1 (zh)
CN (1) CN111988511B (zh)
WO (1) WO2022042671A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111988511B (zh) * 2020-08-31 2021-08-27 展讯通信（上海）有限公司 可穿戴设备及其图像信号处理装置
CN114650364A (zh) * 2020-12-19 2022-06-21 Oppo广东移动通信有限公司 设备控制方法、装置、可穿戴设备和存储介质
CN115460333A (zh) * 2021-06-08 2022-12-09 Oppo广东移动通信有限公司 图像信号处理模块、芯片、电子设备及方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160105162A1 (en) * 2014-10-14 2016-04-14 Uzi Zangi METHODS AND APPARATUSES FOR ULTRA-LOW-POWER SYSTEM ON A CHIP (SoC) ACTIVITY WEARABLE DEVICES
CN105791679A (zh) * 2016-02-29 2016-07-20 广东欧珀移动通信有限公司 一种降低相机功耗的方法、装置及移动终端
CN106506960A (zh) * 2016-11-29 2017-03-15 维沃移动通信有限公司 一种图像数据的处理方法及移动终端
CN106686305A (zh) * 2016-12-22 2017-05-17 深圳众思科技有限公司 电子设备的图像处理方法及电子设备
WO2020077523A1 (zh) * 2018-10-16 2020-04-23 华为技术有限公司 一种用于识别用户行为的方法、芯片、终端
CN111327814A (zh) * 2018-12-17 2020-06-23 华为技术有限公司 一种图像处理的方法及电子设备
CN111988511A (zh) * 2020-08-31 2020-11-24 展讯通信（上海）有限公司 可穿戴设备及其图像信号处理装置

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110133699A (ko) * 2010-06-07 2011-12-14 삼성전자주식회사 휴대용 단말기의 이미지 획득 장치 및 방법
CN104754141B (zh) * 2013-12-30 2018-03-23 展讯通信（上海）有限公司 智能可穿戴设备控制方法和装置
CN105550640B (zh) * 2015-12-07 2019-07-26 联想(北京)有限公司 一种信息处理方法及电子设备
US10592305B2 (en) * 2017-05-16 2020-03-17 Apple Inc. Application session and enhancement for a wearable electronic device
CN107222658A (zh) * 2017-07-28 2017-09-29 合肥芯福传感器技术有限公司 一种降低功耗的装置和方法
CN109979438A (zh) * 2019-04-04 2019-07-05 Oppo广东移动通信有限公司 语音唤醒方法及电子设备
CN110231895A (zh) * 2019-04-25 2019-09-13 努比亚技术有限公司 一种图像通知方法、可穿戴设备及计算机可读存储介质
CN111292488A (zh) * 2020-02-13 2020-06-16 展讯通信（上海）有限公司 图像数据的处理方法、装置及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160105162A1 (en) * 2014-10-14 2016-04-14 Uzi Zangi METHODS AND APPARATUSES FOR ULTRA-LOW-POWER SYSTEM ON A CHIP (SoC) ACTIVITY WEARABLE DEVICES
CN105791679A (zh) * 2016-02-29 2016-07-20 广东欧珀移动通信有限公司 一种降低相机功耗的方法、装置及移动终端
CN106506960A (zh) * 2016-11-29 2017-03-15 维沃移动通信有限公司 一种图像数据的处理方法及移动终端
CN106686305A (zh) * 2016-12-22 2017-05-17 深圳众思科技有限公司 电子设备的图像处理方法及电子设备
WO2020077523A1 (zh) * 2018-10-16 2020-04-23 华为技术有限公司 一种用于识别用户行为的方法、芯片、终端
CN111327814A (zh) * 2018-12-17 2020-06-23 华为技术有限公司 一种图像处理的方法及电子设备
CN111988511A (zh) * 2020-08-31 2020-11-24 展讯通信（上海）有限公司 可穿戴设备及其图像信号处理装置

Also Published As

Publication number Publication date
CN111988511B (zh) 2021-08-27
CN111988511A (zh) 2020-11-24
US20240031672A1 (en) 2024-01-25

Similar Documents

Publication Publication Date Title
WO2022042671A1 (zh) 可穿戴设备及其图像信号处理装置
US20220342475A1 (en) Terminal control method and terminal
US9973702B2 (en) Terminal, and apparatus and method for previewing an image
US9621741B2 (en) Techniques for context and performance adaptive processing in ultra low-power computer vision systems
US6002436A (en) Method and system for auto wake-up for time lapse image capture in an image capture unit
CN105578068B (zh) 一种高动态范围图像的生成方法、装置及移动终端
CN102999298B (zh) 一种摄像模组传输图像数据的方法以及计算机
US8319856B2 (en) Imaging apparatus for calculating a histogram to adjust color balance
JP2003244529A (ja) デジタルカメラ
US10154198B2 (en) Power saving techniques for an image capture device
KR20140128885A (ko) 전력 효율적인 이미지 감지 장치, 이미지 감지 장치의 작동 방법 및 눈/시선 추적 시스템
JP2013115816A (ja) 低電力シャッタラグ除去方法、カメラモジュール及びそれを備えたモバイル機器
CN105812553A (zh) 一种快速拍照方法及移动终端
JP2004015595A (ja) デジタルカメラ
CN103428416A (zh) 一种自动切换终端对焦模式的方法及终端
JP2012519989A (ja) カメラの電池寿命を延ばす方法および機器
US9369633B2 (en) Imaging apparatus, control method, and storage medium storing program
US11304143B2 (en) Terminal device, network device, frame format configuration method, and system
JP2010176061A (ja) 撮影装置、及びプログラム
CN111510629A (zh) 数据显示方法、图像处理器、拍摄装置和电子设备
WO2023077939A1 (zh) 摄像头的切换方法、装置、电子设备及存储介质
CN113259587B (zh) 图像采集设备及其快速启动方法、存储介质、终端
CN112153245B (zh) 控制方法及摄像机
CN202003114U (zh) 镜头自动聚焦控制模块
US10284783B2 (en) Imaging apparatus and control method of imaging apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21860505

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18023437

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21860505

Country of ref document: EP

Kind code of ref document: A1