WO2023225900A1 - Image processing method, mobile terminal and imaging system - Google Patents

Image processing method, mobile terminal and imaging system Download PDF

Info

Publication number
WO2023225900A1
WO2023225900A1 PCT/CN2022/094948 CN2022094948W WO2023225900A1 WO 2023225900 A1 WO2023225900 A1 WO 2023225900A1 CN 2022094948 W CN2022094948 W CN 2022094948W WO 2023225900 A1 WO2023225900 A1 WO 2023225900A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
image data
processing
mobile terminal
processed
Prior art date
Application number
PCT/CN2022/094948
Other languages
French (fr)
Chinese (zh)
Inventor
杨康
刘攀
Original Assignee
北京小米移动软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京小米移动软件有限公司 filed Critical 北京小米移动软件有限公司
Priority to PCT/CN2022/094948 priority Critical patent/WO2023225900A1/en
Publication of WO2023225900A1 publication Critical patent/WO2023225900A1/en

Links

Images

Definitions

  • the present disclosure relates to the field of image processing technology, and specifically to an image processing method, a mobile terminal and an imaging system.
  • images have become an essential information recording carrier for people.
  • electronic devices have the function of capturing images.
  • imaging devices such as portable devices and monitoring equipment that have limited hardware performance or cost, their imaging effects are poor.
  • embodiments of the present disclosure provide an image processing method, a mobile terminal, an imaging system, and a storage medium.
  • an embodiment of the present disclosure provides an image processing method, applied to a mobile terminal, and the method includes:
  • the original image data is processed by the first processing chip of the mobile terminal to obtain image data to be processed; wherein the image data to be processed is data that complies with the communication protocol of the image processing module of the mobile terminal;
  • the target imaging data is sent to the imaging device.
  • the mobile terminal includes a first communication module and a second processing chip; the receiving original image data sent by the imaging device includes:
  • the original image data is processed by the first processing chip of the mobile terminal to obtain the image data to be processed, including:
  • the second processing chip receives the original image data sent by the first communication module, and sends the original image data to the first processing chip through the first data interface;
  • the first processing chip processes the original image data to obtain the image data to be processed.
  • the second processing chip includes the image processing module; performing data processing on the image data to be processed through the image processing module to obtain target imaging data includes:
  • the first processing chip sends the image data to be processed to the second data interface of the second processing chip
  • the image processing module receives the image data to be processed through the second data interface, and performs data processing on the image data to be processed to obtain the target imaging data.
  • sending the target imaging data to the imaging device includes:
  • the image processing module sends the target imaging data to the first processing chip through the second data interface
  • the first processing chip sends the target imaging data to the first data interface of the second processing chip
  • the second processing chip sends the received target imaging data to the imaging device through the first communication module.
  • the receiving original image data sent by the imaging device includes:
  • the original image data is processed by the first processing chip of the mobile terminal to obtain the image data to be processed, including:
  • the first processing chip receives the original image data sent by the first communication module and processes the original image data to obtain the image data to be processed.
  • the image processing module performs data processing on the image data to be processed to obtain target imaging data, including:
  • the image data to be processed is subjected to format conversion processing by the image processing module to obtain the image data to be processed in a target format;
  • the target format is a data format that conforms to the communication protocol of the image processing module.
  • the image processing module performs data processing on the image data to be processed to obtain target imaging data, including:
  • the image processing module performs data processing on the image data to be processed based on a preset 3A algorithm to obtain the target imaging data.
  • an embodiment of the present disclosure provides a mobile terminal, including:
  • a processing system including a first processing chip and a second processing chip, the second processing chip including an image processing module;
  • a storage system stores computer instructions, and the computer instructions are used to cause the processing system to execute the method according to any implementation manner of the first aspect.
  • embodiments of the present disclosure provide an imaging system, including a mobile terminal and an imaging device, where the mobile terminal and the imaging device are communicatively connected;
  • the imaging device includes a camera module, the imaging device obtains original image data through the camera module, and sends the original image data to the mobile terminal;
  • the mobile terminal obtains target imaging data according to the method described in any implementation manner of the first aspect, and sends the target imaging data to the imaging device;
  • the imaging device controls the camera module imaging according to the received target imaging data.
  • the imaging device includes one or more of a wearable device, a monitoring device, or a camera.
  • an embodiment of the present disclosure provides a storage medium that stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any embodiment of the first aspect.
  • the image processing method in the embodiment of the present disclosure includes receiving original image data sent by the imaging device, processing the original image data through the first processing chip of the mobile terminal to obtain image data to be processed, and performing data processing on the image data to be processed through the image processing module Obtain target imaging data and send the target imaging data to the imaging device.
  • relatively high-performance mobile terminal software and hardware are used to improve the imaging quality of the imaging device, and the mobile terminal algorithm can be used to control the camera of the imaging device, which is conducive to opening up the ecological chain of the mobile terminal and the imaging device. Deployment to improve user experience.
  • FIG. 1 is a schematic structural diagram of an imaging system according to some embodiments of the present disclosure.
  • Figure 2 is a structural block diagram of an imaging system according to some embodiments of the present disclosure.
  • FIG. 3 is a flowchart of an image processing method according to some embodiments of the present disclosure.
  • Figure 4 is a schematic diagram of an image processing method according to some embodiments of the present disclosure.
  • Figure 5 is a schematic diagram of an image processing method according to some embodiments of the present disclosure.
  • Figure 6 is a structural block diagram of a mobile terminal according to some embodiments of the present disclosure.
  • the inventor of this case found that with the rapid development of image performance of mobile terminals, mobile terminals have better image processing capabilities than portable devices, monitoring equipment, etc., both in terms of hardware performance and software algorithms. . If the image processing capability of the mobile terminal can be used to process the image data collected by the external imaging device, the imaging effect of the imaging device can obviously be improved.
  • the SoC System on Chip, system-on-chip
  • the SoC does not design a data interface for external data, resulting in the mobile terminal's ISP being unable to process externally acquired image data, such as Qualcomm (Qualcomm) ) platform's SoC cannot directly process image data obtained externally.
  • embodiments of the present disclosure provide an image processing method, a mobile terminal, an imaging system and a storage medium, aiming to realize the acquisition of imaging equipment by using the image processing hardware and algorithms of the mobile terminal through a communicable connection between the mobile terminal and the imaging equipment. Processing of original image data to improve imaging quality and efficiency.
  • FIG. 1 shows a schematic structural diagram of an imaging system in some embodiments of the present disclosure.
  • the imaging system of the example of the present disclosure may include a mobile terminal 100 and an imaging device 200 .
  • the mobile terminal 100 may be a device type such as a smart phone, a tablet computer, etc.
  • the imaging device 200 may be a device type such as a smart watch, a digital camera, a surveillance camera, etc.
  • the imaging device 200 has a camera module 210, so that the imaging device 200 can obtain the original image data of the scene image through the camera module 210.
  • the hardware and software performance of the imaging device 200 are relatively average. Therefore, if the imaging device 200 is used to process the original image data, the final image quality will be poor.
  • the mobile terminal 100 has great advantages over the imaging device 200 in terms of hardware performance and software algorithms.
  • a communicable connection can be established between the mobile terminal 100 and the imaging device 200 in a wireless or wired manner.
  • the two can use wireless communication modules such as Bluetooth, WiFi (wireless fidelity), UWB (Ultra Wide Band, ultra-wideband), etc. Establish a wireless communication connection.
  • the mobile terminal 100 and the imaging device 200 may use, for example, a USB (Universal Serial Bus, Universal Serial Bus) interface to establish a wired communication connection.
  • USB Universal Serial Bus, Universal Serial Bus
  • the device types of the mobile terminal 100 and the imaging device 200 there is no limitation on the device types of the mobile terminal 100 and the imaging device 200 , and they can also be any electronic device type except the example in FIG. 1 .
  • the communicable connection method between the mobile terminal 100 and the imaging device 200 there is no restriction on the communicable connection method between the mobile terminal 100 and the imaging device 200, as long as data transmission between the two can be achieved. This disclosure will not go into details.
  • Figure 2 shows a structural block diagram of an imaging system in some embodiments of the present disclosure.
  • the mobile terminal 100 includes a communication system and a processing system.
  • the communication system includes a first communication module 130 for establishing a communicable connection with the imaging device 200 .
  • the first communication module 130 can be any communication module suitable for establishing a wired or wireless communication connection, including but not limited to USB interface module, WiFi module, Bluetooth module, UWB module, etc., the present disclosure does not make any reference to this. limit.
  • the processing system includes a first processing chip 110 and a second processing chip 120 .
  • the second processing chip 120 may be a system-on-chip SoC (hereinafter referred to as "SoC").
  • the SoC is a highly integrated system-on-chip that implements data processing for the mobile terminal 100. It integrates many system functions such as data processing, image processing, and data storage.
  • common SoCs for mobile terminal devices may include, for example, SoCs based on Qualcomm platforms, SoCs based on IOS platforms, SoCs based on HiSilicon platforms, and so on.
  • the SoC of the mobile terminal 100 is integrated with an image processing module, such as an ISP (Image Signal Process, image signal processing) module.
  • an image processing module such as an ISP (Image Signal Process, image signal processing) module.
  • the ISP module of the SoC can realize data interaction with the local camera module of the mobile terminal through the MIPI bus interface (Mobile Industry Processor Interface) of the SoC, thereby imaging on the mobile terminal 100 .
  • MIPI bus interface Mobile Industry Processor Interface
  • the SoC of the mobile terminal does not reserve an interface for external image data, the ISP module cannot directly process the external image data.
  • the mobile terminal 100 is provided with a first processing chip 110.
  • the first processing chip 110 can be a data conversion chip.
  • the first processing chip 110 can obtain the data sent by the imaging device 200 received through the first communication module 130.
  • Raw image data The original image data is RAW data received by the camera module 210 of the imaging device 200 through the photosensitive device.
  • RAW data refers to unprocessed raw data obtained from the photosensitive device. It has internal colors with a wide color gamut and can be accurately imaged. Adjustments are therefore called raw image data.
  • the first processing chip 110 may perform format conversion on the original image data, and convert the original image data from RAW format to the format required by the image processing module in the second processing chip 120 Target format, in the embodiment of the present disclosure, the image data converted into the target format is defined as the image data to be processed.
  • the second processing chip 120 takes the SoC of the Qualcomm platform as an example.
  • the ISP module in the SoC communicates with the local camera module through the MIPI bus interface. Therefore, in the embodiment of the present disclosure, the first processing chip 110 can obtain the original image data.
  • the RAW format is converted into a MIPI format file and sent to the ISP module through the MIPI bus interface on the second processing chip 120 .
  • the MIPI format image data to be processed sent by the first processing chip 110 is the same as the image data sent by the local camera module, so the ISP module The external image data can be processed in the same way as local camera data.
  • the first processing chip 110 can be, for example, a FPGA (Field Programmable Gate Array) chip.
  • the FPGA chip has the advantages of high computing power and good real-time performance, and can replace the SoC for image data processing. Decompression processing ensures the real-time nature of the imaging process.
  • the imaging device 200 includes a camera module 210 , a control chip 220 , a second communication module 230 and a display module 240 .
  • the camera module 210 is a camera hardware part of the imaging device 200.
  • the camera module 210 may include a lens, a photosensitive device, and peripheral circuits.
  • the control chip 220 is the main processor of the imaging device 200 and is used to implement the sending, receiving and processing of relevant data.
  • the second communication module 230 is used to establish a communicable connection with the first communication module 130 of the mobile terminal 100.
  • the second communication module 230 may use the same wireless or wired communication module as the first communication module 130.
  • the display module 240 may be a display screen, which is used to display the final image.
  • the control chip 220 of the imaging device 200 receives the target imaging data returned by the mobile terminal 100, and performs the processing according to the target
  • the imaging data controls the display module 240 to output a display image.
  • the image processing method of the present disclosure example is executed by the mobile terminal 100, and the image processing method includes:
  • RAW image data that is, the original image data described in this disclosure
  • the imaging device 200 uses the second communication
  • the module 230 sends the original image data to the first communication module 130 of the mobile terminal 100, so that the mobile terminal 100 can receive the original image data.
  • S320 Process the original image data through the first processing chip of the mobile terminal to obtain the image data to be processed.
  • the original image data needs to be processed by the first processing chip 110 to obtain image data to be processed that conforms to the image processing module communication protocol.
  • the data format that conforms to the image processing module communication protocol is defined. for the target format.
  • the ISP module of the current mobile phone is often integrated into the second processing chip 120, and performs data communication with the local camera module through the MIPI bus interface of the second processing chip 120. Therefore, the first processing chip 110 can convert the original image into The data is converted from the RAW format to the MIPI protocol format, and the image data to be processed converted into the MIPI format is sent to the MIPI bus interface of the second processing chip 120.
  • the MIPI protocol format is the target format.
  • the first communication module 130 may directly send the original image data to the first processing chip 110 for processing; it may also send the original image data to the second processing chip 110 for processing.
  • the processing chip 120 is then sent from the second processing chip 120 to the first processing chip 110 for processing.
  • S330 Perform data processing on the image data to be processed through the image processing module to obtain target imaging data.
  • the image processing module is also the ISP module.
  • the image processing module is integrated in the second processing chip 120 and is therefore not shown in the drawing.
  • the image processing module in the present disclosure can also be independent of the second processing chip 120, for example, using an independent ISP chip. Its working principle is the same as the example in Figure 2, and this disclosure does not elaborate on this. Expand.
  • the second processing chip 120 receives the image data to be processed through the MIPI bus interface. It can be understood that for the ISP module in the second processing chip 120, the image data to be processed received through the MIPI bus interface , is the same as the image data sent by the local camera module through the MIPI bus interface, that is, the ISP module can perform data processing on the image data to be processed according to the normal process of processing local data to obtain the target imaging data.
  • the image processing module can call the preset 3A algorithm to process the image data to be processed, and implement image processing processes such as denoising, automatic focus (AF), automatic exposure (AE), automatic white balance (AWB), etc. , to obtain target imaging data with higher image quality.
  • image processing processes such as denoising, automatic focus (AF), automatic exposure (AE), automatic white balance (AWB), etc.
  • AF automatic focus
  • AE automatic exposure
  • AVB automatic white balance
  • the mobile terminal 100 only needs to use existing image processing algorithms to perform relevant processing on the image data to be processed. Those skilled in the art can understand and fully implement it by referring to relevant technologies, and the present disclosure will not elaborate on this. .
  • the target imaging data needs to be sent from the mobile terminal 100 side to the imaging device 200 side, so that the display image is output on the imaging device 200 .
  • the mobile terminal is used to process the image data of the external imaging device, thereby using relatively high-performance mobile terminal software and hardware to improve the imaging quality of the imaging device, and the mobile terminal can be used
  • the camera control of imaging equipment by algorithms is conducive to opening up the ecological chain deployment of mobile terminals and imaging equipment and improving user experience.
  • the first communication module 130 can directly send the original image data to the first processing chip 110 , so that the first processing chip 110
  • the original image data is processed to obtain image data to be processed, and the image data to be processed is sent to the second data interface 112 of the second processing chip 120 .
  • the image processing module in the second processing chip 120 can receive the image data to be processed through the second data interface 112 and perform corresponding data processing on the image data to be processed.
  • the second data interface 112 may be, for example, a MIPI bus interface.
  • the second processing chip 120 is often responsible for receiving and forwarding data. Therefore, in the embodiment of the present disclosure, in order to ensure system stability as much as possible, avoid setting up more peripherals. circuit, to reduce costs, the uplink for external data can be shown in Figure 5.
  • the first communication module 130 may send the original image data to the second processing chip 120 .
  • the second processing chip 120 sends the received original image data to the first processing chip 110 through the first data interface 111 . Therefore, the first processing chip 110 processes the original image data to obtain image data to be processed, and sends the image data to be processed to the second data interface 112 of the second processing chip 120 .
  • the image processing module in the second processing chip 120 can receive the image data to be processed through the second data interface 112 and perform corresponding data processing on the image data to be processed.
  • the first data interface 111 may be, for example, a PCIe (peripheral component interconnect express, high-speed serial computer expansion bus) bus interface
  • the second data interface 112 may be, for example, a MIPI bus interface.
  • the target imaging data can also be sent to the first processing chip 110 through the second data interface 112 of the second processing chip 120 , and then The first processing chip 110 sends the target imaging data to the first data interface 111 of the second processing chip, and then the second processing chip 120 sends the target imaging data to the imaging device 200 through the first communication module 130 .
  • the target imaging data can also be directly processed through the communication link between the second processing chip 120 and the first communication module 130 Sent to the first communication module 130, and sent to the imaging device 200 through the first communication module 130.
  • control chip 220 of the imaging device 200 can control the display module 240 to output a display image according to the target imaging data.
  • the mobile terminal 100 has a better automatic imaging 3A algorithm, so that the target imaging data obtained after the mobile terminal 100 processes the original image data of the imaging device 200 also includes control of the camera module 210 of the imaging device 200 parameters, so that the control chip 220 can adjust parameters of the camera module 210 in real time according to the control parameters, such as automatic focus, automatic exposure, automatic white balance control, etc., to achieve better imaging effects.
  • the control parameters such as automatic focus, automatic exposure, automatic white balance control, etc.
  • the mobile terminal is used to process the image data of the external imaging device, thereby using relatively high-performance mobile terminal software and hardware to improve the imaging quality of the imaging device, and the mobile terminal can be used
  • the camera control of imaging equipment by algorithms is conducive to opening up the ecological chain deployment of mobile terminals and imaging equipment and improving user experience.
  • the mobile terminal 100 may be a smartphone, and the imaging device 200 may be implemented as an external camera provided for the smartphone.
  • the external camera can establish a wireless or wired communicable connection with the smartphone.
  • the image data of the external camera can be processed using the software and hardware of the smartphone, thereby improving the imaging of the external camera. quality.
  • the plug-in camera itself does not require high hardware and algorithm support, it can be implemented at a relatively low cost, thereby quickly laying out the ecological chain deployment of mobile phones and cameras.
  • the full-screen plus external camera can be used to solve the problem of opening the front camera, and the external camera has a higher degree of freedom than the front camera with a fixed opening position.
  • examples of the present disclosure provide a mobile terminal, which may include a processing system and a storage system.
  • the processing system includes the first processing chip 110 and the second processing chip 120 in the above example of FIG. 2, and the second processing chip 120.
  • the processing chip 120 includes an image processing module, such as an ISP module.
  • the storage system may be the memory of the mobile terminal, which stores computer instructions, and the computer instructions are used to cause the processing system to execute the image processing method described in any of the above embodiments.
  • examples of the present disclosure provide a storage medium storing computer instructions for causing a computer to perform the method described in the above embodiments.
  • FIG. 6 shows a structural block diagram of a mobile terminal in some embodiments of the present disclosure.
  • the mobile terminal in some embodiments of the present disclosure will be further described below with reference to FIG. 6 .
  • the mobile terminal 1800 may include one or more of the following components: a processing component 1802, a memory 1804, a power component 1806, a multimedia component 1808, an audio component 1810, an input/output (I/O) interface 1812, a sensor component 1816, and communications component 1818.
  • a processing component 1802 a memory 1804, a power component 1806, a multimedia component 1808, an audio component 1810, an input/output (I/O) interface 1812, a sensor component 1816, and communications component 1818.
  • the processing component 1802 generally controls the overall operations of the mobile terminal 1800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations.
  • Processing component 1802 may include one or more processors 1820 to execute instructions.
  • processing component 1802 may include one or more modules that facilitate interaction between processing component 1802 and other components.
  • processing component 1802 may include a multimedia module to facilitate interaction between multimedia component 1808 and processing component 1802.
  • the processing component 1802 can read executable instructions from the memory to implement electronic device-related functions.
  • the memory 1804 is configured to store various types of data to support operations at the mobile terminal 1800 . Examples of these data include instructions for any application or method operating on the mobile terminal 1800, contact data, phonebook data, messages, pictures, videos, etc.
  • Memory 1804 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EEPROM erasable programmable read-only memory
  • EPROM Programmable read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • magnetic memory flash memory
  • flash memory magnetic or optical disk.
  • Power supply component 1806 provides power to various components of mobile terminal 1800.
  • Power component 1806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to mobile terminal 1800.
  • Multimedia component 1808 includes a display screen that provides an output interface between the mobile terminal 1800 and the user.
  • multimedia component 1808 includes a front-facing camera and/or a rear-facing camera.
  • the front camera and/or the rear camera may receive external multimedia data.
  • Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.
  • Audio component 1810 is configured to output and/or input audio signals.
  • the audio component 1810 includes a microphone (MIC) configured to receive external audio signals when the mobile terminal 1800 is in operating modes, such as call mode, recording mode, and voice recognition mode.
  • the received audio signals may be further stored in memory 1804 or sent via communications component 1818 .
  • audio component 1810 includes a speaker for outputting audio signals.
  • the I/O interface 1812 provides an interface between the processing component 1802 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.
  • Sensor component 1816 includes one or more sensors for providing various aspects of status assessment for mobile terminal 1800 .
  • the sensor component 1816 can detect the open/closed state of the mobile terminal 1800 and the relative positioning of components, such as the display and keypad of the mobile terminal 1800.
  • the sensor component 1816 can detect the mobile terminal 1800 or a component of the mobile terminal 1800.
  • the position changes, the presence or absence of user contact with the mobile terminal 1800, the orientation or acceleration/deceleration of the mobile terminal 1800 and the temperature change of the mobile terminal 1800.
  • Sensor component 1816 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact.
  • Sensor component 1816 may include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor component 1816 may include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 1818 is configured to facilitate wired or wireless communication between the mobile terminal 1800 and other devices.
  • the mobile terminal 1800 can access a wireless network based on communication standards, such as Wi-Fi, 2G, 3G, 4G, 5G or 6G, or a combination thereof.
  • communication component 1818 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.
  • the communications component 1818 includes a near field communications (NFC) module to facilitate short-range communications.
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • the mobile terminal 1800 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable Programmed gate array
  • controller microcontroller, microprocessor or other electronic component implementation.

Landscapes

  • Studio Devices (AREA)

Abstract

The present disclosure relates to the technical field of image processing. Specifically provided are an image processing method, a mobile terminal and an imaging system. The image processing method is applied to the mobile terminal. The method comprises: receiving original image data, which is sent by an imaging device; processing the original image data by means of a first processing chip of a mobile terminal, so as to obtain image data to be processed; performing data processing on said image data by means of an image processing module of the mobile terminal, so as to obtain target imaging data; and sending the target imaging data to the imaging device. By means of the embodiments of the present disclosure, the imaging quality of an imaging device is improved.

Description

图像处理方法、移动终端以及成像系统Image processing method, mobile terminal and imaging system 技术领域Technical field
本公开涉及图像处理技术领域,具体涉及一种图像处理方法、移动终端以及成像系统。The present disclosure relates to the field of image processing technology, and specifically to an image processing method, a mobile terminal and an imaging system.
背景技术Background technique
图像作为视觉信息的基础,已经成为人们必不可少的信息记录载体。目前,越来越多的电子设备具有拍摄图像的功能,但是,对于部分便携设备、监控设备等硬件性能或成本受限的成像设备而言,其成像效果较差。As the basis of visual information, images have become an essential information recording carrier for people. Currently, more and more electronic devices have the function of capturing images. However, for some imaging devices such as portable devices and monitoring equipment that have limited hardware performance or cost, their imaging effects are poor.
发明内容Contents of the invention
为提高成像设备的成像质量,本公开实施方式提供了一种图像处理方法、移动终端、成像系统以及存储介质。In order to improve the imaging quality of the imaging device, embodiments of the present disclosure provide an image processing method, a mobile terminal, an imaging system, and a storage medium.
第一方面,本公开实施方式提供了一种图像处理方法,应用于移动终端,所述方法包括:In a first aspect, an embodiment of the present disclosure provides an image processing method, applied to a mobile terminal, and the method includes:
接收成像设备发送的原始图像数据;Receive raw image data sent by the imaging device;
通过所述移动终端的第一处理芯片对所述原始图像数据进行处理,得到待处理图像数据;其中,所述待处理图像数据为符合所述移动终端的图像处理模块通信协议的数据;The original image data is processed by the first processing chip of the mobile terminal to obtain image data to be processed; wherein the image data to be processed is data that complies with the communication protocol of the image processing module of the mobile terminal;
通过所述图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据;Perform data processing on the image data to be processed by the image processing module to obtain target imaging data;
将所述目标成像数据发送至所述成像设备。The target imaging data is sent to the imaging device.
在一些实施方式中,所述移动终端包括第一通信模块和第二处理芯片;所述接收成像设备发送的原始图像数据,包括:In some embodiments, the mobile terminal includes a first communication module and a second processing chip; the receiving original image data sent by the imaging device includes:
通过所述第一通信模块接收所述成像设备发送的所述原始图像数据;Receive the original image data sent by the imaging device through the first communication module;
所述通过所述移动终端的第一处理芯片对所述原始图像数据进行处理,得到待处理图像数据,包括:The original image data is processed by the first processing chip of the mobile terminal to obtain the image data to be processed, including:
所述第二处理芯片接收所述第一通信模块发送的所述原始图像数据,并将所述原始图像数据通过第一数据接口发送至所述第一处理芯片;The second processing chip receives the original image data sent by the first communication module, and sends the original image data to the first processing chip through the first data interface;
所述第一处理芯片对所述原始图像数据进行处理,得到所述待处理图像数据。The first processing chip processes the original image data to obtain the image data to be processed.
在一些实施方式中,所述第二处理芯片包括所述图像处理模块;所述通过所述图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据,包括:In some embodiments, the second processing chip includes the image processing module; performing data processing on the image data to be processed through the image processing module to obtain target imaging data includes:
所述第一处理芯片将所述待处理图像数据发送至所述第二处理芯片的第二数据接口;The first processing chip sends the image data to be processed to the second data interface of the second processing chip;
所述图像处理模块通过所述第二数据接口接收所述待处理图像数据,并对所述待处理图像处理进行数据处理,得到所述目标成像数据。The image processing module receives the image data to be processed through the second data interface, and performs data processing on the image data to be processed to obtain the target imaging data.
在一些实施方式中,所述将所述目标成像数据发送至所述成像设备,包括:In some embodiments, sending the target imaging data to the imaging device includes:
所述图像处理模块通过所述第二数据接口将所述目标成像数据发送至所述第一处理芯片;The image processing module sends the target imaging data to the first processing chip through the second data interface;
所述第一处理芯片将所述目标成像数据发送至所述第二处理芯片的第一数据接口;The first processing chip sends the target imaging data to the first data interface of the second processing chip;
所述第二处理芯片将接收到的所述目标成像数据通过所述第一通信模块发送至所述成像设备。The second processing chip sends the received target imaging data to the imaging device through the first communication module.
在一些实施方式中,所述接收成像设备发送的原始图像数据,包括:In some embodiments, the receiving original image data sent by the imaging device includes:
通过所述移动终端的第一通信模块接收所述成像设备发送的所述原始图像数据;Receive the original image data sent by the imaging device through the first communication module of the mobile terminal;
所述通过所述移动终端的第一处理芯片对所述原始图像数据进行处理,得到待处理图像数据,包括:The original image data is processed by the first processing chip of the mobile terminal to obtain the image data to be processed, including:
所述第一处理芯片接收所述第一通信模块发送的所述原始图像数据,并对所述原始图像数据进行处理,得到所述待处理图像数据。The first processing chip receives the original image data sent by the first communication module and processes the original image data to obtain the image data to be processed.
在一些实施方式中,所述通过所述图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据,包括:In some embodiments, the image processing module performs data processing on the image data to be processed to obtain target imaging data, including:
通过所述图像处理模块对所述待处理图像数据进行格式转换处理,得到目标格式的所述待处理图像数据;所述目标格式为符合所述图像处理模 块通信协议的数据格式。The image data to be processed is subjected to format conversion processing by the image processing module to obtain the image data to be processed in a target format; the target format is a data format that conforms to the communication protocol of the image processing module.
在一些实施方式中,所述通过图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据,包括:In some embodiments, the image processing module performs data processing on the image data to be processed to obtain target imaging data, including:
所述图像处理模块基于预设的3A算法对所述待处理图像数据进行数据处理,得到所述目标成像数据。The image processing module performs data processing on the image data to be processed based on a preset 3A algorithm to obtain the target imaging data.
第二方面,本公开实施方式提供了一种移动终端,包括:In a second aspect, an embodiment of the present disclosure provides a mobile terminal, including:
处理系统,包括第一处理芯片和第二处理芯片,所述第二处理芯片包括图像处理模块;以及A processing system including a first processing chip and a second processing chip, the second processing chip including an image processing module; and
存储系统,存储有计算机指令,所述计算机指令用于使所述处理系统执行根据第一方面任意实施方式所述的方法。A storage system stores computer instructions, and the computer instructions are used to cause the processing system to execute the method according to any implementation manner of the first aspect.
第三方面,本公开实施方式提供了一种成像系统,包括移动终端和成像设备,所述移动终端和所述成像设备可通信连接;In a third aspect, embodiments of the present disclosure provide an imaging system, including a mobile terminal and an imaging device, where the mobile terminal and the imaging device are communicatively connected;
所述成像设备包括相机模组,所述成像设备通过所述相机模组获取原始图像数据,并向所述移动终端发送所述原始图像数据;The imaging device includes a camera module, the imaging device obtains original image data through the camera module, and sends the original image data to the mobile terminal;
所述移动终端根据第一方面任意实施方式所述的方法得到目标成像数据,并向所述成像设备发送所述目标成像数据;The mobile terminal obtains target imaging data according to the method described in any implementation manner of the first aspect, and sends the target imaging data to the imaging device;
所述成像设备根据接收到的所述目标成像数据控制所述相机模组成像。The imaging device controls the camera module imaging according to the received target imaging data.
在一些实施方式中,所述成像设备包括穿戴设备、监控设备或者相机中的一种或多种。In some embodiments, the imaging device includes one or more of a wearable device, a monitoring device, or a camera.
第四方面,本公开实施方式提供了一种存储介质,存储有计算机指令,所述计算机指令用于使计算机执行根据第一方面任意实施方式所述的方法。In a fourth aspect, an embodiment of the present disclosure provides a storage medium that stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any embodiment of the first aspect.
本公开实施方式的图像处理方法,包括接收成像设备发送的原始图像数据,通过移动终端的第一处理芯片对原始图像数据进行处理得到待处理图像数据,通过图像处理模块对待处理图像数据进行数据处理得到目标成像数据,将目标成像数据发送至成像设备。本公开实施方式中,利用相对较高性能的移动终端软件和硬件,来提高成像设备的成像质量,并且可以利用移动终端算法对成像设备的相机控制,有利于打通移动终端与成像设备的生态链部署,提高用户体验。The image processing method in the embodiment of the present disclosure includes receiving original image data sent by the imaging device, processing the original image data through the first processing chip of the mobile terminal to obtain image data to be processed, and performing data processing on the image data to be processed through the image processing module Obtain target imaging data and send the target imaging data to the imaging device. In the embodiment of the present disclosure, relatively high-performance mobile terminal software and hardware are used to improve the imaging quality of the imaging device, and the mobile terminal algorithm can be used to control the camera of the imaging device, which is conducive to opening up the ecological chain of the mobile terminal and the imaging device. Deployment to improve user experience.
附图说明Description of the drawings
为了更清楚地说明本公开具体实施方式或现有技术中的技术方案,下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图是本公开的一些实施方式,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly explain the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings that need to be used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description The drawings illustrate some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.
图1是根据本公开一些实施方式中成像系统的结构示意图。FIG. 1 is a schematic structural diagram of an imaging system according to some embodiments of the present disclosure.
图2是根据本公开一些实施方式中成像系统的结构框图。Figure 2 is a structural block diagram of an imaging system according to some embodiments of the present disclosure.
图3是根据本公开一些实施方式中图像处理方法的流程图。Figure 3 is a flowchart of an image processing method according to some embodiments of the present disclosure.
图4是根据本公开一些实施方式中图像处理方法的原理图。Figure 4 is a schematic diagram of an image processing method according to some embodiments of the present disclosure.
图5是根据本公开一些实施方式中图像处理方法的原理图。Figure 5 is a schematic diagram of an image processing method according to some embodiments of the present disclosure.
图6是根据本公开一些实施方式中移动终端的结构框图。Figure 6 is a structural block diagram of a mobile terminal according to some embodiments of the present disclosure.
具体实施方式Detailed ways
下面将结合附图对本公开的技术方案进行清楚、完整地描述,显然,所描述的实施方式是本公开一部分实施方式,而不是全部的实施方式。基于本公开中的实施方式,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施方式,都属于本公开保护的范围。此外,下面所描述的本公开不同实施方式中所涉及的技术特征只要彼此之间未构成冲突就可以相互结合。The technical solution of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, but not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this disclosure. In addition, the technical features involved in different embodiments of the present disclosure described below can be combined with each other as long as they do not conflict with each other.
如今,越来越多的电子设备具有拍摄图像的功能,例如智能手机、平板电脑、可穿戴设备、监控设备等,往往均具有相机模组,可以实现图像拍摄。本公开中,定义这些具有图像拍摄功能的电子设备为“成像设备”,但对具体的设备类型不作限制。Nowadays, more and more electronic devices have the function of capturing images, such as smartphones, tablets, wearable devices, monitoring equipment, etc., which often have camera modules that can capture images. In this disclosure, these electronic devices with image capturing functions are defined as "imaging devices", but the specific device type is not limited.
对于部分成像设备,由于受到成本、工业设计等因素的影响,无论硬件性能还是软件算法均较差,导致最终的成像质量较差。例如对于部分可穿戴设备,受限于成本和工业设计,设备的ISP(Image Signal Process,图像信号处理)芯片性能较差,导致最终的成像效果很差,难以满足用户使用。For some imaging equipment, due to factors such as cost and industrial design, both hardware performance and software algorithms are poor, resulting in poor final imaging quality. For example, for some wearable devices, due to cost and industrial design constraints, the device's ISP (Image Signal Process, image signal processing) chip performance is poor, resulting in poor final imaging effects that are difficult to satisfy users.
基于此,本案发明人研究发现,当前,随着移动终端的图像性能飞速发展,移动终端无论在硬件性能,还是软件算法上,相较于便携设备、监控设备等均具有更好的图像处理能力。若是可以利用移动终端的图像处理能力对外部成像设备采集的图像数据进行处理,显然可以提高成像设备的成像效果。Based on this, the inventor of this case found that with the rapid development of image performance of mobile terminals, mobile terminals have better image processing capabilities than portable devices, monitoring equipment, etc., both in terms of hardware performance and software algorithms. . If the image processing capability of the mobile terminal can be used to process the image data collected by the external imaging device, the imaging effect of the imaging device can obviously be improved.
但是,对于移动终端而言,各个平台的SoC(System on Chip,系统级芯片)不会设计针对外部数据的数据接口,导致移动终端的ISP无法对外部获取的图像数据进行处理,例如高通(Qualcomm)平台的SoC,无法直接对外部获取的图像数据进行处理。However, for mobile terminals, the SoC (System on Chip, system-on-chip) of each platform does not design a data interface for external data, resulting in the mobile terminal's ISP being unable to process externally acquired image data, such as Qualcomm (Qualcomm) ) platform's SoC cannot directly process image data obtained externally.
因此,本公开实施方式提供了一种图像处理方法、移动终端、成像系统以及存储介质,旨在通过移动终端与成像设备的可通信连接,实现利用移动终端的图像处理硬件和算法对成像设备获取的原始图像数据的处理,提高成像质量和效率。Therefore, embodiments of the present disclosure provide an image processing method, a mobile terminal, an imaging system and a storage medium, aiming to realize the acquisition of imaging equipment by using the image processing hardware and algorithms of the mobile terminal through a communicable connection between the mobile terminal and the imaging equipment. Processing of original image data to improve imaging quality and efficiency.
图1示出了本公开一些实施方式中成像系统的结构示意图,如图1所示,本公开示例的成像系统可包括移动终端100和成像设备200。移动终端100可以是例如智能手机、平板电脑等设备类型,成像设备200可以是例如智能手表、数码相机、监控摄像头等设备类型。FIG. 1 shows a schematic structural diagram of an imaging system in some embodiments of the present disclosure. As shown in FIG. 1 , the imaging system of the example of the present disclosure may include a mobile terminal 100 and an imaging device 200 . The mobile terminal 100 may be a device type such as a smart phone, a tablet computer, etc., and the imaging device 200 may be a device type such as a smart watch, a digital camera, a surveillance camera, etc.
可以理解,在本公开实施方式中,成像设备200具有相机模组210,从而成像设备200可以通过相机模组210获取场景图像的原始图像数据。但是,成像设备200的硬件和软件性能较为一般,因此若采用成像设备200对原始图像数据进行处理,最终得到的图像质量较差。而移动终端100无论从硬件性能和软件算法上,相较于成像设备200均具有很大优势。It can be understood that in the embodiment of the present disclosure, the imaging device 200 has a camera module 210, so that the imaging device 200 can obtain the original image data of the scene image through the camera module 210. However, the hardware and software performance of the imaging device 200 are relatively average. Therefore, if the imaging device 200 is used to process the original image data, the final image quality will be poor. The mobile terminal 100 has great advantages over the imaging device 200 in terms of hardware performance and software algorithms.
本公开实施方式中,移动终端100与成像设备200之间可以通过无线或者有线的方式建立可通信连接。具体而言,当移动终端100与成像设备200通过无线方式建立通信连接时,两者可以采用例如蓝牙、WiFi(wireless fidelity,无线保真)、UWB(Ultra Wide Band,超宽带)等无线通信模块建立无线通信连接。当移动终端100与成像设备200通过有线方式建立通信连接时,两者可以采用例如USB(Universal Serial Bus,通用串行总线)接口建立有线通信连接。In the embodiment of the present disclosure, a communicable connection can be established between the mobile terminal 100 and the imaging device 200 in a wireless or wired manner. Specifically, when the mobile terminal 100 and the imaging device 200 establish a communication connection wirelessly, the two can use wireless communication modules such as Bluetooth, WiFi (wireless fidelity), UWB (Ultra Wide Band, ultra-wideband), etc. Establish a wireless communication connection. When the mobile terminal 100 and the imaging device 200 establish a communication connection through a wired manner, the two may use, for example, a USB (Universal Serial Bus, Universal Serial Bus) interface to establish a wired communication connection.
总而言之,本公开实施方式中,对移动终端100和成像设备200的设备类型不作任何限制,其还可以是除图1示例之外的任何电子设备类型。另外, 对移动终端100和成像设备200之间的可通信连接方式也不作任何限制,只要能够实现两者之间的数据传输即可。本公开对此不再赘述。In summary, in the embodiments of the present disclosure, there is no limitation on the device types of the mobile terminal 100 and the imaging device 200 , and they can also be any electronic device type except the example in FIG. 1 . In addition, there is no restriction on the communicable connection method between the mobile terminal 100 and the imaging device 200, as long as data transmission between the two can be achieved. This disclosure will not go into details.
图2示出了本公开一些实施方式中成像系统的结构框图,如图2所示,在本公开实施方式中,移动终端100包括通信系统和处理系统。Figure 2 shows a structural block diagram of an imaging system in some embodiments of the present disclosure. As shown in Figure 2, in the embodiment of the present disclosure, the mobile terminal 100 includes a communication system and a processing system.
通信系统包括第一通信模块130,第一通信模块130用于与成像设备200建立可通信连接。在本公开实施方式中,第一通信模块130可以是任何适于建立有线或无线通信连接的通信模块,包括但不限于USB接口模块、WiFi模块、蓝牙模块、UWB模块等,本公开对此不作限制。The communication system includes a first communication module 130 for establishing a communicable connection with the imaging device 200 . In the embodiment of the present disclosure, the first communication module 130 can be any communication module suitable for establishing a wired or wireless communication connection, including but not limited to USB interface module, WiFi module, Bluetooth module, UWB module, etc., the present disclosure does not make any reference to this. limit.
处理系统包括第一处理芯片110和第二处理芯片120。第二处理芯片120可以为系统级芯片SoC(以下简称“SoC”)。The processing system includes a first processing chip 110 and a second processing chip 120 . The second processing chip 120 may be a system-on-chip SoC (hereinafter referred to as "SoC").
SoC为移动终端100实现数据处理的高集成度的片上系统,其集成有例如数据处理、图像处理、数据存储等诸多系统功能。目前,常见的用于移动终端设备的SoC可包括例如基于高通平台的SoC、基于IOS平台的SoC、基于海思平台的SoC等等。The SoC is a highly integrated system-on-chip that implements data processing for the mobile terminal 100. It integrates many system functions such as data processing, image processing, and data storage. Currently, common SoCs for mobile terminal devices may include, for example, SoCs based on Qualcomm platforms, SoCs based on IOS platforms, SoCs based on HiSilicon platforms, and so on.
本公开实施方式中,移动终端100的SoC集成有图像处理模块,例如ISP(Image Signal Process,图像信号处理)模块。相关技术中,SoC的ISP模块可以通过SoC的MIPI总线接口(Mobile Industry Processor Interface,移动产业处理器接口)实现对移动终端的本地相机模块的数据交互,从而在移动终端100上成像。但是,由于移动终端的SoC没有预留针对外部图像数据的接口,因此ISP模块无法实现直接对外部图像数据的处理。In the embodiment of the present disclosure, the SoC of the mobile terminal 100 is integrated with an image processing module, such as an ISP (Image Signal Process, image signal processing) module. In related technology, the ISP module of the SoC can realize data interaction with the local camera module of the mobile terminal through the MIPI bus interface (Mobile Industry Processor Interface) of the SoC, thereby imaging on the mobile terminal 100 . However, since the SoC of the mobile terminal does not reserve an interface for external image data, the ISP module cannot directly process the external image data.
本公开实施方式中,移动终端100中设置有第一处理芯片110,第一处理芯片110可以为数据转换芯片,第一处理芯片110可以获取到通过第一通信模块130接收的成像设备200发送的原始图像数据。原始图像数据为成像设备200的相机模组210通过感光器件接收到的RAW数据,RAW数据是指从感光器件获取的未经处理过的原始数据,具有宽色域的内部色彩,可以进行精确的调整,因此称之为原始图像数据。In the embodiment of the present disclosure, the mobile terminal 100 is provided with a first processing chip 110. The first processing chip 110 can be a data conversion chip. The first processing chip 110 can obtain the data sent by the imaging device 200 received through the first communication module 130. Raw image data. The original image data is RAW data received by the camera module 210 of the imaging device 200 through the photosensitive device. RAW data refers to unprocessed raw data obtained from the photosensitive device. It has internal colors with a wide color gamut and can be accurately imaged. Adjustments are therefore called raw image data.
在一些实施方式中,第一处理芯片110在获取到原始图像数据之后,可以对原始图像数据进行格式转换,将原始图像数据由RAW格式转换为第二处理芯片120中的图像处理模块所需的目标格式,本公开实施方式中,定义转换为目标格式的图像数据为待处理图像数据。In some embodiments, after acquiring the original image data, the first processing chip 110 may perform format conversion on the original image data, and convert the original image data from RAW format to the format required by the image processing module in the second processing chip 120 Target format, in the embodiment of the present disclosure, the image data converted into the target format is defined as the image data to be processed.
例如,第二处理芯片120以高通平台的SoC为例,SoC中的ISP模块通过MIPI总线接口与本地相机模块进行通信,从而本公开实施方式中,第一处理芯片110可将获取的原始图像数据由RAW格式转换为MIPI格式文件,通过第二处理芯片120上的MIPI总线接口发送至ISP模块。可以理解,在此情况下,对于第二处理芯片120中的ISP模块而言,第一处理芯片110发送的MIPI格式的待处理图像数据,与本地相机模块发送的图像数据无异,从而ISP模块即可按照正常处理本地相机数据的方式处理外部图像数据。For example, the second processing chip 120 takes the SoC of the Qualcomm platform as an example. The ISP module in the SoC communicates with the local camera module through the MIPI bus interface. Therefore, in the embodiment of the present disclosure, the first processing chip 110 can obtain the original image data. The RAW format is converted into a MIPI format file and sent to the ISP module through the MIPI bus interface on the second processing chip 120 . It can be understood that in this case, for the ISP module in the second processing chip 120, the MIPI format image data to be processed sent by the first processing chip 110 is the same as the image data sent by the local camera module, so the ISP module The external image data can be processed in the same way as local camera data.
在一些实施方式中,第一处理芯片110可以采用例如FPGA(Field Programmable Gate Array,现场可编程逻辑门阵列)芯片,FPGA芯片具有高运算能力、实时性好的优点,可以代替SoC做图像数据的解压缩处理,保证成像过程的实时性。In some embodiments, the first processing chip 110 can be, for example, a FPGA (Field Programmable Gate Array) chip. The FPGA chip has the advantages of high computing power and good real-time performance, and can replace the SoC for image data processing. Decompression processing ensures the real-time nature of the imaging process.
继续参照图2,成像设备200包括相机模组210、控制芯片220、第二通信模块230以及显示模组240。Continuing to refer to FIG. 2 , the imaging device 200 includes a camera module 210 , a control chip 220 , a second communication module 230 and a display module 240 .
相机模组210为成像设备200的相机硬件部分,例如相机模组210可以包括镜头、感光器件及外围电路等。控制芯片220为成像设备200的主处理器,用于实现相关数据的下发、接收和处理。第二通信模块230用于与移动终端100的第一通信模块130建立可通信连接,第二通信模块230可采用与第一通信模块130相同的无线或者有线通信模块。显示模组240可以为显示屏,其用于显示最终的图像,例如在移动终端100对图像数据处理完成之后,成像设备200的控制芯片220接收到移动终端100返回的目标成像数据,并根据目标成像数据控制显示模组240输出显示图像。The camera module 210 is a camera hardware part of the imaging device 200. For example, the camera module 210 may include a lens, a photosensitive device, and peripheral circuits. The control chip 220 is the main processor of the imaging device 200 and is used to implement the sending, receiving and processing of relevant data. The second communication module 230 is used to establish a communicable connection with the first communication module 130 of the mobile terminal 100. The second communication module 230 may use the same wireless or wired communication module as the first communication module 130. The display module 240 may be a display screen, which is used to display the final image. For example, after the mobile terminal 100 completes processing of the image data, the control chip 220 of the imaging device 200 receives the target imaging data returned by the mobile terminal 100, and performs the processing according to the target The imaging data controls the display module 240 to output a display image.
上述对本公开实施方式的成像系统的结构进行了说明,在上述成像系统的基础上,下面对本公开实施方式的图像处理方法进行说明。The structure of the imaging system according to the embodiment of the present disclosure has been described above. Based on the above imaging system, the image processing method according to the embodiment of the present disclosure will be described below.
如图3所示,在一些实施方式中,本公开示例的图像处理方法由移动终端100执行,图像处理方法包括:As shown in Figure 3, in some implementations, the image processing method of the present disclosure example is executed by the mobile terminal 100, and the image processing method includes:
S310、接收成像设备发送的原始图像数据。S310. Receive original image data sent by the imaging device.
结合图2所示,在成像设备200一侧,相机模组210拍摄场景图像时,通过感光器件可以获取到RAW图像数据,也即本公开所述的原始图像数据,成像设备200通过第二通信模块230将原始图像数据发送至移动终端100的第一通信模块130,从而移动终端100即可接收到原始图像数据。As shown in FIG. 2 , on the side of the imaging device 200 , when the camera module 210 captures a scene image, RAW image data, that is, the original image data described in this disclosure, can be obtained through the photosensitive device. The imaging device 200 uses the second communication The module 230 sends the original image data to the first communication module 130 of the mobile terminal 100, so that the mobile terminal 100 can receive the original image data.
S320、通过移动终端的第一处理芯片对原始图像数据进行处理,得到待处理图像数据。S320: Process the original image data through the first processing chip of the mobile terminal to obtain the image data to be processed.
本公开实施方式中,需要通过第一处理芯片110对原始图像数据进行处理,得到符合图像处理模块通信协议的待处理图像数据,本公开实施方式中,将符合图像处理模块通信协议的数据格式定义为目标格式。In the embodiment of the disclosure, the original image data needs to be processed by the first processing chip 110 to obtain image data to be processed that conforms to the image processing module communication protocol. In the embodiment of the disclosure, the data format that conforms to the image processing module communication protocol is defined. for the target format.
以智能手机为例,目前手机的ISP模块往往集成于第二处理芯片120中,通过第二处理芯片120的MIPI总线接口与本地相机模块进行数据通信,从而,第一处理芯片110可将原始图像数据由RAW格式转换为MIPI协议格式,并将转换为MIPI格式的待处理图像数据发送至第二处理芯片120的MIPI总线接口,MIPI协议格式即为目标格式。Taking smartphones as an example, the ISP module of the current mobile phone is often integrated into the second processing chip 120, and performs data communication with the local camera module through the MIPI bus interface of the second processing chip 120. Therefore, the first processing chip 110 can convert the original image into The data is converted from the RAW format to the MIPI protocol format, and the image data to be processed converted into the MIPI format is sent to the MIPI bus interface of the second processing chip 120. The MIPI protocol format is the target format.
在一些实施方式中,第一通信模块130在接收到原始图像数据(RAW图像数据)之后,可以直接将原始图像数据发送至第一处理芯片110进行处理;也可以将原始图像数据发送至第二处理芯片120,然后由第二处理芯片120发送至第一处理芯片110中进行处理。本公开对此不作限制,并且在下文实施方式中对两种方式分别进行具体说明,在此暂且不表。In some embodiments, after receiving the original image data (RAW image data), the first communication module 130 may directly send the original image data to the first processing chip 110 for processing; it may also send the original image data to the second processing chip 110 for processing. The processing chip 120 is then sent from the second processing chip 120 to the first processing chip 110 for processing. The present disclosure does not limit this, and the two methods will be specifically described in the following embodiments, which are not shown here for the time being.
S330、通过图像处理模块对待处理图像数据进行数据处理,得到目标成像数据。S330: Perform data processing on the image data to be processed through the image processing module to obtain target imaging data.
本公开实施方式中,图像处理模块也即ISP模块。在图2所示中,图像处理模块集成于第二处理芯片120中,因此附图未进行示出。但是,本领域技术人员可以理解,本公开中的图像处理模块也可以独立于第二处理芯片120之外,例如采用独立的ISP芯片,其工作原理与图2示例相同,本公开对此不再展开。In the embodiment of the present disclosure, the image processing module is also the ISP module. In FIG. 2 , the image processing module is integrated in the second processing chip 120 and is therefore not shown in the drawing. However, those skilled in the art can understand that the image processing module in the present disclosure can also be independent of the second processing chip 120, for example, using an independent ISP chip. Its working principle is the same as the example in Figure 2, and this disclosure does not elaborate on this. Expand.
继续参照上述成像系统,第二处理芯片120通过MIPI总线接口接收到待处理图像数据,可以理解,对于第二处理芯片120中的ISP模块来说,其通过MIPI总线接口接收到的待处理图像数据,与本地相机模块通过MIPI总线接口发送的图像数据无异,也即ISP模块可按照正常处理本地数据的流程,对待处理图像数据进行数据处理,得到目标成像数据。Continuing to refer to the above imaging system, the second processing chip 120 receives the image data to be processed through the MIPI bus interface. It can be understood that for the ISP module in the second processing chip 120, the image data to be processed received through the MIPI bus interface , is the same as the image data sent by the local camera module through the MIPI bus interface, that is, the ISP module can perform data processing on the image data to be processed according to the normal process of processing local data to obtain the target imaging data.
在一些实施方式中,图像处理模块可以调用预先设置的3A算法对待处理图像数据进行处理,实现诸如去噪、自动对焦(AF)、自动曝光(AE)、自动白平衡(AWB)等图像处理过程,得到具有较高画质的目标成像数据。可以 理解,本公开实施方式中,移动终端100采用已有的图像处理算法对待处理图像数据进行相关处理即可,本领域技术人员参照相关技术可以理解并充分实现,本公开对此不再具体展开。In some embodiments, the image processing module can call the preset 3A algorithm to process the image data to be processed, and implement image processing processes such as denoising, automatic focus (AF), automatic exposure (AE), automatic white balance (AWB), etc. , to obtain target imaging data with higher image quality. It can be understood that in the embodiments of the present disclosure, the mobile terminal 100 only needs to use existing image processing algorithms to perform relevant processing on the image data to be processed. Those skilled in the art can understand and fully implement it by referring to relevant technologies, and the present disclosure will not elaborate on this. .
S340、将目标成像数据发送至成像设备。S340. Send the target imaging data to the imaging device.
本公开实施方式中,在图像处理模块对待处理图像数据处理完成得到目标成像数据之后,需要将目标成像数据由移动终端100侧发送至成像设备200侧,以使得在成像设备200上输出显示图像。In the embodiment of the present disclosure, after the image processing module completes processing of the image data to be processed to obtain the target imaging data, the target imaging data needs to be sent from the mobile terminal 100 side to the imaging device 200 side, so that the display image is output on the imaging device 200 .
通过上述可知,本公开实施方式中,实现利用移动终端对外部成像设备的图像数据进行处理,从而利用相对较高性能的移动终端软件和硬件,来提高成像设备的成像质量,并且可以利用移动终端算法对成像设备的相机控制,有利于打通移动终端与成像设备的生态链部署,提高用户体验。It can be seen from the above that in the embodiments of the present disclosure, the mobile terminal is used to process the image data of the external imaging device, thereby using relatively high-performance mobile terminal software and hardware to improve the imaging quality of the imaging device, and the mobile terminal can be used The camera control of imaging equipment by algorithms is conducive to opening up the ecological chain deployment of mobile terminals and imaging equipment and improving user experience.
在一些实施方式中,如图4所示,第一通信模块130在接收到原始图像数据(RAW图像数据)之后,可以直接将原始图像数据发送至第一处理芯片110,从而第一处理芯片110对原始图像数据进行处理得到待处理图像数据,并且将待处理图像数据发送至第二处理芯片120的第二数据接口112。第二处理芯片120中的图像处理模块即可通过第二数据接口112接收到待处理图像数据,并对待处理图像数据进行相应的数据处理。第二数据接口112可以是例如MIPI总线接口。In some embodiments, as shown in FIG. 4 , after receiving the original image data (RAW image data), the first communication module 130 can directly send the original image data to the first processing chip 110 , so that the first processing chip 110 The original image data is processed to obtain image data to be processed, and the image data to be processed is sent to the second data interface 112 of the second processing chip 120 . The image processing module in the second processing chip 120 can receive the image data to be processed through the second data interface 112 and perform corresponding data processing on the image data to be processed. The second data interface 112 may be, for example, a MIPI bus interface.
但是,在实际应用场景中,对于移动终端100而言,往往由第二处理芯片120负责数据的接收以及转发,因此本公开实施方式中,为了尽可能保证系统稳定性,避免设置更多的外围电路,降低成本,外部数据的上行链路可如图5所示。However, in actual application scenarios, for the mobile terminal 100, the second processing chip 120 is often responsible for receiving and forwarding data. Therefore, in the embodiment of the present disclosure, in order to ensure system stability as much as possible, avoid setting up more peripherals. circuit, to reduce costs, the uplink for external data can be shown in Figure 5.
在一些实施方式中,如图5所示,第一通信模块130在接收到原始图像数据(RAW图像数据)之后,可以将原始图像数据发送至第二处理芯片120。第二处理芯片120将接收到的原始图像数据通过第一数据接口111发送至第一处理芯片110。从而第一处理芯片110对原始图像数据进行处理得到待处理图像数据,并且将待处理图像数据发送至第二处理芯片120的第二数据接口112。第二处理芯片120中的图像处理模块即可通过第二数据接口112接收到待处理图像数据,并对待处理图像数据进行相应的数据处理。In some implementations, as shown in FIG. 5 , after receiving the original image data (RAW image data), the first communication module 130 may send the original image data to the second processing chip 120 . The second processing chip 120 sends the received original image data to the first processing chip 110 through the first data interface 111 . Therefore, the first processing chip 110 processes the original image data to obtain image data to be processed, and sends the image data to be processed to the second data interface 112 of the second processing chip 120 . The image processing module in the second processing chip 120 can receive the image data to be processed through the second data interface 112 and perform corresponding data processing on the image data to be processed.
第一数据接口111可以是例如PCIe(peripheral component interconnect  express,高速串行计算机扩展总线)总线接口,第二数据接口112可以是例如MIPI总线接口。The first data interface 111 may be, for example, a PCIe (peripheral component interconnect express, high-speed serial computer expansion bus) bus interface, and the second data interface 112 may be, for example, a MIPI bus interface.
在一些实施方式中,继续参照图5所示,在图像处理模块得到目标成像数据之后,同样可以通过第二处理芯片120的第二数据接口112将目标成像数据发送至第一处理芯片110,然后第一处理芯片110将目标成像数据发送至第二处理芯片的第一数据接口111,之后第二处理芯片120将目标成像数据通过第一通信模块130发送至成像设备200。In some embodiments, continuing to refer to FIG. 5 , after the image processing module obtains the target imaging data, the target imaging data can also be sent to the first processing chip 110 through the second data interface 112 of the second processing chip 120 , and then The first processing chip 110 sends the target imaging data to the first data interface 111 of the second processing chip, and then the second processing chip 120 sends the target imaging data to the imaging device 200 through the first communication module 130 .
在另一些实施方式中,考虑到数据传输的效率,在图像处理模块得到目标成像数据之后,也可以通过第二处理芯片120与第一通信模块130之间的通信链路,直接将目标成像数据发送至第一通信模块130,并通过第一通信模块130发送至成像设备200。In other embodiments, considering the efficiency of data transmission, after the image processing module obtains the target imaging data, the target imaging data can also be directly processed through the communication link between the second processing chip 120 and the first communication module 130 Sent to the first communication module 130, and sent to the imaging device 200 through the first communication module 130.
在成像设备200接收到目标成像数据之后,成像设备200的控制芯片220即可根据目标成像数据控制显示模组240输出显示图像.After the imaging device 200 receives the target imaging data, the control chip 220 of the imaging device 200 can control the display module 240 to output a display image according to the target imaging data.
值得说明的是,本公开实施方式中,不仅可以使用移动终端100侧较高性能的系统硬件,同时还可以使用移动终端100侧图像处理模块较好的软件性能。例如,移动终端100具有更好的自动成像3A算法,从而通过移动终端100对成像设备200的原始图像数据进行处理后得到的目标成像数据中,还包括对成像设备200的相机模组210的控制参数,从而控制芯片220可以根据控制参数实时对相机模组210进行参数调整,例如自动对焦、自动曝光、自动白平衡控制等,实现更好的成像效果。It is worth noting that in the embodiment of the present disclosure, not only the higher performance system hardware on the mobile terminal 100 side can be used, but also the better software performance of the image processing module on the mobile terminal 100 side can be used. For example, the mobile terminal 100 has a better automatic imaging 3A algorithm, so that the target imaging data obtained after the mobile terminal 100 processes the original image data of the imaging device 200 also includes control of the camera module 210 of the imaging device 200 parameters, so that the control chip 220 can adjust parameters of the camera module 210 in real time according to the control parameters, such as automatic focus, automatic exposure, automatic white balance control, etc., to achieve better imaging effects.
通过上述可知,本公开实施方式中,实现利用移动终端对外部成像设备的图像数据进行处理,从而利用相对较高性能的移动终端软件和硬件,来提高成像设备的成像质量,并且可以利用移动终端算法对成像设备的相机控制,有利于打通移动终端与成像设备的生态链部署,提高用户体验。It can be seen from the above that in the embodiments of the present disclosure, the mobile terminal is used to process the image data of the external imaging device, thereby using relatively high-performance mobile terminal software and hardware to improve the imaging quality of the imaging device, and the mobile terminal can be used The camera control of imaging equipment by algorithms is conducive to opening up the ecological chain deployment of mobile terminals and imaging equipment and improving user experience.
例如一个示例场景中,移动终端100可以是智能手机,而成像设备200可实现为针对智能手机设置的外挂相机。外挂相机可以与智能手机建立无线或者有线的可通信连接,这样采用本公开上述的图像处理方法,即可实现利用智能手机的软件和硬件对外挂相机的图像数据进行处理,从而提高外挂相机的成像质量。For example, in an example scenario, the mobile terminal 100 may be a smartphone, and the imaging device 200 may be implemented as an external camera provided for the smartphone. The external camera can establish a wireless or wired communicable connection with the smartphone. In this way, using the image processing method described above in the present disclosure, the image data of the external camera can be processed using the software and hardware of the smartphone, thereby improving the imaging of the external camera. quality.
在此场景下,由于外挂相机自身不需要很高的硬件和算法支持,即可以 相对较低的成本落地应用,从而快速铺设手机与相机的生态链部署。而且,对于全面屏手机,即可利用全面屏加上外挂相机的方式,解决前置摄像头开孔的问题,而且外挂相机相较于固定开孔位置的前置摄像头具有更高的自由度。In this scenario, since the plug-in camera itself does not require high hardware and algorithm support, it can be implemented at a relatively low cost, thereby quickly laying out the ecological chain deployment of mobile phones and cameras. Moreover, for full-screen mobile phones, the full-screen plus external camera can be used to solve the problem of opening the front camera, and the external camera has a higher degree of freedom than the front camera with a fixed opening position.
在一些实施方式中,本公开示例提供了一种移动终端,该移动终端可包括处理系统和存储系统,处理系统包括上述图2示例中的第一处理芯片110和第二处理芯片120,第二处理芯片120包括图像处理模块,图像处理模块例如为ISP模块。存储系统可以为移动终端的存储器,其存储有计算机指令,所述计算机指令用于使处理系统执行上述任意实施方式中所述的图像处理方法。In some implementations, examples of the present disclosure provide a mobile terminal, which may include a processing system and a storage system. The processing system includes the first processing chip 110 and the second processing chip 120 in the above example of FIG. 2, and the second processing chip 120. The processing chip 120 includes an image processing module, such as an ISP module. The storage system may be the memory of the mobile terminal, which stores computer instructions, and the computer instructions are used to cause the processing system to execute the image processing method described in any of the above embodiments.
在一些实施方式中,本公开示例提供了一种存储介质,存储介质存储有计算机指令,所述计算机指令用于使计算机执行上述实施方式所述的方法。In some embodiments, examples of the present disclosure provide a storage medium storing computer instructions for causing a computer to perform the method described in the above embodiments.
图6中示出了本公开一些实施方式中的移动终端的结构框图,下面结合图6对本公开一些实施方式的移动终端进一步说明。FIG. 6 shows a structural block diagram of a mobile terminal in some embodiments of the present disclosure. The mobile terminal in some embodiments of the present disclosure will be further described below with reference to FIG. 6 .
参照图6,移动终端1800可以包括以下一个或多个组件:处理组件1802,存储器1804,电源组件1806,多媒体组件1808,音频组件1810,输入/输出(I/O)接口1812,传感器组件1816,以及通信组件1818。Referring to Figure 6, the mobile terminal 1800 may include one or more of the following components: a processing component 1802, a memory 1804, a power component 1806, a multimedia component 1808, an audio component 1810, an input/output (I/O) interface 1812, a sensor component 1816, and communications component 1818.
处理组件1802通常控制移动终端1800的整体操作,诸如与显示,电话呼叫,数据通信,相机操作和记录操作相关联的操作。处理组件1802可以包括一个或多个处理器1820来执行指令。此外,处理组件1802可以包括一个或多个模块,便于处理组件1802和其他组件之间的交互。例如,处理组件1802可以包括多媒体模块,以方便多媒体组件1808和处理组件1802之间的交互。又如,处理组件1802可以从存储器读取可执行指令,以实现电子设备相关功能。The processing component 1802 generally controls the overall operations of the mobile terminal 1800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. Processing component 1802 may include one or more processors 1820 to execute instructions. Additionally, processing component 1802 may include one or more modules that facilitate interaction between processing component 1802 and other components. For example, processing component 1802 may include a multimedia module to facilitate interaction between multimedia component 1808 and processing component 1802. As another example, the processing component 1802 can read executable instructions from the memory to implement electronic device-related functions.
存储器1804被配置为存储各种类型的数据以支持在移动终端1800的操作。这些数据的示例包括用于在移动终端1800上操作的任何应用程序或方法的指令,联系人数据,电话簿数据,消息,图片,视频等。存储器1804可以由任何类型的易失性或非易失性存储设备或者它们的组合实现,如静态随机存取存储器(SRAM),电可擦除可编程只读存储器(EEPROM),可擦除可编程只读存储器(EPROM),可编程只读存储器(PROM),只读存储器(ROM), 磁存储器,快闪存储器,磁盘或光盘。The memory 1804 is configured to store various types of data to support operations at the mobile terminal 1800 . Examples of these data include instructions for any application or method operating on the mobile terminal 1800, contact data, phonebook data, messages, pictures, videos, etc. Memory 1804 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
电源组件1806为移动终端1800的各种组件提供电力。电源组件1806可以包括电源管理系统,一个或多个电源,及其他与为移动终端1800生成、管理和分配电力相关联的组件。 Power supply component 1806 provides power to various components of mobile terminal 1800. Power component 1806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to mobile terminal 1800.
多媒体组件1808包括在所述移动终端1800和用户之间的提供一个输出接口的显示屏。在一些实施例中,多媒体组件1808包括一个前置摄像头和/或后置摄像头。当移动终端1800处于操作模式,如拍摄模式或视频模式时,前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。 Multimedia component 1808 includes a display screen that provides an output interface between the mobile terminal 1800 and the user. In some embodiments, multimedia component 1808 includes a front-facing camera and/or a rear-facing camera. When the mobile terminal 1800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.
音频组件1810被配置为输出和/或输入音频信号。例如,音频组件1810包括一个麦克风(MIC),当移动终端1800处于操作模式,如呼叫模式、记录模式和语音识别模式时,麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器1804或经由通信组件1818发送。在一些实施例中,音频组件1810包括一个扬声器,用于输出音频信号。 Audio component 1810 is configured to output and/or input audio signals. For example, the audio component 1810 includes a microphone (MIC) configured to receive external audio signals when the mobile terminal 1800 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1804 or sent via communications component 1818 . In some embodiments, audio component 1810 includes a speaker for outputting audio signals.
I/O接口1812为处理组件1802和外围接口模块之间提供接口,上述外围接口模块可以是键盘,点击轮,按钮等。这些按钮可包括但不限于:主页按钮、音量按钮、启动按钮和锁定按钮。The I/O interface 1812 provides an interface between the processing component 1802 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.
传感器组件1816包括一个或多个传感器,用于为移动终端1800提供各个方面的状态评估。例如,传感器组件1816可以检测到移动终端1800的打开/关闭状态,组件的相对定位,例如所述组件为移动终端1800的显示器和小键盘,传感器组件1816可以检测移动终端1800或移动终端1800一个组件的位置改变,用户与移动终端1800接触的存在或不存在,移动终端1800方位或加速/减速和移动终端1800的温度变化。传感器组件1816可以包括接近传感器,被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件1816可以包括光传感器,如CMOS或CCD图像传感器,用于在成像应用中使用。在一些实施例中,该传感器组件1816可以包括加速度传感器,陀螺仪传感器,磁传感器,压力传感器或温度传感器。 Sensor component 1816 includes one or more sensors for providing various aspects of status assessment for mobile terminal 1800 . For example, the sensor component 1816 can detect the open/closed state of the mobile terminal 1800 and the relative positioning of components, such as the display and keypad of the mobile terminal 1800. The sensor component 1816 can detect the mobile terminal 1800 or a component of the mobile terminal 1800. The position changes, the presence or absence of user contact with the mobile terminal 1800, the orientation or acceleration/deceleration of the mobile terminal 1800 and the temperature change of the mobile terminal 1800. Sensor component 1816 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor component 1816 may include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1816 may include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
通信组件1818被配置为便于移动终端1800和其他设备之间有线或无线方式的通信。移动终端1800可以接入基于通信标准的无线网络,如Wi-Fi,2G,3G,4G,5G或6G,或它们的组合。在一个示例性实施例中,通信组件 1818经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中,所述通信组件1818包括近场通信(NFC)模块,以促进短程通信。例如,在NFC模块可基于射频识别(RFID)技术,红外数据协会(IrDA)技术,超宽带(UWB)技术,蓝牙(BT)技术和其他技术来实现。The communication component 1818 is configured to facilitate wired or wireless communication between the mobile terminal 1800 and other devices. The mobile terminal 1800 can access a wireless network based on communication standards, such as Wi-Fi, 2G, 3G, 4G, 5G or 6G, or a combination thereof. In one exemplary embodiment, communication component 1818 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communications component 1818 includes a near field communications (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
在示例性实施例中,移动终端1800可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现。In an exemplary embodiment, the mobile terminal 1800 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Programmed gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation.
显然,上述实施方式仅仅是为清楚地说明所作的举例,而并非对实施方式的限定。对于所属领域的普通技术人员来说,在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。而由此所引伸出的显而易见的变化或变动仍处于本公开创造的保护范围之中。Obviously, the above-mentioned embodiments are only examples for clear explanation and are not limitations of the embodiments. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. An exhaustive list of all implementations is neither necessary nor possible. The obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims (11)

  1. 一种图像处理方法,其特征在于,应用于移动终端,所述方法包括:An image processing method, characterized in that it is applied to a mobile terminal, and the method includes:
    接收成像设备发送的原始图像数据;Receive raw image data sent by the imaging device;
    通过所述移动终端的第一处理芯片对所述原始图像数据进行处理,得到待处理图像数据;其中,所述待处理图像数据为符合所述移动终端的图像处理模块通信协议的数据;The original image data is processed by the first processing chip of the mobile terminal to obtain image data to be processed; wherein the image data to be processed is data that complies with the communication protocol of the image processing module of the mobile terminal;
    通过所述图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据;Perform data processing on the image data to be processed by the image processing module to obtain target imaging data;
    将所述目标成像数据发送至所述成像设备。The target imaging data is sent to the imaging device.
  2. 根据权利要求1所述的方法,其特征在于,所述移动终端包括第一通信模块和第二处理芯片;所述接收成像设备发送的原始图像数据,包括:The method of claim 1, wherein the mobile terminal includes a first communication module and a second processing chip; and receiving the original image data sent by the imaging device includes:
    通过所述第一通信模块接收所述成像设备发送的所述原始图像数据;Receive the original image data sent by the imaging device through the first communication module;
    所述通过所述移动终端的第一处理芯片对所述原始图像数据进行处理,得到待处理图像数据,包括:The original image data is processed by the first processing chip of the mobile terminal to obtain the image data to be processed, including:
    所述第二处理芯片接收所述第一通信模块发送的所述原始图像数据,并将所述原始图像数据通过第一数据接口发送至所述第一处理芯片;The second processing chip receives the original image data sent by the first communication module, and sends the original image data to the first processing chip through the first data interface;
    所述第一处理芯片对所述原始图像数据进行处理,得到所述待处理图像数据。The first processing chip processes the original image data to obtain the image data to be processed.
  3. 根据权利要求2所述的方法,其特征在于,所述第二处理芯片包括所述图像处理模块;所述通过所述图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据,包括:The method according to claim 2, wherein the second processing chip includes the image processing module; the image data to be processed is processed by the image processing module to obtain target imaging data, include:
    所述第一处理芯片将所述待处理图像数据发送至所述第二处理芯片的第二数据接口;The first processing chip sends the image data to be processed to the second data interface of the second processing chip;
    所述图像处理模块通过所述第二数据接口接收所述待处理图像数据,并对所述待处理图像处理进行数据处理,得到所述目标成像数据。The image processing module receives the image data to be processed through the second data interface, and performs data processing on the image data to be processed to obtain the target imaging data.
  4. 根据权利要求3所述的方法,其特征在于,所述将所述目标成像数据发送至所述成像设备,包括:The method of claim 3, wherein sending the target imaging data to the imaging device includes:
    所述图像处理模块通过所述第二数据接口将所述目标成像数据发送至所述第一处理芯片;The image processing module sends the target imaging data to the first processing chip through the second data interface;
    所述第一处理芯片将所述目标成像数据发送至所述第二处理芯片的第一数据接口;The first processing chip sends the target imaging data to the first data interface of the second processing chip;
    所述第二处理芯片将接收到的所述目标成像数据通过所述第一通信模块发送至所述成像设备。The second processing chip sends the received target imaging data to the imaging device through the first communication module.
  5. 根据权利要求1所述的方法,其特征在于,所述接收成像设备发送的原始图像数据,包括:The method according to claim 1, wherein the receiving original image data sent by the imaging device includes:
    通过所述移动终端的第一通信模块接收所述成像设备发送的所述原始图像数据;Receive the original image data sent by the imaging device through the first communication module of the mobile terminal;
    所述通过所述移动终端的第一处理芯片对所述原始图像数据进行处理,得到待处理图像数据,包括:The original image data is processed by the first processing chip of the mobile terminal to obtain the image data to be processed, including:
    所述第一处理芯片接收所述第一通信模块发送的所述原始图像数据,并对所述原始图像数据进行处理,得到所述待处理图像数据。The first processing chip receives the original image data sent by the first communication module, and processes the original image data to obtain the image data to be processed.
  6. 根据权利要求1至5任一项所述的方法,其特征在于,所述通过所述图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据,包括:The method according to any one of claims 1 to 5, wherein the image processing module performs data processing on the image data to be processed to obtain target imaging data, including:
    通过所述图像处理模块对所述待处理图像数据进行格式转换处理,得到目标格式的所述待处理图像数据;所述目标格式为符合所述图像处理模块通信协议的数据格式。The image data to be processed is subjected to format conversion processing by the image processing module to obtain the image data to be processed in a target format; the target format is a data format that complies with the communication protocol of the image processing module.
  7. 根据权利要求1至5任一项所述的方法,其特征在于,所述通过图像处理模块对所述待处理图像数据进行数据处理,得到目标成像数据,包括:The method according to any one of claims 1 to 5, characterized in that the image processing module performs data processing on the image data to be processed to obtain target imaging data, including:
    所述图像处理模块基于预设的3A算法对所述待处理图像数据进行数据处理,得到所述目标成像数据。The image processing module performs data processing on the image data to be processed based on a preset 3A algorithm to obtain the target imaging data.
  8. 一种移动终端,其特征在于,包括:A mobile terminal, characterized by including:
    处理系统,包括第一处理芯片和第二处理芯片,所述第二处理芯片包括图像处理模块;以及A processing system including a first processing chip and a second processing chip, the second processing chip including an image processing module; and
    存储系统,存储有计算机指令,所述计算机指令用于使所述处理系统执行根据权利要求1至7任一项所述的方法。A storage system stores computer instructions, which are used to cause the processing system to execute the method according to any one of claims 1 to 7.
  9. 一种成像系统,其特征在于,包括移动终端和成像设备,所述移动终端和所述成像设备可通信连接;An imaging system, characterized in that it includes a mobile terminal and an imaging device, and the mobile terminal and the imaging device are communicably connected;
    所述成像设备包括相机模组,所述成像设备通过所述相机模组获取原始图像数据,并向所述移动终端发送所述原始图像数据;The imaging device includes a camera module, the imaging device obtains original image data through the camera module, and sends the original image data to the mobile terminal;
    所述移动终端根据权利要求1至7任一项所述的方法得到目标成像数据,并向所述成像设备发送所述目标成像数据;The mobile terminal obtains target imaging data according to the method according to any one of claims 1 to 7, and sends the target imaging data to the imaging device;
    所述成像设备根据接收到的所述目标成像数据控制所述相机模组成像。The imaging device controls the camera module imaging according to the received target imaging data.
  10. 根据权利要求9所述的成像系统,其特征在于,The imaging system according to claim 9, characterized in that:
    所述成像设备包括穿戴设备、监控设备或者相机中的一种或多种。The imaging device includes one or more of a wearable device, a monitoring device, or a camera.
  11. 一种存储介质,其特征在于,存储有计算机指令,所述计算机指令用于使计算机执行根据权利要求1至7任一项所述的方法。A storage medium, characterized in that computer instructions are stored, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 7.
PCT/CN2022/094948 2022-05-25 2022-05-25 Image processing method, mobile terminal and imaging system WO2023225900A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/094948 WO2023225900A1 (en) 2022-05-25 2022-05-25 Image processing method, mobile terminal and imaging system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/094948 WO2023225900A1 (en) 2022-05-25 2022-05-25 Image processing method, mobile terminal and imaging system

Publications (1)

Publication Number Publication Date
WO2023225900A1 true WO2023225900A1 (en) 2023-11-30

Family

ID=88918060

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/094948 WO2023225900A1 (en) 2022-05-25 2022-05-25 Image processing method, mobile terminal and imaging system

Country Status (1)

Country Link
WO (1) WO2023225900A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003319311A (en) * 2002-04-23 2003-11-07 Fuji Photo Film Co Ltd Image processing apparatus and system
CN105516423A (en) * 2015-12-24 2016-04-20 努比亚技术有限公司 Mobile terminal, data transmission system and mobile terminal shoot method
CN111164983A (en) * 2018-05-09 2020-05-15 华为技术有限公司 Lending local processing capability by interconnection terminal
CN111726516A (en) * 2019-10-23 2020-09-29 北京小米移动软件有限公司 Image processing method and device
CN111953904A (en) * 2020-08-13 2020-11-17 北京达佳互联信息技术有限公司 Shooting method, shooting device, electronic equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003319311A (en) * 2002-04-23 2003-11-07 Fuji Photo Film Co Ltd Image processing apparatus and system
CN105516423A (en) * 2015-12-24 2016-04-20 努比亚技术有限公司 Mobile terminal, data transmission system and mobile terminal shoot method
CN111164983A (en) * 2018-05-09 2020-05-15 华为技术有限公司 Lending local processing capability by interconnection terminal
CN111726516A (en) * 2019-10-23 2020-09-29 北京小米移动软件有限公司 Image processing method and device
CN111953904A (en) * 2020-08-13 2020-11-17 北京达佳互联信息技术有限公司 Shooting method, shooting device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
JP6945744B2 (en) Shooting methods, devices, and devices
JP7378576B2 (en) Terminal device, method and system for implementing one-touch screen projection using remote control device
JP6238184B2 (en) Imaging device
WO2020244623A1 (en) Air-mouse mode implementation method and related device
US11825539B2 (en) Bluetooth connection method and electronic device
CN112449099B (en) Image processing method, electronic equipment and cloud server
CN112469013B (en) Bluetooth connection method and related device
US11082600B2 (en) Electronic apparatus that performs wireless communication with an image capturing device at two different communication speeds, and method for controlling same
JP2016029788A (en) Communication device, control method of communication device, and program
US10348870B2 (en) Information processing apparatus, electronic apparatus, and control method therefor
US20180063790A1 (en) Electronic device and method for controlling the same
US9560476B2 (en) Wireless communication apparatus capable of communicating with external apparatus, controlling method for wireless communication apparatus, and storage medium
WO2023225900A1 (en) Image processing method, mobile terminal and imaging system
JP2016163270A (en) Display device, and image display method and program
CN111698414B (en) Image signal processing method and device, electronic device and readable storage medium
CN111800836A (en) Communication method, communication device, electronic equipment and storage medium
JP6468827B2 (en) IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, PROGRAM
CN217160027U (en) Electronic device and data communication system
US11503656B2 (en) Communication apparatus, method for controlling communication apparatus, and storage medium
CN117119314B (en) Image processing method and related electronic equipment
US11178343B2 (en) Combining images from different devices according to a determined wipe shape
US20230188835A1 (en) Imaging device, imaging instruction method, and imaging instruction program
US11206345B2 (en) Communication apparatus, control method therefor, and recording medium therefor
WO2023077938A1 (en) Video frame generation method and apparatus, electronic device, and storage medium
JP2018011269A (en) Communication device, control method thereof, and program