WO2022218080A1

WO2022218080A1 - Prepositive image signal processing apparatus and related product

Info

Publication number: WO2022218080A1
Application number: PCT/CN2022/080676
Authority: WO
Inventors: 曾玉宝
Original assignee: Oppo广东移动通信有限公司
Priority date: 2021-04-17
Filing date: 2022-03-14
Publication date: 2022-10-20
Also published as: CN115225828A; CN115225828B

Abstract

Provided in the present application are a prepositive image signal processing apparatus and a related product, wherein the prepositive image signal processing apparatus is coupled between a camera apparatus and an application processing apparatus. The prepositive image signal processing apparatus comprises: an image signal processing module, which is used for generating multi-path image statistical information for image data of a plurality of image channels; a central processing module, which is used for generating comprehensive statistical information of the image channels on the basis of the multi-path image statistical information; and a first transmission port module, which is used for transmitting the comprehensive statistical information of the image channels to an application processing apparatus. By means of the embodiments of the present application, reducing the hardware overheads and power consumption of transmitting statistical information of a plurality of image channels is facilitated, and the resource utilization rate is improved.

Description

Front image signal processing device and related products

technical field

The application belongs to the technical field of image signal processing chips, and in particular relates to a front-end image signal processing device and related products.

Background technique

At present, when the mobile phone camera function is in the blurring mode, it needs to open multiple cameras at the same time, and the image channel statistics of each camera (for example, the automatic exposure AE statistics in the 3A statistics, etc.) will pass through the front image signal processor. The corresponding single channel is sent back to the application processing device. Therefore, the statistical information of multiple image channels corresponding to multiple cameras needs to be turned on at the same time in the front image signal processor to realize the return transmission, which leads to the front image signal. Channel hardware overhead and power consumption in the processor is high.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a pre-image signal processing device and related products, so as to reduce hardware overhead and power consumption for transmitting statistical information of multiple image channels and improve resource utilization.

In a first aspect, an embodiment of the present application provides a front image signal processing device, which is coupled between a camera device and an application processing device, including:

an image signal processing module for generating multi-channel image statistics for image data of multiple image channels;

a central processing module, configured to generate comprehensive statistical information of image channels based on the multi-channel image statistical information;

A first transmission port module, configured to transmit the comprehensive statistical information of the image channel to the application processing device.

It can be seen that, in the embodiment of the present application, since the pre-image signal processing apparatus can combine the statistical information of multiple channels into the comprehensive statistical information of the image channels, and transmit the comprehensive statistical information of the image channels to the application processing apparatus through a single first transmission port module Statistics. Compared with the solution that requires multiple channels to be enabled at the same time to realize the return of multiple image channel statistical information, it does not need to occupy the interface resources of other transmission port modules to transmit image channel statistical information, which is beneficial to reduce the transmission of multiple image channels by the front image signal processing device. Statistics of hardware overhead and power consumption to improve resource utilization.

In a second aspect, an embodiment of the present application provides an image statistical information processing chip, which is connected to a camera device, including:

A front image signal processing device, electrically connected to the camera device, and configured to perform the function of the device according to any one of the first aspects.

The application processing device is electrically connected to the image signal processing device, and is configured to receive the comprehensive statistical information of the image channels, and determine the statistical results of the image channels of each camera unit according to the comprehensive statistical information of the image channels.

It can be seen that, in the embodiment of the present application, since the pre-image signal processing apparatus can combine the statistical information of multiple channels into the comprehensive statistical information of the image channels, and transmit the comprehensive statistical information of the image channels to the application processing apparatus through a single first transmission port module Statistics. Compared with the scheme that requires multiple channels to open at the same time to realize the return of multiple image channel statistics, it does not need to occupy the interface resources of other transmission port modules to transmit image channel statistics, which is beneficial to reduce the image statistics processing chip. Information hardware overhead and power consumption to improve resource utilization. At the same time, the efficiency of the image statistical information processing chip in calculating the statistical results of the image channels of each camera unit is improved.

In a third aspect, an embodiment of the present application provides an electronic device, including:

a camera device for collecting image data of multiple image channels;

An image statistical information processing chip, electrically connected to the camera device, for implementing the function of the chip according to any one of the second aspects.

It can be seen that, in the embodiment of the present application, since the pre-image signal processing apparatus can combine the statistical information of multiple channels into the comprehensive statistical information of the image channels, and transmit the comprehensive statistical information of the image channels to the application processing apparatus through a single first transmission port module Statistics. Compared with the solution that requires multiple channels to open at the same time to realize the backhaul, it does not need to occupy the interface resources of other transmission port modules to transmit the image channel statistics, which is beneficial to reduce the hardware for electronic devices to transmit the statistics of multiple image channels. Overhead and power consumption, improve resource utilization. At the same time, the efficiency of calculating the image channel statistics of each camera unit by the electronic device is improved.

In a fourth aspect, an embodiment of the present application provides an image information transmission method, including:

Generate multi-channel image statistics of multiple image channel image data of multiple camera modules;

generating comprehensive statistical information of image channels based on the multi-channel image statistical information;

Transmitting the image channel synthesis statistics.

It can be seen that, in the embodiment of the present application, since the device can combine the statistical information of the multi-channel image into the comprehensive statistical information of the image channel of a single channel, and transmit the comprehensive statistical information of the image channel. Compared with the scheme that requires multiple channels to be enabled at the same time to realize the transmission of the statistical information of multiple image channels, it does not need to occupy the interface resources of other transmission port modules to transmit the statistical information of the image channel, which is beneficial to reduce the hardware overhead and cost of the device to transmit the statistical information of multiple image channels. power consumption and improve resource utilization.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1a is a schematic structural diagram of a front image signal processing apparatus 100 provided by an embodiment of the present application;

FIG. 1b is a schematic structural diagram of another front image signal processing apparatus 100 provided by an embodiment of the present application;

FIG. 1c is a schematic structural diagram of another front image signal processing apparatus provided by an embodiment of the present application;

1d is a schematic structural diagram of a transmission port module provided by an embodiment of the present application;

FIG. 2 is a schematic structural diagram of an image statistical information processing chip 10 provided by an embodiment of the present application;

3 is a block diagram of functional units of an apparatus for determining a statistical result of an image channel provided by an embodiment of the present application;

4a is a block diagram of functional units of another image channel statistical result determination device provided in an embodiment of the present application;

FIG. 4b is an architecture diagram of a software system provided by an embodiment of the present application for supporting the image information transmission method of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

At present, the mobile phone camera is used in the mode of blurring, which requires multiple cameras to be turned on at the same time. The image channel statistics of each camera must be counted and returned by the image signal processing device. Specifically, the image channel statistics of each camera. It will be counted by the corresponding front ISP, and transmitted to the application processing device through the corresponding transmission port module. For example, for a 3DOL (Sony SONY multi-exposure) camera, it is necessary to output three channels of long, medium and short, and the data of each channel must be counted. 3A information, if there are two 3DOL cameras in the system, 3 channels*2 sensors*3A need to be counted, and at least 18 types of statistical data need to be used and transmitted, corresponding to 2 transmission interfaces corresponding to the 2 3DOL cameras. The module transmits data and occupies a lot of interface resources.

In view of the above problems, the present application proposes a pre-image signal processing device and related products, which will be described in detail below with reference to the accompanying drawings.

Please refer to FIG. 1a. FIG. 1a is a schematic structural diagram of a front image signal processing apparatus 100 provided by an embodiment of the present application. The front image signal processing apparatus 100 is coupled between the camera apparatus 200 and the application processing apparatus 300, and includes :

an image signal processing module 110, configured to generate multi-channel image statistical information for image data of multiple image channels;

a central processing module 120, configured to generate comprehensive statistical information of image channels based on the multi-channel image statistical information;

The first transmission port module 131 is configured to transmit the comprehensive statistical information of the image channel to the application processing apparatus 300 .

The camera device 200 includes a plurality of camera units, and the types of camera units include main camera, wide-angle, telephoto, etc., which are not limited here. The image channels are in one-to-one correspondence with the camera units.

Wherein, the first transmission port module 131 may be, for example, a mobile industry processor interface (Mobile Industry Processor Interface, MIPI).

Wherein, the image statistical information may be AE statistical information and/or automatic white balance AWB statistical information in the 3A statistical information, the AE statistical information includes R, G, B histogram and/or brightness information, and the AWB statistical information may be It includes at least one of the following: the number of dots of red R pixels, green G pixels, and blue B pixels, the ratio of red pixels to green pixels, and the ratio of blue pixels to green pixels.

In a specific implementation, the comprehensive statistical information of the image channel may be the superposition of the statistical information of the multi-channel images, that is, the information content is unchanged, and only superimposed together for transmission, so that the overall data volume is basically not increased, and additional data transmission will not be brought. burden. In addition, the comprehensive statistical information of the image channels may also be achieved by reducing the amount of data through data fusion, so as to improve the overall data transmission efficiency.

Wherein, the image channel statistical results may be AE statistical results and/or AWB statistical results in the 3A statistical results, the AE statistical results may include exposure time and/or exposure gain, and the AWB statistical results may include, for example, at least the following One: R channel gain, G channel gain, B channel gain and color correction matrix.

In a possible example, before generating the comprehensive statistical information of image channels based on the statistical information of the multi-channel images, the central processing module 120 is further configured to: align coefficients based on the capabilities of multiple camera units in the camera device 200, Perform capability alignment processing on the multi-channel image statistical information to obtain the multi-channel image statistical information after capability alignment, wherein the capability alignment coefficient is used to describe the difference in the degree of response to color between the multiple camera units .

In the specific implementation, the capability alignment refers to that for each camera module, the image signal processing apparatus of the electronic device updates the image statistical information of the current camera module according to the preconfigured capability alignment coefficient of the current camera module, so as to obtain the capability alignment After the current camera module image statistics.

The capability alignment coefficient may be a capability alignment coefficient for the image signal of the camera module, or a converted capability alignment coefficient directly for image statistical information, which is not uniquely limited here.

In the specific implementation, the electronic device can perform capability alignment with respect to the statistical information of the image data in the overlapping area of the screen, and specifically can adjust the Tuning (Tuning refers to the Tuning tool specially used for calculating camera-related parameters in the existing solution) conditions for a single shot. Next, confirm the FOV values of the multiple viewing angles corresponding to the multiple camera modules one-to-one, and then determine the overlapping area of the images of the multiple camera modules according to the multiple FOV values.

It can be seen that in this example, due to the performance differences of the camera units, there are obvious differences in the image signals collected by each camera unit, resulting in differences in the corresponding image statistics. Matching to the same standard to overcome the error caused by the difference in sensor performance, thus providing a unified data basis for subsequent data processing and improving accuracy.

In this possible example, the central processing module 120 is further configured to:

Determine whether the camera unit corresponding to the currently processed image statistical information is a reference camera unit;

If so, determining the currently processed image statistical information as the capability-aligned image statistical information;

If not, the information statistics capability alignment coefficient of the camera unit is acquired, and the image statistics information after capability alignment is determined according to the currently processed image statistics information and the information statistics capability alignment coefficient.

In the specific implementation, the information statistical capability alignment of the image statistical information is expressed by the following formula:

Ystats'=K _X *Ystats+B _X ,

Among them, Ystats represents the original image statistical information of the currently processed camera module, Ystats' represents the final image statistical information after the information statistical capability of the currently processed camera module is aligned, and K _X represents the currently processed camera module's information statistical capability alignment The first-order coefficient, B _X represents the alignment compensation coefficient of the information statistics capability of the currently processed camera module.

It can be seen that, in this example, the electronic device aligns the statistical capabilities of the image statistics of other camera modules other than the reference camera module, so that the image statistics after the multiple capabilities of the multiple camera modules are aligned can overcome the performance differences of the sensors. The resulting errors can provide a unified data basis for subsequent data processing and improve accuracy.

In a possible example, the central processing module 120 is further configured to:

acquiring a plurality of image data corresponding to the camera 200;

aligning the responsiveness of the acquisition signal for each image data in the plurality of image data to obtain a plurality of image data after the responsiveness of the acquisition signal is aligned;

Corresponding capability-aligned image statistical information is determined according to each image data in the plurality of image data whose response capabilities of the collected signals are aligned, and the capability-aligned multi-channel image statistical information is obtained.

Determine whether the camera unit corresponding to the currently processed image data is a reference camera unit;

If yes, then determining the currently processed image data as the image data after capability alignment;

If not, obtain the pre-configured acquisition signal responsiveness alignment coefficient of the camera module, and determine the image after the acquisition signal responsiveness alignment according to the currently processed image statistical information and the acquisition signal responsiveness alignment coefficient data.

In the specific implementation, the alignment of the collected signal response capabilities of the image data is expressed by the following formula:

Xstats'=K _X *Xstats+B _X

Among them, Xstats represents the original signal response value of the color X channel in the image data of the currently processed camera module, Xstats' represents the final signal response value after alignment of the acquired signal response capability of the color X channel of the currently processed camera module, K _X represents The first-order coefficient is aligned with the response capability of the acquired signal of the sensor color X channel of the currently processed camera module, and B _X represents the alignment compensation coefficient of the acquired signal response capability of the currently processed color X channel of the camera module.

For example, taking the RGB encoding scheme as an example, the formula can be:

R0=Kr*R1+Br,

G0=Kg*G1+Bg,

B0=Kb*B1+Bb,

Among them, R0 represents the final signal response value after alignment of the acquisition signal responsiveness of the color R channel in the image data of the currently processed camera module, and G0 represents the acquisition signal response ability of the color G channel in the image data of the currently processed camera module after alignment. The final signal response value, B0 represents the final signal response value after alignment of the acquisition signal response capabilities of the color B channel in the image data of the currently processed camera module, and R1 represents the color R channel in the image data of the currently processed camera module. The original signal of the channel Response value, G1 represents the raw signal response value of the color G channel in the image data of the currently processed camera module, B1 represents the raw signal response value of the color B channel in the image data of the currently processed camera module, Kr represents the currently processed camera module. The color R channel is the first-order coefficient of alignment of the acquisition signal response capability, Gr represents the color G channel of the currently processed camera module, the acquisition signal response capability is aligned to the first-order coefficient, and Br represents the currently processed camera module. The color B channel of the acquisition signal response Capability alignment first-order coefficient, Br represents the current processing camera module color R channel acquisition signal response ability alignment compensation coefficient, Gr represents the current processing camera module color G channel acquisition signal response ability alignment compensation coefficient, Bb represents the current processing The responsiveness of the captured signal of the color B channel of the camera module is aligned with the compensation coefficient.

In a possible example, in the aspect of generating comprehensive image channel statistical information based on the multi-channel image statistical information, the central processing module 120 is specifically configured to: superimpose the multi-channel image statistical information into a single image channel comprehensive information Statistics.

In the specific implementation, the image signal processing device of the electronic device specifically adopts the method of expanding the bit width to fuse the two channels of data into one channel of data. For example, multiple camera modules include a main camera and a sub camera. The camera statistics bit width is 12 bits, then the expanded bit width is 24 bits, and the statistics information transmission interface of the transmission port module corresponding to the main camera can use three 8bits bit widths for transmission.

In addition, each corresponding image statistical information can be calibrated according to the preset number of the camera module, so that the preset number in the integrated statistical information of a single image channel after superimposition is associated with the corresponding camera module.

It can be seen that in this example, the multi-channel image statistical information is superimposed into the comprehensive statistical information of a single image channel, that is, the information content is unchanged, and only superimposed together for transmission, so that the overall data volume is basically not increased, and no additional data transmission burden will be brought. . And single-channel transmission saves hardware interface resource overhead and power consumption.

In a possible example, in the aspect of generating comprehensive statistical information of image channels based on the statistical information of the multi-channel images, the central processing module 120 is specifically configured to: acquire the statistics of each camera module in the plurality of camera modules Information configuration weights; weights are configured according to the multi-channel image statistical information and the statistical information of each camera module, and the comprehensive statistical information of the image channels is determined.

It can be seen that in this example, the electronic device can calculate the comprehensive statistical information of the image channel by weighting through the weight allocation ratio, thereby realizing data fusion and data volume convergence, and improving transmission efficiency.

In this possible example, in the aspect of acquiring the statistical information configuration weight of each camera module in the plurality of camera modules, the central processing module 120 is specifically configured to: determine according to the image data of each camera module The average brightness of the images collected by each camera module; the statistical information configuration weight of each camera module is determined according to the average brightness of the images collected by each camera module.

For example, it is assumed that the multiple camera modules include a first camera and a second camera, and the average brightness of image data collected by the first camera is L1, and the average brightness of image data collected by the second camera is L2, L1:L2=2 : 1, the statistical information configuration weight W1 of the first camera may be 0.67, and the statistical information configuration weight W2 of the second camera may be 0.33.

It can be seen that in this example, since the higher the image brightness is, the higher the confidence level of the corresponding statistical information is. Therefore, determining the weight of each camera module based on the average brightness of the image can effectively improve the accuracy of the weighted image channel comprehensive statistical information. Spend.

In a possible example, the first statistical information transmission interface in the first transmission port module 131 is used to transmit the comprehensive statistical information of the image channel.

The transmission port module may be a MIPI transmission port, and the statistical information transmission interface may be a DMA interface.

It can be seen that in this example, the electronic device can transmit the comprehensive statistical information of the image channel through the first statistical information transmission interface in the first transmission port module of the multiple transmission port modules, that is, multiplex the original statistical information transmission interface to transmit multiple Comprehensive statistics of image channels of the camera module, without changing the hardware interface, improve the utilization rate.

In this possible example, as shown in FIG. 1b, the front image signal processing apparatus 100 further includes a second transmission port module 132;

The second transmission port module 132 is at least used for transmitting the image data, and/or the power supply state of the second transmission port module 132 is adjustable.

Wherein, the information other than the comprehensive statistical information of the image channel includes target information for target functions, such as face recognition, and target information such as thumbnails. Improve the comprehensiveness and efficiency of information transmission.

Wherein, the second transmission port module 132 may be single or multiple.

It can be seen that, in this example, the second transmission port module of the electronic device may include a second statistical information transmission interface for transmitting information other than the comprehensive statistical information of the image channel, so as to realize the multiplexing of other information with the original statistical information transmission interface for transmission. , improve the utilization rate of hardware interface resources and the comprehensiveness of transmission information. Alternatively, the second statistical information transmission interface included in the second transmission port module of the electronic device may also be directly in a dormant state, that is, without power supply, so as to save power.

As shown in FIG. 1c, an embodiment of the present application provides a schematic structural diagram of an example of a front image signal processing apparatus 100. The front image signal processing apparatus 100 is coupled between a plurality of camera units 200 and an application processing apparatus 300. The image signal processing apparatus 100 may include a plurality of transmission port modules 110, a front image signal processor ISP120 correspondingly connected to the plurality of camera units 200, a neural network processor (Neural-network Processing Unit, NPU) 130, a front center A processing unit CPU140, a double-rate SDRAM (Double Data Rate SDRAM, DDR) 150 and a power management unit (Power Management Unit, PMU) 160, each front ISP 120 is connected to the NPU 130, and the NPU 130 is connected to the plurality of transmission port modules 110 And DDR150, PMU160 supplies power for NPU130, front CPU140, transmission port module 110, and front CPU140 is connected to multiple front ISP 120, NPU130 and multiple transmission port modules. The application processing device 300 includes a plurality of rear ISPs 310 corresponding to the plurality of transmission port modules 110, and a rear CPU 320, and the front CPU 140 is connected (for example, through a secure digital input and output SDIO interface or a serial external interface). Set the interface SPI interface) the post-installed CPU320.

The pre-CPU 140 is used for interacting with the post-CPU 320 of the application processing apparatus 300 , receiving control commands and data of the application processing apparatus 300 , and sending information back to the application processing apparatus 300 . At the same time, it is responsible for the control of the front ISP120 and NPU130. The front ISP120 is used for basic image processing, and the conventional ISP functions are similar, including dead pixel correction, dark current correction, lens shading correction, digital gain, white balance, demosaic, color correction, gamma, noise reduction and other modules. The NPU 130 is used to run the artificial intelligence AI algorithm to process the preview image data (for example: noise reduction, HDR, super-resolution, etc.). The DDR 150 stores algorithms for NPU 130 calls.

In addition, as shown in FIG. 1d, the transmission port module 110 may include an image data transmission interface 1101 (eg, direct memory access DMA interface) for transmitting image data and a statistics information transmission interface 1102 for transmitting image channel statistics (eg: DMA interface), or only the image data transfer interface 1101 for transferring image data is included.

Please refer to FIG. 2. FIG. 2 is an image statistical information processing chip 10 provided by an embodiment of the present application, which is connected to a camera device 200 and includes:

The front image signal processing device 100 is electrically connected to the camera device 200 for realizing the function of the front image signal processing device 100 in FIG. 1a or FIG. 2;

The application processing device 300 is electrically connected to the image signal processing device, and is configured to receive the comprehensive statistical information of the image channels, and determine the statistical results of the image channels of each camera unit according to the comprehensive statistical information of the image channels.

In a possible example, in the aspect of determining the statistical result of the image channels of each camera unit according to the comprehensive statistical information of the image channels, the application processing device 300 is specifically configured to: make comprehensive statistics of the superimposed single image channel The information is divided into multi-channel image statistical information corresponding to multiple camera units one-to-one; the current environmental parameters are obtained; according to each image statistical information in the split multi-channel image statistical information, the The configuration parameters and the current environment parameters are used to determine the statistical results of the image channels of each camera module.

Wherein, the current environment parameter includes at least one of the following: current environment brightness and color temperature of the current environment. The current ambient brightness is used to calculate the AE statistical result. The color temperature of the current environment is used to calculate the AWB statistical result.

The configuration parameters include at least one of the following: a tuning parameter of AE and a tuning parameter of AWB, the tuning parameter of AE includes an exposure index curve, and the tuning parameter of AWB includes a gamma curve.

It can be seen that in this example, the electronic device can split and process the received comprehensive statistical information of a single image channel through the application processing device, so as to calculate the statistical results of the image channels of each camera module. While calculating the statistical results, it can improve the utilization of hardware interface resources.

In a possible example, in the aspect of determining the statistical result of the image channel of each camera unit according to the comprehensive statistical information of the image channel, the application processing device 300 is specifically configured to: acquire the current environment parameter and the image channel of each camera. The configuration parameters of the module; according to the comprehensive statistical information of the image channels, the configuration parameters of each camera module and the current environment parameters, the statistical results of the image channels of each camera module are determined.

In a specific implementation, the electronic device configures weights according to the statistical information of the multi-channel images and the statistical information of each camera module, and determines the comprehensive statistical information of the image channel, and then passes the comprehensive statistical information of the image channel through the first transmission port. The module is transmitted to the application processing device, and the application processing device receives the comprehensive statistical information of the image channel, and applies the comprehensive statistical information of the image channel to the calculation process of the statistical results of the image channel of each camera module, that is, the application processing device obtains the current environmental parameters. and the configuration parameters of each camera module, according to the comprehensive statistical information of the image channels, the configuration parameters of each camera module and the current environment parameters, to determine the image channel statistical results of each camera module.

It can be seen that in this example, the electronic device can adapt the integrated statistical information of the image channels obtained by fusion into the calculation process of the statistical results of the image channels of each camera unit, so as to improve the calculation efficiency.

Please refer to FIG. 3. FIG. 3 is an electronic device 1 provided by an embodiment of the present application, including:

a camera device 200 for collecting image data of multiple image channels;

The image statistical information processing chip 10 is electrically connected to the camera device 200 and is used to implement the functions of the image statistical information processing chip 10 in FIG. 2 .

Please refer to FIG. 4a. FIG. 4a is a schematic flowchart of an image information transmission method provided by an embodiment of the present application, applied to an electronic device, and the method includes:

Step 401, generating multi-channel image statistical information of multiple image channel image data of multiple camera modules;

Step 402, generating comprehensive statistical information of image channels based on the multi-channel image statistical information;

Step 403, transmitting the comprehensive statistical information of the image channel.

As shown in Figure 4b, the architecture diagram of the image information transmission system used to support the image information transmission method of the present application, the application layer of the software system architecture includes the camera application, the Google framework layer includes the Google Camera HAL3 interface of the Google Camera framework layer, and the hardware The abstraction layer includes the channel Pipeline module, 3A module, the pre-ISP hardware abstraction layer PreISP HAL module, and the kernel Kernel layer includes the PreISP subsystem subsystem, PreISP driver module, serial communication bus I2C driver Driver, device tree Device Tree configuration (including IMX586 and OV23850) and ISP driver.

The embodiments of the present application have been introduced in detail above, and the principles and implementations of the present application are described in this paper by using specific examples. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for Persons of ordinary skill in the art, according to the idea of the present application, will have changes in the specific implementation manner and application scope. In summary, the contents of this specification should not be construed as a limitation on the present application.

The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server or data center Transmission by wire or wireless to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like that contains one or more sets of available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media. The semiconductor medium may be a solid state drive.

Embodiments of the present application further provide a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes the computer to execute part or all of the steps of any method described in the above method embodiments , the above computer includes electronic equipment.

Embodiments of the present application further provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of the method embodiments described above. some or all of the steps of the method. The computer program product may be a software installation package, and the computer includes an electronic device.

It should be understood that, in various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not be dealt with in the embodiments of the present application. implementation constitutes any limitation.

In the several embodiments provided in this application, it should be understood that the disclosed method, apparatus and system may be implemented in other manners. For example, the device embodiments described above are only illustrative; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included individually, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

The above-mentioned integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM for short), Random Access Memory (RAM for short), magnetic disk or CD, etc. that can store program codes medium.

Although the present invention is disclosed above, the present invention is not limited thereto. Any person skilled in the art, without departing from the spirit and scope of the present invention, can easily think of changes or substitutions, and can make various changes and modifications, including the combination of the above-mentioned different functions and implementation steps, including the implementation of software and hardware. The methods are all within the protection scope of the present invention.

Claims

A front image signal processing device, which is coupled between a camera device and an application processing device, characterized in that it includes:

an image signal processing module for generating multi-channel image statistics for image data of multiple image channels;

a central processing module, configured to generate comprehensive statistical information of image channels based on the multi-channel image statistical information;

A first transmission port module, configured to transmit the comprehensive statistical information of the image channel to the application processing device.
The pre-image signal processing device according to claim 1, wherein before generating the comprehensive statistical information of image channels based on the multi-channel image statistical information, the central processing module is further configured to:

Based on capability alignment coefficients of multiple camera units in the camera device, capability alignment processing is performed on the multi-channel image statistical information to obtain the capability-aligned multi-channel image statistical information, wherein the capability alignment coefficient is used to describe Differences in the degree of response to color among the plurality of camera units.
The front image signal processing apparatus according to claim 2, wherein the type of the camera unit includes any one of the following: main camera, wide-angle and telephoto.
The pre-image signal processing apparatus according to claim 3, wherein the image statistical information includes automatic exposure statistical information and/or automatic white balance statistical information.
The pre-image signal processing device according to claim 2, wherein the central processing module is further used for:

Determine whether the camera unit corresponding to the currently processed image statistical information is a reference camera unit;

If so, determining the currently processed image statistical information as the capability-aligned image statistical information;

If not, obtain the information statistics capability alignment coefficient of the camera unit, and determine the capability-aligned image statistics information according to the currently processed image statistics information and the information statistics capability alignment coefficient.
The pre-image signal processing device according to claim 2, wherein the central processing module is further used for:

acquiring a plurality of image data corresponding to the camera;

aligning the responsiveness of the acquisition signal for each image data in the plurality of image data to obtain a plurality of image data after the responsiveness of the acquisition signal is aligned;

According to each image data in the plurality of image data after the response capability alignment of the collected signal, the corresponding image statistical information after the capability alignment is determined, and the multi-channel image statistical information after the capability alignment is obtained.
The pre-image signal processing device according to claim 6, wherein the central processing module is further used for:

Determine whether the camera unit corresponding to the currently processed image data is a reference camera unit;

If yes, then determining the currently processed image data as the image data after capability alignment;

If not, obtain the pre-configured acquisition signal responsiveness alignment coefficient of the camera module, and determine the image after the acquisition signal responsiveness alignment according to the currently processed image statistical information and the acquisition signal responsiveness alignment coefficient data.
The pre-image signal processing device according to claim 1 or 2, wherein, in the aspect of generating comprehensive statistical information of image channels based on the statistical information of the multi-channel images, the central processing module is specifically configured to:

The multi-channel image statistics are overlaid into a single image channel composite statistics.
The pre-image signal processing device according to claim 1 or 2, wherein, in the aspect of generating comprehensive statistical information of image channels based on the statistical information of the multi-channel images, the central processing module is specifically configured to:

Obtain the statistical information configuration weight of each camera module in the plurality of camera modules;

The weight is configured according to the statistical information of the multi-channel images and the statistical information of each camera module, and the comprehensive statistical information of the image channels is determined.
The front-end image signal processing device according to claim 9, wherein, in the aspect of acquiring the statistical information configuration weight of each camera module in the plurality of camera modules, the central processing module is specifically used for:

Determine the average brightness of the images collected by each camera module according to the image data of each camera module;

The statistical information configuration weight of each camera module is determined according to the average brightness of the images collected by each camera module.
The pre-image signal processing apparatus according to any one of claims 1-10, wherein the first statistical information transmission interface in the first transmission port module is used to transmit the comprehensive statistical information of the image channel.
The front-end image signal processing apparatus according to claim 11, wherein the first transmission port module comprises a mobile industry processor interface.
The pre-image signal processing device according to claim 11, wherein the comprehensive statistical information of the image channels includes the superposition of the statistical information of the multi-channel images or the information obtained by performing data fusion on the statistical information of the multi-channel images. .
The pre-image signal processing device according to claim 11, wherein the pre-image signal processing device further comprises a second transmission port module;

The second transmission port module is at least used for transmitting the image data, and/or the power supply state of the second transmission port module is adjustable.
An image statistical information processing chip, which is connected to a camera device, is characterized in that, comprising:

A front image signal processing device, electrically connected to the camera device, for realizing the function of the device according to any one of claims 1-14;

The application processing device is electrically connected to the image signal processing device, and is configured to receive the comprehensive statistical information of the image channels, and determine the statistical results of the image channels of each camera unit according to the comprehensive statistical information of the image channels.
The image statistical information processing chip according to claim 15, wherein, in the aspect of determining the image channel statistical result of each camera unit according to the image channel comprehensive statistical information, the application processing device is specifically configured to:

Splitting the superimposed comprehensive statistical information of the single image channel into multi-channel image statistical information corresponding to multiple camera units one-to-one;

Get the current environment parameters;

According to each image statistical information in the split multi-channel image statistical information, the configuration parameter of each camera module and the current environment parameter, the image channel statistical result of each camera module is determined.
The image statistical information processing chip according to claim 16, wherein, in the aspect of determining the image channel statistical result of each camera unit according to the image channel comprehensive statistical information, the application processing device is specifically configured to:

Obtain the current environment parameters and the configuration parameters of each camera module;

According to the comprehensive statistical information of the image channel, the configuration parameters of each camera module and the current environment parameter, the statistical result of the image channel of each camera module is determined.
The image statistical information processing chip according to any one of claims 15-17, wherein the image channel statistical results include automatic exposure statistical information and/or automatic white balance statistical information.
An electronic device, comprising:

a camera device for collecting image data of multiple image channels;

An image statistical information processing chip, electrically connected to the camera device, for implementing the function of the chip according to any one of claims 15-18.
A method for transmitting image information, comprising:

Generate multi-channel image statistics of multiple image channel image data of multiple camera modules;

generating comprehensive statistical information of image channels based on the multi-channel image statistical information;

Transmitting the image channel synthesis statistics.