WO2023000903A1 - Data transmission method and apparatus, data receiving method and apparatus, chip, device, and storage medium - Google Patents

Abstract

Provided are a data transmission method and apparatus, a data receiving method and apparatus, a chip, a device, and a storage medium. The data transmission method comprises: converting, into N second data sets to be transmitted, M first data sets to be transmitted (201), M being greater than 1, and N being greater than 0 and less than M; packaging said N second data sets according to an MIPI protocol to obtain a first batch of one or more MIPI data packets (202); and transmitting the first batch of one or more MIPI data packets to an application processor by means of N data channels (203).

Description

Data transmission and reception method and device, chip, device, storage medium

Cross References to Related Applications

This application is based on a Chinese patent application with application number 202110839218.6 and a filing date of July 23, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated into this application in its entirety .

technical field

This application relates to electronic technology, involving but not limited to data transmission and reception methods and devices, chips, equipment, and storage media.

Background technique

Mobile Industry Processor Interface (MIPI) has become the main standard for image sensor data transmission. The original intention is to standardize the interface between the camera, display, RF baseband and processor inside the mobile phone to meet the needs of large-capacity image transmission. However, the increase of the data channel of MIPI hardware means the increase of hardware cost.

Contents of the invention

In view of this, the data transmission and receiving method and device, chip, device, and storage medium provided by the embodiments of the present application can reduce the hardware requirements for MIPI, and even if there are more data sets to be transmitted, fewer channels can be used. The data channel completes the data transfer. The data transmission and reception methods and devices, chips, equipment, and storage media provided in the embodiments of the present application are implemented in the following way:

The data transmission method provided by the embodiment of the present application includes: converting the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M; according to the MIPI protocol , packing the second data sets to be transmitted in the N-way to obtain the first batch of one or more MIPI data packets; transmitting the first batch of one or more MIPI data packets to the application processor through the N-way data channels .

The data receiving method provided by the embodiment of the present application includes: determining whether a whole row of data is currently received according to a specific row data length; if a whole row of data is currently received, parsing the currently received row of data to obtain the analyzed data; sending the analyzed data to the corresponding first image processing chip.

The image processing chip provided by the embodiment of the present application includes: a conversion circuit, which is used to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M; MIPI-TX circuit, used to pack the second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and the first batch of one or more data packets MIPI data packets are transmitted to the application processor through N data lanes.

The application processor provided in the embodiment of the present application includes: a MIPI-RX circuit, configured to determine whether a whole row of data is currently received according to a specific row data length; if a whole row of data is currently received, trigger the data parsing module; The data parsing module is used to analyze the currently received row of data to obtain the parsed data, and send the parsed data to the corresponding first image processing chip; the first image processing chip is used to Analyze the data described above and execute specific image processing algorithms.

The electronic device provided in the embodiment of the present application includes: a second image processing chip, at least N data channels, and an application processor, where N is greater than 0; wherein, the conversion circuit of the second image processing chip converts the A data set to be transmitted is converted into an N-way second data set to be transmitted; wherein, M is greater than 1, and N is greater than 0 and less than M; the MIPI-TX circuit of the second image processing chip, according to the MIPI protocol, for the N The second data set to be transmitted is packaged to obtain the first batch of one or more MIPI data packets; and the first batch of one or more MIPI data packets is transmitted to the MIPI-RX of the application processor through N data channels circuit; the MIPI-RX circuit of the application processor determines whether a whole row of data is currently received according to a specific row data length; if a whole row of data is currently received to trigger a data analysis module; the data of the application processor The parsing module is configured to parse a line of data currently received to obtain parsed data, and to send the parsed data to a corresponding first image processing chip; the first image processing chip of the application processor is configured to For each of the analyzed data received, a specific image processing algorithm is executed.

The data transmission device provided by the embodiment of the present application includes: a conversion module, which is used to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M; Packing module, for according to MIPI agreement, the second data set to be transmitted of described N road is packed, obtains first batch of one or more MIPI data packets; Transmission module, for described first batch of one or more Multiple MIPI data packets are transmitted to the application processor through N data lanes.

The data receiving device provided by the embodiment of the present application includes: a receiving module, configured to determine whether a whole row of data is currently received according to a specific row data length; and if a whole row of data is currently received, trigger a data parsing module; A module, configured to analyze a row of currently received data to obtain analyzed data; and send the analyzed data to a corresponding first image processing chip.

The electronic device provided by the embodiment of the present application includes a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements the image processing chip described in the embodiment of the present application when executing the program or, when the processor executes the program, implements the method on the application processor side described in the embodiment of the present application.

The computer-readable storage medium provided by the embodiment of the present application has a computer program stored thereon, and when the computer program is executed by the processor, the method on the image processing chip side described in the embodiment of the present application is implemented, or the computer program is executed by the processor During execution, the method on the application processor side described in the embodiment of the present application is implemented.

In the embodiment of the present application, when receiving the first data set to be transmitted from M channels, the first data set to be transmitted from M channels is not directly packaged according to the MIPI protocol, but the first data set to be transmitted of the M channels is firstly packaged. The transmission data set is organized into N second data sets to be transmitted which are less than M; then, the N second data sets to be transmitted are packaged according to the MIPI protocol; The packaged transmission scheme can transmit the data to the application processor through fewer data channels, that is, transmit the data to the application processor through N channels instead of M channels. It can be seen that the solution provided by the embodiment of the present application reduces the hardware requirements for MIPI. Even if there are more data sets to be transmitted, fewer data channels can be used to complete data transmission, so that even if MIPI has fewer hardware data channels It can also meet the needs of data transmission.

Description of drawings

The accompanying drawings here are incorporated into the specification and constitute a part of the specification. These drawings show embodiments consistent with the application, and are used together with the description to describe the technical solution of the application.

FIG. 1 is a schematic diagram of the basic architecture of data transmission for image processing;

FIG. 2 is a schematic diagram of the implementation flow of the data transmission method provided by the embodiment of the present application;

FIG. 3 is a schematic diagram of an implementation flow of a data transmission and data receiving method according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

Figure 5 is a schematic diagram of the output position of the 3A data in the pipeline of the Image Signal Processor (ISP) (ie, the signal processing flow of the ISP);

FIG. 6 is a schematic diagram of the output position of 3A data in the ISP pipeline (ie, the signal processing flow of the ISP);

7 is a schematic diagram of the data transmission process in the early stage image processing (PreISP) chip solution;

Figure 8 is a schematic diagram of the architecture of the pre-processing ISP chip end of the PreISP chip;

9 is a schematic diagram of the software framework of the application (Application, AP) side platform of the embodiment of the present application;

FIG. 10 is a schematic diagram of an implementation flow of a data transmission and receiving method according to an embodiment of the present application;

Figure 11 is a schematic diagram of the comparison of the results before and after the rearrangement of the embodiment of the present application;

FIG. 12 is a schematic structural diagram of a data transmission device according to an embodiment of the present application;

FIG. 13 is a schematic structural diagram of a data receiving device according to an embodiment of the present application;

FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following examples are used to illustrate the present application, but not to limit the scope of the present application.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

In the following description, references to "some embodiments" describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or a different subset of all possible embodiments, and Can be combined with each other without conflict.

It should be pointed out that the terms "first\second\third" etc. involved in the embodiment of the present application do not represent a specific ordering of objects. It is understandable that "first\second\third" etc. are allowed The specific order or sequencing may be interchanged such that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein.

In order to facilitate the understanding of the embodiment of the present application, a basic architecture of data transmission for image processing is firstly provided. FIG. 1 is a schematic diagram of the architecture, in which only some key modules are shown. As shown in FIG. Two image processing chip 10 and application processor 11; Wherein, the second image processing chip 10 comprises ISP chip 101, MIPI-TX module 102, ISP chip 103 and MIPI-TX module 104; Application processor 11 comprises MIPI-RX module 111 , ISP chip 112, 3A algorithm module 113, MIPI-RX module 114, ISP chip 115 and 3A algorithm module 116; wherein, 3A algorithm includes automatic focus (Automatic Focus, AF) algorithm, automatic exposure (Automatic Exposure, AE) algorithm and Automatic White Balance (AWB) algorithm;

The image sensor 20 transmits the sensing data to the ISP chip 101, and the ISP chip 101 processes it, and transmits the obtained statistical data (hereinafter referred to as 3A data), original (raw) image data and PD data required by the 3A algorithm to the MIPI -TX module 102, the MIPI-TX module 102 transmits the received data sets from each channel to the MIPI-RX module 111 on the application processor side through the MIPI data channel, and the MIPI-RX module 111 transmits 3A data and PD data to 3A The algorithm module 113 transmits the raw image data to the ISP chip 112; wherein, the PD data includes phase information for focusing.

The transmission path of the sensing data output by the image sensor 21 and the type of the image sensor 20 will not be repeated here.

It should be noted that the above architecture 1 is only an example for the convenience of understanding the embodiment of the present application, and cannot limit the scope of the embodiment of the present application. There is no restriction on the structure of the second image processing chip and the application processor applicable to the embodiment of the present application, and there is no restriction on the data content transmitted by the data channel, which is not limited to 3A data, PD data and raw image data, etc. .

The embodiment of the present application provides a data transmission method for transmitting image data between the application processor and the image sensor. FIG. 2 is a schematic diagram of the implementation flow of the data transmission method provided in the embodiment of the present application. As shown in FIG. 2 , the The method may include the following steps 201 to 203:

Step 201, the second image processing chip converts the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M.

It can be understood that in the image processing solution, a lot of data needs to be transmitted through MIPI, for example, 3A data, raw image data, PD data and so on. And these data sets to be transmitted, the data sets to be transmitted of the same type may come from different sources, and correspondingly, the transmission paths are also different. For example, the type of the first data set to be transmitted is 3A data, wherein the statistical data required by the AWB algorithm comes from the LSC module, the statistical data required by the AE algorithm comes from the WB module, and the statistical data required by the AF algorithm comes from the Demosaic module. For another example, the types of the first data set to be transmitted are all raw image data, wherein one channel of raw image data comes from the first image sensor, and the other channel of raw image data comes from the second image sensor; as another example, the first data set to be transmitted The types of sets are all raw image data, but one path is short-exposure data, one path is medium-exposure data and one path is long-exposure data.

It should be noted that the first to-be-transmitted data sets of the M channels may be different types of data sets, or may be the same type of data sets. In the case of the same type, for example, in some embodiments, the first data set to be transmitted includes at least one of the following: image statistical data (abbreviated as 3A data), PD data and original image required to execute the 3A algorithm The data, that is, the M channels of first data sets to be transmitted are all 3A data, or all are PD data, or all are original image data. In the case of different types, for example, the first data set to be transmitted of M channels includes 3A data, PD data, and original image data.

In some embodiments, the first data sets to be transmitted of the M channels are data sets generated at the same moment or in the same time period. The M paths can be understood as M transmission paths or M different modules. The meaning of N channels is similar to that of M channels. For the second image processing chip, the N channels of second data sets to be transmitted may come from N channels of transmission, or may be output by N different modules.

In step 202, the second image processing chip packs the N channels of second data sets to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets.

There may be a situation where the length of a certain second data set to be transmitted exceeds the specified upper limit of the length of the MIPI data packet, so the data set may need to be split into two MIPI data packets.

Step 203, the second image processing chip transmits the first batch of one or more MIPI data packets to the application processor through N data channels.

It can be understood that, in the MIPI protocol, several channels of data are generated at the same time or in the same time period, and usually the same number of data channels are used for transmission. For example, if there are 3 channels of raw image data generated at the same time, the MIPI-TX module following the MIPI protocol will transmit them through 3 channels of data. In this way, higher requirements are put forward for the number of data channels of the MIPI hardware, so that data transmission requirements in various scenarios can be met. For example, the number of data channels is required to be able to cope with the simultaneous generation of 3-way data sets, as well as the transmission requirements of simultaneously generated 5-way, 6-way or even 8-way data sets, then this requires MIPI hardware to have more than 3-way data sets Data channels to meet transmission needs. However, the increase of hardware data channels brings hardware cost and design space cost. Therefore, how to meet the data transmission requirements in different scenarios with low hardware cost in a limited device space has become a technical problem to be solved.

In the embodiment of the present application, when receiving the first data sets to be transmitted from the M channels, these data sets are not directly packaged according to the MIPI protocol, but the first data sets to be transmitted from the M channels are organized into smaller than M's N-way second data sets to be transmitted; then, according to the MIPI protocol, the N-way second data sets to be transmitted are packaged; thus, compared to the scheme of directly packaging and transmitting the M-way first to-be-transmitted sets , enabling the second image processing chip to complete data transmission through fewer data channels, that is, to transmit these data to the application processor through N data channels instead of M data channels.

The embodiment of the present application provides a data transmission and data receiving method. FIG. 3 is a schematic diagram of the implementation flow of the data transmission and data receiving method in the embodiment of the present application. As shown in FIG. 3 , the method may include the following steps 301 to 307:

Step 301, the second image processing chip decomposes the data to be transmitted in the first data set to be transmitted from M channels into at least one row of data according to a specific row data length.

In some embodiments, the length of each decomposed row of data is fixed, and is a specific row data length. Further, in some embodiments, there is a certain interval between adjacent rows of data, that is, there is blank data between adjacent rows of data, so that a certain time can be reserved for the receiving end to process the received row data.

There is no limitation on the size of the specific row data length. For example, the length of each row of data is 1024 bytes or 2048 bytes and so on.

In step 302, the second image processing chip rearranges the at least one row of data to obtain N channels of second data sets to be transmitted; wherein, M is greater than 1, N is greater than 0, and N is less than M.

In some embodiments, the second data set to be transmitted includes not only the content of the data itself, but also the metadata of these data. Understandably, metadata, also known as intermediary data or relay data, is the data describing data (data about data), mainly describing the information of data attributes (property), used to support such as indicating storage location, historical data, resource search , file recording and other functions. In this embodiment, the metadata of the second data set to be transmitted may include, for example, the data length and/or data type of the second data set to be transmitted.

In the embodiment of the present application, the value of N is not limited. It can be set according to the number of MIPI data channels, in a word, N is less than or equal to the number of MIPI data channels. For example, the first data set to be transmitted from 3 channels is organized as the second data set to be transmitted from 1 channel. As another example, the first data sets to be transmitted from 6 channels are organized into the second data sets to be transmitted from 2 channels. M and N may or may not be in multiples.

In some embodiments, the second image processing chip can implement step 302 as follows: determine the first type of each row of data; update the first type of label used to represent the row of data into the row of data; update The last row of data is rearranged to obtain the N-way second data set to be transmitted.

For example, the first type of tag is used to identify whether the corresponding row data is the content of the data itself or metadata. Each line of data is marked with a corresponding first type, so that for the application processor side, when parsing the received MIPI data packet, it is not necessary to analyze and identify the type of the line of data, thereby improving the overall data processing of the application processor efficiency.

Further, in some embodiments, the updated at least one row of data may be rearranged in the following way: determine the second type of each updated row of data; the second type belongs to a subtype of the first type , the first type of tag is used to identify that the corresponding row data is the content or metadata of the data itself; the second type refers to the data type of the corresponding row data; according to the mapping relationship between the second type and row number , rearranging the at least one row of updated data to obtain the N-way second data set to be transmitted; in this way, it is convenient for the application processor to analyze quickly. For example, the application processor may directly determine the second type of the row of data according to the row number of the currently received data.

Furthermore, the second image processing chip can know the second type that each row should correspond to according to the preset mapping table, that is, the mapping relationship between the second type and the row number is recorded in the mapping table. For example, as shown in Table 1, the first row agrees to store long-exposure raw image data, the second row agrees to store short-exposure raw image data, the third row agrees to store mid-exposure raw image data, and the fourth line agrees to store The statistical data required by the AWB algorithm in the 3A data (abbreviated as AWB data), the fifth line agrees to store the statistical data required by the AE algorithm in the 3A data (abbreviated as AE data), and the sixth line agrees to store the statistical data required by the AF algorithm (referred to as AF data), and so on.

Table 1

first row	Long exposure raw image data
second line	Short exposure raw image data
The third row	Raw image data exposed in
fourth row	AWB data
The fifth line	AE data
sixth line	AF data
...	...

The data length of each row is fixed, and the number of rows can be adjusted according to the size of the total data set received. Total data sets of different sizes have different corresponding mapping tables. For example, the first data set to be transmitted in the M channel is raw image data, its total data size is 3K bytes, and the specific row data length is 1024 bytes. The contents of the corresponding mapping table are shown in Table 2. Among them, the first The first row agrees to store long-exposure raw image data, the second row agrees to store mid-exposure raw image data, and the third row agrees to store short-exposure raw image data. As another example, the first data set to be transmitted in M channels is raw image data, its total data size is 6K bytes, and the specific line data length is 1024 bytes, and the contents of the corresponding mapping table are shown in Table 3, wherein, The first and second lines agree to store long-exposure raw image data, the third and fourth lines agree to store medium-exposure raw image data, and the fifth and sixth lines agree to store short-exposure raw image data.

Table 2

first row	Long exposure raw image data
second line	Long exposure raw image data
The third row	Raw image data exposed in
fourth row	Raw image data exposed in
The fifth line	Short exposure raw image data
sixth line	Short exposure raw image data

first row	Long exposure raw image data
second line	Raw image data exposed in
The third row	Short exposure raw image data

When the second image processing chip rearranges the updated at least one row of data, it may call the corresponding The mapping table, and then rearrange the updated at least one row of data according to the second type corresponding to each row agreed in the mapping table.

Step 303 , the second image processing chip packs the N channels of second data sets to be transmitted according to the MIPI protocol to obtain a first batch of one or more MIPI data packets.

It can be understood that packing means encapsulating data, for example, adding header information at the beginning of the second data set to be transmitted, adding packet tail information at the end of the second data set to be transmitted, and so on. It can be seen that the encapsulated second data set to be transmitted, that is, the data length of each line in the MIPI data packet is a specific line data length. In some embodiments, one channel of the second data set to be transmitted is encapsulated into one MIPI data packet.

Step 304, the second image processing chip evenly distributes the first batch of one or more MIPI data packets to the N data channels and transmits them to the application processor through the N data channels.

For example, N data lanes can transmit MIPI packets to the application processor line by line.

The second image processing chip distributes the first batch of one or more MIPI data packets to the N data channels uniformly or non-uniformly. Different data lanes are responsible for transporting assigned MIPI packets. In some embodiments, one data channel can serially transmit MIPI data packets to the application processor line by line; wherein, there is blank data of a fixed size between adjacent line data, that is, blank intervals, and the purpose of this interval is to provide The MIPI-RX module of the application processor reserves a certain amount of time to parse the received row data.

Step 305, the application processor determines whether a whole row of data is currently received according to the specific row data length; if yes, execute step 306; otherwise, return to execute step 305;

In step 306, the application processor parses the currently received row of data to obtain parsed data.

In some embodiments, whenever the MIPI-RX module of the application processor receives a row of data, it generates a row interrupt, thereby triggering the data parsing module of the application processor to parse the currently received row of data; thus, using the row interrupt, Realize parsing while transmitting, no need to wait for all row data (that is, cache data for image processing) to be transmitted and then parse; thus, data transmission can achieve shorter delay transmission or even zero delay transmission, thereby improving the overall performance of the application processor Data processing efficiency. For example, for the AF algorithm, fast acquisition of analytical data can improve the autofocus speed, thereby improving user experience.

Wherein, the data parsing module may be included in the MIPI-RX module of the application processor, or may be another module independent of the MIPI-RX module.

Step 307, the application processor sends the analysis data to the corresponding first image processing chip.

In some embodiments, the application processor can implement step 306 in the following way: determine the line number of the parsed data; determine the second type of the parsed data according to the agreed second type corresponding to each line; For the second type of data, the analysis data is sent to the corresponding first image processing chip.

For example, if the second type of the row data obtained through analysis is 3A data or PD data, the data is transmitted to a 3A algorithm module, which is an image processing chip. For another example, if the second type of the analysis data is raw image data, the data is transmitted to the ISP chip, which is a different chip from the 3A algorithm module.

The embodiment of the present application provides an electronic device. FIG. 4 is a schematic structural diagram of the electronic device in the embodiment of the present application. As shown in FIG. 4 , the electronic device 4 includes a second image processing chip 40, an application processor 41 and at least N channels of data Channel 42; wherein, the second image processing chip 40 includes a conversion circuit 401, a MIPI-TX circuit 402; the application processor 41 includes a MIPI-RX circuit 411, a data analysis module 412 and a first image processing chip 413; wherein,

A conversion circuit 401, configured to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M;

The MIPI-TX circuit 402 is configured to pack the N-way second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and

The first batch of one or more MIPI data packets is transmitted to the MIPI-RX circuit 411 of the application processor through N data channels.

MIPI-RX circuit 411, according to the specific line data length, determines whether a whole line of data is currently received; if a whole line of data is currently received, the data analysis module 412 is triggered; otherwise, continue to receive and judge whether to receive according to the currently received data length Received a whole line of data;

The data parsing module 412 is configured to parse a row of currently received data to obtain parsed data, and send the parsed data to the corresponding first image processing chip 413 .

In some embodiments, the MIPI-RX circuit is used to determine whether an entire line of data is currently received according to a specific line data length; if an entire line of data is currently received, the output line is interrupted to the data analysis module to The data parsing module is triggered to parse the currently received row of data.

In some embodiments, the functions of the data parsing module 412 can be realized by means of software, for example, a CPU can run related programs to realize the functions of the module.

The first image processing chip 413 is configured to execute a specific image processing algorithm according to each received analysis data.

For example, the first image processing chip 413 is a chip for implementing the 3A algorithm. In another example, the first image processing chip 413 is an ISP chip.

In some embodiments, the conversion circuit 401 can decompose the data to be transmitted in the M-way first data set to be transmitted into at least one row of data according to a specific row data length; determine the first type of each row of data; use The label of the first type representing the row data is updated in the row data; the second type of each row of data after the update is determined; the second type belongs to the subtype of the first type; according to the The mapping relationship between the second type and the line number is to rearrange the updated at least one line of data to obtain the N-way second data set to be transmitted; the MIPI-TX circuit 402 is used to transmit the data according to the MIPI protocol. The second data set to be transmitted in N roads is packaged to obtain the first batch of one or more MIPI data packets; RX circuit 411.

Correspondingly, the MIPI-RX circuit 411 can determine whether a whole row of data is currently received according to the specific row data length; if a whole row of data is currently received, the sending row is interrupted to the data parsing module 412; the data parsing module 412, In response to the line interruption, analyze the currently received line of data to obtain the analysis data; determine the line number of the analysis data; determine the second type of the analysis data according to the agreed second type corresponding to each line stored; According to the second type of the analysis data, the analysis data is sent to a corresponding first image processing chip.

It should be noted that the above description about the electronic device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the electronic device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

In the solution of the PreISP chip (that is, an example of the second image processing chip), a lot of data needs to be transmitted through the Mobile Industry Processor Interface (MIPI), such as 3A data, raw image data and PD data. Transport via MIPI.

The relevant introduction about 3A data is as follows:

3A algorithm, that is, automatic exposure (AE) algorithm, automatic white balance (AWB) algorithm and automatic focus (AF) algorithm. The statistical data required by these algorithms are generated in the image signal processing (ISP) chip. The brightness and color of each pixel in the received image data are counted, and the statistical results will be mainly used for the 3A algorithm.

The output position of the 3A data in the ISP pipeline (that is, the signal processing flow of the ISP) is shown in Figure 5, where the statistical data required by the AF algorithm includes: statistics on the images output by the image sensor, and the output of the Degamma module Statistics of images, statistics of images output by Demosaic module, statistics of images output by YUV NR/Sharpening module; statistical data required by AE algorithm include: statistics of images output by LSC module, WB module The statistics of the output image, the statistics of the image output by the CCM module, and the statistics of the image output by the CSM module; the statistical data required by the AWB algorithm include: the statistics of the image output by the LSC module, and the statistics of the Demosaic module The statistics of the output images and the statistics of the images output by the CCM module. It can be seen that there are many statistical positions of AF.

It should be noted that the BLC module is used for black level correction; the Degamma module is used to adjust the contrast of the picture; the NR module is used for noise reduction; the LSC (Lens Shade Correction) module is used for lens shading correction; the WB module, Used for white balance; Demosaic module, used for demosaic; CCM module, used for color correction; Gamma module, used to adjust the contrast of the picture; Sharpening module, used for sharpening; CSM module, used for color space conversion; YUV NR /Sharpening module for noise reduction/sharpening.

In some embodiments, the typical configuration of the output position of 3A data is as shown in Figure 6, wherein the statistical data required by the AWB algorithm is mainly obtained by statistics on the images output by the LSC module, and the statistical data required by the AE algorithm The data is mainly obtained by statistics on the images output by the WB module, and the statistical data required by the AF algorithm is mainly obtained by statistics on the images output by the Demosaic module. As shown in Figure 6, the AE module is mainly used to count the statistical data required by the AE algorithm, the AF module is mainly used to count the statistical data required by the AF algorithm, and the AWB module is mainly used to count the statistical data required by the AWB algorithm.

Among them, the statistical data required by the AE algorithm mainly includes R, G, B histogram and brightness information, and the statistical data is usually the Region Of Interest (ROI) of the image.

The statistical data required by the AWB algorithm mainly includes the statistics of the number of R, G, and B pixel points in the ROI area, the ratio of red to green and the ratio of blue to green.

The statistical data required by the AF algorithm includes the sharpness value of each small block in the ROI area.

Judging from the structure of the current ISP, the ISP will generate more AE statistical data, AWB statistical data and AF statistical data, and these information need to be sent to the 3A algorithm module for processing. When the system includes multiple cameras, the statistical data of each camera will be counted separately, and then sent to the 3A algorithm modules on the independent AP side.

The DOL sensor can simultaneously generate 3 channels of raw image data and PD data; among them, the PD data is phase information for focusing.

Figure 7 shows the data transmission process in the PreISP chip solution. It can be seen from the figure that, limited by the hardware, MIPI will multiplex the data.

In modes such as smooth zoom, where multiple cameras need to be turned on at the same time, the 3A data of each image sensor (sensor) needs to be sent back, and each sensor must be counted, and the amount of data will be doubled; in addition, for 3DOL (snoy multi-exposure ) sensor will output long, medium and short raw image data. For each channel of data, 3A data needs to be counted. If there are 2 channels of 3DOL sensors in the system, it is necessary to count 3 channels × 2 sensors × 3A. It can be seen that at least 18 types of statistical data need to be used and transmitted. Plus (3 channels of Raw image data + 3 channels of PD data) × 2 sensors, a total of 30 channels of data.

However, due to hardware limitations, the above-mentioned PreISP chip solution encounters the following problems:

(1) The number of data channels of MIPI hardware is small, which cannot meet the demand;

(2) After the data is arranged out of order, the parsing performance is poor, which becomes a bottleneck affecting performance.

Based on this, an exemplary application of the embodiment of the present application in an actual application scenario will be described below.

In the embodiment of this application, the MIPI data of the PreISP chip is organized into 1024 bytes per line through the buffer hardware, and different data forms a frame, and different types of data are agreed to be placed in fixed lines. When the platform side (that is, the AP side) receives the MIPI data line by line, and generates a line interruption, the platform side counts the line number and parses the received data according to the line number.

As shown in Figure 8, only the key modules are listed in the figure, the architecture of the pre-processing ISP chip side of the PreISP chip, and the hardware mainly includes: CPU, MIPI, NPU and conversion circuits, etc.; among them,

CPU: Responsible for interacting with the AP. Receive control commands and data sent by AP, and send information back to AP. It is also responsible for the control of modules such as ISP chip and NPU.

ISP chip: responsible for image-based processing. Similar to conventional ISP functions, including bad pixel correction, dark current correction, lens shading correction, digital gain, white balance, Demosaic, color correction, Gamma and noise reduction modules;

Conversion circuit: responsible for organizing multiple channels of MIPI data into one channel of 3A, one channel of raw image data and one channel of PD data.

NPU: run AI algorithms to process preview data, such as noise reduction, HDR and/or super resolution, etc.;

MIPI-TX module: Package the organized data (such as 3A, raw image data and PD data), and then send it to the ISP chip on the AP side through the MIPI protocol, and the ISP chip on the AP side will further process the received data.

AP module:

Data parsing (Unpack) module: It is a software module used to unpack the packaged data of the PreISP chip. After unpacking, the 3A data is divided into statistical data required by the AE, AF, and AWB algorithms. The raw data is divided into long, medium, and Short 3-way data. After unpacking the data, the corresponding algorithm can be used directly.

ISP chip: responsible for image-based processing, mainly used for photo-taking processing in this embodiment of the application. Similar to conventional ISP chips, it has more powerful functions than the ISP chips in PreISP chips, but it cannot be customized and its functions are relatively fixed. An ISP chip includes ISP 0 and ISP 1, and the ISP chip can receive data. In addition, the 3A algorithm is executed on the CPU.

CPU: Responsible for the control of the PreISP chip, such as power-on, power-off, firmware loading, and related control during runtime.

3A algorithm module: used to process 3A data and realize 3A functions.

In the embodiment of this application, the software framework of the AP side platform is shown in Figure 9, wherein,

The kernel (Kernel) layer mainly includes the driver of the device module, such as the driver of the image sensor (sensor driver), I2c driver, SDIO driver and the driver of the PreISP chip on the platform side, etc., through which the device initialization and state switching can be configured. Work;

The HAL layer mainly controls the platform to receive the image data, statistical data and ambient light information output by the PreISP chip through SDIO, and then executes the 3A algorithm; Zero Shutter Lang (ZSL) is responsible for receiving and analyzing raw image data, and raw Further post-processing algorithms for image data and ISP pipeline control for taking pictures.

Framework layer: the original framework of Android, which is the middleware of HAL and APP.

App layer: Mainly use Camera, system Camera or third-party Camera application through Google standard Camera API2;

The operation flowchart of the embodiment of the present application is shown in Figure 10, including the following steps 101 to 106:

Step 101, turn on the dual cameras, and control the pre-processing PreISP chip to start working;

Step 102, the PreISP chip starts to process the image data, and generates 3A data (stats), and sends these statistical data to the conversion circuit.

In step 103, the conversion circuit organizes multiple channels of raw image data, PD data and 3A data into one channel of 3A data, one channel of raw image data and one channel of PD data.

Step 104, these data are packaged as MIPI packets and sent to the AP through the MIPI-TX module;

Step 105, the AP receives the MIPI data packet, and sends the MIPI data packet to the data parsing module for unpacking.

Step 106, send the unpacked raw image data to the ISP chip, and the ISP chip processes it based on the pipeline, and send the unpacked 3A data and PD data to the 3A algorithm module.

The conversion circuit of the PreISP chip is a module realized by hardware, which mainly realizes the following functions:

The raw image data, PD data and generated 3A data of the image sensor are received in the PreISP chip; then, the conversion circuit rearranges these data. As shown in Figure 11, before the rearrangement, the data length of each row is different. After the rearrangement, each row has a fixed data length, that is, 1024 bytes, and there is a data interval of a certain length between adjacent rows ( That is, blank). And after rearranging, the raw image data is organized into one data set, the PD data is organized into one data set, and the 3A data is organized into one data set.

After rearranging the data, each line of data corresponds to fixed-length data, and a fast parsing agreement has been reached. The packaging protocol is as follows:

(1) After packing, the line width is fixed to 1024Bytes.

(2) The packed rows correspond to the agreed mapping table, which is used to agree to allocate fixed data. For example, the first row is long-exposure raw image data, the second row is short-exposure raw image data, and so on.

The data analysis module on the AP side is a software module. Using the same mapping table as the PreISP chip, the received data blocks can be quickly parsed by software. The schematic pseudo code is as follows:

Among them, i represents the line number, line_count_of_pack represents the number of lines of MIPI data packets, the length of each line is 1024, and the value of line_count_of_pack is an integer, which can be adjusted according to the total data size. Each cycle will judge whether the line is a raw image according to the mapping table Data, PD data or 3A data; all packets can be parsed in one cycle, and then sent to the 3A algorithm module or ISP chip for processing after parsing.

The trigger mode of the data parsing module on the AP side can be triggered by a line interrupt mode. In the MIPI protocol on the AP side, it can be set to generate a line interrupt every time a line of data is received. That is, it triggers analysis while transmitting. There is no need to wait for all cached data (buffer) transmission to end before parsing.

In the embodiment of the present application, (1) in the PreISP chip scheme, transmission is carried out according to a fixed line spacing and a fixed number of lines, and the analysis speed is fast; (2) line interruption is used to analyze while transmitting, so that the transmission reaches zero delay Transmission; (3) Transmission of 1024 bytes per line, making the parsing speed on the AP side faster;

In the embodiment of the present application, the raw image data and meta data are packed in a regular arrangement, so that the MIPI receiving end can use less VC_DT and can receive quickly; for the raw image data and meta data, they are decomposed into several lines, each The length of the row is a fixed length. Add a tag in front of each line, and the tag is used to indicate the data type (the types of raw image data and meta image data are inconsistent). Then, pack the data into MIPI packets of the same VC_DT. Send to AP via the same VC_DT of MIPI hardware.

And, use the line interrupt method to trigger the data parsing module to analyze line by line, so as to achieve quasi-zero delay analysis; enable the line interrupt generated when the data packet is received on the AP side, so that when the line interrupt is received, the edge can be real-time Receive and parse line by line. In this way, when processing AF autofocus data, etc., it is possible to receive data and respond faster.

The above solution of the embodiment of the present application can be extended to the transmission of other data, not limited to raw image data, 3A data and PD data. This method can also be used for other newly added data types in the future.

Based on the above-mentioned embodiments, this embodiment of the present application provides a data transmission device, which includes each module included, and each unit included in each module, which can be implemented by a processor; of course, it can also be implemented by a specific logic circuit Implementation; in the process of implementation, the processor can be an image processing chip, a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA).

Fig. 12 is a schematic structural diagram of a data transmission device according to an embodiment of the present application. As shown in Fig. 12, the data transmission device 12 includes:

A conversion module 121, configured to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M;

The packing module 122 is used for packing the second data sets to be transmitted of the N paths according to the MIPI protocol, so as to obtain the first batch of one or more MIPI data packets;

The transmission module 123 is configured to transmit the first batch of one or more MIPI data packets to the application processor through N data channels.

In some embodiments, the conversion module 121 is configured to: decompose the data to be transmitted in the M-way first data set to be transmitted into at least one row of data according to a specific row data length; rearrange the at least one row of data , to obtain the second data sets to be transmitted of the N paths; correspondingly, the transmission module 123 is configured to: distribute the first batch of one or more MIPI data packets evenly to the N paths of data channels and pass through the N data channels are transmitted to the application processor.

In some embodiments, the conversion module 121 is configured to: determine the first type of each row of data; update the first type of label used to characterize the row of data into the row of data; update the updated At least one row of data is rearranged to obtain the N channels of second data sets to be transmitted.

In some embodiments, the conversion module 121 is configured to: determine the second type of each row of data after the update; the second type belongs to a subtype of the first type; The mapping relationship is to rearrange the at least one row of updated data to obtain the N-way second data sets to be transmitted.

In some embodiments, the specific row data length corresponds to the maximum amount of data read from the memory per unit time.

In some embodiments, the first data set to be transmitted includes at least one of the following: statistical data, PD data and original image data required to execute the 3A algorithm.

The embodiment of the present application also provides a data receiving device. FIG. 13 is a schematic structural diagram of the data receiving device in the embodiment of the present application. As shown in FIG. 13 , the data receiving device 13 includes:

The receiving module 131 is used to determine whether a whole row of data is currently received according to a specific row data length; and if a whole row of data is currently received, trigger the data parsing module 132;

The data parsing module 132 is configured to parse the currently received row of data to obtain parsed data; and send the parsed data to the corresponding first image processing chip.

In some embodiments, the data parsing module 132 is configured to: determine the line number of the parsed data; determine the second type of the parsed data according to the agreed second type corresponding to each row; determine the second type of the parsed data according to the parsed data of the second type, sending the analysis data to the corresponding first image processing chip.

In some embodiments, the receiving module 131 is configured to determine whether an entire row of data is currently received according to a specific row data length; and if an entire row of data is currently received, the output row is interrupted to the data parsing module 132 to trigger The data parsing module 132 parses a row of data currently received.

The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

It should be noted that the division of modules by the above device is schematic, and is only a logical function division, and there may be another division method in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or physically exist separately, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. It can also be implemented in the form of a combination of software and hardware.

It should be noted that, in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solutions of the embodiments of the present application or the part that contributes to the related technologies can be embodied in the form of software products. The computer software products are stored in a storage medium and include several instructions to make The electronic device executes all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read-only memory (Read Only Memory, ROM), magnetic disk or optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

The embodiment of the present application provides an electronic device. FIG. 14 is a schematic diagram of the hardware entity of the electronic device in the embodiment of the present application. As shown in FIG. 14 , the electronic device 14 includes a memory 141 and a processor 142, and the memory 141 stores A computer program that can run on the processor 142, and the processor 142 implements the steps in the methods provided in the above-mentioned embodiments when executing the program.

It should be noted that the memory 141 is configured to store instructions and applications executable by the processor 142, and may also cache data to be processed or processed by each module in the processor 142 and the electronic device 14 (for example, image data, audio data, etc. , voice communication data and video communication data), can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM).

An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the methods provided in the foregoing embodiments are implemented.

The embodiment of the present application provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the steps in the method provided by the above method embodiment.

It should be pointed out here that: the descriptions of the above storage medium and device embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to those of the method embodiments. For technical details not disclosed in the storage medium, storage medium, and device embodiments of the present application, please refer to the description of the method embodiment of the present application for understanding.

It should be understood that reference throughout this specification to "one embodiment" or "an embodiment" or "some embodiments" means that a particular feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application . Thus, appearances of "in one embodiment" or "in an embodiment" or "in some embodiments" in various places throughout the specification are not necessarily referring to the same embodiments. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present application. The implementation process constitutes any limitation. The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments. The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments, the same or similar points can be referred to each other, and for the sake of brevity, details are not repeated herein.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, such as object A and/or object B, which can mean: object A exists alone, and object A and object exist at the same time B, there are three situations of object B alone.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article or apparatus comprising that element.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. The above-described embodiments are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be Integrate into another system, or some features may be ignored, or not implemented. In addition, the mutual coupling, or direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms of.

The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed to multiple network units; Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can be used as a single unit, or two or more modules can be integrated into one unit; the above-mentioned integration The modules can be implemented in the form of hardware, or in the form of hardware plus software functional units.

Those of ordinary skill in the art can understand that all or part of the steps to realize the above method embodiments can be completed by hardware related to program instructions, and the aforementioned programs can be stored in computer-readable storage media. When the program is executed, the execution includes The steps of the foregoing method embodiments; and the foregoing storage media include: removable storage devices, read-only memory (Read Only Memory, ROM), magnetic disks or optical disks and other media that can store program codes.

Alternatively, if the above-mentioned integrated units of the present application are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solutions of the embodiments of the present application or the part that contributes to the related technologies can be embodied in the form of software products. The computer software products are stored in a storage medium and include several instructions to make The electronic device executes all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes various media capable of storing program codes such as removable storage devices, ROMs, magnetic disks or optical disks.

The methods disclosed in several method embodiments provided in this application can be combined arbitrarily to obtain new method embodiments under the condition of no conflict. The features disclosed in several product embodiments provided in this application can be combined arbitrarily without conflict to obtain new product embodiments. The features disclosed in several method or device embodiments provided in this application can be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

The above is only the embodiment of the present application, but the scope of protection of the present application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, and should covered within the scope of protection of this application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims (20)

1. A data transmission method for transmitting image data between an application processor and an image sensor, the method comprising:

   Converting the first data set to be transmitted from the M path into the second data set to be transmitted from the N path; wherein, M is greater than 1, and N is greater than 0 and less than M;

   According to the MIPI protocol, the second data sets to be transmitted of the N channels are packaged to obtain the first batch of one or more MIPI data packets;

   and transmitting the first batch of one or more MIPI data packets to the application processor through N data channels.
2. The method according to claim 1, wherein said converting the first data set to be transmitted from M channels into the second data set to be transmitted from N channels comprises:

   Decomposing the data to be transmitted in the M-way first data set to be transmitted into at least one row of data according to a specific row data length;

   Rearranging the at least one row of data to obtain the N-way second data set to be transmitted;

   Wherein, the transmitting the first batch of one or more MIPI data packets to the application processor through N-way data channels includes: evenly distributing the first batch of one or more MIPI data packets to the N-way The data channels are transmitted to the application processor through the N data channels.
3. The method according to claim 2, wherein said rearranging said at least one row of data to obtain said N-way second data sets to be transmitted comprises:

   Determine the first type of data for each row;

   updating the first type of label used to characterize the row data into the row data;

   Rearranging the at least one row of updated data to obtain the N channels of second data sets to be transmitted.
4. The method according to claim 3, wherein said rearranging said at least one row of updated data to obtain said N-way second data sets to be transmitted comprises:

   Determine the second type of each row of data after the update; wherein, the second type belongs to a subtype of the first type; the first type of label is used to identify that the corresponding row data is the content or metadata of the data itself Data; the second type refers to the data type of the corresponding row data;

   According to the mapping relationship between the second type and the row number, the updated at least one row of data is rearranged to obtain the N channels of second data sets to be transmitted.
5. The method according to any one of claims 2 to 4, wherein the specific row data length corresponds to the maximum amount of data read from the memory within a unit time.
6. The method according to any one of claims 2 to 4, wherein there is blank data between adjacent rows of data.
7. The method according to claim 1, wherein the first data set to be transmitted includes at least one of the following: image statistical data, PD data, and original image data required to execute the 3A algorithm.
8. A data receiving method, the method comprising:

   According to the specific row data length, determine whether a whole row of data is currently received;

   If a whole row of data is currently received, analyze the currently received row of data to obtain the parsed data;

   Send the analysis data to the corresponding first image processing chip.
9. The method according to claim 8, wherein, if a whole row of data is currently received, parsing the currently received row of data to obtain the parsed data includes:

   If a whole row of data is currently received, generate a row interrupt;

   According to the line interruption, the currently received line of data is analyzed to obtain the analysis data.
10. The method according to claim 8, wherein the sending the analysis data to the corresponding first image processing chip comprises:

    determining a line number of the parsed data;

    Determine the second type of the parsed data according to the agreed second type corresponding to each row;

    According to the second type of the analysis data, the analysis data is sent to a corresponding first image processing chip.
11. An image processing chip, the chip comprising:

    A conversion circuit, configured to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M;

    A MIPI-TX circuit, configured to package the N-way second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and

    and transmitting the first batch of one or more MIPI data packets to the application processor through N data channels.
12. The chip according to claim 11, wherein the conversion circuit is configured to:

    Decomposing the data to be transmitted in the M-way first data set to be transmitted into at least one row of data according to a specific row data length;

    Rearranging the at least one row of data to obtain the N-way second data set to be transmitted;

    The MIPI-TX circuit is configured to: evenly distribute the first batch of one or more MIPI data packets to the N data channels and transmit them to the application processor through the N data channels.
13. The chip according to claim 12, wherein the conversion circuit is used for:

    Determine the first type of data for each row;

    updating the first type of label used to characterize the row data into the row data;

    Rearranging the at least one row of updated data to obtain the N channels of second data sets to be transmitted.
14. An application processor, the application processor includes:

    The MIPI-RX circuit is used to determine whether a whole line of data is currently received according to a specific line data length; if a whole line of data is currently received, the data analysis module is triggered to analyze the currently received line of data to obtain the analysis data ;

    The data analysis module is further configured to send the analysis data to the corresponding first image processing chip;

    The first image processing chip is configured to execute a specific image processing algorithm according to each received analysis data.
15. The application processor according to claim 14, wherein the MIPI-RX circuit is used to determine whether a whole row of data is currently received according to a specific row data length; if a whole row of data is currently received, the output row is interrupted to The data parsing module is configured to trigger the data parsing module to parse a row of data currently received.
16. An electronic device, the device comprising: a second image processing chip, at least N data channels and an application processor, where N is greater than 0; wherein,

    The conversion circuit of the second image processing chip is used to convert the first data set to be transmitted from M channels into the second data set to be transmitted from N channels; wherein, M is greater than 1, and N is greater than 0 and less than M;

    The MIPI-TX circuit of the second image processing chip is used to package the N-way second data sets to be transmitted according to the MIPI protocol to obtain the first batch of one or more MIPI data packets; and the first batch of one or more MIPI data packets; A batch of one or more MIPI data packets is transmitted to the MIPI-RX circuit of the application processor through N data channels;

    The MIPI-RX circuit of the application processor is used to determine whether an entire row of data is currently received according to a specific row data length; if an entire row of data is currently received, trigger the data analysis module of the application processor to Analyzing the currently received line of data to obtain the analyzed data, and sending the analyzed data to the corresponding first image processing chip;

    The first image processing chip of the application processor is configured to execute a specific image processing algorithm according to each received analysis data.
17. A data transmission device, said device comprising:

    A conversion module, configured to convert the first data set to be transmitted from the M path into the second data set to be transmitted from the N path; wherein, M is greater than 1, and N is greater than 0 and less than M;

    A packaging module, configured to package the second data sets to be transmitted of the N paths according to the MIPI protocol, so as to obtain the first batch of one or more MIPI data packets;

    A transmission module, configured to transmit the first batch of one or more MIPI data packets to the application processor through N data channels.
18. A data receiving device, the device comprising:

    The receiving module is used to determine whether a whole row of data is currently received according to a specific row data length; and if a whole row of data is currently received, trigger the data parsing module to parse the currently received row of data to obtain parsing data; and sending the analyzed data to the corresponding first image processing chip.
19. An electronic device, comprising a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements the method according to any one of claims 1 to 7 when executing the program, or , the processor implements the method according to any one of claims 8 to 10 when executing the program.
20. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is realized, or, when the computer program is executed by the processor, the method as described in The method according to any one of claims 8 to 10.

Images