WO2020215220A1

WO2020215220A1 - Interface timing conversion device, image data processing method, transmission method and system, and storage medium

Info

Publication number: WO2020215220A1
Application number: PCT/CN2019/083949
Authority: WO
Inventors: 黄帅凯; 谢骞
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-04-23
Filing date: 2019-04-23
Publication date: 2020-10-29
Also published as: CN111373736A

Abstract

Provided by the embodiments of the present application are an interface timing conversion device, an image data processing method, a transmission method and system, and a storage medium. The present invention may solve problems in HDR image transmission and timing conversion, ensure image transmission quality and improve system performance by means of extracting a first exposure value data packet and a second exposure value data packet from image data, and allocating same to a first transmission path and a second transmission path respectively to perform different timing conversion processing.

Description

Interface timing conversion device, image data processing method, transmission method and system, and storage medium

Technical field

This application relates to the technical field of image data processing and transmission, and in particular to an interface timing conversion device and an image data processing method thereof, an image data transmission method and an image data transmission system, and a storage medium.

Background technique

At present, with the growing demand for high frame rate and high resolution in the image sensor market, Sony has introduced a new generation of image data transmission interface standard "Scalable Low Voltage Signaling With Embedded Clock" (Scalable Low Voltage Signaling). Clock, SLVS-EC). The SLVS-EC interface protocol mainly defines the physical layer (data serial transmission specification, electrical characteristics of the sender and receiver), protocol layer (image data sending format and packetization protocol) and system topology.

Three typical system topologies (ie, transmission modes) are defined in the SLVS-EC interface protocol, as follows.

As shown in Figure 1, the first system topology is the single-port mode (Basic Topology). The working principle of the main control chip docking the SLVS-EC interface image sensor includes: the image sensor sends the pixels stored in the pixel array in order. The end circuit is transmitted to the receiving end circuit of the main control chip, and the receiving end circuit parses the data according to the format specified by the SLVS-EC protocol, and converts it into VBUS interface timing (private transmission protocol defined in the main control chip) and transmits it to the subsequent module image signal Processor (Image Signal Processor, ISP). Figure 1 shows a typical application scenario of the SLVS-EC interface. The SLVS-EC interface protocol stipulates that a port has a maximum of 8 digital channels, and the bandwidth brought by 8 data channels can basically meet the typical application requirements in the camera market.

However, in the high-end camera market, a larger bandwidth (more than 8 data channels) is required to meet customer needs, so the second system topology "Multiple I/F Topology" (Multiple I/F Topology) came into being. As shown in Figure 2, a single image sensor sends image data through two SLVS-EC sending ports, in which odd-numbered columns and even-numbered columns of pixels in a row are distinguished for transmission.

In addition, in some applications, there are scenarios where two image sensors need to be connected at the same time, that is, the third system topology "Multiple CIS Topology" (Multiple CIS Topology), or 2 Sensor mode, as shown in Figure 3.

However, with the high pursuit of image quality, High Dynamic Range (HDR) has developed rapidly. HDR is a technology commonly used in photography, which can overcome the shortcomings of most camera sensors with limited dynamic range. Multiple images with different exposure values are superimposed and processed to synthesize a high-contrast image, thereby bringing higher image quality. The HDR function of the image sensor of the SLVS-EC interface will produce two images with different exposure values in one exposure, referred to as long exposure frame and short exposure frame. In the HDR mode of the image sensor, there are two different image transmission methods, called HDR 1Stream and HDR 2Stream. The two HDR modes can cooperate with the packet format of the protocol layer to complete the image transmission. However, the existing image receiving modules in FIGS. 1 to 3 only support single port and dual port modes. If the image is transmitted in HDR, the existing image receiving modules cannot support it. For example, for the HDR 1Stream mode, the existing image receiving module sends the data packets interleaved with long and short exposure frames to the ISP through the same port (such as VBUS), but the ISP cannot deal with such data packets with different exposure values and interleaved together. Processing, ISP can effectively handle the transmission of only one complete frame of image on the same VBUS port (that is, the exposure value needs to be the same). Therefore, the existing technology system topology can no longer meet the needs of technological development, affecting the quality of image transmission and System performance.

Summary of the invention

Based on this, this application provides an interface timing conversion device, image data processing method, transmission method and system, and storage medium, aiming to solve the technical problem that the existing system topology cannot meet the needs of image technology development and ensure the quality of image transmission , Improve system performance.

In the first aspect, this application provides an image data processing method, including:

Obtain image data;

Extracting the first exposure value data packet and the second exposure value data packet in the image data;

The first exposure value data packet is allocated to a first transmission path, and the second exposure value data packet is allocated to a second transmission path, so as to compare the first exposure value data packet and the second exposure value The data packets perform different timing conversion processing.

In the second aspect, this application also provides an interface timing conversion device, including a storage circuit and a processing circuit;

The storage circuit is used to store a computer program:

The processing circuit is used to execute the computer program and, when the computer program is executed, implement the following steps:

Obtain image data;

In the third aspect, this application also provides an image data transmission method, including:

The image sensor sends image data;

The interface timing conversion device obtains image data from the image sensor;

The interface timing conversion device extracts the first exposure value data packet and the second exposure value data packet in the image data;

The interface timing conversion device allocates the first exposure value data packet to a first transmission path, and allocates the second exposure value data packet to a second transmission path;

The interface timing conversion device performs different timing conversion processing on the first exposure value data packet and the second exposure value data packet;

The interface timing conversion device is transmitted to the subsequent image signal processing device through different interfaces.

In a fourth aspect, this application also provides an image data transmission system, including an image sensor, an interface timing conversion device, and an image signal processing device:

The image sensor is used to send image data;

The interface timing conversion device is used to obtain image data from the image sensor;

The interface timing conversion device is used to extract a first exposure value data packet and a second exposure value data packet in the image data;

The interface timing conversion device is configured to allocate the first exposure value data packet to a first transmission path, and allocate the second exposure value data packet to a second transmission path;

The interface timing conversion device is configured to perform different timing conversion processing on the first exposure value data packet and the second exposure value data packet;

The interface timing conversion device is used to transmit to the image signal processing device through different interfaces.

In a fifth aspect, the present application also provides a storage medium that stores a computer program, and when the computer program is executed by the processing circuit, the processing circuit realizes:

Obtain image data;

The embodiment of the application provides an interface timing conversion device, an image data processing method, a transmission method and system, and a storage medium, by extracting a first exposure value data packet and a second exposure value data packet in the image data, and respectively Allocate to the first transmission path and the second transmission path to perform different timing conversion processing, which can solve the transmission and timing conversion problems of HDR images, ensure image transmission quality, and improve system performance.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic diagram of the image transmission principle in the single-port mode in the prior art;

FIG. 2 is a schematic diagram of the image transmission principle in the dual-port mode in the prior art;

3 is a schematic diagram of the image transmission principle in the dual sensor mode in the prior art;

4 is a schematic flowchart of steps of an image data processing method provided by an embodiment of the present application;

Figure 5 is a schematic diagram of the basic format of the SLVS-EC interface data packet;

6 is a schematic diagram of the principle of sending/receiving data packets in the first HDR mode according to an embodiment of the present application;

Fig. 7 is a schematic diagram of the principle of sending/receiving data packets in the second HDR mode according to an embodiment of the present application;

FIG. 8 is a schematic flowchart of steps in a specific embodiment of extracting a first exposure value data packet and a second exposure value data packet in the image data shown in FIG. 4;

9 is a schematic flowchart of steps of a specific embodiment of extracting a first exposure value data packet and a second exposure value data packet in the image data shown in FIG. 4;

FIG. 10 is a schematic flowchart of steps of a specific embodiment of extracting a first exposure value data packet and a second exposure value data packet in the image data shown in FIG. 4;

11 is a schematic flowchart of steps of a specific embodiment of extracting a first exposure value data packet and a second exposure value data packet in the image data shown in FIG. 4;

FIG. 12 is a schematic flowchart of steps of a specific embodiment of extracting a first exposure value data packet and a second exposure value data packet in the image data shown in FIG. 4;

FIG. 13 is a schematic diagram of functional modules of an interface timing conversion device provided by an embodiment of the present application;

14 is a schematic diagram of the circuit connection of a specific embodiment of the interface timing conversion device shown in FIG. 13, in which the image sensor and the ISP are also shown;

FIG. 15 is a schematic flowchart of steps of an image data transmission method provided by an embodiment of the present application;

FIG. 16 is a schematic diagram of functional modules of a specific embodiment of an image data transmission system provided by an embodiment of the present application;

FIG. 17 is a schematic flowchart of steps implemented when a computer program stored in a storage medium is executed by a processing circuit according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is merely an illustration, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Hereinafter, some embodiments of the application will be described in detail with reference to the drawings. In the case of no conflict, the following embodiments and the features in the embodiments can be combined with each other.

Please refer to FIG. 4, which is a schematic flowchart of steps of an image data processing method provided by an embodiment of the present application. The image data processing method of this embodiment can be applied to various terminal devices for automatic image post-processing . Among them, terminal equipment includes mobile phones, tablets, laptops and drones.

Specifically, as shown in FIG. 4, the image data processing method may include step S101 to step S103.

S101. Acquire image data.

The image may be an original picture taken by the terminal device, or one or more pictures selected by the user in the graphics library of the terminal device.

S102: Extract a first exposure value data packet and a second exposure value data packet in the image data.

It should be particularly noted that this embodiment can be applied to the processing of the existence of the first exposure value data packet and/or the second exposure value data packet. For example, the following three cases can be handled: the first exposure value data packet and the second exposure value data packet exist simultaneously. The exposure value data packet, or only the first exposure value data packet, or only the second exposure value data packet. In addition, the first exposure value and the second exposure value are relative terms, and the specific exposure time value is not limited.

S103. Allocate the first exposure value data packet to a first transmission path, and allocate the second exposure value data packet to a second transmission path, so as to compare the first exposure value data packet and the second transmission path. The exposure value data packet executes different timing conversion processing.

As mentioned above, in this embodiment, different transmission paths can be used to transmit different exposure value data packets. When one of them is zero, it means that there is no corresponding exposure value data packet. When both of them exist, then two All transmission paths are used to transmit data packets. It should be pointed out that the two transmission paths in this embodiment are only an example. With the development of technology, if there are multiple exposure value data packets in the same image, this embodiment can correspond to multiple transmission paths. And can perform a variety of sequential conversion processes.

It needs to be explained in detail here that compared with the prior art, this embodiment can implement two additional system topologies. The first one is defined as HDR 1Stream mode (or as the first HDR mode), which corresponds to The image sensor set in the previous stage is an HDR 1Stream mode image sensor; the second is defined as HDR 2Stream mode (or defined as the second HDR mode), which corresponds to an HDR 2Stream mode set in the previous stage The image sensor; both modes can be implemented based on the three system structures of Figures 1 to 3 of the prior art with the packet format of the protocol layer to improve the compatibility of this application. In other words, this embodiment can implement at least five system topology functions.

Specifically, the working principle of its specific cooperation transmission is shown in Figure 5, which is the packet format of the protocol layer. The first thing to note is that in the SLVS-EC interface protocol, one line of pixel data of a frame of image is used as the basic element of a data packet. On this basis, the SLVS-EC interface transmitter circuit will add header information to the data packet. The information mainly includes some key image synchronization information, and the receiving circuit used in the SLVS-EC interface can correctly obtain and parse the image related information.

Among them, the HDR 1Stream mode of this embodiment is based on the single-port mode similar to the above-mentioned Figure 1, but the difference is that the data packets transmitted at this time are in the form of interleaving long exposure frames and short exposure frames, as shown in Figure 6 Example. The SLVS-EC interface transmitter circuit can use different data ID identification bits to distinguish the data packets of the long exposure frame and the short exposure frame.

Similarly, as shown in Figure 7, the HDR 2Stream of this embodiment is similar to the dual-port mode shown in Figure 2, but the difference is that the two SLVS-EC interface transmitter circuits transmit independent long The pixels of the exposure frame and the short exposure frame are not the pixels of the odd and even columns of the single frame image.

It is not difficult to understand that the receiving end of the SLVS-EC interface of this embodiment can complete the conversion of the SLVS-EC standard serial data stream to parallel data, and can complete the analysis of the data packet according to the protocol layer specification, and then convert it to the ISP. VBUS interface timing that can be received.

Through this embodiment, this application solves the problems of HDR image transmission and timing conversion, guarantees image transmission quality, and improves system performance.

Please refer to FIG. 8 in conjunction with the above embodiment. In order to realize the transmission of HDR images, the extraction of the first exposure value data packet and the second exposure value data packet in the image data described in the embodiment of the present application may specifically include steps S1021 and S1022.

S1021: Determine the current working mode.

S1022: If it is determined that the working mode is the first HDR mode, split the image data to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the first exposure The value data packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.

Furthermore, when the image sensor is in HDR mode, the data packet readout time of the short exposure frame will be later than that of the long exposure frame. During this interval, the image sensor will send padding packets, and these padding packets are not valid. These useless filling data packets will affect the processing efficiency of the subsequent ISP. Therefore, this embodiment can remove these filling data packets in the previous stage in advance. Specifically, the allocating the second exposure value data packet to the second transmission path may further include: removing the padding data packet in the second exposure value data packet; wherein, the padding data packet is located in The fixed frame position in the short exposure value data packet or the specified frame position where the filling data packet is marked in the header information of the short exposure value data packet, so as to correspond according to a preset rule in the first HDR mode Remove the padding data packet at the fixed frame position or the designated frame position.

It should be noted that the preset rules can be set in advance, or during the transmission of HDR images, when a short exposure value data packet is detected, the header information is automatically identified, and when it is identified that there is padding When the packet is marked, the padding packet is automatically removed.

Correspondingly, acquiring the image data in this embodiment may specifically include: acquiring the first exposure value data packet and the second exposure value data packet through the same input path. In other words, for HDR images, this embodiment can obtain the first exposure value data packet and the second exposure value data packet through the same SLVS-EC interface receiving end circuit, where the long exposure value data packet and the all The image data constructed by the short exposure value data packet is high dynamic range HDR image data.

For example, in this scenario, this application can be connected to an image sensor in HDR 1Stream mode, which is processed by the receiving end circuit of the SLVS-EC interface and converted into a data packet interleaved with long/short exposure frames, and then through the data packet header data ID The analysis of the flag bit completes the shunt operation of the long exposure frame and the short exposure frame data packet. The long exposure frame data packet will enter a VBUS IF GEN circuit for processing, and then output through a VBUS port, while the short exposure data packet You can remove the invalid stuffing packet first, then send it to another VBUS IF GEN circuit for processing, and finally output it to the subsequent ISP through another VBUS port for processing.

Next, referring to FIG. 9, the extraction of the first exposure value data packet and the second exposure value data packet in the image data according to the embodiment of the present application also includes steps S1021 and S1023.

S1021: Determine the current working mode.

S1023: If it is determined that the working mode is the second HDR mode, obtain the first exposure value data packet through the first input path, and obtain the second exposure value data packet through the second input path.

Correspondingly, the first exposure value data packet in this embodiment is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value data packet and the short exposure value data packet are The image data constructed by the data package is high dynamic range HDR image data.

It is not difficult to understand that the HDR 2Stream of this embodiment is similar to the dual-port mode shown in Figure 2, but the difference is that at this time, the two SLVS-EC interface transmitter circuits transmit independent long exposure frames and short exposures. Frame pixels instead of pixels in odd and even columns of a single frame image. This application obtains the first exposure value data packet through the first input path, and obtains the second exposure value data packet through the second input path, which can effectively implement image data processing in the HDR 2Stream mode.

For example, in this scenario, this application can be connected to an HDR 2Stream mode image sensor, and the long exposure frame/short exposure frame data passes through two SLVS-EC interface receiving end circuits (first input path, second input Path) transmission, complete the timing conversion through, for example, two VBUS IF GEN circuits, and finally output to the subsequent ISP for processing through two VBUS ports.

Next, referring to FIG. 10, the extraction of the first exposure value data packet and the second exposure value data packet in the image data in the embodiment of the present application may further include steps S1021 and S1024.

S1021: Determine the current working mode.

S1024. If it is determined that the working mode is the single-port mode, obtain the image data through the same input path to extract the first exposure value data packet and the second exposure value data packet, where the first exposure value data packet and the second exposure value data packet One of the value packets is zero.

It is easy to understand that this embodiment is aimed at mode selection in the case of single-port mode, and accordingly can be understood as a special scene in which the exposure value data packet is zero.

For example, in this scenario, this application can be connected to an image sensor in single-port mode, and an SLVS-EC interface receiver circuit (same input path) parses high-speed serial data into basic pixel data packets and transmits them Give the interface timing conversion circuit, and then the data packet can be processed by, for example, the VBUS IF GEN circuit, converted into the VBUS interface timing, and finally sent to the subsequent ISP through the VBUS port for processing.

Next, referring to FIG. 11, the extraction of the first exposure value data packet and the second exposure value data packet in the image data according to the embodiment of the present application may further include steps S1021, S1025, and S1026.

S1021: Determine the current working mode.

S1025: If it is determined that the working mode is the dual-port mode, obtain the odd-numbered column serial data in the image data through the first input path, and obtain the even-numbered column serial data in the image data through the second input path.

S1026. Splicing the odd column serial data and the even column serial data to restore the first exposure value data packet and the second exposure value data packet extracted from the image data, wherein the first exposure value data packet in the image data One of the one exposure value data packet and the second exposure value data packet is zero.

For example, in this scenario, this application can be connected to an image sensor in dual port mode, and the two SLVS-EC interface receiving end circuits (first input path and second input path) will respectively connect the serial numbers of the odd and even columns. The data is parsed into corresponding pixel data packets and transmitted to the interface timing conversion circuit. After the two-way data packet is spliced, the odd and even pixels are spliced back to a complete line of pixels, and then converted into VBUS interface timing, and finally sent to the subsequent ISP processing through a VBUS port.

Next, referring to FIG. 12, the extraction of the first exposure value data packet and the second exposure value data packet in the image data according to the embodiment of the present application may further include steps S1021 and S1027.

S1021: Determine the current working mode.

S1027: If it is determined that the working mode is the dual sensor mode, obtain the first exposure value data packet through the first input path, and obtain the second exposure value data packet through the second input path.

It should be noted that the first exposure value data packet and the second exposure value data packet in this embodiment are a short exposure value data packet, a long exposure value data packet or a combination of both.

For example, in this scenario, this application can interface with two image sensors in single-port mode. The data packet of one image sensor is processed by one of the SLVS-EC interface receiving end circuits and one of the VBUSIF GEN circuits. One of the VBUS ports is output to the ISP; and the data packet of the other image sensor is processed by another SLVS-EC interface receiving end circuit, and another VBUS IF GEN circuit is processed and output to the ISP through the other VBUS port.

It is not difficult to see that the image data obtained in the embodiments of the present application can all be obtained through the low-voltage signal SLVS-EC interface of the expandable embedded clock.

At the same time, the execution of different time sequence conversion processing on the first exposure value data packet and the second exposure value data packet in this application may include: using a first interface time sequence conversion module to convert the first exposure value data packet The timing of the data packet is converted into a VBUS interface timing, and the second interface timing conversion module is used to convert the timing of the second exposure value data packet into a VBUS interface timing to transmit to the subsequent image signal processing device through different VBUS interfaces.

Through the above specific embodiments, this application can avoid the problems of too many interface receiving circuits occupying resources and area and system consumption under the premise of using only two SLVS-EC interface receiving end circuits. By improving the way of interface timing conversion, The image receiving module can work correctly in five modes: single port, dual port, HDR1Stream, HDR 2Stream, dual sensor, etc., and transmit complete image data packets to the ISP, especially in HDR 1Stream mode. Packet header analysis, divide long and short exposure frames into different VBUS interfaces for output, and discard padding packets before short exposure frames, thereby improving ISP processing efficiency.

In order to make this application easier to understand and explain, please refer to FIG. 13, which is a schematic diagram of modules of an interface timing conversion device used to implement one of the embodiments of the above-mentioned image data processing method.

Wherein, the interface timing conversion device described in this embodiment may include a storage circuit 21 and a processing circuit 22.

In this embodiment, the storage circuit 21 is used to store computer programs.

In this embodiment, the processing circuit 22 is used to execute the computer program, and when the computer program is executed, the following steps are implemented:

Obtain image data;

As one of the embodiments, when the processing circuit 22 implements the extraction of the first exposure value data packet and the second exposure value data packet in the image data, it is used to implement: judging the current working mode; If it is determined that the working mode is the first HDR mode, the image data is split to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the first exposure value data The packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.

In this embodiment, when the processing circuit 22 implements the allocation of the second exposure value data packet to the second transmission path, it is also used to implement: remove the padding in the second exposure value data packet data pack.

In this embodiment, the padding data packet is located at a fixed frame position in the short exposure value data packet, or the specified frame position where the padding data packet is located is marked in the header information of the short exposure value data packet to In the first HDR mode, the padding data packet at the fixed frame position or the designated frame position is correspondingly removed according to a preset rule.

It should be noted that the preset rules can be set in advance, or in the process of HDR image transmission, when a short exposure value data packet is detected, the header information is automatically identified, and when it is identified When filling the mark of the data packet, the filling data packet is automatically removed.

As an optional embodiment, in the process of removing the padding packet, the number of rows n of the padding packet may be recorded and sent to the picture trimming circuit, that is, the PictureClip circuit, and the PictureClip circuit removes the n rows of padding packets.

As an optional embodiment, in each row of the padding packet generated by the sensor, there is a special header file to indicate that the packet is a data row of the data padding packet, and the Picture Clip circuit can remove the data row of the padding packet according to the header file.

As an optional embodiment, the data stuffing packet can be removed in the PictureClip circuit by recording the number of rows of the stuffing packet and the header file of the data stuffing packet.

In this embodiment, when the processing circuit 22 realizes the acquisition of image data, it is used to realize: acquiring the first exposure value data packet and the second exposure value data packet through the same input path.

Wherein, the image data constructed by the long exposure value data packet and the short exposure value data packet are high dynamic range HDR image data.

As one of the embodiments, when the processing circuit 22 implements the extraction of the first exposure value data packet and the second exposure value data packet in the image data, it is used to implement: judging the current working mode; If it is determined that the working mode is the second HDR mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.

Wherein, the first exposure value data packet is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value data packet and the short exposure value data packet are constructed The image data is high dynamic range HDR image data.

As one of the embodiments, when the processing circuit 22 implements the extraction of the first exposure value data packet and the second exposure value data packet in the image data, it is used to implement: judging the current working mode; If it is determined that the working mode is the single-port mode, the image data is obtained through the same input path to extract the first exposure value data packet and the second exposure value data packet, wherein the first exposure value data packet and the second exposure value data packet One of the packets is zero.

As one of the embodiments, when the processing circuit 22 implements the extraction of the first exposure value data packet and the second exposure value data packet in the image data, it is used to implement: judging the current working mode; If it is determined that the working mode is the dual-port mode, obtain the odd-numbered column serial data in the image data through the first input path, and obtain the even-numbered column serial data in the image data through the second input path; The odd column serial data and the even column serial data are used to restore the first exposure value data packet and the second exposure value data packet extracted from the image data, wherein the first exposure value data packet in the image data And one of the second exposure value data packets is zero.

As one of the embodiments, when the processing circuit 22 implements the extraction of the first exposure value data packet and the second exposure value data packet in the image data, it is used to implement: judging the current working mode; If it is determined that the working mode is the dual sensor mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.

Wherein, the first exposure value data packet and the second exposure value data packet are a short exposure value data packet, a long exposure value data packet or a combination of both.

It is worth mentioning that when the processing circuit 22 realizes the acquisition of image data, it is used to realize: acquiring image data through a low-voltage signal SLVS-EC interface with an expandable embedded clock.

In this embodiment, when the processing circuit 22 implements different time sequence conversion processing on the first exposure value data packet and the second exposure value data packet, it is used to implement: The module converts the timing of the first exposure value data packet into a VBUS interface timing, and uses a second timing conversion module to convert the timing of the second exposure value data packet into a VBUS interface timing, so as to be transmitted through different VBUS interfaces. To the subsequent image signal processing device.

Please refer to FIG. 14, which is a schematic diagram of the circuit connection of a specific implementation of the interface timing conversion device shown in FIG. 13. For ease of description, the first image sensor, the second image sensor and the ISP of the image data transmission system are also shown.

As shown in FIG. 14, the interface timing conversion device of this embodiment may specifically include a first SLVS-EC interface receiving module, a second SLVS-EC interface receiving module, a setting circuit module, a pixel mosaic circuit Pixel Merge, and a data distribution circuit Packet Distribute, the first MUX circuit module, the second MUX circuit module, the image trimming circuit PictureClip, the first VBUS IF GEN circuit module, the second VBUS IF GEN circuit module, the first VBUS port, and the second VBUS port.

Among them, the Setting circuit module is used to store various parameters configured by the processing circuit 22, and control which mode the interface timing conversion device specifically works in.

The pixel splicing circuit is used to splice odd and even pixels back into a complete row of pixels, and it can take effect in dual-port mode.

The data distribution circuit is used to distinguish the data packets interleaved with long and short exposure frames according to the data ID flag in the packet header, which can be effective in HDR 1Stream mode.

The image trimming circuit can be used to remove the useless padding data packets before the short exposure frame, and it can take effect in HDR 1Stream mode. In this embodiment, the padding data packet is located at a fixed frame position in the short exposure value data packet, or the specified frame position where the padding data packet is located is marked in the header information of the short exposure value data packet to In the HDR 1Stream mode, correspondingly remove the padding data packet at the fixed frame position or the designated frame position according to the preset rule. It should be noted that the preset rules can be set in advance, or in the process of HDR image transmission, when a short exposure value data packet is detected, the header information is automatically identified, and when it is identified When filling the mark of the data packet, the filling data packet is automatically removed.

As an optional embodiment, in the process of removing padding packets, the number of rows of padding packets n may be recorded and sent to the image trimming circuit, and the image trimming circuit removes n rows of padding packets.

As an optional embodiment, in each row of the padding packet generated by the sensor, there will be a special header file to indicate that the packet is a data row of the data padding packet, and the image trimming circuit can remove the data row of the padding packet according to the header file .

As an optional embodiment, the data padding packet can be removed in the image trimming circuit by recording the number of rows of the padding packet and the header file of the data padding packet.

The first MUX circuit module and the second MUX circuit module may be multiple data stream selectors.

The first VBUS IF GEN circuit module and the second VBUS IF GEN circuit module are used to convert the timing of the input data packet into a standard VBUS interface timing.

In the specific working process, in the multiple modes described in the foregoing embodiment, it specifically includes the following working principles.

Single port mode: In this scenario, this application can be connected to a first image sensor in single port mode, and the first SLVS-EC interface receiving module parses high-speed serial data into basic pixel data packets, and then the data packets can be After processing such as the data distribution circuit, the first MUX circuit module, and the first VBUS IF GEN circuit module, it is converted into a VBUS interface sequence, and finally sent to the subsequent ISP for processing through the first VBUS port.

Dual port mode: In this scenario, this application can be connected to a first image sensor and a second image sensor in dual port mode. The first SLVS-EC interface receiving module and the second SLVS-EC interface receiving module will be respectively Analyze the serial data of the odd and even column into corresponding pixel data packets. The two data packets are spliced by the pixel splicing circuit, and the odd and even pixels are spliced back into a complete row of pixels, and then converted into VBUS interface timing, and finally passed through the first MUX circuit module , The first VBUS IF GEN circuit module and the first VBUS port are sent to the subsequent ISP processing.

The dual sensor mode is the 2 Sensor mode. In this scenario, this application can interface with the first image sensor and the second image sensor in two single-port modes, where the data packets of the first image sensor are received through the first SLVS-EC interface The module, data distribution circuit, first MUX circuit module, and first VBUS IF GEN circuit module are processed and output to the ISP through the first VBUS port; and the data packet of the second image sensor passes through the second SLVS-EC interface receiving module and the second The MUX circuit module and the second VBUS IF GEN circuit module process the output to the ISP through the second VBUS port.

The first HDR mode is HDR 1Stream mode. In this scenario, this application can be connected to a first image sensor in HDR 1Stream mode, which is processed by the first SLVS-EC interface receiving module and converted into long/short exposure frame interleaved data Then, the data distribution circuit analyzes the ID flag of the packet header to complete the shunt operation of the long exposure frame and the short exposure frame data packet. The long exposure frame data packet will enter the first MUX circuit module and the first VBUS IF The GEN circuit module performs processing and then outputs it through the first VBUS port. The short-exposure data packets can be first removed by the image trimming circuit to remove invalid filling data packets, and then sent to the second MUX circuit module and the second VBUSIF GEN circuit module for processing , And finally output to the subsequent ISP for processing through the second VBUS port.

The second HDR mode is HDR 2Stream mode. In this scenario, this application can interface with a first image sensor and a second image sensor in HDR 2Stream mode, and the long exposure frame/short exposure frame data passes through the first SLVS-EC interface respectively The receiving module and the second SLVS-EC interface receiving module transmit, after passing through the first VBUS IF GEN circuit module and the second VBUS IF GEN circuit module to complete the timing conversion, and finally output to the subsequent ISP through the first VBUS port and the second VBUS port To process.

Through the above specific embodiments, this application can not only avoid the problems of too many interface receiving circuits occupying resources and area and system consumption under the premise of using only two SLVS-EC interface circuits, but also by improving the way of interface timing conversion , So that the image receiving module can work correctly in five modes: single port, dual port, HDR 1Stream, HDR 2Stream, dual sensor, etc., and transmit complete image data packets to the ISP, especially in HDR 1Stream mode. The packet header is analyzed, the long and short exposure frames are divided and output on different VBUS interfaces, and the padding data packets before the short exposure frames are discarded, thereby improving the processing efficiency of the ISP.

Please refer to FIG. 15, which is a schematic flowchart of steps of an image data transmission method according to an embodiment of the present application. The image data transmission method of this embodiment can be applied to various terminal devices for automatic image post-processing. Among them, terminal equipment includes mobile phones, tablets, laptops and drones.

As shown in FIG. 15, the image data transmission method in this embodiment may include step S201 to step S206.

S201: The image sensor sends image data.

S202: The interface timing conversion device obtains image data from the image sensor.

S203: The interface timing conversion device extracts a first exposure value data packet and a second exposure value data packet in the image data.

S204: The interface timing conversion device allocates the first exposure value data packet to a first transmission path, and allocates the second exposure value data packet to a second transmission path.

S205: The interface timing conversion device performs different timing conversion processing on the first exposure value data packet and the second exposure value data packet.

S206: The interface timing conversion device transmits to a subsequent image signal processing device through a different interface.

As one of the embodiments, the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data may include: judging the current working mode; if it is judged that the working mode is In the first HDR mode, the image data is split by a data packet distribution circuit to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the first exposure value data packet Is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.

Wherein, the interface timing conversion device of this embodiment allocates the second exposure value data packet to the second transmission path, and further includes: removing the second exposure allocated by the data packet distribution circuit through an image trimming circuit The padding packet in the value packet.

Wherein, the interface timing conversion apparatus of this embodiment obtains image data from the image sensor, including: obtaining the first exposure value data packet and the second exposure value data packet through the input path of the same interface receiving module.

It is easy to understand that the image data constructed by the long exposure value data packet and the short exposure value data packet in this embodiment are high dynamic range HDR image data.

As one of the embodiments, the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data may include: judging the current working mode; if it is judged that the working mode is In the second HDR mode, the first exposure value data packet is acquired through the input path of the first interface receiving module, and the second exposure value data packet is acquired through the input path of the second interface receiving module.

Wherein, the first exposure value data packet in this embodiment is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value data packet and the short exposure value data packet The image data constructed by the package is high dynamic range HDR image data.

As one of the embodiments, the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data may include: judging the current working mode; if it is judged that the working mode is In single port mode, the image data is acquired through the input path of the receiving module of the same interface to extract the first exposure value data packet and the second exposure value data packet, wherein the first exposure value data packet and the second exposure value data packet are One of them is zero.

As one of the embodiments, the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data may include: judging the current working mode; if it is judged that the working mode is In the dual-port mode, the odd-numbered serial data in the image data is acquired through the input path of the first interface receiving module, and the even-numbered serial data in the image data is acquired through the input path of the second interface receiving module; The pixel stitching circuit stitches the odd column serial data of the first interface receiving module and the even column serial data of the second interface receiving module to restore the first exposure value data packet and the second exposure extracted from the image data A value data packet, wherein one of the first exposure value data packet and the second exposure value data packet in the image data is zero.

As one of the embodiments, the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data includes: judging the current working mode; if it is judged that the working mode is dual In the sensor mode, the first exposure value data packet is acquired through the input path of the first interface receiving module, and the second exposure value data packet is acquired through the input path of the second interface receiving module.

Wherein, the first exposure value data packet and the second exposure value data packet in this embodiment are a short exposure value data packet, a long exposure value data packet or a combination of both.

It is worth mentioning that the interface timing conversion device of this embodiment obtains image data from the image sensor, which may include: obtaining image data through a low-voltage signal SLVS-EC interface receiving module with an expandable embedded clock.

In this embodiment, the interface timing conversion device performs different timing conversion processing on the first exposure value data packet and the second exposure value data packet, including: using a first interface timing conversion module to convert the second The timing of an exposure value data packet is converted into a VBUS interface timing, and the second interface timing conversion module is used to convert the timing of the second exposure value data packet into a VBUS interface timing, so as to be transmitted to the subsequent stage through different VBUS interfaces. Image signal processing device.

Please refer to FIG. 16 in conjunction with the foregoing multiple embodiments. This application provides an image data transmission system. In this embodiment, it may include an image sensor 31, an interface timing conversion device 32, and an image signal processing device 33 (ie ISP).

In this embodiment, the image sensor 31 is used to send image data;

The interface timing conversion device 32 is configured to obtain image data from the image sensor 31;

The interface timing conversion device 32 is configured to extract a first exposure value data packet and a second exposure value data packet in the image data;

The interface timing conversion device 32 is configured to allocate the first exposure value data packet to a first transmission path, and allocate the second exposure value data packet to a second transmission path;

The interface timing conversion device 32 is configured to perform different timing conversion processing on the first exposure value data packet and the second exposure value data packet;

The interface timing conversion device 32 is configured to transmit to the image signal processing device 33 through different interfaces.

Specifically, the interface timing conversion device 32 includes a processing circuit and a data distribution circuit. When extracting the first exposure value data packet and the second exposure value data packet in the image data: the processing circuit is used to determine The current working mode; if it is determined that the working mode is the first HDR mode, the data packet distribution circuit is configured to split the image data to extract the first exposure value data packet and the The second exposure value data packet, wherein the first exposure value data packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.

Wherein, the interface timing conversion device 32 includes an image trimming circuit, when the second exposure value data packet is allocated to the second transmission path: the image trimming circuit is used to remove the data packet distribution circuit allocated The padding data packet in the second exposure value data packet.

In this embodiment, the interface timing conversion device 32 includes an interface receiving module. When acquiring image data from the image sensor 31: the processing circuit acquires the first exposure through the input path of the same interface receiving module. Value data packet and the second exposure value data packet.

It should be noted that the interface timing conversion device 32 includes a processing circuit, a first interface receiving module, and a second interface receiving module. When extracting the first exposure value data packet and the second exposure value data packet in the image data : The processing circuit is used to determine the current working mode; if it is determined that the working mode is the second HDR mode, the processing circuit obtains the first exposure value data through the input path of the first interface receiving module And obtain the second exposure value data packet through the input path of the second interface receiving module.

It should be correspondingly explained that the first interface receiving module and the second interface receiving module may correspond to the first SLVS-EC interface receiving module and the second SLVS-EC interface receiving module described in the previous embodiment.

It should be noted that the interface timing conversion device 32 includes a processing circuit and an interface receiving module. When extracting the first exposure value data packet and the second exposure value data packet in the image data: the processing circuit is used for Determine the current working mode; if it is determined that the working mode is single-port mode, the processing circuit is used to obtain the image data through the input path of the same interface receiving module to extract the first exposure value data packet and the first Two exposure value data packets, wherein one of the first exposure value data packet and the second exposure value data packet is zero.

It should be noted that the interface timing conversion device 32 includes a processing circuit, a first interface receiving module, a second interface receiving module, and a pixel stitching circuit. The first exposure value data packet and the second exposure data packet in the image data are extracted. Value data packet: the processing circuit is used to determine the current working mode; if it is determined that the working mode is the dual-port mode, the processing circuit is used to obtain the input path of the first interface receiving module The odd-numbered column of serial data in the image data, and the even-numbered column of serial data in the image data is obtained through the input path of the second interface receiving module; the processing circuit is used for splicing all the data through the pixel splicing circuit The odd column serial data of the first interface receiving module and the even column serial data of the second interface receiving module are used to restore the first exposure value data packet and the second exposure value data packet extracted from the image data, wherein , One of the first exposure value data packet and the second exposure value data packet in the image data is zero.

It should be noted that the interface timing conversion device 32 includes a processing circuit, a first interface receiving module, and a second interface receiving module. When extracting the first exposure value data packet and the second exposure value data packet in the image data : The processing circuit is used to determine the current operating mode; if it is determined that the operating mode is the dual sensor mode, the processing circuit is used to obtain the first exposure value through the input path of the first interface receiving module Data packet, and obtain the second exposure value data packet through the input path of the second interface receiving module.

As mentioned above, the interface timing conversion device 32 includes a low-voltage signal SLVS-EC interface receiving module with an expandable embedded clock for acquiring image data from the image sensor 31.

Wherein, the interface timing conversion device 32 is configured to use a first interface timing conversion module to convert the timing of the first exposure value data packet into a VBUS interface timing, and use a second interface timing conversion module to convert the second exposure The time sequence of the value data packet is converted into the VBUS interface time sequence to be transmitted to the subsequent image signal processing device 33 through different VBUS interfaces.

Please refer to FIG. 17 next. This application also provides a storage medium that stores a computer program, and when the computer program is executed by a processing circuit, the processing circuit implements steps S301 to S303.

S301: Acquire image data.

S302: Extract a first exposure value data packet and a second exposure value data packet in the image data.

S303. Allocate the first exposure value data packet to a first transmission path, and allocate the second exposure value data packet to a second transmission path to compare the first exposure value data packet and the second transmission path. The exposure value data packet executes different timing conversion processing.

It should be noted that, when the processing circuit implements the extraction of the first exposure value data packet and the second exposure value data packet in the image data, it is used to implement:

Determine the current working mode;

If it is determined that the working mode is the first HDR mode, the image data is split to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the first exposure value data The packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.

Wherein, when the processing circuit implements the allocation of the second exposure value data packet to the second transmission path, it is also used to implement: remove the padding data packet in the second exposure value data packet.

Wherein, the padding data packet is located at a fixed frame position in the short exposure value data packet, or the specified frame position where the padding data packet is located is marked in the header information of the short exposure value data packet, so as to be in the first HDR In the mode, the padding data packet at the fixed frame position or the designated frame position is correspondingly removed according to the preset rule.

It is worth mentioning that the preset rules can be set in advance, or during the transmission of HDR images, when a short exposure value data packet is detected, the packet header information is automatically identified, and when it is identified When filling the mark of the data packet, the filling data packet is automatically removed.

Wherein, when the processing circuit implements the acquisition of image data, it is used to implement: acquiring the first exposure value data packet and the second exposure value data packet through the same input path.

It should be noted that the image data constructed by the long exposure value data packet and the short exposure value data packet are high dynamic range HDR image data.

Determine the current working mode;

If it is determined that the working mode is the second HDR mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.

Determine the current working mode;

If it is determined that the working mode is the single-port mode, the image data is obtained through the same input path to extract the first exposure value data packet and the second exposure value data packet, wherein the first exposure value data packet and the second exposure value data packet One of the packets is zero.

Determine the current working mode;

If it is determined that the working mode is the dual-port mode, obtain the odd-numbered serial data in the image data through the first input path, and obtain the even-numbered serial data in the image data through the second input path;

Splicing the odd column serial data and the even column serial data to restore the first exposure value data packet and the second exposure value data packet extracted from the image data, wherein the first exposure value in the image data One of the value data packet and the second exposure value data packet is zero.

Determine the current working mode;

If it is determined that the working mode is the dual sensor mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.

As mentioned above, when the processing circuit realizes the acquisition of image data, it is used to realize the acquisition of image data through the low-voltage signal SLVS-EC interface of the expandable embedded clock.

It is worth mentioning that, when the processing circuit implements different time sequence conversion processing on the first exposure value data packet and the second exposure value data packet, it is used to implement: adopting a first time sequence conversion module The time sequence of the first exposure value data packet is converted into the VBUS interface time sequence, and the second time sequence conversion module is used to convert the time sequence of the second exposure value data packet into the VBUS interface time sequence, so as to be transmitted to the VBUS interface through different VBUS interfaces. The subsequent image signal processing device.

Wherein, the storage medium may be a computer-readable storage medium, which may be an internal storage unit of the apparatus/device described in any of the foregoing embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk equipped on the device, a smart memory card (Smart Media Card, SMC), a Secure Digital (SD) card, and a flash card (Flash Card). )Wait.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

An image data processing method, characterized in that it comprises:

Obtain image data;

Extracting the first exposure value data packet and the second exposure value data packet in the image data;

The first exposure value data packet is allocated to a first transmission path, and the second exposure value data packet is allocated to a second transmission path, so as to compare the first exposure value data packet and the second exposure value The data packets perform different timing conversion processing.
The method according to claim 1, wherein said extracting a first exposure value data packet and a second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the first HDR mode, the image data is split to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the first exposure value data The packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.
The method according to claim 2, wherein the allocating the second exposure value data packet to a second transmission path further comprises:

Removing the padding data packet in the second exposure value data packet.
The method according to claim 2, wherein said acquiring image data comprises:

Obtain the first exposure value data packet and the second exposure value data packet through the same input path.
The method according to claim 4, wherein the image data constructed by the long exposure value data packet and the short exposure value data packet are high dynamic range HDR image data.
The method according to claim 1, wherein the extracting a first exposure value data packet and a second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the second HDR mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.
The method according to claim 6, wherein the first exposure value data packet is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value data packet The image data constructed by the packet and the short exposure value data packet are high dynamic range HDR image data.
The method according to claim 1, wherein said extracting a first exposure value data packet and a second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the single-port mode, the image data is obtained through the same input path to extract the first exposure value data packet and the second exposure value data packet, wherein the first exposure value data packet and the second exposure value data packet One of the packets is zero.
The method according to claim 1, wherein said extracting a first exposure value data packet and a second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the dual-port mode, obtain the odd-numbered serial data in the image data through the first input path, and obtain the even-numbered serial data in the image data through the second input path;

Splicing the odd column serial data and the even column serial data to restore the first exposure value data packet and the second exposure value data packet extracted from the image data, wherein the first exposure value in the image data One of the value data packet and the second exposure value data packet is zero.
The method according to claim 1, wherein the extracting a first exposure value data packet and a second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the dual sensor mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.
The method according to claim 10, wherein the first exposure value data packet and the second exposure value data packet are a short exposure value data packet, a long exposure value data packet, or a combination of both.
The method according to claim 1, 2, 6, 8, 9 or 10, wherein said acquiring image data comprises:

The image data is acquired through the low-voltage signal SLVS-EC interface with an expandable embedded clock.
The method according to claim 12, wherein the performing different time sequence conversion processing on the first exposure value data packet and the second exposure value data packet comprises:

The first interface timing conversion module is used to convert the timing of the first exposure value data packet into the VBUS interface timing, and the second interface timing conversion module is used to convert the timing of the second exposure value data packet into the VBUS interface timing to They are respectively transmitted to the subsequent image signal processing device through different VBUS interfaces.
An interface timing conversion device, characterized in that it comprises a storage circuit and a processing circuit;

The storage circuit is used to store a computer program:

The processing circuit is used to execute the computer program and, when the computer program is executed, implement the following steps:

Obtain image data;

Extracting the first exposure value data packet and the second exposure value data packet in the image data;

The first exposure value data packet is allocated to a first transmission path, and the second exposure value data packet is allocated to a second transmission path, so as to compare the first exposure value data packet and the second exposure value The data packets perform different timing conversion processing.
The device according to claim 14, wherein the processing circuit is used to implement the following when extracting the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the first HDR mode, the image data is split to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the first exposure value data The packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.
The device according to claim 15, wherein the processing circuit is further configured to implement the following when implementing the allocation of the second exposure value data packet to the second transmission path:

Removing the padding data packet in the second exposure value data packet.
The device according to claim 15, wherein the processing circuit is used to realize:

Obtain the first exposure value data packet and the second exposure value data packet through the same input path.
The device according to claim 17, wherein the image data constructed by the long exposure value data packet and the short exposure value data packet is high dynamic range HDR image data.
The device according to claim 14, wherein the processing circuit is used to implement the following when extracting the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the second HDR mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.
The device according to claim 19, wherein the first exposure value data packet is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value data packet The image data constructed by the packet and the short exposure value data packet are high dynamic range HDR image data.
The device according to claim 14, wherein the processing circuit is used to implement the following when extracting the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the single-port mode, the image data is obtained through the same input path to extract the first exposure value data packet and the second exposure value data packet, wherein the first exposure value data packet and the second exposure value data packet One of the packets is zero.
The device according to claim 14, wherein the processing circuit is used to implement the following when extracting the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the dual-port mode, obtain the odd-numbered serial data in the image data through the first input path, and obtain the even-numbered serial data in the image data through the second input path;

Splicing the odd column serial data and the even column serial data to restore the first exposure value data packet and the second exposure value data packet extracted from the image data, wherein the first exposure value in the image data One of the value data packet and the second exposure value data packet is zero.
The device according to claim 14, wherein the processing circuit is used to implement the following when extracting the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the dual sensor mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.
The device according to claim 23, wherein the first exposure value data packet and the second exposure value data packet are short exposure value data packets, long exposure value data packets, or a combination of both.
The device according to claim 14, 15, 19, 21, 22 or 23, wherein the processing circuit is used to implement the following when implementing the image data acquisition:

The image data is acquired through the low-voltage signal SLVS-EC interface with an expandable embedded clock.
The device according to claim 25, wherein the processing circuit is configured to implement different time sequence conversion processing on the first exposure value data packet and the second exposure value data packet when implementing :

The first timing conversion module is used to convert the timing of the first exposure value data packet into the VBUS interface timing, and the second timing conversion module is used to convert the timing of the second exposure value data packet into the VBUS interface timing to pass through Different VBUS interfaces are transmitted to the subsequent image signal processing device.
An image data transmission method, characterized in that it comprises:

The image sensor sends image data;

The interface timing conversion device obtains image data from the image sensor;

The interface timing conversion device extracts the first exposure value data packet and the second exposure value data packet in the image data;

The interface timing conversion device allocates the first exposure value data packet to a first transmission path, and allocates the second exposure value data packet to a second transmission path;

The interface timing conversion device performs different timing conversion processing on the first exposure value data packet and the second exposure value data packet;

The interface timing conversion device is transmitted to the subsequent image signal processing device through different interfaces.
The method according to claim 27, wherein the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the first HDR mode, the image data is split by a data packet distribution circuit to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the The first exposure value data packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.
The method according to claim 28, wherein the interface timing conversion device assigns the second exposure value data packet to a second transmission path, further comprising:

The padding data packet in the second exposure value data packet allocated by the data packet distribution circuit is removed by the image trimming circuit.
The method according to claim 28, wherein the interface timing conversion device acquiring image data from the image sensor comprises:

The first exposure value data packet and the second exposure value data packet are acquired through the input path of the same interface receiving module.
The method according to claim 30, wherein the image data constructed by the long exposure value data packet and the short exposure value data packet is high dynamic range HDR image data.
The method according to claim 27, wherein the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the second HDR mode, the first exposure value data packet is acquired through the input path of the first interface receiving module, and the second exposure value data packet is acquired through the input path of the second interface receiving module.
The method according to claim 32, wherein the first exposure value data packet is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value data packet The image data constructed by the packet and the short exposure value data packet are high dynamic range HDR image data.
The method according to claim 27, wherein the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the single-port mode, the image data is obtained through the input path of the receiving module of the same interface to extract the first exposure value data packet and the second exposure value data packet, wherein the first exposure value data packet and the second exposure value data packet One of the two exposure value data packets is zero.
The method according to claim 27, wherein the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the dual-port mode, obtain the odd-numbered serial data in the image data through the input path of the first interface receiving module, and obtain the even-numbered serial data in the image data through the input path of the second interface receiving module Column serial data;

The odd-numbered column serial data of the first interface receiving module and the even-numbered column serial data of the second interface receiving module are spliced by a pixel splicing circuit to restore the first exposure value data packet and the second data packet extracted from the image data. The exposure value data packet, wherein one of the first exposure value data packet and the second exposure value data packet in the image data is zero.
The method according to claim 27, wherein the interface timing conversion device extracting the first exposure value data packet and the second exposure value data packet in the image data comprises:

Determine the current working mode;

If it is determined that the working mode is the dual sensor mode, the first exposure value data packet is acquired through the input path of the first interface receiving module, and the second exposure value data packet is acquired through the input path of the second interface receiving module.
The method according to claim 36, wherein the first exposure value data packet and the second exposure value data packet are a short exposure value data packet, a long exposure value data packet or a combination of both.
The method according to claim 27, 28, 32, 34, 35, or 36, wherein the interface timing conversion device acquiring image data from the image sensor comprises:

The image data is obtained through the low-voltage signal SLVS-EC interface receiving module with expandable embedded clock.
The method according to claim 38, wherein the interface timing conversion device performs different timing conversion processing on the first exposure value data packet and the second exposure value data packet, comprising:

The first interface timing conversion module is used to convert the timing of the first exposure value data packet into the VBUS interface timing, and the second interface timing conversion module is used to convert the timing of the second exposure value data packet into the VBUS interface timing to They are respectively transmitted to the subsequent image signal processing device through different VBUS interfaces.
An image data transmission system, characterized in that it comprises an image sensor, an interface timing conversion device and an image signal processing device:

The image sensor is used to send image data;

The interface timing conversion device is used to obtain image data from the image sensor;

The interface timing conversion device is used to extract a first exposure value data packet and a second exposure value data packet in the image data;

The interface timing conversion device is configured to allocate the first exposure value data packet to a first transmission path, and allocate the second exposure value data packet to a second transmission path;

The interface timing conversion device is configured to perform different timing conversion processing on the first exposure value data packet and the second exposure value data packet;

The interface timing conversion device is used to transmit to the image signal processing device through different interfaces.
The system according to claim 40, wherein the interface timing conversion device comprises a processing circuit and a data distribution circuit, and when extracting the first exposure value data packet and the second exposure value data packet in the image data:

The processing circuit is used to determine the current working mode;

If it is determined that the working mode is the first HDR mode, the data packet distribution circuit is configured to split the image data to extract the first exposure value data packet and the second exposure value data packet respectively, Wherein, the first exposure value data packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.
The system according to claim 41, wherein the interface timing conversion device comprises an image trimming circuit, and when the second exposure value data packet is allocated to the second transmission path:

The image trimming circuit is used to remove the padding data packet in the second exposure value data packet distributed by the data packet distribution circuit.
The system according to claim 41, wherein the interface timing conversion device comprises an interface receiving module, and when acquiring image data from the image sensor:

The processing circuit obtains the first exposure value data packet and the second exposure value data packet through the input path of the same interface receiving module.
The system according to claim 43, wherein the image data constructed by the long exposure value data packet and the short exposure value data packet are high dynamic range HDR image data.
The system according to claim 40, wherein the interface timing conversion device comprises a processing circuit, a first interface receiving module and a second interface receiving module, in extracting the first exposure value data packet and the second interface receiving module in the image data. When the second exposure value data packet:

The processing circuit is used to determine the current working mode;

If it is determined that the working mode is the second HDR mode, the processing circuit obtains the first exposure value data packet through the input path of the first interface receiving module, and obtains all data packets through the input path of the second interface receiving module. The second exposure value data packet.
The system according to claim 45, wherein the first exposure value data packet is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value data packet The image data constructed by the packet and the short exposure value data packet are high dynamic range HDR image data.
The system according to claim 40, wherein the interface timing conversion device comprises a processing circuit and an interface receiving module, and when extracting the first exposure value data packet and the second exposure value data packet in the image data:

The processing circuit is used to determine the current working mode;

If it is determined that the working mode is the single-port mode, the processing circuit is configured to obtain the image data through the input path of the same interface receiving module to extract the first exposure value data packet and the second exposure value data packet, wherein, One of the first exposure value data packet and the second exposure value data packet is zero.
The system according to claim 40, wherein the interface timing conversion device includes a processing circuit, a first interface receiving module, a second interface receiving module, and a pixel stitching circuit, and the first exposure in extracting the image data Value data packet and second exposure value data packet:

The processing circuit is used to determine the current working mode;

If it is determined that the working mode is the dual-port mode, the processing circuit is configured to obtain the odd-numbered serial data in the image data through the input path of the first interface receiving module, and receive the serial data of the module through the second interface The input path acquires even-numbered serial data in the image data;

The processing circuit is used to splice the odd-numbered serial data of the first interface receiving module and the even-numbered serial data of the second interface receiving module through the pixel splicing circuit to restore the extracted image data The first exposure value data packet and the second exposure value data packet, wherein one of the first exposure value data packet and the second exposure value data packet in the image data is zero.
The system according to claim 40, wherein the interface timing conversion device comprises a processing circuit, a first interface receiving module and a second interface receiving module, in extracting the first exposure value data packet and the second interface receiving module in the image data. When the second exposure value data packet:

The processing circuit is used to determine the current working mode;

If it is determined that the working mode is the dual sensor mode, the processing circuit is configured to obtain the first exposure value data packet through the input path of the first interface receiving module, and obtain the first exposure value data packet through the input path of the second interface receiving module The second exposure value data packet.
The system according to claim 49, wherein the first exposure value data packet and the second exposure value data packet are a short exposure value data packet, a long exposure value data packet, or a combination of both.
The system according to claim 40, 41, 45, 47, 48, or 49, wherein the interface timing conversion device comprises a low-voltage signal SLVS-EC interface receiving module with an expandable embedded clock for receiving from The image sensor acquires image data.
The system according to claim 51, wherein the interface timing conversion device is configured to use a first interface timing conversion module to convert the timing of the first exposure value data packet into a VBUS interface timing, and use a second The interface timing conversion module converts the timing of the second exposure value data packet into a VBUS interface timing, so as to be transmitted to the subsequent image signal processing device through different VBUS interfaces.
A storage medium, characterized in that the storage medium stores a computer program, and when the computer program is executed by a processing circuit, the processing circuit realizes:

Obtain image data;

Extracting the first exposure value data packet and the second exposure value data packet in the image data;

The first exposure value data packet is allocated to a first transmission path, and the second exposure value data packet is allocated to a second transmission path, so as to compare the first exposure value data packet and the second exposure value The data packets perform different timing conversion processing.
The storage medium according to claim 53, wherein the processing circuit is used to implement the extraction of the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the first HDR mode, the image data is split to extract the first exposure value data packet and the second exposure value data packet respectively, wherein the first exposure value data The packet is a long exposure value data packet, and the second exposure value data packet is a short exposure value data packet.
The storage medium according to claim 54, characterized in that, when the processing circuit realizes the distribution of the second exposure value data packet to the second transmission path, it is further configured to realize:

Removing the padding data packet in the second exposure value data packet.
The storage medium according to claim 54, characterized in that, when the processing circuit implements the acquisition of image data, it is used to implement:

Obtain the first exposure value data packet and the second exposure value data packet through the same input path.
The storage medium according to claim 56, wherein the image data constructed by the long exposure value data packet and the short exposure value data packet is high dynamic range HDR image data.
The storage medium according to claim 53, wherein the processing circuit is used to implement the extraction of the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the second HDR mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.
The storage medium according to claim 58, wherein the first exposure value data packet is a long exposure value data packet, the second exposure value data packet is a short exposure value data packet, and the long exposure value The image data constructed by the data packet and the short exposure value data packet is high dynamic range HDR image data.
The storage medium according to claim 53, wherein the processing circuit is used to implement the extraction of the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the single-port mode, the image data is obtained through the same input path to extract the first exposure value data packet and the second exposure value data packet, wherein the first exposure value data packet and the second exposure value data packet One of the packets is zero.
The storage medium according to claim 53, wherein the processing circuit is used to implement the extraction of the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the dual-port mode, obtain the odd-numbered serial data in the image data through the first input path, and obtain the even-numbered serial data in the image data through the second input path;

Splicing the odd column serial data and the even column serial data to restore the first exposure value data packet and the second exposure value data packet extracted from the image data, wherein the first exposure value in the image data One of the value data packet and the second exposure value data packet is zero.
The storage medium according to claim 53, wherein the processing circuit is used to implement the extraction of the first exposure value data packet and the second exposure value data packet in the image data:

Determine the current working mode;

If it is determined that the working mode is the dual sensor mode, the first exposure value data packet is acquired through the first input path, and the second exposure value data packet is acquired through the second input path.
The storage medium according to claim 62, wherein the first exposure value data packet and the second exposure value data packet are a short exposure value data packet, a long exposure value data packet, or a combination of both.
The storage medium according to claim 53, 54, 58, 60, 61, or 62, wherein the processing circuit is used to realize:

The image data is acquired through the low-voltage signal SLVS-EC interface with an expandable embedded clock.
The storage medium according to claim 64, wherein the processing circuit is configured to perform different time sequence conversion processing on the first exposure value data packet and the second exposure value data packet when implementing achieve:

The first timing conversion module is used to convert the timing of the first exposure value data packet into the VBUS interface timing, and the second timing conversion module is used to convert the timing of the second exposure value data packet into the VBUS interface timing to pass through Different VBUS interfaces are transmitted to the subsequent image signal processing device.