TW202110144A - Data transmission system and data transmission method based on Mobile Industry Processor Interface capable of simultaneously transmitting image and multiple sensor data through camera serial interface of Mobile Industry Processor Interface - Google Patents

Abstract

The invention provides a data transmission system and data transmission method based on Mobile Industry Processor Interface. A pre-processing unit receives the image data from an image capture device through a Camera Serial Interface (CSI) of Mobile Industry Processor Interface (MIPI) and also receives the sensor data from one or multiple sensors through a serial communication interface, such as I2C, I2S, and SPI. Then, the pre-processing unit may transmit the image data after pre-processing through a first virtual channel, such as virtual channel 0, included in the camera serial interface to a central processing unit, and may also simultaneously transmit the sensor data after pre-processing through a second virtual channel, such as virtual channel 1, included in the camera serial interface to the central processing unit. The initialization of the image capture device and each of the sensors may be all performed by a Camera Control Interface (CCI) of Mobile Industry Processor Interface.

Description

Data transmission system and data transmission method based on mobile industry processor interface

The present invention relates to the technical field of data transmission, in particular to a data transmission system and data transmission method capable of simultaneously transmitting images and multiple sensor information through a camera serial interface of a mobile industrial processor interface.

With the development of technology, the functions of mobile devices have become more diversified and mature. Accordingly, components such as camera modules, UHD displays, GPS positioning chips, and sensors from different vendors must be integrated into the mobile device. As shown in "Figure 1", conventional sensors (such as cameras, audio devices, displays, Netcom chips, etc.) usually use different transmission interfaces (or interfaces) to transmit data to the CPU (central processing unit). For example, cameras can transmit image information to the CPU through the MIPI-CSI interface, audio devices (such as microphones and audio encoders) can transmit audio information to the CPU through the I2S interface, and other sensors can use, for example, SPI or I2C The interface transfers the sensing data to the CPU, and then the CPU performs subsequent processing on the data (for example, image processing is continued on the image information).

However, with the increasing number of sensors connected to CPUs today and the increasing complexity of various sensors, it is really necessary to use a pre-processing chip connected to different sensors to collect data. The received sensing data is preprocessed and then transmitted to the CPU through different transmission interfaces. However, in this way, the CPU will have to support the transmission interfaces of different sensors, which will increase the system cost and power consumption. No inventor has proposed a solution that can simplify the transmission channel between the pre-processing chip and the CPU, or propose a unified sensor transmission standard interface.

Although some inventors have proposed that the image data captured by each camera of the image capture device (which is a kind of sensor) is transmitted to the main controller through their respective MIPI interfaces, the system will occupy system resources in terms of control. The problem of repeatedly sending related or the same settings to different cameras, causing the main controller to burden the main controller. If the main controller needs to receive the image data from different cameras at the same time, it must open different MIPI interfaces to receive one-to-one reception. That is, as disclosed in the Republic of China Invention Patent No. TWI637632B "Image Capture Device and Its Control Method", the previous case of the patent mainly proposed that the main controller only needs to set the camera parameters for a single camera, and other sub cameras can be based on the main camera. The camera parameters and their respective corresponding parameter tables are used to set the camera parameters to reduce the burden of the main controller. Therefore, the previous patent does not address the transmission of each sensor as for other sensors that are not image capture devices. Propose solutions to problems with different interfaces.

In summary, for sensors that use different transmission interfaces, how to propose a data transmission system and transmission that can simplify the transmission of different sensing data between the pre-processing chip and the CPU in order to reduce system cost and power consumption The method is a problem to be solved.

To achieve the above objective, the present invention provides a data transmission system and a data transmission method based on a mobile industrial processor interface for transmitting an image information of an image capturing device, and also for transmitting one of one or more sensors Sensor information. The data transmission system includes a pre-processing unit and a central processing unit coupled to the pre-processing unit; the pre-processing unit is used to pre-process and temporarily store image information and sensor information; The capture device and one or more sensors are respectively coupled to the preprocessing unit, and the preprocessing unit is configured to receive from the image capture device via a camera serial interface (CSI) of a mobile industry processor interface (MIPI) The image information is also configured to receive sensor information from the sensor through a serial communication interface; the preprocessing unit is further configured to place the preprocessed image information on a first part of the camera serial interface. The virtual channel (Virtual Channel) is transmitted to the central processing unit, and is also configured to transmit the preprocessed sensor information to a second virtual channel included in the camera serial interface to the central processing unit.

In addition, in the data transmission system based on the mobile industrial processor interface proposed by the present invention, the central processing unit can be configured to send a configuration parameter (such as a camera control interface (CCI)) included in a camera serial interface (CSI) (Initialization parameters) to the preprocessing unit, so that the preprocessing unit writes the configuration parameters to a register of the image capture device and the sensor, for the image capture device and the sensor to load and act according to the configuration parameters.

It can be seen that, after the present invention is implemented, by simplifying the transmission channel between the pre-processing chip and the central processing unit, images and other sensor information can be transmitted on different MIPI-CSI virtual channels, thereby, the central processing The unit does not need to process a large amount of sensor information from different transmission interfaces, and can at least achieve the beneficial effects of reducing system cost and power consumption.

In order to enable your reviewer to have a clear understanding of the purpose, technical features and effects of the present invention after implementation, the following descriptions and illustrations are used for explanation, please refer to them.

Please refer to "Figure 2", which is a schematic diagram of the system architecture and information flow of the present invention. The present invention discloses a data transmission system 1 based on a mobile industrial processor interface, which can transmit an image information of an image capture device 10, and also It can transmit a sensor information of one or more sensors (20, 20'...), including a pre-processing unit 30 and a central processing unit 40, among which: (1) The image capture device 10 can be operated through a mobile A Camera Serial Interface (CSI) of the Mobile Industry Processor Interface (MIPI) transmits image information (in the form of MIPI signals) to the preprocessing unit 30, where the camera serial interface can be Compliant with the transmission protocol of CSI-1, CSI-2 or CSI-3, but not limited to this; in addition, the image capturing device 10 may at least include a lens and a photosensitive element, wherein the photosensitive element is used to generate an image, It can be a charge-coupled device (CCD), a complementary metal oxide semiconductor (CMOS) device or other photosensitive device; (2) The sensor 20 can be performed through a serial communication interface (Serial Communication) and the preprocessing unit 30 Sensor information transmission, in which the serial communication interface can be in line with an internal integrated circuit (Inter-Integrated Circuit Bus, I2C), a serial peripheral interface (Serial Peripheral Interface Bus, SPI), and an internal integrated circuit sound Serial interface (Integrated Interchip Sound, I2S), a universal asynchronous receiver transmitter (UART), a microwire (Microwire) one of the interfaces or a combination of communication protocols, but not limited to this; (3) The pre-processing unit 30 can pre-process and temporarily store the received image information and sensor information, and the pre-processing unit 30 can be configured to retrieve the image from the image capture device 10 through a camera serial interface of a mobile industrial processor interface. Receiving image information can also be configured to receive sensor information from one or more sensors (20, 20'...) through a serial communication interface. Furthermore, the preprocessing unit 30 can be further configured to The preprocessed image information is transmitted to the central processing unit 40 through a first virtual channel VC_0 included in the camera serial interface; the preprocessing unit 30 can also be configured to transfer the preprocessed sensor information to the camera string A second virtual channel VC_1 included in the serial interface is transmitted to the central processing unit 40; (4) The central processing unit 40 is coupled to the preprocessing unit 30, and the central processing unit 40 can be configured as a camera included in the camera serial interface The control interface (Camera Serial Interface, CCI) sends a configuration parameter to the preprocessing unit 30 for the preprocessing unit 30 to write the configuration parameters respectively. Set the parameters to the relevant addresses of the register 101 of the image capture device 10 and the register 201 of the sensor 20, so that the image capture device 10 and the sensor 20 can load the configuration parameters according to the configuration parameters For example, the configuration parameters written into the register 101 of the image capturing device 10 can be, for example, an initialization setting, an image output specification setting, a focus setting, an exposure setting, a white balance setting, etc. But it is not limited to this; (5) As mentioned above, the central processing unit 40 may have functions such as logic operations, temporarily storing operation results, and saving the position of execution instructions, and the central processing unit 40 may be, for example, a central processing unit (CPU), A microprocessor (MPU), a microcontroller (MCU), an application processor (AP), an embedded processor (Embedded Processor), a digital signal processor (DSP), a special application integrated circuit ( ASIC) or a programmable logic device (PLD), but not limited to this.

Please continue to refer to the system implementation flow chart of "Figure 3" and refer to "Figure 2". The data transmission system 1 based on the mobile industrial processor interface of the present invention can execute a data transmission method S, which includes the following steps: (1) Write configuration parameters (step S10): A central processing unit 40 uses a camera control interface included in a camera serial interface of a mobile industrial processor interface to send a configuration parameter to a preprocessing unit 30, and then preprocess The processing unit 30 can access and control a register (101, 201) of the image capture device 10 and the sensor 20 to write configuration parameters to the registers (101, 201) respectively, so that the image capture device 10 And the sensor 20 can be actuated according to the configuration parameters; (2) Receive image information through the MIPI interface (step S20): the preprocessing unit 30 receives the image information from the image capture device 10 through the camera serial interface of the mobile industrial processor interface; (3) Receive sensor information through a serial communication interface (step S30): The preprocessing unit 30 receives sensor information from one or more sensors (20, 20'...) through a serial communication interface, and The serial communication interface of this embodiment takes the internal integrated circuit, serial peripheral interface, and an internal integrated circuit sound serial interface as examples. However, each sensor (20, 20'...) can also be connected through other serial interfaces. The communication interface and the pre-processing unit 30 perform two-way data transmission, and are not limited to the aforementioned serial communication interface; (4) Signal pre-processing (step S40): the pre-processing unit 30 separately performs processing on the received image information and the sensor The information is preprocessed and temporarily stored. The aforementioned preprocessing may include aggregating, filtering or otherwise enhancing the raw image information and sensor information; (5) Simultaneously transmit images and sensor information on different virtual channels of the MIPI-CSI interface (step S50): the preprocessing unit 30 transmits the preprocessed image information to a first virtual channel VC_0 included in the camera serial interface to the center The processing unit 40 simultaneously transmits the preprocessed sensor information to the central processing unit 40 via a second virtual channel VC_1 included in the camera serial interface. (6) In conclusion, if the aforementioned configuration parameters are not initialization parameters, step S10 (write configuration parameters) in this embodiment can also be performed after step S50.

Please refer to "Figure 4", which is a schematic diagram of the sensor packet format of the present invention. Please also refer to "Figure 2" to "Figure 3". The information transmission system disclosed in this embodiment implements the data transmission method When S, when step S50 (synchronous transmission of image and sensor information) is executed, one or more sensor information can be respectively encapsulated as a sensor package P_S, so that each sensor information can be packaged in the form of ( Packet) is transmitted to the central processing unit 40 on the second virtual channel VC_1, and the sensor packet P_S of this embodiment may at least include a header (Header), a time stamp information (Time Stamp), and a packet length (length) , A load data (PayLoad) and an error correction code (ECC), where the header can be used to distinguish information from different sensors (20, 20'...), and the timestamp information can be used to distinguish information from the same sensor The sequence of the sensor information of 20, the packet length is used to indicate the data length of the carried data, and the error correction code is used to verify and correct the sensor information to ensure the integrity and reliability of the packet data.

Please refer to "Figure 5", which is a schematic diagram of the MIPI-CSI packet format of another embodiment (1) of the present invention. Please also refer to "Figure 2" ~ "Figure 3". As shown in Figure 4, the main difference is that one or more sensor packets P_S in the form of packets are encapsulated in step S50. In this embodiment, a MIPI-CSI packet P_CSI can be further formed for the second virtual channel. VC_1 is transferred to the central processing unit 40, and the MIPI-CSI packet P_CSI can at least further include a MIPI packet header and a MIPI packet footer, so that the sensor packet can be simulated as image information to integrate each sensor The sensor information is better embedded in the MIPI-CSI transmission protocol for transmitting images; please continue to refer to "Figure 6", which is a schematic diagram of the MIPI-CSI packet format of another embodiment (2) of the present invention. This implementation The example is related to the technology disclosed in "Figure 5", and the aforementioned MIPI-CSI packet P_CSI may further include one or a combination of a filler and a dummy packet (Dummy Packet), if the MIPI-CSI packet P_CSI Contains padding data, the packet format is as shown in "Figure 5". If the MIPI-CSI packet P_CSI contains a dummy packet, the packet format is as shown in "Figure 6".

Please refer to "Figure 7", which is another embodiment of the present invention (3) a schematic diagram of interleaving transmission in units of MIPI-CSI packets, and please refer to "Figure 2" ~ "Figure 4" for this implementation The example is related to that disclosed in "Figure 5". When step S50 of this embodiment is executed, the multiple sensor packets P_S transmitted on the second virtual channel VC_1 can form the data format of the MIPI-CSI packet P_CSI, so The image information generated by the image capture device 10 and the sensor information generated by one or more sensors (20, 20'...) are all used in MIPI-CSI packets as a unit, and are used in different virtual channels (VC_0). , VC_1) is interleaved and sent to the central processing unit 40. In more detail, MIPI-CSI packets transmitting image information on the first virtual channel VC_0 and MIPI-CSI packets transmitting sensor information on the second virtual channel VC_1 can be A data frame defined between the frame end code (FE_0, FE_1) and the frame start code (FS_0, FS_1) is interspersed and sent, and a frame blanking period (frame blanking period) can be defined outside the data frame. ) And a line blanking interval (line blanking). When transmitting packet data, when the MIPI-CSI packet becomes a frame blank interval or a line blank interval, the clock signal will be gated, or the clock signal will be stopped to be transmitted. This realizes the data interleaving supported by the MIPI-CSI communication protocol.

Please refer to "Figure 8", which is a schematic diagram of another embodiment (4) of the interleaving transmission in the unit of frame of the present invention. Please also refer to "Figure 2" ~ "Figure 5" in conjunction with this embodiment. The main difference as shown in Fig. 7 is that when step S50 is executed, the multiple sensor packets P_S transmitted on the second virtual channel VC_1 can form the data format of MIPI-CSI packet P_CSI, so the image is captured The image information generated by the device 10 and the sensor information generated by one or more sensors (20, 20'...) are interleaved and sent to the central processing unit 40 in units of frames. In more detail , Multiple MIPI-CSI packets that convey image information on the first virtual channel VC_0 can be sent in the frame (Frame_VC0) defined between the frame end code (FE_0) and the frame start code (FS_0). Similarly, in the second The MIPI-CSI packet that the virtual channel VC_1 transmits sensor information can be sent in the frame (Frame_VC1) defined between the frame end code (FE_1) and the frame start code (FS_1) to realize the MIPI-CSI communication protocol. Supported Data Interleaving, where the frame sending rate of the first virtual channel and the second virtual channel (VC_0, VC_1) need not be the same, and the number of MIPI-CSI packets per frame of the first virtual channel VC_0 and MIPI- The CSI packet length can be different from the second virtual channel VC_1, but within each virtual channel (VC_0 or VC_1), the number of MIPI-CSI packets per frame and the length of each MIPI-CSI packet must be the same.

Please refer to "Figure 9", which is a schematic diagram of the frame information format of the second virtual channel temporarily stored in the frame buffer of another embodiment (4) of the present invention. Please also refer to "Figure 2" in conjunction with this embodiment. As shown in "Figure 8", as shown in the figure, the MIPI-CSI receiving end (not shown in the figure) of the central processing unit 40 of this embodiment receives image information and sensor information from different virtual channels (VC_0, VC_1) After that, they are respectively stored in a memory (not shown in the figure) of the central processing unit 40, and the memory of this embodiment can be further divided into two frame buffers (Frame Buffer, also called frame buffer), It can store the image information from the first virtual channel VC_0 and the multiple sensor information from the second virtual channel VC_1 respectively. Among them, the frame buffer B_VC1 is used to store the multiple sensor information of the second virtual channel VC_1. , It can temporarily store each sensor packet in the frame information format, and each sensor packet of this embodiment (sensor 0 data packet, sensor 1 data packet...) can be constructed separately MIPI-CSI packets, so each MIPI-CSI packet of this embodiment may further include a fill data (Filler) or a dummy packet (Dummy Packet), as shown in "Figure 9", but it is not limited to this .

Please refer to "Figure 10", which is another embodiment of the present invention (5) of the MIPI-CSI receiver driver structure, and please refer to "Figure 2", this embodiment and "Figure 9" Related to the technology, as shown in the figure, when the MIPI-CSI Rx Driver of the central processing unit 40 in this embodiment stores multiple image information of the first virtual channel VC_0 into the aforementioned one frame buffer ( After Virtual Channel 0 Frame Buffer), a video driver (Video Driver) in the upper layer of a driver structure D_S can be used to directly provide a video driver API interface (Video API) to the corresponding application (Application); relatively, When the MIPI-CSI receiver driver of the central processing unit 40 of this embodiment transfers multiple MIPI-CSI packets of the second virtual channel VC_1 (which mainly include multiple sensor information, multiple padding data, or dummy packets) After storing the frame information format in another frame buffer (Virtual Channel 1 Frame Buffer), the upper multi-sensor driver (Multi Sensor Driver) is required to extract the information of different sensors (20, 20'...) Each sensor information is sent to the corresponding sensor driver (such as sensor 0 Driver, sensor 1 Driver...Sensor N Driver) for processing. Then, each sensor driver can pass through each sensor driver. The sensor driver APIs (such as sensor 0 API, sensor 1 API,...sensor N API) transmit the information of each sensor to the corresponding application for processing.

In summary, by simplifying the transmission channel between the pre-processing chip and the central processing unit, the present invention can transmit images and other sensor information on different MIPI-CSI virtual channels, which is different from the conventional central processing unit that must process data from The method of transmitting sensor information with different interfaces can at least achieve the beneficial effects of reducing system cost and power consumption.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention; anyone who is familiar with this technique can make equal changes and modifications without departing from the spirit and scope of the present invention. It should be covered in the scope of the patent of the present invention.

To sum up, the present invention has patent requirements such as "industrial applicability", "novelty" and "advancedness"; the applicant filed an application for a patent for invention with the Bureau in accordance with the provisions of the Patent Law.

1: Data transmission system based on mobile industry processor interface 10: Image capture device 101: scratchpad 20: Sensor 201: Register 30: preprocessing unit 40: Central Processing Unit VC_0: The first virtual channel VC_1: second virtual channel P_S: Sensor packet P_CSI: MIPI-CSI packet B_VC1: frame buffer FS_0: Frame start code FS_1: Frame start code FE_0: Frame end code FE_1: Frame end code D_S: driver structure S: Data transmission method S10: Write configuration parameters S20: Receive image information through MIPI interface S30: Receive sensor information through serial communication interface S40: Signal preprocessing S50: Synchronous transmission of images and sensor information on different virtual channels of the MIPI-CSI interface

Figure 1 is a schematic diagram of the system architecture and information flow of conventional sensor data transmission. Figure 2 is a schematic diagram of the system architecture and information flow of the present invention. Figure 3 is a flow chart of the system implementation of the present invention. Figure 4 is a schematic diagram of the sensor packet format of the present invention. Figure 5 is a schematic diagram of the MIPI-CSI packet format of another embodiment (1) of the present invention. Figure 6 is a schematic diagram of the MIPI-CSI packet format of another embodiment (2) of the present invention. Figure 7 is a schematic diagram of another embodiment (3) of the interleaving transmission using MIPI-CSI packets as a unit of the present invention. Figure 8 is a schematic diagram of another embodiment (4) of the interleaving transmission in frame units of the present invention. Figure 9 is a schematic diagram of the frame information format of the second virtual channel temporarily stored in the frame buffer of another embodiment (5) of the present invention. Figure 10 shows the driver structure of the MIPI-CSI receiving end of another embodiment (5) of the present invention.

1: Data transmission system based on mobile industry processor interface

10: Image capture device

101: scratchpad

20: Sensor

201: Register

20’: Sensor

30: preprocessing unit

40: Central Processing Unit

VC_0: The first virtual channel

VC_1: second virtual channel

Claims (12)

A data transmission system based on a mobile industrial processor interface is used to transmit an image information of an image capture device and also a sensor information of one or more sensors, including: A preprocessing unit for preprocessing and temporarily storing the image information and the sensor information; A central processing unit coupled to the preprocessing unit; The image capturing device is coupled to the preprocessing unit, and the preprocessing unit is used for receiving the image information from the image capturing device through a camera serial interface of a mobile industrial processor interface; The sensor is coupled to the preprocessing unit, and the preprocessing unit is configured to receive the sensor information from the sensor through a serial communication interface; The preprocessing unit is configured to transmit the preprocessed image information through a first virtual channel included in the camera serial interface to the central processing unit, and the preprocessing unit is also configured to transmit the preprocessed image information to the central processing unit. The sensor information of is transmitted to the central processing unit via a second virtual channel included in the camera serial interface; and The central processing unit is used for a camera control interface included in the camera serial interface to send a configuration parameter to the preprocessing unit, so that the preprocessing unit writes the configuration parameter to the image capture device and the sensor respectively The image capturing device and the sensor can be actuated according to the configuration parameter. For example, the data transmission system based on the mobile industry processor interface in the first patent application, wherein the serial communication interface complies with an internal integrated circuit, a serial peripheral interface, an internal integrated circuit audio serial interface, and a general purpose A communication protocol for one or a combination of an asynchronous receiver transmitter and a micro wire. For example, the data transmission system based on the mobile industry processor interface of the first patent application, in which the one or more sensor information is encapsulated as a sensor packet, which is transmitted on the second virtual channel in the form of a packet To the central processing unit, and the one or more sensor packets include a header, a timestamp information, a packet length, a carrying data, and an error correction code. For example, the data transmission system based on the mobile industry processor interface of the third item of the patent application, wherein the one or more sensor packets constitute a MIPI-CSI packet, and the MIPI-CSI packet includes at least one MIPI packet label Header and footer of a MIPI packet. For example, the data transmission system based on the mobile industry processor interface of the fourth patent application, wherein the MIPI-CSI packet further includes one of a padding data or a dummy packet or a combination thereof. For example, the data transmission system based on the mobile industrial processor interface of the 4th or 5th patent application, wherein the image information transmitted by the first virtual channel and the sensor information transmitted by the second virtual channel , Is sent to the central processing unit in the unit of the MIPI-CSI packet or interspersed in the unit of the frame. A data transmission method based on a mobile industrial processor interface is used to transmit an image information of an image capture device and also a sensor information of one or more sensors, including: A step of writing configuration parameters: a central processing unit uses a camera control interface included in a camera serial interface of a mobile industrial processor interface to send a configuration parameter to a preprocessing unit, and the preprocessing unit writes the configuration parameter A register to the image capture device and the sensor, so that the image capture device and the sensor can act according to the configuration parameter; A step of receiving image information through the MIPI interface: the preprocessing unit receives the image information from the image capturing device through the camera serial interface; A step of receiving sensor information through a serial communication interface: the preprocessing unit receives the sensor information from the sensor through a serial communication interface; A signal preprocessing step: the preprocessing unit preprocesses the image information and the sensor information and temporarily stores them; and A step of synchronously transmitting images and sensor information on different virtual channels of the MIPI-CSI interface: the preprocessing unit transmits the preprocessed image information to a first virtual channel included in the camera serial interface to the center The processing unit simultaneously transmits the preprocessed sensor information to the central processing unit via a second virtual channel included in the camera serial interface. For example, the data transmission method based on the mobile industry processor interface of the 7th patent application, in which, when the step of receiving sensor information through the serial communication interface is executed, the serial communication interface conforms to an internal integrated circuit and a serial communication interface. A communication protocol of one or a combination of a peripheral interface, an internal integrated circuit sound serial interface, a universal asynchronous receiver transmitter, and a microwire. For example, the data transmission method based on the mobile industry processor interface of the 7th patent application, in which the different virtual channels of the MIPI-CSI interface synchronously transmit images and sensor information when the step is executed, the one or more sensors The sensor information is transmitted to the central processing unit in the second virtual channel in the form of a sensor packet, and the one or more sensor packets include a header, a timestamp information, a packet length, and a bearer Data length, a carrying data and an error correction code. For example, the data transmission method based on the mobile industry processor interface in the scope of patent application, wherein the different virtual channels on the MIPI-CSI interface synchronously transmit images and sensor information when the step is executed, the one or more sensors The device packet constitutes a MIPI-CSI packet, and the MIPI-CSI packet at least further includes a MIPI packet header and a MIPI packet footer. For example, the data transmission method based on the mobile industry processor interface of the 10th patent application, in which the different virtual channels on the MIPI-CSI interface synchronously transmit images and sensor information when the step is executed, the MIPI-CSI packet further includes One or a combination of a padding data or a dummy packet. For example, the data transmission method based on the mobile industry processor interface of the 10th or 11th patent application, in which the different virtual channels of the MIPI-CSI interface synchronously transmit images and sensor information when the step is executed, the first The image information transmitted by the virtual channel and the sensor information transmitted by the second virtual channel are interleaved and sent to the central processing unit in units of the MIPI-CSI packet or in units of frames.

Images