KR102107299B1 - Method of transmitting image data and apparatuses performing the same - Google Patents

Abstract

Disclosed is an image data transmission method and apparatuses performing the same. An image data transmission method according to an embodiment may include obtaining a plurality of image data from a plurality of cameras, generating one image data packet by integrating the plurality of image data, and the one image data packet. It includes the step of transmitting.

Description

METHOD OF TRANSMITTING IMAGE DATA AND APPARATUSES PERFORMING THE SAME}

The following embodiments relate to an image data transmission method and apparatuses performing the same.

Recently, a lot of research and investment in ADAS (advanced driver assistance systems) has been made. However, commercialized products are limited to one or two locations worldwide.

In developing ADAS technology, the role of the sensor to recognize the external environment is important. Therefore, the sensor market for ADAS cameras, radars and lidars is expected to grow rapidly.

As the image analysis technology using the ADAS camera needs to analyze the image under various conditions, a technique for processing multiple camera image data is required.

In particular, the image analysis technology is determined to standardize a mobile industry processor interface (MIPI) camera serial interface (CSI), which is a camera interface standard used in mobile, to use for image analysis.

Embodiments are a single device for relaying communication between a plurality of cameras and an image processing device by transmitting a single image data packet, a synchronization signal and camera information in which a plurality of image data transmitted from a plurality of cameras are integrated to the image processing device. (Or a single interface).

An image data transmission method according to an embodiment may include obtaining a plurality of image data from a plurality of cameras, generating one image data packet by integrating the plurality of image data, and the one image data packet. It includes the step of transmitting.

The acquiring step and the transmitting step may be performed through one camera serial interface (CSI).

The method may further include transmitting at least one of camera identification (ID) and camera location information of each of the plurality of cameras.

The at least one may be included in a data identifier (ID) included in a packet header of the one video data packet.

The at least one of the data types of the data ID may be included in user defined byte-based data.

The at least one may be transmitted through a camera control interface (CCI).

1 is a schematic block diagram of an image data transmission system according to an embodiment.
FIG. 2 shows a schematic block diagram of the transmitting and receiving device shown in FIG. 1.
3 shows an example for describing the CSI transmitter module and the CSI receiver module shown in FIG. 2.
4 shows an example for describing one video data packet according to an embodiment.
5 shows the structure of the long packet carried in FIG. 4.
FIG. 6 shows the data identifier shown in FIG. 5.
FIG. 7 shows the data type shown in FIG. 6.
8 is a flowchart illustrating an operation of the transceiver shown in FIG. 1.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, various changes may be made to the embodiments, and the scope of the patent application right is not limited or limited by these embodiments. It should be understood that all modifications, equivalents, or substitutes for the embodiments are included in the scope of rights.

The terms used in the examples are for illustrative purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts or combinations thereof described herein, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

The terms first or second may be used to describe various components, but the components should not be limited by the terms. The terms are for the purpose of distinguishing one component from another component, for example, without departing from the scope of rights according to the concept of the embodiment, the first component may be referred to as the second component, and similarly The second component may also be referred to as the first component.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the embodiment belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter of the embodiments, detailed descriptions thereof will be omitted.

A module in the present specification may mean hardware capable of performing functions and operations according to each name described in the present specification, or computer program code capable of performing specific functions and operations. Or, it may mean an electronic recording medium on which computer program code capable of performing a specific function and operation is mounted, for example, a processor or a microprocessor.

In other words, the module may mean a functional and / or structural combination of hardware for performing the technical idea of the present invention and / or software for driving the hardware.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. The same reference numerals in each drawing denote the same members.

1 is a schematic block diagram of an image data transmission system according to an embodiment.

Referring to FIG. 1, an image data transmitting system 10 includes a plural cameras 100-1 to 100-n, a transceiver 300, and an image processing device. (an image processing apparatus; 500).

The plurality of cameras 100-1 to 100-n may transmit a plurality of image data, a synchronization signal, and camera information to the transceiver 300. The plurality of cameras 100-1 to 100-n may be multiple cameras for ADAS. The plurality of cameras 100-1 to 100-n may be cameras having a MIPI CSI standard. n may be a natural number of 1 or more.

For example, the plurality of cameras 100-1 to 100-n may transmit image data and synchronization signals captured by the plurality of cameras to the transceiver 300. The image data may be various images captured by a plurality of cameras 100-1 to 100-n, such as a vehicle interior, an exterior, and a surrounding external environment. The synchronization signal may include a horizontal synchronization signal (HSYNC) and a vertical synchronization signal (VSYNC).

The plurality of cameras 100-1 to 100-n may transmit camera information of the plurality of cameras 100-1 to 100-n to the transceiver 300. The camera information may include at least one of camera ID, camera location information, camera pixel size, and image data information for each of the plurality of cameras 100-1 to 100-n. Information about the image data may include horizontal and vertical sizes of the image.

Also, the plurality of cameras 100-1 to 100-n may be controlled according to a control signal transmitted by the image processing apparatus 500 through the transmission / reception device 300.

The transmitting and receiving device 300 receives a plurality of image data, a synchronization signal, and camera information transmitted from a plurality of cameras 100-1 to 100-n, and a single image data packet in which a plurality of image data are integrated, synchronization Signals and camera information may be transmitted (or transmitted) to the image processing apparatus 500. At this time, the transmission / reception device 300 may include camera information in one image data packet.

That is, the transmission / reception device 300 may be a single device (or a single interface) that relays communication between a plurality of cameras 100-1 to 100-n and the image processing device 500.

The image processing apparatus 500 may be various apparatuses for processing images, such as an application processor (AP), a digital signal processor, and a display.

The image processing apparatus 500 may process image data transmitted from the transmission / reception apparatus 300. For example, the image processing apparatus 500 may acquire (or receive) image data and camera information transmitted from the transmission / reception apparatus 300. The image processing apparatus 500 may process and / or display image data provided by the transmission / reception apparatus 300 based on camera information.

In addition, the image processing apparatus 500 transmits a control signal for controlling the plurality of cameras 100-1 to 100-n through the transceiver 300 to the plurality of cameras 100-1 to 100-n. You can. For example, the image processing apparatus 500 may acquire (or receive) a synchronization signal transmitted from the transmission / reception apparatus 300. The image processing apparatus 500 may transmit a control signal to the plurality of cameras 100-1 to 100-n based on the synchronization signal.

FIG. 2 shows a schematic block diagram of the transmitting and receiving device shown in FIG. 1.

Referring to FIG. 2, the transmission / reception device 300 includes a transmitter 310 and a receiver 330.

The transmitter 310 may include a camera serial interface (CSI) transmitter module 311 and a control interface (CCI) slave module 313.

The CSI transmitter module 311 may be a single CSI module that transmits (or transmits) a plurality of image data, synchronization signals, and camera information. CSI may be a standard for an interface connecting an AP and a camera.

The CSI transmitter module 311 acquires (or receives) a plurality of image data transmitted from a plurality of cameras 100-1 to 100-n, and integrates the plurality of image data to generate one image data packet. You can. For example, one integrated data packet may be a packet conforming to the MIPI CSI packet standard.

Also, the CSI transmitter module 311 may generate one image data packet including camera information of a plurality of cameras 100-1 to 100-n. For example, the camera information may include at least one of camera ID and camera location information. At this time, at least one may be included in user defined byte-based data of a data ID (data identifier) included in the packet header of one video data packet.

The CSI transmitter module 311 may transmit one image data packet to the CSI receiver module 331 through a data lane. For example, a plurality of data lines (eg, data 1 to data N) may be configured. N may be a natural number of 1 or more. Data lines are + and? It can be composed of 2 pins for each signal.

The CSI transmitter module 311 may transmit (or transmit) a synchronization signal to the CSI receiver module 331. For example, the CSI transmitter module 311 may transmit a synchronization signal to the CSI receiver module 331 through a clock lane. The clock lines are + and? It can be composed of 2 pins for each signal.

The CCI slave module 313 may be a single CCI module that transmits (or transmits) camera information to the CCI master module 333 of the receiver 300. For example, the camera information may be information for distinguishing a plurality of image data included in one image data packet transmitted through the CSI transmitter module 311. Even if the camera ID and the camera location information are not included or transmitted in one video data packet, the camera ID and the camera location information may be transmitted to the CCI master module 333 through the CCI slave module 313.

The receiver 330 may include a CSI receiver module 331 and a CCI master module 333.

The CSI receiver module 331 may be a single CSI module that receives one image data packet, a synchronization signal, and camera information.

The CSI receiver module 331 may distinguish a plurality of image data included in one image data packet using camera information, and transmit the plurality of image data to the image processing apparatus 500. For example, the CSI receiver module 331 may depacketize one image data packet into a plurality of image data.

The CSI receiver module 331 may distinguish a plurality of image data using at least one of camera information included in one image data packet and camera information received through the CCI master module 333. For example, any one image data among a plurality of image data may be distinguished by matching the camera information of the camera that transmitted the one image data.

The CSI receiver module 331 may transmit a plurality of distinct image data to the image processing apparatus 500.

The CCI master module 333 may be a single CCI module that receives only camera information transmitted from the CCI slave module 313.

3 illustrates an example for describing the CSI transmitter module and the CSI receiver module illustrated in FIG. 2, and FIG. 4 illustrates an example for describing one image data packet according to an embodiment.

3 and 4, the CSI transmitter module 311 may be composed of an application layer, a pixel to byte packing formats layer, a low level protocol layer, a lane management layer, and a phy layer.

The pixel to byte packing formats layer may be a layer for integrating a plurality of image data transmitted from a plurality of cameras 100-1 to 100-n. For example, the pixel to byte packing formats layer may convert a plurality of image data by converting 8-bit image data into 32-bit image data or converting 32-bit image data into 8-bit image data.

The low level protocol layer may be a layer for packetizing a plurality of integrated image data into one image data packet and transmitting one image data packet to one physical channel.

The low level protocol can have six characteristics. The low level protocol can transmit independent payload (arbitrary data). The low level protocol can provide 8-bit word size. The low level protocol can support up to four interleaved virtual channels on the same link. The low level protocol can provide special packets for frame start, frame end, line start and line end information. The low level protocol can provide a descriptor for the type, pixel depth, and format of application specific payload data. The low level protocol can provide a 16 bit checksum code for error detection.

As illustrated in FIG. 4, one video data packet may be any one of a short packet and a long packet divided into LPS (low power state). However, one video data packet may be mainly a long packet. The long packet can be distinguished from the start and end of the packet by star of transmission (ST) and end of transmission (ET).

The lane management layer may be a layer for managing data lines and clock lines.

The phy layer is a D-PHY and can perform a basic role played by a general physical layer (PHY). For example, the phy layer may be a physical layer that transmits one image data packet at high speed. The D-PHY may add ST and ET indicating the start and end of one video data packet. The D-PHY may convert parallel data in series or serial data in parallel to transmit and receive one video data packet. The data signal of D-PHY may be either low power (LP) mode or high speed (HP) mode.

The CSI receiver module 331 may include an application layer, a byte to pixel unpacking formats layer, a low level protocol layer, a lane management layer, and a phy layer.

The application layer, byte to pixel unpacking formats layer, low level protocol layer, lane management layer and phy layer of the CSI receiver module 331 are the application layer, pixel to byte unpacking formats layer, low level of the CSI transmitter module 311 described above. It may be the same as the protocol layer, lane management layer, and phy layer.

However, the low level protocol layer of the CSI receiver module 313 performs depacketization, not packetization, and the byte to pixel unpacking formats layer only distinguishes a plurality of image data without integrating them.

FIG. 5 shows the structure of the long packet shown in FIG. 4, FIG. 6 shows the data identifier shown in FIG. 5, and FIG. 7 shows the data type shown in FIG.

5 to 7, the long packet may include packet header (PH), packet data, and packet footer (PF).

The packet header may include a data identifier (DI) composed of a byte structure. For example, the DI may include a virtual channel indentifier (VC) and a data type (DT). The data type (DT) may represent various data types as shown in FIG. 7. For example, DT (data type) represents synchronization short packet data types in the 0x00 to 0x07 region, generic short packet data types in the 0x08 to 0x0F region, and generic long packet data types in the 0x10 ro 0x17 region, and 0x18 to Yx data in the 0x1F area, RGC data in the 0x20 to 0x27 area, RAW data in the 0x28 to 0x2F area, user defined byte-based data in the 0x30 to 0x37 areas, and reserved in the 0x38 to 0x3F areas. Can be represented.

User defined byte-based data in the region of 0x30 to 0x37 may be defined to distinguish a plurality of image data. For example, 0x30 displays camera ID and camera location information of the first camera 100-1 corresponding to the image data of the first camera 100-1 among the plurality of cameras 100-1 to 100-n. Included, 0x31 includes the camera ID and camera position information of the second camera 100-2 corresponding to the image data of the second camera 100-2 among the plurality of cameras 100-1 to 100-n can do.

The packet data may include a plurality of image data of a plurality of cameras 100-1 to 100-n and a word count of a plurality of image data. For example, data 1-0, data1-1, data 1-2, and data1-3 of FIG. 5 are image data of the first camera 100-1, and data n-0, data n-1, and data n -2 and data n-3 may be image data of the n-th camera 100-n. data WC-1 is a word count of image data of the first camera 100-1, data WC-2 is a word count of image data of the second camera 100-2, and data WC-3 is a third camera The word count of the image data of (100-3), and the data WC-4 may be the word count of the image data of the fourth camera 100-4.

The packet footer may include a 16 bit checksum code for error detection.

8 is a flowchart illustrating an operation of the transceiver shown in FIG. 1.

Referring to FIG. 8, the transmitter 310 may acquire (or receive) a plurality of image data of a plurality of cameras 100-1 to 100-n (S110).

The transmitter 310 may generate a single image data packet by integrating a plurality of image data (S130). In this case, the transmitter 310 may include camera ID and camera location information of each of the plurality of cameras 100-1 to 100-n in one image data packet.

The transmitter 330 may transmit (or transmit) one video data packet to the receiver 330 (S150).

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed on networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (5)

Obtaining a plurality of image data from a plurality of cameras;
Generating one image data packet by integrating the plurality of image data; And
Transmitting the one video data packet
Including,
The acquiring step and the transmitting step are performed through one camera serial interface (CSI),
The one image data packet includes camera information of each of the plurality of cameras,
The camera information allows the plurality of image data to be distinguished from the one image data packet,
Transmitting camera identification (ID) and camera location information of each of the plurality of cameras
Further comprising,
The camera ID and the camera location information are included in a data ID (data identifier) included in the packet header of the one video data packet,
The camera ID and the camera location information are included in user defined byte-based data among the data types of the data ID.
delete delete delete According to claim 1,
The camera ID and the location information of the camera is a video data transmission method transmitted through a camera control interface (CCI).

Images