KR20210094762A - System and method for logging and replaying data for automatic driving - Google Patents

Abstract

Disclosed are a system for collecting and playing data for automatic driving and a method thereof. The system includes a data collection device which acquires and stores an image output from a camera attached to a vehicle and a data playing device which acquires the image stored in the data collection device to transmit the image to a target device for algorithm verification.

Description

Data collection and replay system and method for autonomous driving {System and method for logging and replaying data for automatic driving}

The present invention relates to a data collection and reproduction system and method for autonomous driving, and more particularly, to collect and store image data from a camera through a MIPI interface, and to receive data through a USB method and transmit it to a target device It relates to a data collection and reproduction system and method therefor.

The self-driving car that drives itself even if the driver does not drive the vehicle has emerged as a big topic in the recent automobile market. In the case of autonomous vehicles, advanced sensors that can recognize surrounding objects and high-performance graphic processing units are required.

need more

The existing data collection device transmits the image received from the camera to the storage through 10G Ethernet, transmits the stored image to the HILS device through 10G Ethernet, and transmits the image from the HILS device to the target device. In this case, by using 10G Ethernet, there is a problem in that a number of communication conversion steps must be performed and the equipment is expensive.

In order to solve this problem, the present invention does not use 10G Ethernet, but uses the USB method to reduce the communication conversion step, and generates a plurality of images to enable autonomous driving algorithms to be verified in multiple target devices. To provide a data collection and reproduction system and method for

According to an embodiment of the present invention for solving the above technical problem, a data collection apparatus for autonomous driving includes a receiver configured to receive data in a Bayer image format from a camera attached to a vehicle through a predetermined interface; a first image copy unit for copying the received Bayer image format data when the predetermined interface is a MIPI interface; an image format converter converting the Bayer image format data received from the first image copying part into YUV image format data; and a storage unit for storing the Bayer image format data and the YUV image format data.

When the predetermined interface is an interface other than the MIPI interface, the data collection device for autonomous driving further includes a first interface conversion unit that converts the received data in the Bayer image format into data in the Bayer image format received by the MIPI interface. Including, the first image copy unit may copy the converted Bayer image format data.

According to another embodiment of the present invention for solving the above technical problem, a data reproducing apparatus for autonomous driving includes: a USB controller for receiving Bayer image format data or YUV image format data using a MIPI interface through a USB method; a second image copy unit that copies the received data in the Bayer image format or the data in the YUV image format to correspond to the number of target devices that have reproduced the data; and a transmitter configured to transmit the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data to the target device.

The data reproducing apparatus for autonomous driving applies the received Bayer image format data or the YUV image format data and the copied Bayer image format data or the YUV image format data by the MIPI interface to the target device Further comprising a second interface conversion unit for converting data of a predetermined interface to be a target device, wherein the transmitting unit transmits Bayer image format data or YUV image format data and copied Bayer image format data by the converted interface to the target device can be sent to

According to another embodiment of the present invention for solving the above technical problem, a data collection and reproduction system for autonomous driving includes a receiver configured to receive data in a Bayer image format from a camera attached to a vehicle through a predetermined interface; When the predetermined interface is a MIPI interface, a first image copying unit for copying the received Bayer image format data, and converting the Bayer image format data received from the first image copying part into YUV image format data a data collection device including an image format conversion unit and a storage unit for storing data in the Bayer image format and data in the YUV image format; and a USB controller for receiving the Bayer image format data or the YUV image format data by the MIPI interface stored in the storage unit through a USB method, and the received Bayer image format data or the YUV image format data a second image copy unit that copies to correspond to the number of target devices to be used, and the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data and a data reproducing apparatus including a transmitter for transmitting to a target apparatus.

A data collection and reproduction system for autonomous driving receives data in Bayer image format or data in YUV image format from the target device, and transmits the received data in Bayer image format or data in YUV image format to the receiver. may further include.

According to another embodiment of the present invention for solving the above technical problem, a data collection method for autonomous driving includes: receiving data in a Bayer image format from a camera attached to a vehicle through a predetermined interface; when the predetermined interface is a MIPI interface, copying the received Bayer image format data; converting the data in the Bayer image format into data in the YUV image format; and storing the data in the Bayer image format and the data in the YUV image format.

According to another embodiment of the present invention for solving the above technical problem, the data reproduction method for autonomous driving comprises the steps of receiving data in the Bayer image format or data in the YUV image format by the stored MIPI interface through a USB method. ; copying data in the Bayer image format received through the USB method or the number of target devices using the YUV image format data; and transmitting the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data to the target device.

According to another embodiment of the present invention for solving the above technical problem, a data verification method for autonomous driving receives data in a Bayer image format or data in a YUV image format from a target device, and transmitting data or data in YUV image format.

The present invention has the advantage of being able to measure the recognition rate of an image or an image at a low cost by reducing the conventional communication conversion step.

need more

1 is a block diagram of a data collection and reproduction system for autonomous driving according to an embodiment of the present invention.
2 is a block diagram of an apparatus for collecting data for autonomous driving according to an embodiment of the present invention.
3 is a block diagram of an apparatus for reproducing data for autonomous driving according to an embodiment of the present invention.
4 is a flowchart illustrating a data collection method for autonomous driving according to an embodiment of the present invention.
5 is a flowchart illustrating a data reproduction method for autonomous driving according to an embodiment of the present invention.
6 is a flowchart illustrating a data verification method for autonomous driving according to an embodiment of the present invention.

A preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a data collection and reproduction system for autonomous driving according to an embodiment of the present invention, and FIG. 2 is a block diagram of a data collection apparatus for autonomous driving according to an embodiment of the present invention; 3 is a block diagram of an apparatus for reproducing data for autonomous driving according to an embodiment of the present invention.

1 and 3 , the data collection and reproduction system 100 for autonomous driving includes a data collection device 10 , a data reproduction device 20 , and a transceiver 50 , and the data collection device 10 . includes a receiving unit 11 , a first image copying unit 12 , a first interface converting unit 13 , a storage 14 and an image format converting unit 15 , and the data reproduction device 20 is a USB control unit 21 , a second image copying unit 22 , a transmitting unit 23 and a second interface converting unit 24 .

Hereinafter, a data collection and reproduction system for autonomous driving and an apparatus included therein will be described with reference to FIGS. 1 to 3 .

In the data collection and reproduction system 100 for autonomous driving, the data collection device 10 acquires and stores the image output from the camera 30 attached to the vehicle, and the data reproduction device 20 stores the data collection device The image stored in 10 is acquired and transmitted to the target device 40 to verify the algorithm, and the transceiver 50 receives the image from the target device 40 and sends it back to the data collection device 10 . By sending it, the algorithm can be verified again.

First, the data collection device 10 for autonomous driving will be described.

Referring to FIG. 1 , the receiver 11 receives data in Bayer image format output from the camera 30 . The Bayer image format consists of RGB values. The data may be an image or an image.

The receiver 11 receives data in the Bayer image format input from the camera 30 through a predetermined interface. The predetermined interface is an output interface of the camera 30 , and includes an LVDS interface, a DVP interface, and a MIPI interface. The predetermined interface is not limited as long as it is an output interface of the camera 30 . In the present invention, the camera output interface is preferably a MIPI CSI-2 interface.

When the predetermined interface above is an interface other than the MIPI interface, for example, an LVDS interface or a DVP interface, the first interface conversion unit 12 converts the received Bayer image format data into the Bayer image format received by the MIPI interface. Convert to data.

Since the data acquisition device 10 stores image data through the MIPI interface, image data through other interfaces must be converted into image data through the MIPI interface.

The first image copy unit 13 receives Bayer image format data from the receiving unit 11 when the predetermined interface is the MIPI interface, and converts the interface from the first interface conversion unit 12 when it is not the MIPI interface. Receives Bayer image format data. The first image copying unit 13 copies at least one or more received Bayer image format data. The reason for copying one input image data is to convert the copied image data to obtain image data of a different format.

The image format conversion unit 15 receives Bayer image format data from the first image copying unit 13 and converts the received Bayer image format data into YUV image format data. In the YUV image format, color is composed of brightness (Y), blue color difference (U), and red color difference (V) information.

The storage unit 14 receives and stores Bayer image format data from the first image copy unit 13 , and receives and stores YUV image format data from the image format converter 15 .

Hereinafter, the data reproducing apparatus 20 for autonomous driving will be described.

Referring to FIG. 2 , the USB control unit 21 receives data in the Bayer image format or the data in the YUV image format through the MIPI interface from the storage unit 14 of the data collection device 10 through the USB method. USB (Universal Serial Bus) is one of serial bus standards for connecting peripheral devices to information devices such as computers, and in the present invention, it is preferable to use the USB 3.0 standard.

The second image copy unit 22 copies the received data in the Bayer image format or the data in the YUV image format to correspond to the number of target devices to be used. The target device is a device that verifies the received image format data using an algorithm. When a plurality of target devices are used, verification can be performed by applying various algorithms to the same data. Since data in an image format is verified in a plurality of target devices, a plurality of image format data is required. In order to obtain a plurality of image format data, the second image copy unit 22 copies at least one data of the received image format.

When the interface applied by the target device is not the MIPI interface, the second interface conversion unit 24 receives the Bayer image format data or the YUV image format data from the second image copy unit 22, and sends them to the target device. It can be converted into data of a predetermined interface to be applied.

The transmitter 23 receives data in the Bayer image format or data in the YUV image format from the second image copying unit 22 or the second interface converter 24 and transmits the data to the target device 40 .

The number of target devices 40 is at least one.

The transceiver 50 receives the Bayer image format data or the YUV image format data from the target device 40 , and transmits it back to the receiver 11 of the data collection device 10 . In this case, the algorithm can be verified again with the same image data.

4 is a flowchart illustrating a data collection method for autonomous driving according to an embodiment of the present invention.

Referring to FIG. 4 , in step 410 , an apparatus for performing a data collection method (hereinafter, a data execution apparatus) receives data in a Bayer image format output from a camera attached to a vehicle. The data may be an image or an image.

The data execution device receives data in the Bayer image format input through a predetermined interface from the camera. The predetermined interface is an output interface of the camera, and includes an LVDS interface, a DVP interface, and a MIPI interface. In the present invention, the camera output interface is preferably a MIPI CSI-2 interface.

In step 420, the data performing apparatus determines in step 410 whether a predetermined interface is a MIPI interface. In the case of the MIPI interface, the process proceeds to step 440, and if it is not the MIPI interface, the process proceeds to step 430.

In step 430, when the predetermined interface is the LVDS interface or the DVP interface, the data performing apparatus converts the received Bayer image format data into Bayer image format data received by the MIPI interface. Since the data performing apparatus stores image data by the MIPI interface, image data by an interface other than the MIPI interface must be converted into image data by the MIPI interface.

In operation 440, the data performing apparatus copies at least one or more data of the received Bayer image format. The reason for copying one input image data is to convert the copied image data to obtain image data of a different format.

In step 450, some of the data of the plurality of Bayer image formats are converted into data of the YUV image format.

In step 460, the data performing apparatus stores the data of the Bayer image format and the data of the converted YUV image format in the storage.

5 is a flowchart illustrating a data reproduction method for autonomous driving according to an embodiment of the present invention. 5 may be connected and proceeded after step 460 of FIG. 4 .

Referring to FIG. 5 , in step 510 , the device performing the data reproducing method (hereinafter, the data reproducing device) transmits Bayer image format data or YUV image format data stored in the data collection device through the MIPI interface through the USB method. receive USB (Universal Serial Bus) is one of serial bus standards for connecting peripheral devices to information devices such as computers, and in the present invention, it is preferable to use the USB 3.0 standard.

In step 520, the data reproducing device copies data in the Bayer image format or data in the YUV image format received through the USB method to correspond to the number of target devices to be used. The target device is a device that verifies the received image format data using an algorithm. When a plurality of target devices are used, verification can be performed by applying various algorithms to the same data. Since data in an image format is verified in a plurality of target devices, a plurality of image format data is required. Accordingly, in order to acquire data in a plurality of image formats, the data reproducing apparatus copies at least one data in the received image format.

In step 530, if the interface applied by the target device is not the MIPI interface, the data reproducing device receives Bayer image format data or YUV image format data by the MIPI interface and Bayer image format data or YUV copied in step 520 The image format data is converted into data of a predetermined interface applied to the target device. In one embodiment of the present invention, step 530 may be omitted.

In step 540, the data reproducing device transmits the received Bayer image format data or YUV image format data and copied Bayer image format data or YUV image format data to at least one target device.

6 is a flowchart illustrating a data verification method for autonomous driving according to an embodiment of the present invention. 6 may be connected and proceeded after step 540 of FIG. 5 .

Referring to FIG. 6 , in step 610, the transceiver receives data in a Bayer image format or data in a YUV image format from a target device.

In step 620, the transceiver transmits the data in the Bayer image format or the data in the YUV image format back to the data acquisition device. In this case, the method of Figs. 4 and 5 proceeds again, so that the method of Figs. 4 to 6 continues to cycle, so that the algorithm can be verified with the same image data.

The data collection method, data reproduction method, and data verification method for autonomous driving as described above can also be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes any type of recording medium in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium is distributed in a computer system connected to a network, so that the computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the disk management method can be easily inferred by programmers in the technical field to which the present invention pertains.

So far, the present invention has been looked at with respect to preferred embodiments thereof. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Data Collection and Replay Systems for Autonomous Driving: 100
Data Acquisition Devices: 10
Receiving unit: 11 First interface converting unit: 12
First image copy unit: 13 Storage unit: 14
Image format converter: 15
Data playback devices: 20
USB control unit: 21 Second image copy unit: 22
Transmitting unit: 23 Second interface converting unit: 24
Camera: 30 Target Device: 40
Transceiver: 50

Claims (12)

a receiver for receiving data in Bayer image format from a camera attached to the vehicle through a predetermined interface;
a first image copy unit for copying the received Bayer image format data when the predetermined interface is a MIPI interface;
an image format converter converting the Bayer image format data received from the first image copying part into YUV image format data; and
and a storage unit for storing the Bayer image format data and the YUV image format data. The method of claim 1,
When the predetermined interface is an interface other than the MIPI interface, further comprising a first interface converter for converting the received Bayer image format data into Bayer image format data received by the MIPI interface,
The data collecting device for autonomous driving, characterized in that the first image copy unit copies the converted Bayer image format data. a USB controller for receiving Bayer image format data or YUV image format data through a MIPI interface through a USB method;
a second image copying unit that copies the received data in the Bayer image format or the data in the YUV image format to correspond to the number of regenerated target devices; and
and a transmitter configured to transmit the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data to the target device. data playback device. 4. The method of claim 3,
Converts the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data by the MIPI interface into data of a predetermined interface applied to the target device It further comprises a second interface conversion unit,
and the transmitter transmits Bayer image format data or YUV image format data and copied Bayer image format data through the converted interface to the target device. a receiving unit for receiving data in Bayer image format from a camera attached to the vehicle through a predetermined interface;
a first image copying unit for copying the received Bayer image format data when the predetermined interface is a MIPI interface;
an image format converter converting the Bayer image format data received from the first image copying part into YUV image format data; and
a data collection device including a storage unit configured to store the Bayer image format data and the YUV image format data; and
a USB control unit for receiving the Bayer image format data or the YUV image format data stored in the storage unit through the USB method through the MIPI interface;
a second image copying unit that copies the received Bayer image format data or the YUV image format data to correspond to the number of target devices using the data; and
and a data reproducing apparatus including a transmitter for transmitting the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data to the target device Data collection and playback system for autonomous driving. 6. The method of claim 5,
When the predetermined interface is an interface other than the MIPI interface, the data collection device further comprises a first interface converter for converting the received Bayer image format data into Bayer image format data received by the MIPI interface, The first image copy unit copies the converted Bayer image format data,
The data reproducing device is configured to apply the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data through the MIPI interface to the target device. Further comprising a second interface conversion unit for converting the data of the interface, wherein the transmitting unit transmits Bayer image format data or YUV image format data and copied Bayer image format data by the converted interface to the target device Data collection and reproduction system for autonomous driving, characterized in that. 7. The method of claim 6,
Autonomous driving, characterized in that it further comprises a transceiver for receiving Bayer image format data or YUV image format data from the target device, and transmitting the received Bayer image format data or YUV image format data to the receiver data collection and replay system for Receiving data in Bayer image format from a camera attached to the vehicle through a predetermined interface;
when the predetermined interface is a MIPI interface, copying the received Bayer image format data;
converting the data in the Bayer image format into data in the YUV image format; and
and storing the data in the Bayer image format and the data in the YUV image format. 9. The method of claim 8,
If the predetermined interface is an interface other than the MIPI interface, converting the received Bayer image format data into Bayer image format data received by the MIPI interface,
Copying the data in the received Bayer image format includes copying the converted Bayer image format data. 10. The method of claim 9,
Receiving the stored Bayer image format data or YUV image format data through the MIPI interface through a USB method;
copying data in the Bayer image format received through the USB method or the number of target devices using the YUV image format data; and
and transmitting the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data to the target device. How to play. 11. The method of claim 10,
Converts the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data by the MIPI interface into data of a predetermined interface applied to the target device further comprising the step of
Transmitting the received Bayer image format data or YUV image format data and the copied Bayer image format data or YUV image format data to the target device is a Bayer image format by the converted interface. Data collection and reproduction method for autonomous driving, comprising the step of transmitting the data of the YUV image format and the copied Bayer image format data to the target device. 12. The method of claim 11,
Data for autonomous driving, characterized in that the method further comprises receiving data in the Bayer image format or data in the YUV image format from the target device, and transmitting the received data in the Bayer image format or data in the YUV image format. How to collect and play.

Images