WO2019047663A1 - Video format-based end-to-end automatic driving data storage method and device - Google Patents

Video format-based end-to-end automatic driving data storage method and device Download PDF

Info

Publication number
WO2019047663A1
WO2019047663A1 PCT/CN2018/099391 CN2018099391W WO2019047663A1 WO 2019047663 A1 WO2019047663 A1 WO 2019047663A1 CN 2018099391 W CN2018099391 W CN 2018099391W WO 2019047663 A1 WO2019047663 A1 WO 2019047663A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
video
image data
automatic driving
reading
Prior art date
Application number
PCT/CN2018/099391
Other languages
French (fr)
Chinese (zh)
Inventor
闫泳杉
郁浩
郑超
唐坤
张云飞
姜雨
Original Assignee
百度在线网络技术(北京)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 百度在线网络技术(北京)有限公司 filed Critical 百度在线网络技术(北京)有限公司
Publication of WO2019047663A1 publication Critical patent/WO2019047663A1/en

Links

Images

Classifications

    • GPHYSICS
    • G07CHECKING-DEVICES
    • G07CTIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C5/00Registering or indicating the working of vehicles
    • G07C5/08Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
    • G07C5/0841Registering performance data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/174Redundancy elimination performed by the file system
    • G06F16/1744Redundancy elimination performed by the file system using compression, e.g. sparse files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/71Indexing; Data structures therefor; Storage structures

Definitions

  • the present invention relates to the field of computers, and in particular, to a method and apparatus for storing end-to-end automatic driving data based on a video format.
  • an automatic driving system generally uses a model established by data acquired in real time in front, output steering angle, and speed to perform deep learning. The more data collected, the more favorable the generated model is for deep learning. However, these data usually need to be stored in a specific file, which requires a large storage space, which limits the development of deep learning in the field of automatic driving.
  • One of the technical problems solved by the present invention is that the data collected in front of the automatic driving system needs to occupy a large storage space.
  • a method for storing end-to-end automatic driving data based on a video format including:
  • the image data is stored as a video file using the video compression parameters.
  • a storage device for end-to-end automatic driving data based on a video format including:
  • the present embodiment stores the read posture data as image data in the order of the time stamp and stores it as a video file, the storage space occupied by the data can be reduced, and the amount of access of the network I/O can also be reduced to establish a more A good autonomous driving data model, which in turn improves the learning efficiency of deep learning in the field of automatic driving.
  • FIG. 1 shows a flow chart of a method for storing end-to-end autopilot data based on a video format in accordance with an embodiment of the present invention.
  • FIG. 2 is a flow chart showing a method for storing end-to-end automatic driving data based on a video format according to Embodiment 1 of the present invention.
  • FIG. 3 is a flow chart showing a method for storing end-to-end automatic driving data based on a video format according to Embodiment 2 of the present invention.
  • FIG. 4 is a block diagram showing a storage device for end-to-end automatic driving data based on a video format in accordance with an embodiment of the present invention.
  • FIG. 5 is a block diagram showing a storage device for end-to-end automatic driving data based on a video format according to Embodiment 3 of the present invention.
  • Fig. 6 is a block diagram showing a storage device for end-to-end automatic driving data based on a video format proposed in Embodiment 4 of the present invention.
  • Computer device also referred to as “computer” in the context, is meant an intelligent electronic device that can perform predetermined processing, such as numerical calculations and/or logical calculations, by running a predetermined program or instruction, which can include a processor and The memory is executed by the processor to execute a predetermined process pre-stored in the memory to execute a predetermined process, or is executed by hardware such as an ASIC, an FPGA, a DSP, or the like, or a combination of the two.
  • Computer devices include, but are not limited to, servers, personal computers, notebook computers, tablets, smart phones, and the like.
  • the computer device includes a user device and a network device.
  • the user equipment includes, but is not limited to, a computer, a smart phone, a PDA, etc.
  • the network device includes but is not limited to a single network server, a server group composed of multiple network servers, or a cloud computing based computer Or a cloud composed of a network server, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers.
  • the computer device can be operated separately to implement the present invention, and can also access the network and implement the present invention by interacting with other computer devices in the network.
  • the network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like.
  • the user equipment, the network equipment, the network, and the like are merely examples, and other existing or future possible computer equipment or networks, such as those applicable to the present invention, are also included in the scope of the present invention. It is included here by reference.
  • FIG. 1 is a flow chart of a method of storing end-to-end autopilot data based on a video format, in accordance with one embodiment of the present invention.
  • the video format-based end-to-end automatic driving data storage method includes the following steps:
  • the video compression parameter is first determined.
  • the test data may be subjected to a compression test by a compression parameter, and then determined as a video compression parameter according to the compression ratio of the compression test.
  • the compression parameters therein include, but are not limited to, at least one of a codec, an inter-frame allocation code rate (crf), or a color space.
  • the gesture data is read.
  • the attitude data output by the predetermined automatic driving system can be read in real time, and the posture data is stored as a time-stamped data sequence.
  • step S120 after the posture data is read, the image data is sequentially read according to the time stamp of the posture data.
  • the image data may be read in the order of the time stamp of the posture data, and the image data may be stored as an image data sequence in the order of the time stamp.
  • the read image data can be stored as a video file using the video compression parameters described above.
  • the image data may be compression-stored into a video file by a predetermined video format, and each frame of the video file corresponds to an image in the image data.
  • the embodiment further stores the video file on the predetermined server according to the type of the posture data output by the automatic driving system.
  • the types of gesture data include, but are not limited to, speed data, steering angle data, road network data, and the like.
  • the storage space occupied by the data can be reduced, and the access of the network I/O can also be reduced.
  • Volume to build a better autonomous driving data model which in turn improves the learning efficiency of deep learning in the field of automatic driving.
  • the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software.
  • This method will result in an HDF5 file storing images that is too large and will significantly increase the overhead of network I/O, so the traditional data acquisition method is not conducive to deep learning of the automatic driving system.
  • this embodiment proposes another storage method for the end-to-end automatic driving data based on the video format. As shown in FIG. 2, the method includes the following steps:
  • the color space can be used as a compression parameter due to less landscape changes on both sides of the road.
  • the color characteristics of snow, desert, forest, etc. the same in multiple image data. The color is uniformly compressed, and only the changes in the road surface are stored separately.
  • an inter-frame allocation code rate can be used as a compression parameter.
  • By dividing the code rate between frames it is possible to analyze which are important frames and which are secondary frames, and important frames get more bytes.
  • an object that is not moving in the image or a moving object that is far away is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame. This can give a clearer feeling and significantly reduce the size of the video file, because usually the human eye only pays attention to the moving object, and does not recognize the background.
  • the automatic driving system outputs a set of posture data every predetermined time, and the posture data usually includes image data, speed data, steering angle data, and road network data.
  • This embodiment mainly reads image data therein.
  • the image data is read in the order of time stamps of the posture data.
  • the attitude data output by the autopilot system is time stamped, which can be used to indicate the order in which the gesture data is generated, and the image data storage in a chronological order can more accurately characterize the image acquired by the autopilot system.
  • all the posture data are read in the order of time stamps to ensure that the posture data is consistent with the time stamp of the image data.
  • the image data is stored as a data sequence for subsequent steps to be called.
  • This embodiment generates a video file in the FFmpeg format.
  • FFmpeg can be used to record, convert, digital audio, video, and convert these into streams.
  • FFmpeg can not only compress multiple image data to generate video files, but also convert between multiple video formats.
  • the number of image data used to generate a video file is different each time according to different compression parameters.
  • the color space is used as the compression parameter, it is possible to compress 10,000 images each time to generate a 24 frame/second video file, the length of the video file is 7 minutes, the occupied space is generally 20-50M, and the original image is occupied by storage.
  • the space is about 1G, and the compressed video file not only occupies less storage space, but also has low network I/O overhead.
  • the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, therefore, can improve the depth learning efficiency of the automatic driving system.
  • the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software.
  • This method will cause the HDF5 file to store images to be too large, and will obviously increase the overhead of network I/O. Image storage will also result in too many files being stored, which is not conducive to editing and management, so the traditional data acquisition method is not conducive to automatic driving. Deep learning of the system.
  • this embodiment proposes a storage method for end-to-end automatic driving data based on a video format. As shown in FIG. 3, the method includes the following steps:
  • different parameters can be used on the test data, such as codec, inter-frame allocation code rate, color space, etc., to compare the compression of these compression parameters and the image clarity after compression.
  • an inter-frame allocation code rate can be used as a compression parameter.
  • By dividing the code rate between frames it is possible to analyze which are important frames and which are secondary frames.
  • a non-moving object in the image or a moving object farther away from the image is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame.
  • the image thus compressed can highlight a moving object, that is, an object that has an image for automatic driving, and other objects that do not move will not occupy more storage space.
  • the compression effects of the other two compression parameters are significantly worse, so for the road conditions in the urban area, the embodiment preferably uses the inter-frame allocation code rate as the compression parameter.
  • the autopilot system outputs a set of pose data every predetermined time, and each set of pose data is time stamped, which can be used to indicate the order in which the pose data is generated, and the image data storage can be more accurately characterized in chronological order.
  • the image captured by the autopilot system is a set of pose data every predetermined time, and each set of pose data is time stamped, which can be used to indicate the order in which the pose data is generated, and the image data storage can be more accurately characterized in chronological order.
  • all the posture data are read in the order of time stamps to ensure that the posture data is consistent with the time stamp of the image data.
  • the image data is stored as a data sequence for subsequent steps to be called.
  • the image data is generated into a video file by using an AVC encoding format
  • the video file has a code rate of 208 kbps, a frame rate of 14 fps, and a resolution of 448 ⁇ 336.
  • the inter-frame allocation code rate is used as the compression parameter
  • the length of the video file generated by compressing 10,000 images is 12 minutes
  • the occupied space is generally 40-70M
  • the storage space occupied by the original image is about 1G, after compression.
  • Video files not only take up less storage space, but also have lower network I/O overhead.
  • the type of the attitude data generally includes speed data, steering angle data, road network data, and the like. Therefore, in this embodiment, the attitude data is divided into two categories for storage, the first type is dynamic data, including speed data, steering angle data, motor vehicle data, etc., and the second is static data, including building data, real-time road condition data, Traffic signal data, etc. Video files stored in this category are easy to edit and manage, improving the efficiency of deep learning.
  • the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, and easy to edit and manage, avoiding additional decompression process, thus improving the deep learning efficiency of the autopilot system.
  • FIG. 4 is a block diagram of a storage device for end-to-end automatic driving data based on a video format, in accordance with one embodiment of the present invention.
  • the video format-based end-to-end automatic driving data storage device (hereinafter referred to as “storage device”) includes the following devices:
  • Means for determining video compression parameters and reading posture data (hereinafter referred to as “compressed reading device") 410;
  • image reading device Means for reading image data according to the time stamp of the posture data (hereinafter referred to as "image reading device") 420;
  • Means for storing the image data as a video file using the video compression parameter (hereinafter referred to as "video generating device") 430.
  • the video compression parameters are first determined by the compressed reading device 410.
  • the test data may be subjected to a compression test by the compression reading device 410 through a compression parameter, and then determined as a video compression parameter according to the compression ratio of the compression test.
  • the compression parameters therein include, but are not limited to, at least one of a codec, an inter-frame allocation code rate (crf), or a color space.
  • the gesture data is read.
  • the attitude data output by the predetermined automatic driving system can be read in real time by the compression reading device 410, and the posture data is stored as a time-stamped data sequence.
  • the image data is read by the image reading device 420 in accordance with the time stamp of the posture data.
  • the image data may be read by the image reading device 420 in the order of the time stamp of the posture data, and the image data may be stored as an image data sequence in the order of the time stamp.
  • the read image data can be stored as a video file by the video generating device 430.
  • the image data may be compression-stored into a video file by the video generating device 430 through a predetermined video format, and each frame of the video file corresponds to one image in the image data.
  • the embodiment further stores the video file on the predetermined server according to the type of the posture data output by the automatic driving system by the video generating device 430.
  • the types of gesture data include, but are not limited to, speed data, steering angle data, road network data, and the like.
  • the storage space occupied by the data can be reduced, and the access of the network I/O can also be reduced.
  • Volume to build a better autonomous driving data model which in turn improves the learning efficiency of deep learning in the field of automatic driving.
  • the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software.
  • This method will result in an HDF5 file storing images that is too large and will significantly increase the overhead of network I/O, so the traditional data acquisition method is not conducive to deep learning of the automatic driving system.
  • this embodiment proposes another storage device for end-to-end automatic driving data based on the video format, as shown in FIG. 5, including the following devices:
  • compression parameter determining device Means for determining a video compression parameter (hereinafter referred to as "compression parameter determining device”) 510;
  • Attitude data reading device a device for reading posture data (hereinafter referred to as "attitude data reading device") 520;
  • first image data reading device Means for reading image data in order of time stamps of the posture data
  • first video file generating device Means for generating a video file by using the image data in a predetermined format
  • the compression parameter determining device 510 When choosing compression parameters, consider the environment in which the autonomous driving system is located. For example, when driving on a sparsely populated highway, the color space can be used as a compression parameter due to less landscape changes on both sides of the road. For the color characteristics of snow, desert, forest, etc., the compression parameter determining device 510 will be more The same color in the image data is uniformly compressed, and only the changes in the road surface are separately stored.
  • an inter-frame allocation code rate can be used as a compression parameter.
  • By assigning a bit rate between frames it is possible to analyze which are important frames and which are secondary frames, and important frames get more bytes.
  • an object that is not moving in the image or a moving object that is far away is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame. This can give a clearer feeling and significantly reduce the size of the video file, because usually the human eye only pays attention to the moving object, and does not recognize the background.
  • the automatic driving system outputs a set of posture data every predetermined time, and the posture data usually includes image data, speed data, steering angle data, and road network data.
  • the present embodiment mainly reads image data therein by the posture data reading device 520.
  • the attitude data output by the autopilot system is time stamped, which can be used to indicate the order in which the gesture data is generated, and the image data storage in a chronological order can more accurately characterize the image acquired by the autopilot system.
  • all the posture data are read by the first image data reading device 530 in the order of time stamps to ensure that the posture data coincides with the time stamp of the image data.
  • the image data is stored as a data sequence for subsequent steps to be called.
  • This embodiment generates a video file in the FFmpeg format.
  • FFmpeg can be used to record, convert, digital audio, video, and convert these into streams.
  • FFmpeg can not only compress multiple image data to generate video files, but also convert between multiple video formats.
  • the number of image data used to generate a video file is different each time according to different compression parameters.
  • the first video file generating device 540 can select to compress 10,000 images each time to generate a video file of 24 frames/second, and the length of the video file is 7 minutes, and the occupied space is generally 20 -50M, the original image occupies about 1G of storage space.
  • the compressed video file not only occupies less storage space, but also has lower network I/O overhead.
  • the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, therefore, can improve the depth learning efficiency of the automatic driving system.
  • the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software.
  • This method will cause the HDF5 file to store images to be too large, and will obviously increase the overhead of network I/O. Image storage will also result in too many files being stored, which is not conducive to editing and management, so the traditional data acquisition method is not conducive to automatic driving. Deep learning of the system.
  • the present embodiment proposes a storage device for end-to-end automatic driving data based on a video format, as shown in FIG. 6, including the following devices:
  • compression and reading device Means for determining video compression parameters and reading posture data (hereinafter referred to as “compression and reading device”) 610;
  • Second image data reading device Means for reading image data according to the time stamp of the posture data (hereinafter referred to as "second image data reading device") 620;
  • Means for generating a video file by using the image data in a predetermined format hereinafter referred to as "second video file generating device" 630;
  • Means for storing the type of the posture data output by the automatic driving system of the video file on a predetermined server hereinafter referred to as "classification storage device" 640.
  • the compression and reading device 610 can use different parameters on the test data, such as codec, inter-frame allocation code rate, color space, etc., to compare the compression and compression of these compression parameters. After the image clarity.
  • an inter-frame allocation code rate can be used as a compression parameter.
  • By dividing the code rate between frames it is possible to analyze which are important frames and which are secondary frames.
  • a non-moving object in the image or a moving object farther away from the image is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame.
  • the thus compressed image can highlight a moving object, that is, an object that has an image for automatic driving, and other immovable objects do not occupy more storage space.
  • the compression effects of the other two compression parameters are significantly worse, so for the road conditions in the urban area, the embodiment preferably uses the inter-frame allocation code rate as the compression parameter.
  • the autopilot system outputs a set of pose data every predetermined time, and each set of pose data is time stamped, which can be used to indicate the order in which the pose data is generated, and the image data storage can be more accurately characterized in chronological order.
  • the image captured by the autopilot system is a set of pose data every predetermined time, and each set of pose data is time stamped, which can be used to indicate the order in which the pose data is generated, and the image data storage can be more accurately characterized in chronological order.
  • all the posture data are read by the second image data reading device 620 in the order of time stamps to ensure that the posture data coincides with the time stamp of the image data.
  • the image data is stored as a data sequence for subsequent steps to be called.
  • the second video file generating device 630 generates the video file by using the AVC encoding format, and the video file has a code rate of 208 kbps, a frame rate of 14 fps, and a resolution of 448 ⁇ 336.
  • the inter-frame allocation code rate is used as the compression parameter, the length of the video file generated by compressing 10,000 images is 12 minutes, the occupied space is generally 40-70M, and the storage space occupied by the original image is about 1G, after compression.
  • Video files not only take up less storage space, but also have lower network I/O overhead.
  • the type of the attitude data generally includes speed data, steering angle data, road network data, and the like. Therefore, the embodiment stores the posture data into two categories by the classification storage device 640.
  • the first category is dynamic data, including speed data, steering angle data, motor vehicle data, etc.
  • the second is static data, including building data. , real-time traffic data, traffic signal data, etc.
  • Video files stored in this category are easy to edit and manage, improving the efficiency of deep learning.
  • the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, and easy to edit and manage, avoiding additional decompression process, thus improving the deep learning efficiency of the autopilot system.
  • the present invention can be implemented in software and/or a combination of software and hardware.
  • the various devices of the present invention can be implemented using an application specific integrated circuit (ASIC) or any other similar hardware device.
  • the software program of the present invention may be executed by a processor to implement the steps or functions described above.
  • the software program (including related data structures) of the present invention can be stored in a computer readable recording medium such as a RAM memory, a magnetic or optical drive or a floppy disk and the like.
  • some of the steps or functions of the present invention may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A video format-based end-to-end automatic driving data storage method and device, the method comprising: determining a video compression parameter and reading attitude data (S110); reading image data according to the order of timestamps of the attitude data (S120); and using the video compression parameter to store the video file in a pre-determined server according to the type of the attitude data outputted by the automatic driving system, and store the image data as a video file (S130). The method stores the read attitude data as image data according to the order of timestamps and storing same as a video file, and thus can reduce storage space occupied by the data, and also reduce the amount of access to the network I/O, to establish a better automatic driving data model, further improving the learning efficiency of deep learning in the field of automatic driving.

Description

一种基于视频格式的端到端自动驾驶数据的存储方法及装置Method and device for storing end-to-end automatic driving data based on video format
本专利申请要求于2017年9月5日提交的、申请号为201710792055.4、申请人为百度在线网络技术(北京)有限公司、发明名称为“一种基于视频格式的端到端自动驾驶数据的存储方法及装置”的中国专利申请的优先权,该申请的全文以引用的方式并入本申请中。This patent application claims to be submitted on September 5, 2017, the application number is 201710792055.4, the applicant is Baidu Online Network Technology (Beijing) Co., Ltd., and the invention name is "a storage method based on video format for end-to-end automatic driving data. The priority of the Chinese Patent Application, the entire disclosure of which is incorporated herein by reference.
技术领域Technical field
本发明涉及计算机领域,尤其涉及一种基于视频格式的端到端自动驾驶数据的存储方法及装置。The present invention relates to the field of computers, and in particular, to a method and apparatus for storing end-to-end automatic driving data based on a video format.
背景技术Background technique
随着深度学习的迅速发展以及人工智能的深入研究,汽车工业发生了革命性的变化,通过端到端的深度学习实现自动驾驶便是自动驾驶领域的一个主要研究方向。在现有技术中,自动驾驶系统通常采用通过前方实时采集的图像、输出转向角和速度等数据建立的模型进行深度学习。采集的数据越多,则生成的模型越有利于深度学习。但这些数据通常需要存储在特定的文件中,需要占用较大的存储空间,从而限制了深度学习在自动驾驶领域的发展。With the rapid development of deep learning and the in-depth study of artificial intelligence, the automotive industry has undergone revolutionary changes. Automated driving through end-to-end deep learning is a major research direction in the field of automatic driving. In the prior art, an automatic driving system generally uses a model established by data acquired in real time in front, output steering angle, and speed to perform deep learning. The more data collected, the more favorable the generated model is for deep learning. However, these data usually need to be stored in a specific file, which requires a large storage space, which limits the development of deep learning in the field of automatic driving.
发明内容Summary of the invention
本发明解决的技术问题之一是自动驾驶系统前方采集的数据需要占用较大的存储空间。One of the technical problems solved by the present invention is that the data collected in front of the automatic driving system needs to occupy a large storage space.
根据本发明一方面的一个实施例,提供了一种基于视频格式的端到端自动驾驶数据的存储方法,包括:According to an embodiment of an aspect of the present invention, a method for storing end-to-end automatic driving data based on a video format is provided, including:
确定视频压缩参数并读取姿态数据;Determining video compression parameters and reading gesture data;
根据所述姿态数据的时间戳顺序读取图像数据;Reading image data according to a time stamp sequence of the gesture data;
使用所述视频压缩参数将所述图像数据存储为视频文件。The image data is stored as a video file using the video compression parameters.
根据本发明另一方面的一个实施例,提供了一种基于视频格式的端到端自动驾驶数据的存储装置,包括:According to an embodiment of another aspect of the present invention, a storage device for end-to-end automatic driving data based on a video format is provided, including:
用于确定视频压缩参数并读取姿态数据的装置;Means for determining video compression parameters and reading gesture data;
用于根据所述姿态数据的时间戳顺序读取图像数据的装置;Means for reading image data in accordance with a time stamp of the gesture data;
用于使用所述视频压缩参数将所述图像数据存储为视频文件的装置。Means for storing the image data as a video file using the video compression parameters.
由于本实施例通过将读取的姿态数据按照时间戳的顺序存储为图像数据并存储为视频文件,从而能够减少数据占用的存储空间,并且还能够减少网络I/O的访问量,以建立更好的自动驾驶数据模型,进而提高的深度学习在自动驾驶领域的学习效率。Since the present embodiment stores the read posture data as image data in the order of the time stamp and stores it as a video file, the storage space occupied by the data can be reduced, and the amount of access of the network I/O can also be reduced to establish a more A good autonomous driving data model, which in turn improves the learning efficiency of deep learning in the field of automatic driving.
本领域普通技术人员将了解,虽然下面的详细说明将参考图示实施例、附图进行,但本发明并不仅限于这些实施例。而是,本发明的范围是广泛的,且意在仅通过后附的权利要求限定本发明的范围。Those skilled in the art will appreciate that although the following detailed description is made with reference to the illustrated embodiments and drawings, the invention is not limited to these embodiments. Rather, the scope of the invention is intended to be limited the scope of the invention
附图说明DRAWINGS
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述,本发明的其它特征、目的和优点将会变得更明显:Other features, objects, and advantages of the present invention will become more apparent from the Detailed Description of Description
图1示出了根据本发明一实施例中的基于视频格式的端到端自动驾驶数据的存储方法的流程图。1 shows a flow chart of a method for storing end-to-end autopilot data based on a video format in accordance with an embodiment of the present invention.
图2示出了本发明的实施例一提出的基于视频格式的端到端自动驾驶数据的存储方法的流程图。FIG. 2 is a flow chart showing a method for storing end-to-end automatic driving data based on a video format according to Embodiment 1 of the present invention.
图3示出了本发明的实施例二提出的基于视频格式的端到端自动驾驶数据的存储方法的流程图。FIG. 3 is a flow chart showing a method for storing end-to-end automatic driving data based on a video format according to Embodiment 2 of the present invention.
图4示出了根据本发明一实施例中的基于视频格式的端到端自动驾驶数据的存储装置的框图。4 is a block diagram showing a storage device for end-to-end automatic driving data based on a video format in accordance with an embodiment of the present invention.
图5示出了本发明的实施例三提出的基于视频格式的端到端自动驾驶数据的存储装置的框图。FIG. 5 is a block diagram showing a storage device for end-to-end automatic driving data based on a video format according to Embodiment 3 of the present invention.
图6示出了本发明的实施例四提出的基于视频格式的端到端自动驾驶 数据的存储装置的框图。Fig. 6 is a block diagram showing a storage device for end-to-end automatic driving data based on a video format proposed in Embodiment 4 of the present invention.
附图中相同或相似的附图标记代表相同或相似的部件。The same or similar reference numerals in the drawings denote the same or similar components.
具体实施方式Detailed ways
在更加详细地讨论示例性实施例之前应当提到的是,一些示例性实施例被描述成作为流程图描绘的处理或方法。虽然流程图将各项操作描述成顺序的处理,但是其中的许多操作可以被并行地、并发地或者同时实施。此外,各项操作的顺序可以被重新安排。当其操作完成时所述处理可以被终止,但是还可以具有未包括在附图中的附加步骤。所述处理可以对应于方法、函数、规程、子例程、子程序等等。Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as a process or method depicted as a flowchart. Although the flowcharts describe various operations as a sequential process, many of the operations can be implemented in parallel, concurrently or concurrently. In addition, the order of operations can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the figures. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and the like.
在上下文中所称“计算机设备”,也称为“电脑”,是指可以通过运行预定程序或指令来执行数值计算和/或逻辑计算等预定处理过程的智能电子设备,其可以包括处理器与存储器,由处理器执行在存储器中预存的存续指令来执行预定处理过程,或是由ASIC、FPGA、DSP等硬件执行预定处理过程,或是由上述二者组合来实现。计算机设备包括但不限于服务器、个人电脑、笔记本电脑、平板电脑、智能手机等。By "computer device", also referred to as "computer" in the context, is meant an intelligent electronic device that can perform predetermined processing, such as numerical calculations and/or logical calculations, by running a predetermined program or instruction, which can include a processor and The memory is executed by the processor to execute a predetermined process pre-stored in the memory to execute a predetermined process, or is executed by hardware such as an ASIC, an FPGA, a DSP, or the like, or a combination of the two. Computer devices include, but are not limited to, servers, personal computers, notebook computers, tablets, smart phones, and the like.
所述计算机设备包括用户设备与网络设备。其中,所述用户设备包括但不限于电脑、智能手机、PDA等;所述网络设备包括但不限于单个网络服务器、多个网络服务器组成的服务器组或基于云计算(Cloud Computing)的由大量计算机或网络服务器构成的云,其中,云计算是分布式计算的一种,由一群松散耦合的计算机集组成的一个超级虚拟计算机。其中,所述计算机设备可单独运行来实现本发明,也可接入网络并通过与网络中的其他计算机设备的交互操作来实现本发明。其中,所述计算机设备所处的网络包括但不限于互联网、广域网、城域网、局域网、VPN网络等。The computer device includes a user device and a network device. The user equipment includes, but is not limited to, a computer, a smart phone, a PDA, etc.; the network device includes but is not limited to a single network server, a server group composed of multiple network servers, or a cloud computing based computer Or a cloud composed of a network server, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. Wherein, the computer device can be operated separately to implement the present invention, and can also access the network and implement the present invention by interacting with other computer devices in the network. The network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like.
需要说明的是,所述用户设备、网络设备和网络等仅为举例,其他现有的或今后可能出现的计算机设备或网络如可适用于本发明,也应包含在本发明保护范围以内,并以引用方式包含于此。It should be noted that the user equipment, the network equipment, the network, and the like are merely examples, and other existing or future possible computer equipment or networks, such as those applicable to the present invention, are also included in the scope of the present invention. It is included here by reference.
后面所讨论的方法(其中一些通过流程图示出)可以通过硬件、软件、固件、中间件、微代码、硬件描述语言或者其任意组合来实施。当用软件、 固件、中间件或微代码来实施时,用以实施必要任务的程序代码或代码段可以被存储在机器或计算机可读介质(比如存储介质)中。(一个或多个)处理器可以实施必要的任务。The methods discussed below, some of which are illustrated by flowcharts, can be implemented in hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to carry out the necessary tasks can be stored in a machine or computer readable medium, such as a storage medium. The processor(s) can perform the necessary tasks.
这里所公开的具体结构和功能细节仅仅是代表性的,并且是用于描述本发明的示例性实施例的目的。但是本发明可以通过许多替换形式来具体实现,并且不应当被解释成仅仅受限于这里所阐述的实施例。The specific structural and functional details disclosed are merely representative and are for the purpose of describing exemplary embodiments of the invention. The present invention may, however, be embodied in many alternative forms and should not be construed as being limited only to the embodiments set forth herein.
应当理解的是,虽然在这里可能使用了术语“第一”、“第二”等等来描述各个单元,但是这些单元不应当受这些术语限制。使用这些术语仅仅是为了将一个单元与另一个单元进行区分。举例来说,在不背离示例性实施例的范围的情况下,第一单元可以被称为第二单元,并且类似地第二单元可以被称为第一单元。这里所使用的术语“和/或”包括其中一个或更多所列出的相关联项目的任意和所有组合。It should be understood that although the terms "first," "second," etc. may be used herein to describe the various elements, these elements should not be limited by these terms. These terms are used only to distinguish one unit from another. For example, a first unit could be termed a second unit, and similarly a second unit could be termed a first unit, without departing from the scope of the exemplary embodiments. The term "and/or" used herein includes any and all combinations of one or more of the associated listed items.
应当理解的是,当一个单元被称为“连接”或“耦合”到另一单元时,其可以直接连接或耦合到所述另一单元,或者可以存在中间单元。与此相对,当一个单元被称为“直接连接”或“直接耦合”到另一单元时,则不存在中间单元。应当按照类似的方式来解释被用于描述单元之间的关系的其他词语(例如“处于...之间”相比于“直接处于...之间”,“与...邻近”相比于“与...直接邻近”等等)。It will be understood that when a unit is referred to as "connected" or "coupled" to another unit, it can be directly connected or coupled to the other unit, or an intermediate unit can be present. In contrast, when a unit is referred to as being "directly connected" or "directly coupled" to another unit, there is no intermediate unit. Other words used to describe the relationship between the units should be interpreted in a similar manner (eg "between" and "directly between" and "adjacent to" Than "directly adjacent to", etc.).
这里所使用的术语仅仅是为了描述具体实施例而不意图限制示例性实施例。除非上下文明确地另有所指,否则这里所使用的单数形式“一个”、“一项”还意图包括复数。还应当理解的是,这里所使用的术语“包括”和/或“包含”规定所陈述的特征、整数、步骤、操作、单元和/或组件的存在,而不排除存在或添加一个或更多其他特征、整数、步骤、操作、单元、组件和/或其组合。The terminology used herein is for the purpose of describing the particular embodiments, The singular forms "a", "an", It is also to be understood that the terms "comprising" and """ Other features, integers, steps, operations, units, components, and/or combinations thereof.
还应当提到的是,在一些替换实现方式中,所提到的功能/动作可以按照不同于附图中标示的顺序发生。举例来说,取决于所涉及的功能/动作,相继示出的两幅图实际上可以基本上同时执行或者有时可以按照相反的顺序来执行。It should also be noted that in some alternative implementations, the functions/acts noted may occur in a different order than that illustrated in the drawings. For example, two figures shown in succession may in fact be executed substantially concurrently or sometimes in the reverse order, depending on the function/acts involved.
下面结合附图对本发明作进一步详细描述。The invention is further described in detail below with reference to the accompanying drawings.
图1是根据本发明一个实施例的基于视频格式的端到端自动驾驶数据的存储方法的流程图。1 is a flow chart of a method of storing end-to-end autopilot data based on a video format, in accordance with one embodiment of the present invention.
结合图1中所示,本实施例所述的基于视频格式的端到端自动驾驶数据的存储方法包括如下步骤:As shown in FIG. 1, the video format-based end-to-end automatic driving data storage method according to this embodiment includes the following steps:
S110、确定视频压缩参数并读取姿态数据;S110. Determine a video compression parameter and read the posture data.
S120、根据所述姿态数据的时间戳顺序读取图像数据;S120. Read image data according to a timestamp sequence of the posture data.
S130、使用所述视频压缩参数将所述图像数据存储为视频文件。S130. Store the image data as a video file using the video compression parameter.
下面对各步骤做进一步详细介绍。The steps are further described in detail below.
步骤S110中,首先确定视频压缩参数。在本实施例中可通过压缩参数对测试数据进行压缩测试,然后根据该压缩测试的压缩率确定为视频压缩参数。其中的压缩参数包括但不限于采用编解码器、帧间分配码率(crf)或颜色空间中的至少一种。In step S110, the video compression parameter is first determined. In this embodiment, the test data may be subjected to a compression test by a compression parameter, and then determined as a video compression parameter according to the compression ratio of the compression test. The compression parameters therein include, but are not limited to, at least one of a codec, an inter-frame allocation code rate (crf), or a color space.
在确定视频压缩参数后,再读取姿态数据。在本实施例中可实时读取预定自动驾驶系统输出的姿态数据,并将该姿态数据存储为带有时间戳的数据序列。After determining the video compression parameters, the gesture data is read. In the present embodiment, the attitude data output by the predetermined automatic driving system can be read in real time, and the posture data is stored as a time-stamped data sequence.
步骤S120中,在读取姿态数据之后,再根据姿态数据的时间戳顺序读取图像数据。在本实施例中可按照该姿态数据的时间戳的顺序读取图像数据,并将该图像数据按照时间戳的顺序存储为图像数据序列。In step S120, after the posture data is read, the image data is sequentially read according to the time stamp of the posture data. In the present embodiment, the image data may be read in the order of the time stamp of the posture data, and the image data may be stored as an image data sequence in the order of the time stamp.
步骤S130中,可使用上述的视频压缩参数将读取的图像数据存储为视频文件。在本实施例中可通过预定视频格式将该图像数据压缩存储为视频文件,该视频文件的每一帧对应为该图像数据中的一幅图像。In step S130, the read image data can be stored as a video file using the video compression parameters described above. In this embodiment, the image data may be compression-stored into a video file by a predetermined video format, and each frame of the video file corresponds to an image in the image data.
可选的,本实施例还将该视频文件按照自动驾驶系统输出的姿态数据的类型存储在预定服务器上。该姿态数据的类型包括但不限于速度数据、转向角数据、路网数据等。Optionally, the embodiment further stores the video file on the predetermined server according to the type of the posture data output by the automatic driving system. The types of gesture data include, but are not limited to, speed data, steering angle data, road network data, and the like.
采用本实施例提出的技术方案,通过将读取的姿态数据按照时间戳的顺序存储为图像数据并存储为视频文件,从而能够减少数据占用的存储空间,并且还能够减少网络I/O的访问量,以建立更好的自动驾驶数据模型,进而提高的深度学习在自动驾驶领域的学习效率。With the technical solution proposed in this embodiment, by storing the read posture data as image data in the order of time stamp and storing it as a video file, the storage space occupied by the data can be reduced, and the access of the network I/O can also be reduced. Volume to build a better autonomous driving data model, which in turn improves the learning efficiency of deep learning in the field of automatic driving.
实施例一Embodiment 1
在本领域的现有技术中,通过将传感器采集的图像存储在HDF5文件中的方式供机器学习和控制软件使用。该方法会导致存储图像的HDF5文件过于庞大,并且会明显增加网络I/O的开销,所以传统的数据采集方法不利于自动驾驶系统的深度学习。In the prior art in the art, the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software. This method will result in an HDF5 file storing images that is too large and will significantly increase the overhead of network I/O, so the traditional data acquisition method is not conducive to deep learning of the automatic driving system.
因此,本实施例提出了又一种基于视频格式的端到端自动驾驶数据的存储方法,结合图2中所示,包括如下步骤:Therefore, this embodiment proposes another storage method for the end-to-end automatic driving data based on the video format. As shown in FIG. 2, the method includes the following steps:
S210、确定视频压缩参数。S210. Determine a video compression parameter.
在选取压缩参数时,可以考虑自动驾驶系统所处的环境。例如在人烟稀少的高速公路上行驶时,由于路两侧的风景变化较少,则可采用颜色空间作为压缩参数,针对雪地、沙漠、森林等地区的颜色特点,将多幅图像数据中的相同颜色进行统一压缩,而只对路面的变化情况进行单独存储。When choosing compression parameters, consider the environment in which the autonomous driving system is located. For example, when driving on a sparsely populated highway, the color space can be used as a compression parameter due to less landscape changes on both sides of the road. For the color characteristics of snow, desert, forest, etc., the same in multiple image data. The color is uniformly compressed, and only the changes in the road surface are stored separately.
又如,在市区中行驶时,可采用帧间分配码率作为压缩参数。通过帧间分配码率能够分析哪些是重要帧,哪些是次要帧,重要帧会得到更多的字节。例如将图像中不动的物体或距离较远的移动的物体设置为次要帧,只有当距离小于阈值的移动物体才会以重要帧的形式出现在压缩参数中。这样既能够给出一种比较清晰的感觉,又能显著减小视频文件的体积,因为通常情况下人眼只会关注移动的物体,而不会去辨认背景。For example, when traveling in an urban area, an inter-frame allocation code rate can be used as a compression parameter. By dividing the code rate between frames, it is possible to analyze which are important frames and which are secondary frames, and important frames get more bytes. For example, an object that is not moving in the image or a moving object that is far away is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame. This can give a clearer feeling and significantly reduce the size of the video file, because usually the human eye only pays attention to the moving object, and does not recognize the background.
S220、读取姿态数据。S220. Read posture data.
自动驾驶系统会在每隔预定时间输出一组姿态数据,该姿态数据通常包括图像数据、速度数据、转向角数据以及路网数据等。本实施例主要读取其中的图像数据。The automatic driving system outputs a set of posture data every predetermined time, and the posture data usually includes image data, speed data, steering angle data, and road network data. This embodiment mainly reads image data therein.
S230、按照姿态数据的时间戳的顺序读取图像数据。S230. The image data is read in the order of time stamps of the posture data.
自动驾驶系统输出的姿态数据都带有时间戳,该时间戳可用于表示姿态数据产生的顺序,按照时间顺序将图像数据存储能够更准确的表征该自动驾驶系统采集的图像。The attitude data output by the autopilot system is time stamped, which can be used to indicate the order in which the gesture data is generated, and the image data storage in a chronological order can more accurately characterize the image acquired by the autopilot system.
因此本实施例将所有的姿态数据按照时间戳的顺序读取图像数据,以保证姿态数据与图像数据的时间戳一致。读取预定数量的图像数据后将这些图像数据存储为数据序列,以供后续步骤调用。Therefore, in this embodiment, all the posture data are read in the order of time stamps to ensure that the posture data is consistent with the time stamp of the image data. After reading a predetermined number of image data, the image data is stored as a data sequence for subsequent steps to be called.
S240、以预定格式将该图像数据生成视频文件。S240. Generate the video file into the video file in a predetermined format.
本实施例采用FFmpeg格式生成视频文件。FFmpeg可用于记录、转换数字音频、视频,并能将这些内容转化为流。FFmpeg不仅能够实现将多幅图像数据压缩生成视频文件,而且能够实现多种视频格式之间的相互转换。另外,对于选定的视频,还能够截取指定时间的缩略图以及获取静态图和动态图。This embodiment generates a video file in the FFmpeg format. FFmpeg can be used to record, convert, digital audio, video, and convert these into streams. FFmpeg can not only compress multiple image data to generate video files, but also convert between multiple video formats. In addition, for selected videos, it is also possible to capture thumbnails of a specified time and acquire still and dynamic images.
根据不同的压缩参数,每次选取用于生成视频文件的图像数据的数量也不相同。当采用颜色空间作为压缩参数时,可选择每次将1万张图像压缩生成24帧/秒的视频文件,该视频文件的长度为7分钟,占用空间一般为20-50M,原始图像占用的存储空间约为1G左右,压缩后的视频文件不仅占用存储空间较小,而且网络I/O开销也较低。The number of image data used to generate a video file is different each time according to different compression parameters. When the color space is used as the compression parameter, it is possible to compress 10,000 images each time to generate a 24 frame/second video file, the length of the video file is 7 minutes, the occupied space is generally 20-50M, and the original image is occupied by storage. The space is about 1G, and the compressed video file not only occupies less storage space, but also has low network I/O overhead.
在本实施例中,将自动驾驶系统输出的姿态数据按照预定压缩参数和视频格式压缩存储为视频文件,能够显著减小姿态数据占用的存储空间,并且还能够保证存储的视频文件的清晰度和完整性,因此可以提高自动驾驶系统的深度学习效率。In this embodiment, the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, therefore, can improve the depth learning efficiency of the automatic driving system.
实施例二Embodiment 2
在本领域的现有技术中,通过将传感器采集的图像存储在HDF5文件中的方式供机器学习和控制软件使用。该方法会导致存储图像的HDF5文件过于庞大,并且会明显增加网络I/O的开销,图像存储还会导致存储的文件过多,不利于编辑和管理,所以传统的数据采集方法不利于自动驾驶系统的深度学习。In the prior art in the art, the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software. This method will cause the HDF5 file to store images to be too large, and will obviously increase the overhead of network I/O. Image storage will also result in too many files being stored, which is not conducive to editing and management, so the traditional data acquisition method is not conducive to automatic driving. Deep learning of the system.
虽然可以通过压缩图像的方式减小占用的存储空间,但是当需要读取这些文件时,还需要额外的解压缩过程,难以提高深度学习的效率。因此,本实施例提出了一种基于视频格式的端到端自动驾驶数据的存储方法,结合图3中所示,包括如下步骤:Although the occupied storage space can be reduced by compressing the image, when these files need to be read, an additional decompression process is required, and it is difficult to improve the efficiency of deep learning. Therefore, this embodiment proposes a storage method for end-to-end automatic driving data based on a video format. As shown in FIG. 3, the method includes the following steps:
S310、确定视频压缩参数并读取姿态数据。S310. Determine a video compression parameter and read the posture data.
在确定视频压缩参数之前,可在测试数据上使用不同的参数,例如编解码器、帧间分配码率、颜色空间等进行测试,对比这些压缩参数的压缩情况及压缩后的图像清晰度。Before determining the video compression parameters, different parameters can be used on the test data, such as codec, inter-frame allocation code rate, color space, etc., to compare the compression of these compression parameters and the image clarity after compression.
在选取压缩参数时,可以考虑自动驾驶系统所处的环境。例如在市区中行驶时,由于公路两侧的建筑、行人、车辆等都很多而且颜色各异,但建筑物都是不动的,而且一部分行人和车辆也是不动的。因此可采用帧间分配码率作为压缩参数。通过帧间分配码率能够分析哪些是重要帧,哪些是次要帧。将图像中不动的物体或距离较远的移动的物体设置为次要帧,只有当距离小于阈值的移动物体才会以重要帧的形式出现在压缩参数中。这样压缩出的图像能够将移动的物体,即对自动驾驶有影像的物体进行突出显示,而其它不动的物体也不会占用较多的存储空间。相比之下,另外两种压缩参数的压缩效果要明显差一些,因此对于市区中的路况,本实施例优选的采用帧间分配码率作为压缩参数。When choosing compression parameters, consider the environment in which the autonomous driving system is located. For example, when driving in an urban area, because there are many buildings, pedestrians, vehicles, etc. on both sides of the road, and the colors are different, the buildings are not moving, and some pedestrians and vehicles are not moving. Therefore, an inter-frame allocation code rate can be used as a compression parameter. By dividing the code rate between frames, it is possible to analyze which are important frames and which are secondary frames. A non-moving object in the image or a moving object farther away from the image is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame. The image thus compressed can highlight a moving object, that is, an object that has an image for automatic driving, and other objects that do not move will not occupy more storage space. In contrast, the compression effects of the other two compression parameters are significantly worse, so for the road conditions in the urban area, the embodiment preferably uses the inter-frame allocation code rate as the compression parameter.
S320、根据姿态数据的时间戳顺序读取图像数据。S320. Read image data according to timestamp order of the posture data.
自动驾驶系统会在每隔预定时间输出一组姿态数据,并且每组姿态数据都带有时间戳,该时间戳可用于表示姿态数据产生的顺序,按照时间顺序将图像数据存储能够更准确的表征该自动驾驶系统采集的图像。The autopilot system outputs a set of pose data every predetermined time, and each set of pose data is time stamped, which can be used to indicate the order in which the pose data is generated, and the image data storage can be more accurately characterized in chronological order. The image captured by the autopilot system.
因此本实施例将所有的姿态数据按照时间戳的顺序读取图像数据,以保证姿态数据与图像数据的时间戳一致。读取预定数量的图像数据后将这些图像数据存储为数据序列,以供后续步骤调用。Therefore, in this embodiment, all the posture data are read in the order of time stamps to ensure that the posture data is consistent with the time stamp of the image data. After reading a predetermined number of image data, the image data is stored as a data sequence for subsequent steps to be called.
S330、以预定格式将该图像数据生成视频文件。S330. Generate the video file by using the image data in a predetermined format.
本实施例采用AVC编码格式将图像数据生成视频文件,该视频文件的码率为208kbps,帧率为14fps,分辨率为448x 336。当采用帧间分配码率作为压缩参数时,将1万张图像压缩生成的视频文件的长度为12分钟,占用空间一般为40-70M,原始图像占用的存储空间约为1G左右,压缩后的视频文件不仅占用存储空间较小,而且网络I/O开销也较低。In this embodiment, the image data is generated into a video file by using an AVC encoding format, and the video file has a code rate of 208 kbps, a frame rate of 14 fps, and a resolution of 448×336. When the inter-frame allocation code rate is used as the compression parameter, the length of the video file generated by compressing 10,000 images is 12 minutes, the occupied space is generally 40-70M, and the storage space occupied by the original image is about 1G, after compression. Video files not only take up less storage space, but also have lower network I/O overhead.
S340、将视频文件所述自动驾驶系统输出的姿态数据的类型存储在预定服务器上。S340. Store the type of the gesture data output by the autopilot system of the video file on a predetermined server.
该姿态数据的类型通常包括速度数据、转向角数据、路网数据等。因此本实施例将姿态数据分为两类进行存储,第一类为动态数据,包括速度数据、转向角数据、机动车数据等,第二来为静态数据,包括建筑物数据、实时路况数据、交通信号灯数据等。按照该分类存储的视频文件易于编辑 和管理,能够提高深度学习的效率。The type of the attitude data generally includes speed data, steering angle data, road network data, and the like. Therefore, in this embodiment, the attitude data is divided into two categories for storage, the first type is dynamic data, including speed data, steering angle data, motor vehicle data, etc., and the second is static data, including building data, real-time road condition data, Traffic signal data, etc. Video files stored in this category are easy to edit and manage, improving the efficiency of deep learning.
在本实施例中,将自动驾驶系统输出的姿态数据按照预定压缩参数和视频格式压缩存储为视频文件,能够显著减小姿态数据占用的存储空间,并且还能够保证存储的视频文件的清晰度和完整性,而且便于编辑和管理,避免额外的解压缩过程,因此可以提高自动驾驶系统的深度学习效率。In this embodiment, the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, and easy to edit and manage, avoiding additional decompression process, thus improving the deep learning efficiency of the autopilot system.
图4是根据本发明一个实施例的基于视频格式的端到端自动驾驶数据的存储装置的框图。4 is a block diagram of a storage device for end-to-end automatic driving data based on a video format, in accordance with one embodiment of the present invention.
结合图4中所示,本实施例所述的基于视频格式的端到端自动驾驶数据的存储装置(以下简称“存储装置”),包括如下装置:As shown in FIG. 4, the video format-based end-to-end automatic driving data storage device (hereinafter referred to as "storage device") includes the following devices:
用于确定视频压缩参数并读取姿态数据的装置(以下简称“压缩读取装置”)410;Means for determining video compression parameters and reading posture data (hereinafter referred to as "compressed reading device") 410;
用于根据所述姿态数据的时间戳顺序读取图像数据的装置(以下简称“图像读取装置”)420;Means for reading image data according to the time stamp of the posture data (hereinafter referred to as "image reading device") 420;
用于使用所述视频压缩参数将所述图像数据存储为视频文件的装置(以下简称“视频生成装置”)430。Means for storing the image data as a video file using the video compression parameter (hereinafter referred to as "video generating device") 430.
下面对各装置做进一步详细介绍。The device will be further described in detail below.
首先通过压缩读取装置410确定视频压缩参数。在本实施例中可由压缩读取装置410通过压缩参数对测试数据进行压缩测试,然后根据该压缩测试的压缩率确定为视频压缩参数。其中的压缩参数包括但不限于采用编解码器、帧间分配码率(crf)或颜色空间中的至少一种。The video compression parameters are first determined by the compressed reading device 410. In the present embodiment, the test data may be subjected to a compression test by the compression reading device 410 through a compression parameter, and then determined as a video compression parameter according to the compression ratio of the compression test. The compression parameters therein include, but are not limited to, at least one of a codec, an inter-frame allocation code rate (crf), or a color space.
在确定视频压缩参数后,再读取姿态数据。在本实施例中可通过压缩读取装置410实时读取预定自动驾驶系统输出的姿态数据,并将该姿态数据存储为带有时间戳的数据序列。After determining the video compression parameters, the gesture data is read. In the present embodiment, the attitude data output by the predetermined automatic driving system can be read in real time by the compression reading device 410, and the posture data is stored as a time-stamped data sequence.
在读取姿态数据之后,再通过图像读取装置420根据姿态数据的时间戳顺序读取图像数据。在本实施例中可由图像读取装置420按照该姿态数据的时间戳的顺序读取图像数据,并将该图像数据按照时间戳的顺序存储为图像数据序列。After the posture data is read, the image data is read by the image reading device 420 in accordance with the time stamp of the posture data. In the present embodiment, the image data may be read by the image reading device 420 in the order of the time stamp of the posture data, and the image data may be stored as an image data sequence in the order of the time stamp.
在读取图像数据之后,可通过视频生成装置430将读取的图像数据存 储为视频文件。在本实施例中可由视频生成装置430通过预定视频格式将该图像数据压缩存储为视频文件,该视频文件的每一帧对应为该图像数据中的一幅图像。After the image data is read, the read image data can be stored as a video file by the video generating device 430. In the present embodiment, the image data may be compression-stored into a video file by the video generating device 430 through a predetermined video format, and each frame of the video file corresponds to one image in the image data.
可选的,本实施例还通过视频生成装置430将该视频文件按照自动驾驶系统输出的姿态数据的类型存储在预定服务器上。该姿态数据的类型包括但不限于速度数据、转向角数据、路网数据等。Optionally, the embodiment further stores the video file on the predetermined server according to the type of the posture data output by the automatic driving system by the video generating device 430. The types of gesture data include, but are not limited to, speed data, steering angle data, road network data, and the like.
采用本实施例提出的技术方案,通过将读取的姿态数据按照时间戳的顺序存储为图像数据并存储为视频文件,从而能够减少数据占用的存储空间,并且还能够减少网络I/O的访问量,以建立更好的自动驾驶数据模型,进而提高的深度学习在自动驾驶领域的学习效率。With the technical solution proposed in this embodiment, by storing the read posture data as image data in the order of time stamp and storing it as a video file, the storage space occupied by the data can be reduced, and the access of the network I/O can also be reduced. Volume to build a better autonomous driving data model, which in turn improves the learning efficiency of deep learning in the field of automatic driving.
实施例三Embodiment 3
在本领域的现有技术中,通过将传感器采集的图像存储在HDF5文件中的方式供机器学习和控制软件使用。该方法会导致存储图像的HDF5文件过于庞大,并且会明显增加网络I/O的开销,所以传统的数据采集方法不利于自动驾驶系统的深度学习。In the prior art in the art, the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software. This method will result in an HDF5 file storing images that is too large and will significantly increase the overhead of network I/O, so the traditional data acquisition method is not conducive to deep learning of the automatic driving system.
因此,本实施例提出了又一种基于视频格式的端到端自动驾驶数据的存储装置,结合图5中所示,包括如下装置:Therefore, this embodiment proposes another storage device for end-to-end automatic driving data based on the video format, as shown in FIG. 5, including the following devices:
用于确定视频压缩参数的装置(以下简称“压缩参数确定装置”)510;Means for determining a video compression parameter (hereinafter referred to as "compression parameter determining device") 510;
用于读取姿态数据的装置(以下简称“姿态数据读取装置”)520;a device for reading posture data (hereinafter referred to as "attitude data reading device") 520;
用于按照姿态数据的时间戳的顺序读取图像数据的装置(以下简称“第一图像数据读取装置”)530;Means for reading image data in order of time stamps of the posture data (hereinafter referred to as "first image data reading device") 530;
用于以预定格式将该图像数据生成视频文件的装置(以下简称“第一视频文件生成装置”)540;Means for generating a video file by using the image data in a predetermined format (hereinafter referred to as "first video file generating device") 540;
在选取压缩参数时,可以考虑自动驾驶系统所处的环境。例如在人烟稀少的高速公路上行驶时,由于路两侧的风景变化较少,则可采用颜色空间作为压缩参数,针对雪地、沙漠、森林等地区的颜色特点,通过压缩参数确定装置510将多幅图像数据中的相同颜色进行统一压缩,而只对路面的变化情况进行单独存储。When choosing compression parameters, consider the environment in which the autonomous driving system is located. For example, when driving on a sparsely populated highway, the color space can be used as a compression parameter due to less landscape changes on both sides of the road. For the color characteristics of snow, desert, forest, etc., the compression parameter determining device 510 will be more The same color in the image data is uniformly compressed, and only the changes in the road surface are separately stored.
又如,在市区中行驶时,可采用帧间分配码率作为压缩参数。通过帧 间分配码率能够分析哪些是重要帧,哪些是次要帧,重要帧会得到更多的字节。例如将图像中不动的物体或距离较远的移动的物体设置为次要帧,只有当距离小于阈值的移动物体才会以重要帧的形式出现在压缩参数中。这样既能够给出一种比较清晰的感觉,又能显著减小视频文件的体积,因为通常情况下人眼只会关注移动的物体,而不会去辨认背景。For example, when traveling in an urban area, an inter-frame allocation code rate can be used as a compression parameter. By assigning a bit rate between frames, it is possible to analyze which are important frames and which are secondary frames, and important frames get more bytes. For example, an object that is not moving in the image or a moving object that is far away is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame. This can give a clearer feeling and significantly reduce the size of the video file, because usually the human eye only pays attention to the moving object, and does not recognize the background.
自动驾驶系统会在每隔预定时间输出一组姿态数据,该姿态数据通常包括图像数据、速度数据、转向角数据以及路网数据等。本实施例主要通过姿态数据读取装置520读取其中的图像数据。The automatic driving system outputs a set of posture data every predetermined time, and the posture data usually includes image data, speed data, steering angle data, and road network data. The present embodiment mainly reads image data therein by the posture data reading device 520.
自动驾驶系统输出的姿态数据都带有时间戳,该时间戳可用于表示姿态数据产生的顺序,按照时间顺序将图像数据存储能够更准确的表征该自动驾驶系统采集的图像。The attitude data output by the autopilot system is time stamped, which can be used to indicate the order in which the gesture data is generated, and the image data storage in a chronological order can more accurately characterize the image acquired by the autopilot system.
因此本实施例通过第一图像数据读取装置530将所有的姿态数据按照时间戳的顺序读取图像数据,以保证姿态数据与图像数据的时间戳一致。读取预定数量的图像数据后将这些图像数据存储为数据序列,以供后续步骤调用。Therefore, in this embodiment, all the posture data are read by the first image data reading device 530 in the order of time stamps to ensure that the posture data coincides with the time stamp of the image data. After reading a predetermined number of image data, the image data is stored as a data sequence for subsequent steps to be called.
本实施例采用FFmpeg格式生成视频文件。FFmpeg可用于记录、转换数字音频、视频,并能将这些内容转化为流。FFmpeg不仅能够实现将多幅图像数据压缩生成视频文件,而且能够实现多种视频格式之间的相互转换。另外,对于选定的视频,还能够截取指定时间的缩略图以及获取静态图和动态图。This embodiment generates a video file in the FFmpeg format. FFmpeg can be used to record, convert, digital audio, video, and convert these into streams. FFmpeg can not only compress multiple image data to generate video files, but also convert between multiple video formats. In addition, for selected videos, it is also possible to capture thumbnails of a specified time and acquire still and dynamic images.
根据不同的压缩参数,每次选取用于生成视频文件的图像数据的数量也不相同。当采用颜色空间作为压缩参数时,可通过第一视频文件生成装置540选择每次将1万张图像压缩生成24帧/秒的视频文件,该视频文件的长度为7分钟,占用空间一般为20-50M,原始图像占用的存储空间约为1G左右,压缩后的视频文件不仅占用存储空间较小,而且网络I/O开销也较低。The number of image data used to generate a video file is different each time according to different compression parameters. When the color space is used as the compression parameter, the first video file generating device 540 can select to compress 10,000 images each time to generate a video file of 24 frames/second, and the length of the video file is 7 minutes, and the occupied space is generally 20 -50M, the original image occupies about 1G of storage space. The compressed video file not only occupies less storage space, but also has lower network I/O overhead.
在本实施例中,将自动驾驶系统输出的姿态数据按照预定压缩参数和视频格式压缩存储为视频文件,能够显著减小姿态数据占用的存储空间,并且还能够保证存储的视频文件的清晰度和完整性,因此可以提高自动驾 驶系统的深度学习效率。In this embodiment, the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, therefore, can improve the depth learning efficiency of the automatic driving system.
实施例四Embodiment 4
在本领域的现有技术中,通过将传感器采集的图像存储在HDF5文件中的方式供机器学习和控制软件使用。该方法会导致存储图像的HDF5文件过于庞大,并且会明显增加网络I/O的开销,图像存储还会导致存储的文件过多,不利于编辑和管理,所以传统的数据采集方法不利于自动驾驶系统的深度学习。In the prior art in the art, the image acquired by the sensor is stored in the HDF5 file for use by machine learning and control software. This method will cause the HDF5 file to store images to be too large, and will obviously increase the overhead of network I/O. Image storage will also result in too many files being stored, which is not conducive to editing and management, so the traditional data acquisition method is not conducive to automatic driving. Deep learning of the system.
虽然可以通过压缩图像的方式减小占用的存储空间,但是当需要读取这些文件时,还需要额外的解压缩过程,难以提高深度学习的效率。因此,本实施例提出了一种基于视频格式的端到端自动驾驶数据的存储装置,结合图6中所示,包括如下装置:Although the occupied storage space can be reduced by compressing the image, when these files need to be read, an additional decompression process is required, and it is difficult to improve the efficiency of deep learning. Therefore, the present embodiment proposes a storage device for end-to-end automatic driving data based on a video format, as shown in FIG. 6, including the following devices:
用于确定视频压缩参数并读取姿态数据的装置(以下简称“压缩及读取装置”)610;Means for determining video compression parameters and reading posture data (hereinafter referred to as "compression and reading device") 610;
用于根据姿态数据的时间戳顺序读取图像数据的装置(以下简称“第二图像数据读取装置”)620;Means for reading image data according to the time stamp of the posture data (hereinafter referred to as "second image data reading device") 620;
用于以预定格式将该图像数据生成视频文件的装置(以下简称“第二视频文件生成装置”)630;Means for generating a video file by using the image data in a predetermined format (hereinafter referred to as "second video file generating device") 630;
用于将视频文件所述自动驾驶系统输出的姿态数据的类型存储在预定服务器上的装置(以下简称“分类存储装置”)640。Means for storing the type of the posture data output by the automatic driving system of the video file on a predetermined server (hereinafter referred to as "classification storage device") 640.
在确定视频压缩参数之前,可通过压缩及读取装置610在测试数据上使用不同的参数,例如编解码器、帧间分配码率、颜色空间等进行测试,对比这些压缩参数的压缩情况及压缩后的图像清晰度。Before determining the video compression parameters, the compression and reading device 610 can use different parameters on the test data, such as codec, inter-frame allocation code rate, color space, etc., to compare the compression and compression of these compression parameters. After the image clarity.
在选取压缩参数时,可以考虑自动驾驶系统所处的环境。例如在市区中行驶时,由于公路两侧的建筑、行人、车辆等都很多而且颜色各异,但建筑物都是不动的,而且一部分行人和车辆也是不动的。因此可采用帧间分配码率作为压缩参数。通过帧间分配码率能够分析哪些是重要帧,哪些是次要帧。将图像中不动的物体或距离较远的移动的物体设置为次要帧,只有当距离小于阈值的移动物体才会以重要帧的形式出现在压缩参数中。这样压缩出的图像能够将移动的物体,即对自动驾驶有影像的物体进行突 出显示,而其它不动的物体也不会占用较多的存储空间。相比之下,另外两种压缩参数的压缩效果要明显差一些,因此对于市区中的路况,本实施例优选的采用帧间分配码率作为压缩参数。When choosing compression parameters, consider the environment in which the autonomous driving system is located. For example, when driving in an urban area, because there are many buildings, pedestrians, vehicles, etc. on both sides of the road, and the colors are different, the buildings are not moving, and some pedestrians and vehicles are not moving. Therefore, an inter-frame allocation code rate can be used as a compression parameter. By dividing the code rate between frames, it is possible to analyze which are important frames and which are secondary frames. A non-moving object in the image or a moving object farther away from the image is set as a secondary frame, and only when the distance is less than the threshold, the moving object appears in the compression parameter as an important frame. The thus compressed image can highlight a moving object, that is, an object that has an image for automatic driving, and other immovable objects do not occupy more storage space. In contrast, the compression effects of the other two compression parameters are significantly worse, so for the road conditions in the urban area, the embodiment preferably uses the inter-frame allocation code rate as the compression parameter.
自动驾驶系统会在每隔预定时间输出一组姿态数据,并且每组姿态数据都带有时间戳,该时间戳可用于表示姿态数据产生的顺序,按照时间顺序将图像数据存储能够更准确的表征该自动驾驶系统采集的图像。The autopilot system outputs a set of pose data every predetermined time, and each set of pose data is time stamped, which can be used to indicate the order in which the pose data is generated, and the image data storage can be more accurately characterized in chronological order. The image captured by the autopilot system.
因此本实施例通过第二图像数据读取装置620将所有的姿态数据按照时间戳的顺序读取图像数据,以保证姿态数据与图像数据的时间戳一致。读取预定数量的图像数据后将这些图像数据存储为数据序列,以供后续步骤调用。Therefore, in this embodiment, all the posture data are read by the second image data reading device 620 in the order of time stamps to ensure that the posture data coincides with the time stamp of the image data. After reading a predetermined number of image data, the image data is stored as a data sequence for subsequent steps to be called.
本实施例通过第二视频文件生成装置630采用AVC编码格式将图像数据生成视频文件,该视频文件的码率为208kbps,帧率为14fps,分辨率为448x 336。当采用帧间分配码率作为压缩参数时,将1万张图像压缩生成的视频文件的长度为12分钟,占用空间一般为40-70M,原始图像占用的存储空间约为1G左右,压缩后的视频文件不仅占用存储空间较小,而且网络I/O开销也较低。In this embodiment, the second video file generating device 630 generates the video file by using the AVC encoding format, and the video file has a code rate of 208 kbps, a frame rate of 14 fps, and a resolution of 448×336. When the inter-frame allocation code rate is used as the compression parameter, the length of the video file generated by compressing 10,000 images is 12 minutes, the occupied space is generally 40-70M, and the storage space occupied by the original image is about 1G, after compression. Video files not only take up less storage space, but also have lower network I/O overhead.
该姿态数据的类型通常包括速度数据、转向角数据、路网数据等。因此本实施例通过分类存储装置640将姿态数据分为两类进行存储,第一类为动态数据,包括速度数据、转向角数据、机动车数据等,第二来为静态数据,包括建筑物数据、实时路况数据、交通信号灯数据等。按照该分类存储的视频文件易于编辑和管理,能够提高深度学习的效率。The type of the attitude data generally includes speed data, steering angle data, road network data, and the like. Therefore, the embodiment stores the posture data into two categories by the classification storage device 640. The first category is dynamic data, including speed data, steering angle data, motor vehicle data, etc., and the second is static data, including building data. , real-time traffic data, traffic signal data, etc. Video files stored in this category are easy to edit and manage, improving the efficiency of deep learning.
在本实施例中,将自动驾驶系统输出的姿态数据按照预定压缩参数和视频格式压缩存储为视频文件,能够显著减小姿态数据占用的存储空间,并且还能够保证存储的视频文件的清晰度和完整性,而且便于编辑和管理,避免额外的解压缩过程,因此可以提高自动驾驶系统的深度学习效率。In this embodiment, the attitude data output by the automatic driving system is compressed and stored as a video file according to a predetermined compression parameter and a video format, which can significantly reduce the storage space occupied by the posture data, and can also ensure the clarity of the stored video file. Integrity, and easy to edit and manage, avoiding additional decompression process, thus improving the deep learning efficiency of the autopilot system.
需要注意的是,本发明可在软件和/或软件与硬件的组合体中被实施,例如,本发明的各个装置可采用专用集成电路(ASIC)或任何其他类似硬件设备来实现。在一个实施例中,本发明的软件程序可以通过处理器执行以实现上文所述步骤或功能。同样地,本发明的软件程序(包括相关的数 据结构)可以被存储到计算机可读记录介质中,例如,RAM存储器,磁或光驱动器或软磁盘及类似设备。另外,本发明的一些步骤或功能可采用硬件来实现,例如,作为与处理器配合从而执行各个步骤或功能的电路。It should be noted that the present invention can be implemented in software and/or a combination of software and hardware. For example, the various devices of the present invention can be implemented using an application specific integrated circuit (ASIC) or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Likewise, the software program (including related data structures) of the present invention can be stored in a computer readable recording medium such as a RAM memory, a magnetic or optical drive or a floppy disk and the like. Additionally, some of the steps or functions of the present invention may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.
对于本领域技术人员而言,显然本发明不限于上述示范性实施例的细节,而且在不背离本发明的精神或基本特征的情况下,能够以其他的具体形式实现本发明。因此,无论从哪一点来看,均应将实施例看作是示范性的,而且是非限制性的,本发明的范围由所附权利要求而不是上述说明限定,因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本发明内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外,显然“包括”一词不排除其他单元或步骤,单数不排除复数。系统权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第一,第二等词语用来表示名称,而并不表示任何特定的顺序。It is apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and the present invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the invention is defined by the appended claims instead All changes in the meaning and scope of equivalent elements are included in the present invention. Any reference signs in the claims should not be construed as limiting the claim. In addition, it is to be understood that the word "comprising" does not exclude other elements or steps. A plurality of units or devices recited in the system claims can also be implemented by a unit or device by software or hardware. The first, second, etc. words are used to denote names and do not denote any particular order.
虽然前面特别示出并且描述了示例性实施例,但是本领域技术人员将会理解的是,在不背离权利要求书的精神和范围的情况下,在其形式和细节方面可以有所变化。这里所寻求的保护在所附权利要求书中做了阐述。While the invention has been shown and described with reference to the embodiments of the embodiments of the invention The protection sought herein is set forth in the appended claims.

Claims (15)

  1. 一种基于视频格式的端到端自动驾驶数据的存储方法,包括:A storage method for end-to-end automatic driving data based on a video format, comprising:
    确定视频压缩参数并读取姿态数据;Determining video compression parameters and reading gesture data;
    根据所述姿态数据的时间戳顺序读取图像数据;Reading image data according to a time stamp sequence of the gesture data;
    使用所述视频压缩参数将所述图像数据存储为视频文件。The image data is stored as a video file using the video compression parameters.
  2. 根据权利要求1所述的方法,确定视频压缩参数的步骤包括:The method of claim 1 wherein the step of determining video compression parameters comprises:
    通过压缩参数对测试数据进行压缩测试,根据所述压缩测试的压缩率确定为所述视频压缩参数。The test data is subjected to a compression test by a compression parameter, and the video compression parameter is determined according to the compression ratio of the compression test.
  3. 根据权利要求2所述的方法,所述压缩参数包括编解码器、帧间分配码率或颜色空间中的至少一种。The method of claim 2, the compression parameter comprising at least one of a codec, an inter-frame allocation code rate, or a color space.
  4. 根据权利要求1所述的方法,读取姿态数据的步骤包括:The method of claim 1 wherein the step of reading the pose data comprises:
    实时读取预定自动驾驶系统输出的姿态数据,并将所述姿态数据存储为带有时间戳的数据序列。The attitude data output by the predetermined automatic driving system is read in real time, and the posture data is stored as a time-stamped data sequence.
  5. 根据权利要求1所述的方法,根据所述姿态数据的时间戳顺序读取图像数据的步骤包括:The method according to claim 1, wherein the step of reading the image data according to the time stamp of the posture data comprises:
    按照所述姿态数据的时间戳的顺序读取图像数据,并将所述图像数据按照所述时间戳的顺序存储为图像数据序列。The image data is read in the order of the time stamp of the pose data, and the image data is stored as an image data sequence in the order of the time stamp.
  6. 根据权利要求1所述的方法,使用所述视频压缩参数将所述图像数据存储为视频文件的步骤还包括:The method of claim 1, the step of storing the image data as a video file using the video compression parameter further comprises:
    使用所述视频压缩参数将所述视频文件按照所述自动驾驶系统输出的姿态数据的类型存储在预定服务器上。The video file is stored on the predetermined server in accordance with the type of the gesture data output by the automatic driving system using the video compression parameter.
  7. 一种基于视频格式的端到端自动驾驶数据的存储装置,包括:A storage device for end-to-end automatic driving data based on a video format, comprising:
    用于确定视频压缩参数并读取姿态数据的装置;Means for determining video compression parameters and reading gesture data;
    用于根据所述姿态数据的时间戳顺序读取图像数据的装置;Means for reading image data in accordance with a time stamp of the gesture data;
    用于使用所述视频压缩参数将所述图像数据存储为视频文件的装置。Means for storing the image data as a video file using the video compression parameters.
  8. 根据权利要求7所述的装置,在所述用于确定视频压缩参数并读取姿态数据的装置中包括:The apparatus according to claim 7, wherein said means for determining a video compression parameter and reading the attitude data comprises:
    用于通过压缩参数对测试数据进行压缩测试,根据所述压缩测试的压缩率确定为所述视频压缩参数的装置。And a device for performing compression testing on the test data by a compression parameter, and determining the video compression parameter according to a compression ratio of the compression test.
  9. 根据权利要求8所述的装置,在所述用于确定视频压缩参数并读取姿态数据的装置中,所述压缩参数包括编解码器、帧间分配码率或颜色空间中的至少一种。The apparatus of claim 8, wherein in the means for determining a video compression parameter and reading the pose data, the compression parameter comprises at least one of a codec, an inter-frame allocation code rate, or a color space.
  10. 根据权利要求7所述的装置,在所述用于确定视频压缩参数并读取姿态数据的装置中还包括:The apparatus according to claim 7, further comprising: in the means for determining a video compression parameter and reading the attitude data:
    用于实时读取预定自动驾驶系统输出的姿态数据,并将所述姿态数据存储为带有时间戳的数据序列的装置。The means for reading the attitude data output by the predetermined automatic driving system in real time and storing the posture data as a time-stamped data sequence.
  11. 根据权利要求7所述的装置,在所述用于根据所述姿态数据的时间戳顺序读取图像数据的装置中包括:The apparatus according to claim 7, wherein the means for sequentially reading image data according to a time stamp of the gesture data comprises:
    按照所述姿态数据的时间戳的顺序读取图像数据,并将所述图像数据按照所述时间戳的顺序存储为图像数据序列Reading image data in the order of time stamps of the posture data, and storing the image data as an image data sequence in the order of the time stamps
  12. 根据权利要求7所述的装置,在所述用于使用所述视频压缩参数将所述图像数据存储为视频文件的装置中包括:The apparatus according to claim 7, wherein said means for storing said image data as a video file using said video compression parameter comprises:
    用于使用所述视频压缩参数将所述视频文件按照所述自动驾驶系统输出的姿态数据的类型存储在预定服务器上的装置。Means for storing the video file on a predetermined server in accordance with the type of gesture data output by the automatic driving system using the video compression parameter.
  13. 一种计算机可读存储介质,所述计算机可读存储介质存储有计算机代码,当所述计算机代码被执行时,如权利要求1至6中任一项所述的方法被执行。A computer readable storage medium storing computer code, the method of any one of claims 1 to 6 being executed when the computer code is executed.
  14. 一种计算机程序产品,当所述计算机程序产品被计算机设备执行时,如权利要求1至6中任一项所述的方法被执行。A computer program product, when the computer program product is executed by a computer device, the method of any one of claims 1 to 6 being performed.
  15. 一种计算机设备,所述计算机设备包括:A computer device, the computer device comprising:
    一个或多个处理器;One or more processors;
    存储器,用于存储一个或多个计算机程序;a memory for storing one or more computer programs;
    当所述一个或多个计算机程序被所述一个或多个处理器执行时,使得所述一个或多个处理器实现如权利要求1至6中任一项所述的方法。When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method of any one of claims 1 to 6.
PCT/CN2018/099391 2017-09-05 2018-08-08 Video format-based end-to-end automatic driving data storage method and device WO2019047663A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710792055.4A CN107767486A (en) 2017-09-05 2017-09-05 A kind of storage method and device of the end-to-end automatic Pilot data based on video format
CN201710792055.4 2017-09-05

Publications (1)

Publication Number Publication Date
WO2019047663A1 true WO2019047663A1 (en) 2019-03-14

Family

ID=61265036

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/099391 WO2019047663A1 (en) 2017-09-05 2018-08-08 Video format-based end-to-end automatic driving data storage method and device

Country Status (2)

Country Link
CN (1) CN107767486A (en)
WO (1) WO2019047663A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107767486A (en) * 2017-09-05 2018-03-06 百度在线网络技术(北京)有限公司 A kind of storage method and device of the end-to-end automatic Pilot data based on video format
CN110033780B (en) * 2019-04-07 2020-12-08 西安电子科技大学 Audio and video data transmission method based on FFmpeg and EMIF drive
CN112204975A (en) * 2019-04-29 2021-01-08 百度时代网络技术(北京)有限公司 Time stamp and metadata processing for video compression in autonomous vehicles
JP2022045521A (en) * 2020-09-09 2022-03-22 トヨタ自動車株式会社 Information management system, and mobile terminal and image management server used therein

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8126276B2 (en) * 2001-02-21 2012-02-28 International Business Machines Corporation Business method for selectable semantic codec pairs for very low data-rate video transmission
CN104284233A (en) * 2009-10-19 2015-01-14 鹰图公司 Data search, parser and synchronization of video and telemetry data
US20160173883A1 (en) * 2014-12-16 2016-06-16 Sean J. Lawrence Multi-focus image data compression
CN106060468A (en) * 2016-06-23 2016-10-26 乐视控股(北京)有限公司 Video collection device, video transmission system and video transmission method
CN107767486A (en) * 2017-09-05 2018-03-06 百度在线网络技术(北京)有限公司 A kind of storage method and device of the end-to-end automatic Pilot data based on video format

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8126276B2 (en) * 2001-02-21 2012-02-28 International Business Machines Corporation Business method for selectable semantic codec pairs for very low data-rate video transmission
CN104284233A (en) * 2009-10-19 2015-01-14 鹰图公司 Data search, parser and synchronization of video and telemetry data
US20160173883A1 (en) * 2014-12-16 2016-06-16 Sean J. Lawrence Multi-focus image data compression
CN106060468A (en) * 2016-06-23 2016-10-26 乐视控股(北京)有限公司 Video collection device, video transmission system and video transmission method
CN107767486A (en) * 2017-09-05 2018-03-06 百度在线网络技术(北京)有限公司 A kind of storage method and device of the end-to-end automatic Pilot data based on video format

Also Published As

Publication number Publication date
CN107767486A (en) 2018-03-06

Similar Documents

Publication Publication Date Title
US20210160556A1 (en) Method for enhancing resolution of streaming file
WO2019047663A1 (en) Video format-based end-to-end automatic driving data storage method and device
WO2017005149A1 (en) Game acceleration method and device
JP5478047B2 (en) Video data compression pre-processing method, video data compression method and video data compression system using the same
CN103024437B (en) Video data integrity detection method
CN102231820B (en) Monitoring image processing method, device and system
CN116188821B (en) Copyright detection method, system, electronic device and storage medium
WO2023179161A1 (en) Video frame rate control method and apparatus, and electronic device and storage medium
US20230086804A1 (en) Selectively identifying data based on motion data from a digital video to provide as input to an image processing model
Balchandani et al. A deep learning framework for smart street cleaning
CN113688839B (en) Video processing method and device, electronic equipment and computer readable storage medium
US10924637B2 (en) Playback method, playback device and computer-readable storage medium
CN112182289B (en) Data deduplication method and device based on Flink frame
KR102130076B1 (en) Method for improving the resolution of streaming files based on the learning importance of feature areas
Singla et al. HEVC based tampered video database development for forensic investigation
US11042274B2 (en) Extracting demonstrations from in-situ video content
CN106203244B (en) A kind of determination method and device of lens type
KR101212845B1 (en) Method And System For Sampling Moving Picture
CN109886234B (en) Target detection method, device, system, electronic equipment and storage medium
CN112347996A (en) Scene state judgment method, device, equipment and storage medium
CN117176979B (en) Method, device, equipment and storage medium for extracting content frames of multi-source heterogeneous video
KR101174176B1 (en) Method And System For Sampling Moving Picture
KR102130078B1 (en) System for changing AI parameters based on resolution improvement
KR102130077B1 (en) System to improve the resolution based on pattern information of grid generation
CN104574343A (en) Method and equipment for extracting images from videos

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18852969

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC , EPO FORM 1205A DATED 05.08.2020.

122 Ep: pct application non-entry in european phase

Ref document number: 18852969

Country of ref document: EP

Kind code of ref document: A1