WO2019047607A1

WO2019047607A1 - Data processing method and device for end-to-end automatic driving system

Info

Publication number: WO2019047607A1
Application number: PCT/CN2018/094494
Authority: WO
Inventors: 闫泳杉; 郁浩; 郑超; 唐坤; 张云飞; 姜雨
Original assignee: 百度在线网络技术（北京）有限公司
Priority date: 2017-09-05
Filing date: 2018-07-04
Publication date: 2019-03-14
Also published as: CN107808360B; CN107808360A

Abstract

Provided by the present invention are a data processing method and device for an end-to-end automatic driving system, wherein the method comprises: converting a plurality of images acquired in real time into a pre-determined resolution, and then storing the images in a Hierarchical Data Format version 5 (HDF5) file; storing in the HDF5 file the Coordinated Universal Time of a predetermined navigation system, a Gaussian projection corresponding to a speed value extracted from the predetermined navigation system and a curvature value corresponding to Global Positioning System (GPS) data extracted from the predetermined navigation system. By means of storing in an HDF5 file images having a reduced resolution, the Coordinated Universal Time of a navigation system, a Gaussian projection corresponding to a speed value and a curvature value corresponding to GPS data, the present invention may store a large amount of data by using a smaller storage space so as to establish a better automatic driving data model, and thereby increase the learning efficiency of deep learning in the field of automatic driving.

Description

Data processing method and device for end-to-end automatic driving system

This patent application claims to be submitted on September 5, 2017, the application number is 201710792451.7, the applicant is Baidu Online Network Technology (Beijing) Co., Ltd., and the invention name is "a data processing method and device for an end-to-end automatic driving system" Priority of the Chinese Patent Application, the entire contents of which is hereby incorporated by reference.

Technical field

The present invention relates to the field of computers, and in particular, to a data processing method and apparatus for an end-to-end automatic driving system.

Background technique

With the rapid development of deep learning and the in-depth study of artificial intelligence, the automotive industry has undergone revolutionary changes. Automated driving through end-to-end deep learning is a major research direction in the field of automatic driving. In the prior art, an automatic driving system generally uses a model established by data acquired in real time in front, output steering angle, and speed to perform deep learning. The more data collected, the more favorable the generated model is for deep learning. Since the number of images collected in real time in front of the automatic driving system is large and difficult to store in a limited storage space, the development of deep learning in the field of automatic driving is limited.

Summary of the invention

One of the technical problems solved by the present invention is that the number of images collected in real time in front of the existing automatic driving system is large and difficult to store in a limited storage space.

According to an embodiment of an aspect of the present invention, a data processing method for an end-to-end automatic driving system is provided, including:

Converting a plurality of real-time acquired images into a predetermined resolution and storing them in an HDF5 file;

The coordinated world time of the predetermined navigation system, the Gaussian projection corresponding to the speed value extracted by the predetermined navigation system, and the curvature value corresponding to the GPS data extracted by the predetermined navigation system are stored in the HDF5 file.

According to an embodiment of another aspect of the present invention, a data processing apparatus for an end-to-end automatic driving system is provided, including:

Means for converting a plurality of real-time acquired images into a predetermined resolution and storing them in an HDF5 file;

Means for storing the coordinated world time of the predetermined navigation system, the Gaussian projection corresponding to the speed value extracted by the predetermined navigation system, and the curvature value corresponding to the GPS data extracted by the predetermined navigation system in the HDF5 file.

Since the image of the reduced resolution, the coordinated world time of the navigation system, the Gaussian projection corresponding to the velocity value, and the curvature value corresponding to the GPS data are all stored in the HDF5 file, the present embodiment can store a large amount of data with less storage space. In order to establish a better automatic driving data model, and thus improve the learning efficiency of deep learning in the field of automatic driving.

Those skilled in the art will appreciate that although the following detailed description is made with reference to the illustrated embodiments and drawings, the invention is not limited to these embodiments. Rather, the scope of the invention is intended to be limited the scope of the invention

DRAWINGS

Other features, objects, and advantages of the present invention will become more apparent from the Detailed Description of Description

1 is a flow chart showing a data processing method of an end-to-end automatic driving system in accordance with an embodiment of the present invention.

FIG. 2 is a flow chart showing a data processing method of the end-to-end automatic driving system according to Embodiment 1 of the present invention.

FIG. 3 is a flow chart showing a data processing method of the end-to-end automatic driving system according to Embodiment 2 of the present invention.

4 is a block diagram of a data processing apparatus of an end-to-end automatic driving system in accordance with an embodiment of the present invention.

Fig. 5 is a block diagram showing the data processing apparatus of the end-to-end automatic driving system proposed in the third embodiment of the present invention.

Fig. 6 is a block diagram showing a data processing apparatus of the end-to-end automatic driving system proposed in the fourth embodiment of the present invention.

The same or similar reference numerals in the drawings denote the same or similar components.

Detailed ways

Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as a process or method depicted as a flowchart. Although the flowcharts describe various operations as a sequential process, many of the operations can be implemented in parallel, concurrently or concurrently. In addition, the order of operations can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the figures. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

By "computer device", also referred to as "computer" in the context, is meant an intelligent electronic device that can perform predetermined processing, such as numerical calculations and/or logical calculations, by running a predetermined program or instruction, which can include a processor and The memory is executed by the processor to execute a predetermined process pre-stored in the memory to execute a predetermined process, or is executed by hardware such as an ASIC, an FPGA, a DSP, or the like, or a combination of the two. Computer devices include, but are not limited to, servers, personal computers, notebook computers, tablets, smart phones, and the like.

The computer device includes a user device and a network device. The user equipment includes, but is not limited to, a computer, a smart phone, a PDA, etc.; the network device includes but is not limited to a single network server, a server group composed of multiple network servers, or a cloud computing based computer Or a cloud composed of a network server, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. Wherein, the computer device can be operated separately to implement the present invention, and can also access the network and implement the present invention by interacting with other computer devices in the network. The network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like.

It should be noted that the user equipment, the network equipment, the network, and the like are merely examples, and other existing or future possible computer equipment or networks, such as those applicable to the present invention, are also included in the scope of the present invention. It is included here by reference.

The methods discussed below, some of which are illustrated by flowcharts, can be implemented in hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to carry out the necessary tasks can be stored in a machine or computer readable medium, such as a storage medium. The processor(s) can perform the necessary tasks.

The specific structural and functional details disclosed are merely representative and are for the purpose of describing exemplary embodiments of the invention. The present invention may, however, be embodied in many alternative forms and should not be construed as being limited only to the embodiments set forth herein.

It should be understood that although the terms "first," "second," etc. may be used herein to describe the various elements, these elements should not be limited by these terms. These terms are used only to distinguish one unit from another. For example, a first unit could be termed a second unit, and similarly a second unit could be termed a first unit, without departing from the scope of the exemplary embodiments. The term "and/or" used herein includes any and all combinations of one or more of the associated listed items.

It will be understood that when a unit is referred to as "connected" or "coupled" to another unit, it can be directly connected or coupled to the other unit, or an intermediate unit can be present. In contrast, when a unit is referred to as being "directly connected" or "directly coupled" to another unit, there is no intermediate unit. Other words used to describe the relationship between the units should be interpreted in a similar manner (eg "between" and "directly between" and "adjacent to" Than "directly adjacent to", etc.).

The terminology used herein is for the purpose of describing the particular embodiments, The singular forms "a", "an", It is also to be understood that the terms "comprising" and """ Other features, integers, steps, operations, units, components, and/or combinations thereof.

It should also be noted that in some alternative implementations, the functions/acts noted may occur in a different order than that illustrated in the drawings. For example, two figures shown in succession may in fact be executed substantially concurrently or sometimes in the reverse order, depending on the function/acts involved.

The invention is further described in detail below with reference to the accompanying drawings.

1 is a flow chart of a data processing method of an end-to-end automatic driving system in accordance with one embodiment of the present invention.

As shown in FIG. 1, the data processing method of the end-to-end automatic driving system described in this embodiment includes the following steps:

S110. Convert multiple images acquired in real time into a predetermined resolution and store them in an HDF5 file.

S120. Store the coordinate time value corresponding to the coordinated world time of the predetermined navigation system, the Gaussian projection corresponding to the speed value extracted from the predetermined navigation system, and the GPS data extracted from the predetermined navigation system in the HDF5 file.

The steps are further described in detail below.

In step S110, the resolution of the original image data is first scaled down to ensure that the data model can be trained normally in a limited storage space. In the present embodiment, including but not limited to, the original image data is scaled down by 1/3, and the reduced image data can be stored in the HDF5 file with the suffix of .h5. And multiple image files can also be stored in an HDF5 file, reducing network I/O traffic.

Optionally, after the reduced image data is stored in the HDF5 file, the image acquisition time in the HDF5 file and the time interval in the predetermined time array may be used as the coordinated world time to satisfy the time accuracy. Moreover, the image RGB data corresponding to the image acquisition time can also be written into the HDF5 file.

In step S120, the attitude data may first be read from the ieout data outputted by the high-precision collection vehicle, and the readable attitude data includes, but is not limited to, GPS week, weekly seconds, speed value, and GPS data.

Among them, the GPS week and the intra-week second can be converted into Coordinated World Time (UTC Time), and the positive east speed value, the north direction speed value, the elevation (H-Ell) value, and the vertical direction in the extracted speed value. The heading values are converted to Gaussian projections, and the GPS data is obtained by the difference calculation to obtain the curvature. Finally, the above-mentioned coordinated world time, Gaussian projection and curvature values are written into the HDF5 file.

Optionally, the embodiment may further construct a data model according to the image transformed image data, the coordinated world time, the Gaussian projection, and the curvature value. The data model is an end-to-end data model that provides effective deep learning material for automated driving systems.

According to the technical solution proposed by the embodiment, the reduced resolution image, the coordinated world time of the navigation system, the Gaussian projection corresponding to the velocity value, and the curvature value corresponding to the GPS data are all stored in the HDF5 file, thereby enabling less The storage space stores a large amount of data to create a better autopilot data model, which in turn improves the learning efficiency of deep learning in the field of autonomous driving.

Embodiment 1

In the prior art in the art, the images acquired by the high precision collection vehicle are stored in the HDF5 file for use by the machine learning and control software. This method will cause the HDF5 file storing images to occupy more storage space, and will significantly increase the overhead of network I/O, so the traditional data processing method is not conducive to deep learning of the automatic driving system.

Therefore, this embodiment proposes another data processing method of the end-to-end automatic driving system, as shown in FIG. 2, comprising the following steps:

S210. Convert the plurality of real-time acquired images into a predetermined resolution and store them in the HDF5 file.

The resolution of the original image data is scaled down by 1/3 to ensure that the data model can be trained normally in a limited storage space. The reduced image data can be stored in an HDF5 file with the .h5 extension. And multiple image files can also be stored in an HDF5 file, reducing network I/O traffic.

S220. Convert the GPS standard time into coordinated world time.

It can be converted to Coordinated Universal Time after reading the GPS standard time. The content to be converted includes GPS week and week seconds, as shown in the following table:

Since the GPS standard time reference is consistent with the Coordinated World Time at 0 o'clock on January 6, 1980, and accumulates the time in seconds after the atomic time, the Coordinated World Time values after the above four sets of time conversions are as follows:

S230. Convert the extracted velocity value into a Gaussian projection.

The speed value may include a positive east speed value, a northward speed value, an elevation value, and an angle value with a vertical direction, and the specific values are as follows:

Gaussian projection of the above four sets of data can be done through the transform function in the Python module. The basic principle of Gaussian projection calculation using Python module is to calculate from latitude and longitude coordinates to projected coordinates. Gaussian projection needs to determine the central longitude and projection ellipsoid parameter information after projection. Since the Python module is based on the scripting language, the Gaussian projection calculation in Python uses the basic function provided by it, that is, the transform function can complete the calculation.

S240. Obtain a curvature of the GPS data by using a difference calculation.

The curvature of the GPS data can be calculated by the following calculation formula:

Where x' denotes the first derivative of y with respect to x, x`` denotes the second derivative of y with respect to x, x' denotes the first derivative of x with respect to y, and x`` denotes the second derivative of x with respect to y.

S250: Write the coordinated world time value, the Gaussian projection value, and the curvature value into the HDF5 file.

The above calculations can obtain four sets of coordinated world time values, Gaussian projection values and curvature values, which can be written into one HDF5 file, and more posture data can also be written into an HDF5 file, which can significantly reduce the storage occupied by the posture data. Space, therefore, can improve the depth learning efficiency of the automatic driving system.

Embodiment 2

In the prior art in the art, the images acquired by the high precision collection vehicle are stored in the HDF5 file for use by the machine learning and control software. This method will cause the HDF5 file to store images to occupy more storage space, and will obviously increase the overhead of network I/O. Image storage will also result in too many stored files, which is not conducive to editing and management, so the traditional data processing method Not conducive to deep learning of the automatic driving system.

Although the occupied storage space can be reduced by compressing the image, when these files need to be read, an additional decompression process is required, and it is difficult to improve the efficiency of deep learning. Therefore, this embodiment proposes a data processing method for an end-to-end automatic driving system, which is combined with FIG. 3, and includes the following steps:

S310. Adjust the original image collected in real time.

The resolution of the original image is usually 960*640, which ensures that the data model can be trained normally in a limited storage space. Each image can be adjusted to a resolution of 320*320, and then the adjusted image is written to .h5. In the HDF5 file with the suffix name.

The time between the image acquisition time and the time array [0.0, 125.0, 250.0, 375.0, 500.0, 625.0, 750.0, 875.0, 1000.0] in the HDF5 file can be used as the coordinated world time, and the image at that time can be written at the same time. RGB data.

S320: Convert the read posture data into the HDF5 file.

The conversion process includes converting the GPS week and the intra-week second into the coordinated world time, and converting the positive east velocity value, the northward velocity value, the elevation value, and the vertical angle value into the Gaussian projection among the extracted velocity values. And the GPS data is obtained by the difference calculation to obtain the curvature. Finally, the above-mentioned coordinated world time value, Gaussian projection value and curvature value are written into an HDF5 file.

When the image file is compressed by the traditional ZIP method, the storage space occupied by 8 thousand image data is about 16 GB, and the storage space occupied by HDF5 storing 100,000 image data is only 15 GB. The image file stored in HDF5 format not only occupies storage. Smaller space and fewer files are stored, which is not only easy to edit and manage, but also has low network I/O overhead.

S330, constructing a data model.

The data collected from the high-precision acquisition vehicle is image-transformed to obtain image data, and then the above-mentioned coordinated world time value, Gaussian projection value and curvature value are used to construct the data model, and the end-to-end data model is modeled. The data model is an end-to-end data model that provides effective deep learning material for automated driving systems.

4 is a block diagram of a data processing apparatus of an end-to-end automatic driving system in accordance with one embodiment of the present invention.

As shown in FIG. 4, the data processing device (hereinafter referred to as "data processing device") of the end-to-end automatic driving system according to this embodiment includes the following devices:

a device (hereinafter referred to as "image conversion device") 410 for converting a plurality of real-time acquired images into a predetermined resolution and storing them in an HDF5 file;

Means for storing a coordinated world time of a predetermined navigation system, a Gaussian projection corresponding to a speed value extracted from the predetermined navigation system, and a curvature value corresponding to GPS data extracted from the predetermined navigation system in the HDF5 file ( Hereinafter referred to as "data storage device" 420.

The device will be further described in detail below.

The resolution of the original image data is first scaled down by the image transforming device 410 to ensure that the data model can be trained normally in a limited storage space. In the present embodiment, the original image data is reduced by a ratio of 1/3 by the image conversion device 410, and the reduced image data may be stored in the HDF5 file with the suffix of .h5. And multiple image files can also be stored in an HDF5 file, reducing network I/O traffic.

Optionally, after the reduced image data is stored in the HDF5 file, the image acquisition time in the HDF5 file and the time closest to the predetermined time array may be used as the coordinated world time by the image conversion device 410. The accuracy of time. Further, the image RGB data at the time corresponding to the image capturing time can be written into the HDF5 file by the image converting device 410.

The gesture data can then be read from the ieout data output from the high-precision collection vehicle by the data storage device 420, including but not limited to GPS weeks, weekly seconds, speed values, and GPS data.

The data storage device 420 can convert the GPS week and the weekly seconds into a coordinated world time (UTC Time), and the forward east speed value, the north direction speed value, and the elevation (H-Ell) value in the extracted speed value. Both the heading and the vertical direction are converted to a Gaussian projection, and the GPS data is obtained by a difference operation to obtain a curvature. Finally, the above-described coordinated world time, Gaussian projection and curvature values are written into the HDF5 file by the data storage device 420.

Optionally, in this embodiment, the data model may be constructed by the model construction device according to the image transformed image data, the coordinated world time, the Gaussian projection, and the curvature value. The data model is an end-to-end data model that provides effective deep learning material for automated driving systems.

Embodiment 3

Therefore, the present embodiment proposes another data processing apparatus of the end-to-end automatic driving system, as shown in FIG. 5, including the following apparatus:

a device (hereinafter referred to as "transition storage device") 510 for converting a plurality of real-time acquired images into a predetermined resolution and storing them in an HDF5 file;

Means for converting GPS standard time into coordinated world time (hereinafter referred to as "time conversion device") 520;

Means for converting the extracted velocity value into a Gaussian projection (hereinafter referred to as "projection conversion device") 530;

Means for obtaining the curvature of the GPS data by the difference operation (hereinafter referred to as "difference operation device") 540;

A device (hereinafter referred to as "data writing device") 550 for writing the coordinated world time value, the Gaussian projection value, and the curvature value into the HDF5 file.

The resolution of the original image data is scaled by a scale of 1/3 by the transform storage device 510 to ensure that the data model can be trained normally in a limited storage space. The reduced image data can be stored in an HDF5 file with the .h5 extension. And multiple image files can also be stored in an HDF5 file, reducing network I/O traffic.

In the process of converting the read GPS standard time into the coordinated world time by the time conversion device 520, the content to be converted includes the GPS week and the week second, as shown in the following table:

Gaussian projection of the above four sets of data through the projection conversion device 530 can be accomplished by a transform function in the Python module. The basic principle of Gaussian projection calculation using Python module is to calculate from latitude and longitude coordinates to projected coordinates. Gaussian projection needs to determine the central longitude and projection ellipsoid parameter information after projection. Since the Python module is based on the scripting language, the Gaussian projection calculation in Python uses the basic function provided by it, that is, the transform function can complete the calculation.

The curvature of the GPS data can be calculated by the difference operation means 540 by the following calculation formula:

Where x' denotes the first derivative of y with respect to x, x`` denotes the second derivative of y with respect to x, x` denotes the first derivative of x with respect to y, and x`` denotes the second derivative of x with respect to y.

The above calculation obtains four sets of coordinated world time values, Gaussian projection values and curvature values which can be written into an HDF5 file by the data writing device 550, and more posture data can also be written into an HDF5 file, which can be significantly reduced. The storage space occupied by the small gesture data can improve the deep learning efficiency of the automatic driving system.

Embodiment 4

Although the occupied storage space can be reduced by compressing the image, when these files need to be read, an additional decompression process is required, and it is difficult to improve the efficiency of deep learning. Therefore, the present embodiment proposes a data processing apparatus for an end-to-end automatic driving system, as shown in FIG. 6, comprising the following means:

Means for adjusting the original image acquired in real time (hereinafter referred to as "adjustment collection device") 610;

a device (hereinafter referred to as "data conversion device") 620 for converting the read posture data into HDF5 file after conversion processing;

A device for constructing a data model (hereinafter referred to as "model construction device") 630.

The resolution of the original image is usually 960*640, which ensures that the data model can be trained normally in a limited storage space. Each image can be adjusted to a resolution of 320*320 by adjusting the acquisition device 610, and then the acquisition device 610 is adjusted. The adjusted image is written to the HDF5 file with the .h5 extension.

The time between the image acquisition time in the HDF5 file and the time array [0.0, 125.0, 250.0, 375.0, 500.0, 625.0, 750.0, 875.0, 1000.0] can be adjusted by the acquisition device 610 as the coordinated world time, and simultaneously written. Enter the image RGB data at that moment.

The GPS week and the intra-week second are converted into the coordinated world time by the data conversion device 620, and the positive east velocity value, the northward velocity value, the elevation value, and the angle value with the vertical direction among the extracted velocity values are converted into Gaussian projection, and the GPS data is obtained by the difference operation to obtain the curvature. Finally, the above-mentioned coordinated world time value, Gaussian projection value and curvature value are written into an HDF5 file.

The data collected from the high-precision acquisition vehicle is image-transformed to obtain image data, and then the model construction device 630 constructs the data model by using the above-mentioned coordinated world time value, Gaussian projection value and curvature value to complete the modeling of the end-to-end data model. . The data model is an end-to-end data model that provides effective deep learning material for automated driving systems.

It should be noted that the present invention can be implemented in software and/or a combination of software and hardware. For example, the various devices of the present invention can be implemented using an application specific integrated circuit (ASIC) or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Likewise, the software program (including related data structures) of the present invention can be stored in a computer readable recording medium such as a RAM memory, a magnetic or optical drive or a floppy disk and the like. In addition, some of the steps or functions of the present invention may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.

It is apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and the present invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the invention is defined by the appended claims instead All changes in the meaning and scope of equivalent elements are included in the present invention. Any reference signs in the claims should not be construed as limiting the claim. In addition, it is to be understood that the word "comprising" does not exclude other elements or steps. A plurality of units or devices recited in the system claims can also be implemented by a unit or device by software or hardware. The first, second, etc. words are used to denote names and do not denote any particular order.

While the invention has been shown and described with reference to the embodiments of the embodiments of the invention The protection sought herein is set forth in the appended claims.

Claims

A data processing method for an end-to-end automatic driving system, comprising:

Converting a plurality of real-time acquired images into a predetermined resolution and storing them in an HDF5 file;

The coordinated world time of the predetermined navigation system, the Gaussian projection corresponding to the speed value extracted from the predetermined navigation system, and the curvature value corresponding to the GPS data extracted from the predetermined navigation system are stored in the HDF5 file.
The method according to claim 1, wherein the step of converting the plurality of real-time acquired images into a predetermined resolution and storing the images in the HDF5 file comprises:

The image acquisition time in the HDF5 file is the closest to the interval in the predetermined time array as the coordinated world time.
The method of claim 2, the step of converting the plurality of real-time captured images into a predetermined resolution and storing the images in the HDF5 file further comprises:

The image RGB data corresponding to the image acquisition time is written in the HDF5 file.
The method of claim 1 wherein the step of determining a coordinated world time of said predetermined navigation system comprises:

The GPS week and the intra-week of the predetermined navigation system are converted to coordinated world time.
The method of claim 1, the step of determining a Gaussian projection corresponding to the velocity value extracted by the predetermined navigation system comprises:

The positive east velocity value, the normal north velocity value, the elevation value, and the angle value with the vertical direction extracted from the predetermined navigation system are both converted into Gaussian projections.
The method according to claim 1, wherein the step of determining a curvature value corresponding to the GPS data extracted by the predetermined navigation system comprises:

The GPS data is obtained by a difference operation to obtain a curvature value.
The method of any one of claims 1 to 6, the method further comprising:

A data model is constructed from the image transformed image data, the coordinated world time, the Gaussian projection, and the curvature value.
A data processing device for an end-to-end automatic driving system, comprising:

Means for converting a plurality of real-time acquired images into a predetermined resolution and storing them in an HDF5 file;

Means for storing a coordinated world time of a predetermined navigation system, a Gaussian projection corresponding to a speed value extracted from the predetermined navigation system, and a curvature value corresponding to GPS data extracted from the predetermined navigation system in the HDF5 file.
The apparatus according to claim 8, wherein the means for storing the plurality of real-time acquired images into the HDF5 file after converting the image to the predetermined resolution comprises:

A device for synchronizing the image acquisition time in the HDF5 file with the time interval in the predetermined time array as the coordinated world time.
The apparatus according to claim 9, further comprising: in the apparatus for storing the plurality of real-time captured images into a predetermined resolution, and storing the information in the HDF5 file:

Means for performing data persistence processing on the data information with a predetermined period and partitioning rules.
The apparatus according to claim 9, wherein said Gaussian projection corresponding to a coordinated world time of a predetermined navigation system, a speed value extracted by said predetermined navigation system, and a curvature value corresponding to GPS data extracted by said predetermined navigation system The device stored in the HDF5 file includes:

Means for converting the GPS week and the week of the predetermined navigation system into coordinated world time.
The apparatus according to claim 9, wherein said Gaussian projection corresponding to a coordinated world time of a predetermined navigation system, a speed value extracted by said predetermined navigation system, and a curvature value corresponding to GPS data extracted by said predetermined navigation system The device stored in the HDF5 file further includes:

Means for converting both the forward east velocity value, the normal north velocity value, the elevation value, and the angle value from the vertical direction extracted from the predetermined navigation system into a Gaussian projection.
The apparatus according to claim 9, wherein said Gaussian projection corresponding to a coordinated world time of a predetermined navigation system, a speed value extracted by said predetermined navigation system, and a curvature value corresponding to GPS data extracted by said predetermined navigation system The device stored in the HDF5 file further includes:

Means for obtaining the curvature value by performing the difference calculation on the GPS data.
The apparatus according to any one of claims 8 to 13, the apparatus further comprising:

Means for constructing a data model based on image transformed image data, the coordinated world time, the Gaussian projection, and the curvature values.
A computer readable storage medium storing computer code, the method of any one of claims 1 to 7 being executed when the computer code is executed.
A computer program product, when the computer program product is executed by a computer device, the method of any one of claims 1 to 7 being performed.
A computer device, the computer device comprising:

One or more processors;

a memory for storing one or more computer programs;

When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method of any one of claims 1 to 7.