CN117880185A

CN117880185A - Data analysis method for automatic driving vehicle

Info

Publication number: CN117880185A
Application number: CN202410086084.9A
Authority: CN
Inventors: 刘灿; 曾超; 牟剑秋; 许正昊; 陈子健; 李涛; 张显宏; 衡量
Original assignee: Shanghai Youdao Zhitu Technology Co Ltd
Current assignee: Shanghai Youdao Zhitu Technology Co Ltd
Priority date: 2024-01-22
Filing date: 2024-01-22
Publication date: 2024-04-12

Abstract

The invention discloses a data analysis method of an automatic driving vehicle, which is based on a Protobuf serialized ROS2 data packet, carries out topic release in a self-defined msg data structure form by creating a message relay forwarding node, then recognizes, receives and analyzes self-defined msg data by configuring a MATLAB environment, and then carries out visual presentation and preliminary analysis on the analyzed data by a GUI interface or a drawing script; and meanwhile, the analyzed data is imported into a control model built in the Simulink, the control module is recharged and debugged, and the problem investigation is completed.

Description

Data analysis method for automatic driving vehicle

Technical Field

The invention belongs to the technical field of automatic driving, relates to vehicle control, and in particular relates to a data analysis method of an automatic driving vehicle.

Background

Autopilot systems are generally divided into four basic modules, sensing, predicting, planning and controlling. The control module is a part directly connected and communicated with the vehicle, and all instruction requests which need to act on the vehicle in the intelligent driving system are required to be processed by the control module and then issued to the vehicle end. In the automatic driving system, the environment platform form operated by the control module can be in the SoC end as well as the planning, and can also be operated in the MCU end, and the data flow interaction between the planning and the control is completed by means of bottom software. When at the SoC end, the strategy development of the control module is usually based on ROS2 (Robot Operating System, ROS 2) platform, using the c++/Python development language; when the scheme of the control module is developed at the MCU end, MATLAB is the most commonly used development tool for developers, and is also widely applied to the automatic driving departments of a plurality of large-scale automobile enterprises at present.

When testing problem analysis and debugging result verification are carried out on an automatic driving system developed based on ROS2, a method is generally adopted, wherein the analysis and the treatment are carried out by means of rviz/rqt data visualization tools. rviz can integrate with ROS2 systems, directly subscribe to and display ROS 2's message data, view models, data, map lines, etc. in three dimensions. In addition, the rosbag instruction can be used for printing parameters in the data packet in the terminal interface for analysis and verification.

For a control module, upstream and downstream of the planning module and the vehicle itself, it may not be necessary to pay excessive attention to parameters issued by other modules other than the planning and vehicle itself, which are not in direct communication with the control. The main functions of the control are to control the transverse and longitudinal movement, and the accuracy and stability of input parameters, the calculation rationality of a movement control strategy, the post-processing effect of a control instruction and the like are the parts of control comparison attention. The use of rivz2 in the control module is therefore relatively redundant and not sufficiently compact.

When debugging the control module internal logic and functions, when paying attention to some or a large amount of intermediate amount which is not directly output as output quantity, the ROS2 interaction interface needs to be modified or a plurality of output prints need to be added. The operation process is complicated, and the printed parameters are not intuitive.

Rviz2 can create some overhead on system performance when handling large amounts of data or complex scenarios. This can affect real-time, especially on embedded systems where resources are limited.

More custom plug-ins and functionality may be required in some specific application scenarios, while the rviz2 version may not fully meet all of these requirements, while rviz2 has a powerful plug-in system, it may be relatively cumbersome to configure and create custom plug-ins for some highly customized visualization requirements.

For some special data types or visualization requirements, the built-in functionality of rviz2 has some limitations, and the developer needs additional work to achieve a specific visualization effect.

Disclosure of Invention

Aiming at the problems, the main purpose of the invention is to design a data analysis method of an automatic driving vehicle, which solves the problems of complex operation, insufficient simplification and complex configuration and creation of customized plug-ins on visual requirements in the process of debugging a control module by converting the ROS2 data structure type and matching with a visual interface in MATLAB and a control model built in a Simulink dynamic simulation system.

The invention adopts the following technical scheme for realizing the purposes:

a data analysis method of an autonomous vehicle, comprising the steps of:

step 1: based on the Protobuf serialized ROS2 data packet, broadcasting and publishing are carried out through a broadcasting node;

step 2: the method comprises the steps that topic release is carried out on data of a playing node in a self-defined msg data structure mode in a mode of creating a message relay forwarding node;

step 3: configuring an MATLAB environment, and identifying, receiving and analyzing the msg data customized in the step 2 through the MATLAB;

step 4: performing visual presentation and preliminary analysis on the data analyzed in the step 3 through a visual interface in MATLAB;

step 5: importing the elimination data analyzed in the step 3 into a control model built in a Simulink dynamic simulation system, and recharging and debugging the control module;

step 6: and (5) integrating the visual result of the step (4) and the recharging and debugging result of the step (5) to perform the problem investigation of the automatic driving vehicle.

As a further description of the present invention, topic publication is performed by creating a message relay forwarding node, including the following steps:

s201: creating an msg file based on a proto file before data serialization, and defining a structure type for accepting data after deserialization through the created msg file;

the proto file and the msg file are compiled in a combined mode in an ROS2 environment to generate a header file which can be called;

s202: setting a relay forwarding node name and a cycle execution period of the relay forwarding node, and setting topics which the relay forwarding node needs to subscribe and publish in the execution period based on the header file, wherein the topics comprise topic names and data structure types of messages in the topics;

s203: assigning data in topic messages subscribed to the received proto data structure type to topic messages of the msg data structure type in a one-to-one correspondence manner and publishing the topic messages;

s204: in the compiling environment, compiling the compiled whole program, executing the compiled program, and starting the established relay forwarding node to forward the message.

As a further description of the present invention, the relay forwarding node includes forwarding a proto data structure type message to an msg data structure type message in ROS2 data, and forwarding the msg data structure type message to the proto data structure type message, so as to implement data closed loop.

As a further description of the present invention, forwarding of a Proto data structure type message to an msg data structure type message in ROS2 data includes:

the relay forwarding node subscribes to receive the play node data in the ROS2 data packet, and then reissues the play node data in the ROS2 data packet according to a predefined msg data structure form after the play node data is deserialized;

the play node data in the ROS2 data packet includes: perception information, decision information, planning information, and vehicle information.

As a further description of the present invention, the forwarding of msg data structure type messages to proto data structure type messages includes:

the relay forwarding node subscribes to receive the message in the form of msg data structure and forwards the message in the form of Protobuf serialized proto data structure for release.

As a further description of the present invention, the configuration of the MATLAB environment includes compiling a software version and setting an adaptation language thereof, where the MATLAB supports communication with the relay forwarding node;

wherein the MATLAB comprises an ROS Toolbox, and the ROS Toolbox is integrated in the Simulink dynamic simulation system.

As a further description of the present invention, the ROS Toolbox communicates with the relay forwarding node using custom-written MATLAB nodes as an API interface function.

As further described in the invention, a publishing and subscribing module is created in the Simulink dynamic simulation system, ROS2 topic messages are subscribed and published, and debugging analysis is performed by means of the Simulink dynamic simulation system;

the method comprises the steps of receiving a message in the form of an msg data structure issued by an ROS2 node except MATLAB, inputting the message into a control model of a Simulink dynamic simulation system, and verifying the behavior of the message through simulation to realize data recharging and problem checking verification of the control model.

As a further description of the invention, the MATLAB constructs a visual interface through a GUI tool box or writing an m script, and the visual interface comprises a graph of a bicycle state and a graph of a intelligent driving instruction.

Compared with the prior art, the invention has the technical effects that:

the invention provides a data analysis method of an automatic driving vehicle, which is based on a ROS2 data packet after Protobuf serialization, carries out topic release in a self-defined msg data structure mode by creating a message relay forwarding node, then recognizes, receives and analyzes self-defined msg data by configuring a MATLAB environment, and then carries out visual presentation and preliminary analysis on the analyzed data by a GUI interface or a drawing script; and meanwhile, the analyzed data is imported into a control model built in the Simulink, the control module is recharged and debugged, and the problem investigation is completed.

Drawings

FIG. 1 is a schematic flow chart of the overall method of the present invention;

fig. 2 is a message forwarding schematic diagram of a relay forwarding node in the present invention;

FIG. 3 is a schematic diagram of an example visual interface layout display of the present invention;

FIG. 4 is a diagram of a data recharging framework in the Simulink dynamic simulation system of the present invention;

FIG. 5 is a schematic diagram of MATLAB environment configuration references of the present invention;

FIG. 6 is a schematic diagram of the primary function of constructing ROS2 nodes in MATLAB of the present invention;

fig. 7 is a schematic diagram showing MATLAB command line results of the present invention.

Detailed Description

The invention is described in detail below with reference to the attached drawing figures:

in one embodiment of the present invention, a data analysis method for an autonomous vehicle is disclosed, referring to fig. 1 to 7, comprising the steps of:

ROS2 is an open source framework and software development platform for building robotic applications, employing data distribution services (Data Distribution Service, DDS), provides more flexible and efficient distributed communications, and supports multi-node communications such that different nodes can be distributed across different physical or virtual machines, thereby better accommodating the building of complex systems. ROS2 also provides a new set of debug and test tools, including rqt, ROS2cli, test frames, etc., to help developers better debug and verify.

In ROS2, a Message (msg) is a structured data format for transferring data between ROS nodes, and the definition of a Message is typically stored in files in the format ". Msg", which describe the data type and structure of the Message, including the data type and name of the field.

ROS2 communicates through Topic, service or Action three types of interfaces using an interface definition language (Interface Definition Language, IDL). The most common communication method is Topic communication. A Node (Node) transmits and receives a message through communication between a Publisher (Publisher) and a Subscriber (Subscriber), the Publisher publishes the message to a specified topic, and the Subscriber receives the message from the topic.

In ROS2, data between topics of communication can be saved in packets in the format ". Db3" by instructions, and experiments can be reproduced off-line, and without limitation of actual hardware equipment, which is very helpful for locating and repairing problems and verifying the performance of algorithms in different scenarios.

As mentioned above, the data type in ROS2 is generally defined in an msg file, and in addition, a Protocol buffer (Protocol Buffers) may be used to define the data type. Protobuf uses a syntax similar to a structure to define a data structure, and may write a ". Proto" file in a variety of languages such as c++, java, python, go, etc. to define a message structure including field names, field types, and other meta information, etc.

Compared with msg files, protobuf can add or change field types without destroying backward compatibility, and supports iteration, evolution and update of a message structure.

The Protobuf encoder can sequence structured data into binary format, and the sequence data is compact, lighter and more efficient than msg, and is particularly suitable for network communication and a scene for storing a large amount of structured data. However, the Protobuf-serialized data has poor readability, and is basically not directly used in the observation of ROS2 data packets. The use of msg documents may be more intuitive and may also be easier to integrate for certain specific ROS2 functions and tools.

Specifically, in the embodiment, in the step 2, topic distribution is performed by creating a message relay forwarding node, which includes the following steps:

From the above, it can be seen that the binary format data after Protobuf serialization is lighter and more efficient, and is suitable for communication and mass storage, but the readability of the data after data serialization is poor, and the binary format data can not be directly used in the data printing of ROS2 basically; and MATLAB only supports message type definition using msg file format, it cannot be directly used in MATLAB. The relay forwarding node is an ROS2 node, and the ROS2 has friendly support and compatibility to MATLAB, so that the MATLAB can communicate with the relay forwarding node.

The relay forwarding node comprises the step of forwarding the proto data structure type message to the msg data structure type message in the ROS2 data, and the step of forwarding the msg data structure type message to the proto data structure type message, so that the data closed loop is realized.

Specifically, in this embodiment, forwarding the Proto data structure type message to the msg data structure type message in the ROS2 data includes:

the relay forwarding node subscribes to receive the play node data in the ROS2 data packet, and then reissues the play node data in the reverse serialization mode according to a predefined msg data structure form.

Forwarding of msg data structure type messages to proto data structure type messages, comprising:

As can be seen from the foregoing, in this embodiment, a Message relay forwarding node may be created based on ROS2 using an adapted programming language such as c++ or python, where the node may first subscribe to receive the Protobuf-serialized data, then reverse-serialize the data, and then re-issue the data in msg form according to the data type in the msg file that is predefined and adapted, so as to complete forwarding from Protobuf type Message in ROS2 to Message. Similarly, the relay forwarding node can also receive msg type information and forward the msg type information into a Protobuf serialized data format for release, so that data closed loop is realized. The overall flow is shown in fig. 2.

In fig. 2, a ROSbag playing node is a node for broadcasting and publishing data in an ROS2 data packet, where the playing node data in the ROS2 data packet includes: perception information, decision information, planning information, vehicle information, and other message data. The data are serialized proto data, are internally processed into msg type data and are published, and are subscribed for use and message feedback by other nodes with use requirements.

In the embodiment, in step 3, the parsed data may be stored in MATLAB working space in the format of ". Mat", which is a data format built in MATLAB, and both the m script and the Simulink dynamic simulation system may be directly invoked.

In this embodiment, a message of the ROS2 custom data type is identified, and an environment configuration needs to be performed on MATLAB, where the configuration of MATLAB environment includes compiling a software version and setting of an adaptation language thereof, taking MATLAB r2021b as an example, and the environment needing to be configured is as shown in fig. 5. The function supporting the recognition of the ROS2 custom type in the MATLAB is 'ROS 2 genemsg', and after the custom msg is created, the function is operated to complete the recognition of the MATLAB on the custom data type;

in the command line interface of MATLAB, the data types contained in the topic message currently supported by MATLAB can be checked using the "ros2 msg list" instruction, and the result of the instruction displays the list shown in fig. 7. The message type in the figure is actually a built-in MATLAB, or a standard msg message type in ROS2, so long as it is listed in the list, MATLAB can subscribe to or publish the message topic of that data type.

The above mentions that the serialized proto message is forwarded via the relay node as a message of predefined adapted msg data structure type. These ROS2 standard message types can be used directly to socket and forward these data. However, in practical engineering application, the proto data structure defined by the developer according to the actual situation may be complex in type, various and nested layer by layer, and at this time, the standard message is used for forwarding, which is not beneficial to data analysis, and the management and maintenance of the module content may be more complicated. Thus, the corresponding msg file may be created and defined to adapt forwarding in the form of a proto data structure. Because these msg documents are defined by the developer's own creation, MATLAB needs to identify such custom msg message types and display them in a list for communication of custom msg type messages.

In addition, the MATLAB includes an ROS Toolbox, and the ROS Toolbox is integrated in the Simulink dynamic simulation system, so that the ROS communication system may be constructed in a more intuitive module form, and a node may be created using the ROS Toolbox, and this node may be used to issue and configure parameters of a node block, a name of a designated node, a node type, a communication manner of a node, and the like, which are published and fixed to an ROS2 message.

Specifically, in this embodiment, the ROS Toolbox enables the MATLAB to publish and subscribe ROS2 messages, call ROS2 services, and communicate with ROS2 nodes, which provides some convenience for integrating algorithms running in the MATLAB with the ROS2 system, and the ROS Toolbox communicates with the API interface function of the relay forwarding node, and uses the custom-written MATLAB node to complete communication with the relay forwarding node. The function of the MATLAB node is to subscribe and receive the topic message of the msg data structure type published by the relay forwarding node, and then store the topic message into the working space of the MATLAB, wherein the working space of the MATLAB is also a subscribing node with the MATLAB. The functions used mainly to construct ROS2 nodes in MATLAB are shown in fig. 6.

In this embodiment, a publish and subscribe module is created in the Simulink dynamic simulation system described above: subscribe and public, MATLAB completes the identification of the custom data types, and a control module is built by using Simulink, so that ROS2 topic messages are subscribed and released in the control model, and debugging analysis (such as a data checker, step debugging and the like) is carried out by means of a Simulink dynamic simulation system, so that data recharging and problem checking verification of the control model are rapidly realized, as shown in FIG. 4.

It should also be noted that if the control model is set to be real-time simulation, the control model can subscribe and process topic messages in real time; the ROS data packet module can also be directly used for directly loading the existing ROS2 data packet, and the data types in the ROS2 data packet must be all data types which can be identified by MATLAB, otherwise, the ROS2 data packet cannot be loaded.

In addition, in the embodiment, a message in the form of an msg data structure issued by an ROS2 node other than MATLAB is received and is input into a control model of a Simulink dynamic simulation system, and the behavior of the control model is verified through simulation, so that the data recharging and the problem checking verification of the control model are realized. In addition, in the Simulink dynamic simulation system, there are plentiful debugging tools and visual interfaces to monitor the information of the observed topic messages and the logic and functions of the control model of the Simulink dynamic simulation system. When the control model meets the performance requirement after debugging, the executable code which can run on the target hardware can be generated by using a Simulink code or an Embedded code, and the code can be regarded as a node and can be deployed on an actual hardware system.

In this embodiment, after the relay forwarding node is created to send and receive the message, and the MATLAB is configured to identify the message of the custom msg data type, and the receiving node is created in the MATLAB, the data in the ROS2 data packet can be received and visually displayed. The MATLAB can construct a visual interface through a GUI tool box or writing m scripts, including a graph of the vehicle state and a graph of the intelligent driving instruction, as shown in fig. 3.

In particular, the development environment GUIDE is a visual design tool of GUI toolkits for creating and editing GUIs. Through GUIDE, the user can drag and drop controls, such as buttons, text boxes, check boxes, list boxes, sliders, graphical windows, etc., and set their properties, and then generate MATLAB codes through simple operations. These controls may also be customized through a visual interface or code to meet the needs of a particular application. Various events in the GUI (e.g., button clicks, numerical changes, etc.) define callback functions that trigger when a user interacts with the GUI, performing the corresponding operations, thereby implementing the functions of the application.

App Designer is another GUI design tool for creating more complex GUI applications. It provides a visual editor for designing the layout and behavior of an application program, and can render data into curves, scatter plots, images, etc. through a visual component. A basic example of a simple visual display is shown in fig. 4, which includes the position of the vehicle itself and other obstacles and the relative position of the vehicle, the relative motion state, the display of key parameters, and the plotting of important signals, etc., and in fact, the GUI interface may be more rich in elements.

The MATLAB m script drawing function is very powerful, and the m script file is used for writing codes, so that various patterns including graphs, scatter diagrams, bar charts, contour diagrams and the like can be drawn, and the creation of nodes and the subscription and release of messages can be synchronously integrated

Through the content of the embodiment, a specific technical scheme of the invention is disclosed, and compared with the prior art, the invention has the following advantages:

1. the method supports the analysis of the serialized data, has strong adaptability, can customize the message types and the visual display interface, and has high flexibility.

Application specific: custom ROS messages enable a user to define message types that exactly match their application requirements; in this way, users can more accurately represent and communicate their data without being limited by standard message types;

data structure flexibility: the user may define messages containing various data types and structures, including scalars, arrays, nested structures, and the like; this flexibility allows users to better adapt to their data formats and to integrate with other systems more easily;

increasing readability: the use of custom messages may make the communication more readable. The field names and structures of the messages can reflect the meaning of the data, so that the ROS communication is more visual when being read and understood;

simplifying data processing: by defining the custom message, the user can more easily process and parse the data; the field structure of the message can be mapped to the data structure of the user program, so that the processing and operation of the data are more visual;

code reusability is improved: the user can share custom messages among multiple ROS nodes, thereby improving code reusability. By using the same custom messages in different ROS packages, a user can more easily build a modular and scalable system;

support special sensors and hardware: in some cases, special sensors or hardware may output non-standard data types. By customizing the ROS messages, users can easily adapt to these non-standard data types, making it easier for ROS systems to integrate various hardware and sensors;

language independence: custom ROS messages are a standard way of ROS communication, regardless of the programming language used to implement the ROS node. This language independence allows easy communication between ROS nodes written in different languages.

And the MATLAB script language and the GUI configuration are simple, and the use and maintenance cost is low.

The method is user-friendly: the GUI provides an intuitive graphical interface allowing a user to interact with a program through a mouse and a keyboard without having to memorize and input complex commands. This makes MATLAB more suitable for engineers, scientists and researchers who are not familiar with programming;

and (3) visualization: MATLAB GUI makes data visualization and graphical presentation easier. The user can draw charts, graphs and various visual elements through simple drag-and-drop operation or controls on the graphical interface, and a large number of codes are not required to be written;

interactivity: through the GUI, the user may interact directly with graphics, charts, and control elements, such as dragging a slider, clicking a button, or selecting an item in a list. This provides the user with the ability to feed back and adjust parameters in real time, thereby making it easier to understand and adjust the behavior of the program;

simplifying complex tasks: the GUI allows complex tasks to be accomplished in a number of simple steps. By dividing the functionality into modules, the user can complete the task step by step without having to know the underlying technical details;

error reduction: using the GUI may reduce the likelihood of user input errors. By providing visualization elements and options, the GUI reduces the risk of user error, reducing the occurrence of errors;

customizability: MATLAB GUI provides a wide range of customizability, and users can design and adjust interfaces according to their own needs. The GUI can adapt to various application scenes, so that the flexibility of MATLAB is improved;

deployment and sharing: with MATLAB's deployment tool, users can deploy their GUI applications as standalone desktop applications, web applications, or embedded into other systems. This enables users to easily share and distribute their applications without the need for a license for MATLAB.

The above embodiments are only for illustrating the technical solution of the present invention, but not for limiting, and other modifications and equivalents thereof by those skilled in the art should be included in the scope of the claims of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A data analysis method of an autonomous vehicle, comprising the steps of:

2. A data analysis method of an autonomous vehicle according to claim 1, characterized in that: topic release is carried out by creating a message relay forwarding node, and the method comprises the following steps:

3. A data analysis method of an autonomous vehicle according to claim 2, characterized in that: the relay forwarding node comprises the step of forwarding the proto data structure type message to the msg data structure type message in the ROS2 data, and the step of forwarding the msg data structure type message to the proto data structure type message, so that the data closed loop is realized.

4. A data analysis method of an autonomous vehicle according to claim 3, characterized in that: forwarding of Proto data structure type messages to msg data structure type messages in ROS2 data, comprising:

5. A data analysis method of an autonomous vehicle according to claim 3, characterized in that: forwarding of msg data structure type messages to proto data structure type messages, comprising:

6. A data analysis method of an autonomous vehicle according to claim 1, characterized in that: the configuration of the MATLAB environment comprises compiling software version and setting of an adaptation language thereof, wherein the MATLAB supports communication with the relay forwarding node;

7. A data analysis method of an autonomous vehicle according to claim 6, wherein: and the API interface function of the ROS Toolbox for communicating with the relay forwarding node uses a custom-written MATLAB node to complete the communication with the relay forwarding node.

8. A data analysis method of an autonomous vehicle according to claim 6, wherein: the system comprises a Simulink dynamic simulation system, a publishing and subscribing module, a ROS2 topic message subscribing and publishing module and a debugging analysis module, wherein the Simulink dynamic simulation system is used for debugging analysis;

9. A data analysis method of an autonomous vehicle according to claim 1, characterized in that: and constructing a visual interface in the MATLAB through a GUI tool box or writing an m script, wherein the visual interface comprises a graph of a vehicle state and a graph of an intelligent driving instruction.