KR20220165316A - Method and system for collecting large amounts of data in autonomous vehicles - Google Patents

Abstract

Disclosed are a method for collecting large amounts of data in an autonomous vehicle and a system thereof. The system for collecting the large amounts of the data in the autonomous vehicle of the present invention comprises: a control unit which transmits data related to autonomous driving to a conversion/transmission unit; and the conversion/transmission unit which creates a file using the received data, generates meta data related to the created file, and transmits the meta data to a server.

Description

Method and system for collecting large amounts of data in autonomous vehicles}

The present invention relates to a method and system for collecting large amounts of data in an autonomous vehicle.

Autonomous vehicles have a plurality of sensors such as lidar and cameras. In addition, in an autonomous vehicle, various control data may be generated in addition to sensing data obtained through a sensor. These data are also stored in the form of files by the controller of the autonomous vehicle. The above file is a large-capacity data file, and is important not only in the development stage of a typical autonomous driving technique, but also in providing commercialized services such as an autonomous shuttle or robotaxi. When a problem occurs, the time for service interruption can be minimized only when large data files are quickly and accurately analyzed. In other words, fast data collection and the ability to collect data without omission are required. To realize this, the generated files need to be collected from the server as quickly as possible through a wireless network.

However, since a very large number of self-driving vehicles may exist on the road, there is a practical difficulty in transmitting all large files to a server through a limited bandwidth. In order to solve this situation, a new data collection method should be proposed, not simply a method of transmitting large files over a wireless network.

In addition, all data generated by an autonomous vehicle may not always be stored in the form of a file. Therefore, it is necessary to ensure that only data at an important point in time, advantageously in the autonomous driving mode, is saved as a file.

The present invention aims to address the aforementioned needs and/or problems.

In addition, an object of the present invention is to implement a method and system for collecting large amounts of data in an autonomous vehicle.

In order to achieve the above object, an autonomous vehicle of the present invention includes a control unit that transmits data related to autonomous driving to a conversion/transmission unit; and a conversion/transmission unit that generates a file using the received data, generates meta data related to the created file, and transmits the meta data to a server. The data related to the autonomous driving are data for checking the autonomous driving state.

The conversion and transmission unit checks the size of the data received from the control unit, and if the size exceeds a preset threshold, deletes old data from the buffer, shifts the remaining data, stores the received data in the buffer, and receives data from the control unit. The size of the recorded data is checked, and if the size is less than a preset threshold, the data recorded in the buffer is shifted without deleting the data recorded in the buffer, and then the received data is stored in the buffer.

When receiving data including a storage start command from the controller, the conversion/transmission unit creates and opens a new file, and records the current data and the data previously stored in the memory into the file.

The conversion control unit receives data from the control unit after receiving the data including the save start command and before receiving the save end command and records the data in the first file, and if the size of the first file exceeds the threshold value, The recording of the first file is terminated, and data is recorded by creating a new second file. The conversion control unit checks the size of the file being recorded when the data including the save end command is received, terminates the file when the size of the file exceeds a threshold value, records additional data in the file, and stores meta data for the file. is generated and sent to the monitoring server.

The meta data includes information related to file size, creation time, and release of the autonomous driving mode. The meta data is for configuring the display screen of the data list, and the arrangement position on the display screen is changed based on at least one of the file size, creation time, and information related to release of the autonomous driving mode.

The autonomous driving mode may be released based on a dangerous situation, and the control unit may include at least one of brake pressure, accelerator pedal pressure, steering angle, steering torque value, autonomous driving release button, autonomous driving release command, vehicle position, and vehicle speed. A probability value of a dangerous situation is calculated based on , and a dangerous situation is determined based on the calculated probability value.

The control unit transmits and stores data to the conversion transmission unit through a local connection.

The conversion/transmission unit stores data in synchronization with the data storage in the control unit, and stores the data in such a way as to add extra data before and after the storage section of the control unit.

Data transmission in response to the user request is performed using the GHz band, and data transmission other than data transmission in response to the user request is performed by wire or using the mmWave band at a specific location, and the specific location is a gas station or garage , is one of the electric vehicle charging stations.

A method for collecting large amounts of data from an autonomous vehicle includes transmitting data related to autonomous driving from a control unit to a conversion/transmission unit; generating a file using the received data by the conversion transmission unit; Generating meta data related to the created file and transmitting it to the monitoring server. Data related to autonomous driving are data for checking the state of autonomous driving.

The conversion transmission unit checks the size of the data received from the control unit, and if the size exceeds a preset threshold, the old data is deleted from the buffer, the remaining data is shifted, and the received data is stored in the buffer. If it is less than the set threshold, the received data is stored in the buffer after shifting the remaining data without deleting the data recorded in the buffer.

When receiving data including a storage start command from the controller, the conversion/transmission unit creates and opens a new file, and records the current data and the data previously stored in the memory into the file.

The conversion control unit receives data from the control unit after receiving the data including the storage start command and before receiving the storage end command and records the data in a first file, and if the size of the first file exceeds the threshold value, the first file size exceeds the threshold value. The recording of the first file is finished, and a new second file is created to record the data.

When the conversion control unit receives the data including the save end command, the size of the file being recorded is checked, and if the size of the file exceeds the threshold, the file is terminated, additional data is recorded in the file, and meta data for the file is determined. Generates data and transmits it to the monitoring server.

The present invention also includes a computer system-readable recording medium on which a program for executing the method in a computer system is recorded.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide examples of the present invention and, together with the detailed description, describe the technical features of the present invention.
1 is a flowchart of a method for collecting large-volume data according to an embodiment of the present invention.
FIG. 2 is a flowchart for explaining a detailed operation when data is received from the control unit before receiving a storage start command in S10 of FIG. 1 .
FIG. 3 is a flowchart for explaining a detailed operation when data including a storage start command is received from the control unit in S10 of FIG. 1 .
FIG. 4 is a flowchart for explaining a detailed operation when data is received from the controller before receiving a storage end command in S10 of FIG. 1 .
FIG. 5 is a flowchart for explaining a detailed operation when data including a storage end command is received from the control unit in S10 of FIG. 1 .
6 illustrates a format of metadata for a file in which data related to autonomous driving is recorded, and FIG. 7 illustrates a format of a file in which data related to autonomous driving is recorded.
FIG. 8 is a diagram showing an implementation example of a list of data related to autonomous driving in S40 of FIG. 1 .
FIG. 9 is a diagram showing another implementation example of a list of data related to autonomous driving in S40 of FIG. 1 .
10 is a diagram for exemplarily explaining a method for utilizing mmWave when transmitting an urgent file through a wireless network.

Hereinafter, the embodiments disclosed in the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the embodiments disclosed in the present invention, the detailed description will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present invention, the technical idea disclosed in the present invention is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

1 is a flow chart of a method for large-volume data collection according to an embodiment of the present invention. Each step can be concretely realized by the control unit and the conversion/transmission unit included in the vehicle.

The controller transmits data related to autonomous driving to the conversion/transmission unit (S10). The conversion transmission unit creates a file using data related to autonomous driving (S20). The conversion transmission unit transmits metadata about the created file to the server before transmitting the file created using the data related to autonomous driving to the server (S30). The server creates a list of the files based on metadata (S40). In response to receiving the file request (S50) from the server, the conversion transmission unit transmits the requested file to the server (S60).

A detailed description of each step will be described with reference to the following drawings.

FIG. 2 is a flowchart for explaining a detailed operation when data is received from the control unit before receiving a storage start command in S10 of FIG. 1 .

The conversion/transmission unit stores data in synchronization with the control unit, but may also store data for a certain period of time prior to the start command of the control unit in a file. This is to facilitate the analysis of the data included in the file later. The conversion/transmission unit may temporarily store data prior to the save command in a memory in order to store data for a predetermined time prior to the save start command together in a file.

Specifically, in response to receiving data from the controller, the conversion/transmission unit checks the size of data accumulated in the buffer (S11a). If the size of data accumulated in the buffer exceeds a set threshold, the conversion/transmission unit deletes the oldest data among data accumulated in the buffer and shifts the data stored in the buffer (S12a: YES, S13a). Also, if the size of the data accumulated in the buffer is less than or equal to the set threshold, the conversion/transfer unit shifts the data stored in the buffer without deleting the data accumulated in the buffer (S12a: NO, S14a). The conversion transmission unit stores the received data in the buffer after shifting the data stored in the buffer (S15a).

Meanwhile, these steps are repeatedly performed even after data is received from the control unit and written to a file. In other words, there are cases in which the file is immediately terminated shortly after the start of writing to the file. Under any circumstances, even when a file is created, a part of the file can always be appended to the front of the file.

FIG. 3 is a flowchart for explaining a detailed operation when data including a storage start command is received from the control unit in S10 of FIG. 1 .

Upon receiving a storage start command from the control unit, the conversion/transmission unit creates a new file and records the received data and the data previously stored in the buffer into the file.

Specifically, in response to receiving data including a storage start command from the control unit, the conversion/transmission unit creates and opens a new file (S11b). The conversion transmission unit creates and records the header of the opened file (S12b). The conversion and transmission unit records the data stored in the buffer together with the received data in the open file (S13b). In one embodiment, to describe S13b in more detail, the conversion and transmission unit may write the data in the memory buffer to an open file, and then write the data received together with the save start command to the open file. .

FIG. 4 is a flowchart for explaining a detailed operation when data is received from the controller before receiving a storage end command in S10 of FIG. 1 .

If the size of the file is too large, it takes an excessively long time to load the file when analyzing data related to autonomous driving included in the file. Accordingly, a file must be created with a size smaller than or equal to a preset size. If the save end command is not received, the conversion/transmission unit checks the size of the file and continues to record the received data in the file only if the size of the file is less than a preset threshold, otherwise, after creating and opening a new file, , writes the corresponding data to a newly opened new file.

Specifically, the conversion/transmission unit terminates the file being recorded when the size of the file being recorded exceeds a preset threshold, and creates and opens a new file (S11c: YES, S12c). Also, if the size of the file being recorded is less than or equal to a preset threshold, the conversion/transmission unit records the received data for the corresponding file (S11c: NO, S13c).

FIG. 5 is a flowchart for explaining a detailed operation when data including a storage end command is received from the control unit in S10 of FIG. 1 .

The conversion/transmission unit further records data of a predetermined size into a file even after receiving a storage end command from the control unit, thereby facilitating analysis of a problem situation or the like.

Specifically, in response to receiving a storage end command from the control unit, the conversion/transmission unit checks the size of the additional storage data received together with the storage end command, and if the size of the additional storage data exceeds a preset threshold, the file being recorded is deleted. It ends (S11d: YES, S12d). If the size of the additional storage data is less than a preset threshold value, the conversion/transmission unit records the additional storage data in the open file instead of ending the file being recorded (S11d: NO, S13d). On the other hand, after the file is finished, the conversion/transmission unit generates meta data related to the file whose recording is finished (S14d). In addition, the conversion transmission unit transmits the generated meta data to the monitoring server (S15d). For reference, the monitoring server refers to a server for receiving data related to autonomous driving and monitoring an abnormal state of autonomous driving using the received data.

6 illustrates a format of metadata for a file in which data related to autonomous driving is recorded, and FIG. 7 illustrates a format of a file in which data related to autonomous driving is recorded.

Referring to FIG. 6, the format of metadata includes at least one of a metadata header, a data file name, a data file creation date, a data file size (size), whether or not the autonomous driving mode is canceled, a reason for releasing the autonomous driving mode, and an abnormal state probability. do.

Referring to FIG. 7 , the format of a data file includes a file header, a first frame payload header, a first frame payload, a second frame payload header, and a second frame payload.

On the other hand, the first frame payload header has information such as the payload length of the corresponding frame, and the first frame payload is recognition, determination, and control data, and includes information on an autonomous driving mode, a storage start command, and a storage end command. include

The first frame payload refers to data received from the control unit at t=n, for example. The second frame payload, for example, refers to data received from the controller at t=n+ (transmission period of data related to autonomous driving, eg, 100 ms).

FIG. 8 is a diagram showing an implementation example of a list of data related to autonomous driving in S40 of FIG. 1 .

The monitoring server receives metadata about a large-capacity file related to autonomous driving that is transmitted whenever the data conversion/transmission unit in the autonomous vehicle completes file generation. In addition, the monitoring server manages a list of files of all data generated in the self-driving vehicle based on the received metadata. Newly received files are added to the list and can be displayed in the form of a list. As such, the developer of the self-driving service can check/inspect files downloaded to his/her computing device or directly uploaded to the server through the server.

Each item of the list may be determined or changed based on the received metadata. For example, each item in the list may include at least one of a large data file name, creation date, data size, whether or not the autonomous driving mode is disabled, description, reception request status, collection path, and file reception progress status. It can be set differently based on the received meta data.

In general, files can be transferred to a server in more than one way. One or more methods may include, for example, a method of transmitting through a wireless network, a method of recording in an external storage device and transmitting to a server, and a method of transmitting through a wired network. The server displays the file transfer method in a list, and manages the files by classifying them differently for each collection path.

Descriptions of large data file names can be registered in the list. The descriptions indicate information transmitted by including them in meta data in conjunction with separate software, or operators testing autonomous driving register situations through the web. , or finally, after a remote autonomous driving developer analyzes the file, it can be registered or modified.

Meanwhile, in the case of an urgent file, it may be preferentially transmitted to a server through a wireless network. A file requiring urgency refers to a file classified as, for example, an autonomous driving mode is released. Files requiring urgency are preferentially transmitted to the server through a wireless network, preferably first. This will allow self-driving developers to quickly identify critical files and resolve issues.

In this way, by managing files in which data related to autonomous driving is recorded in the server in the form of a list, autonomous driving developers can quickly receive and review files that require urgency, as well as properly manage data without omission. .

FIG. 9 is a diagram showing another implementation example of a list of data related to autonomous driving in S40 of FIG. 1 .

Unlike the list of FIG. 8 , the list of FIG. 9 further includes release probabilities due to risk. The release probability by risk is a value between 0% and 100%.

According to an embodiment of FIG. 8 , when it is confirmed that the autonomous driving mode is released through meta data for a specific file, such a file can be regarded as a file for which urgency of analysis is required. However, it cannot be said that the release information of the autonomous driving mode is a completely urgent file. For example, when a driver gets on board and conducts an autonomous driving test, there may be cases where the driver feels the vehicle's behavior is strange and steps on the steering, accelerator, or brake pedal to release the autonomous driving mode, but simply ends the test. This is because there may be cases where the self-driving mode is released after doing so. An indicator to distinguish them is required.

According to another embodiment according to FIG. 9 , 'release probability due to risk' may be additionally included in the list mentioned in FIG. 8 . The release probability due to danger refers to the probability that the autonomous driving mode is in an abnormal state, that is, represents a probability value that the autonomous driving mode is released due to danger. This probability value may be calculated through patterns such as a steering angle of a steering wheel, steering speed, and brake pedal pressure that may appear in an emergency situation. A release probability due to danger may be calculated from input information consisting of at least one of a steering angle of a steering wheel, a steering speed, and brake pedal pressure using a rule-based or deep learning-based prediction module. For reference, the deep learning-based prediction module has one or more hidden layers, and may be composed of a neural network model having an input layer and an output layer, and the weights between the nodes constituting each layer are the input information and the input information Annotation It is learned using training data composed of labeled information. Learning means that the weight is updated to an appropriate value through a predetermined learning cycle.

Meanwhile, as information that is the basis for determining the release probability due to the risk, not only the steering angle, steering speed, and brake pedal pressure of the steering wheel, but also the current vehicle speed and current vehicle position may be considered. For example, if the current speed is low or the current location of the vehicle is within a predetermined distance from a pre-input place (eg, a laboratory), it may be determined that the test has ended normally.

10 is a diagram for exemplarily explaining a method for utilizing mmWave when transmitting an urgent file through a wireless network.

As described above in FIG. 8, files requiring urgency are transmitted to the server through a wireless network with high priority. In this case, according to one embodiment, the urgent file may be transmitted using a band of 6 GHz or less, that is, a sub-6 GHz band. In the 3.5GHz frequency band, if a certain base station is built, there is no problem in transmitting the necessary files to the server. Such high-priority files need to be transmitted through a wireless network of a band below 6 GHz upon request, and the remaining low-priority files can be transferred or transmitted to the server in other ways, such as being moved to an external storage device.

However, if the number of self-driving vehicles increases significantly in the future, there is a limit to transmitting the remaining low-priority files using the above-described external storage device or through a wired network. Therefore, according to an embodiment of the present invention, a method of transmitting files with low priority by utilizing the mmWave band is proposed. However, since mmWave requires a number of base stations to be installed and LOS conditions must be supported, it is difficult to transmit in reality. Therefore, the method of transmitting using mmWave is used in specific locations where LOS is guaranteed, such as garages or electric vehicle charging stations or gas stations. It can be. When the self-driving vehicle arrives at a specific location, such as the garage, electric vehicle charging station, or gas station, the self-driving vehicle can transmit low-priority files using the mmWave band.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet). Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (21)

In autonomous vehicles,
a controller that transmits data related to autonomous driving to a conversion/transmission unit;
a conversion/transmission unit that creates a file using the received data, generates meta data related to the created file, and transmits the meta data to a server;
Including, self-driving vehicle.
According to claim 1,
Data related to autonomous driving is data for checking the state of autonomous driving, an autonomous vehicle.
According to claim 1,
The conversion transmission unit,
Checking the size of the data received from the controller, and if the size exceeds a preset threshold, deletes old data from the buffer, shifts the remaining data, and stores the received data in the buffer;
An autonomous vehicle that checks the size of the data received from the controller and, if the size is less than a preset threshold, shifts the remaining data without deleting the data recorded in the buffer and then stores the received data in the buffer.
According to claim 1,
When receiving data including a storage start command from the control unit, the conversion transmission unit creates and opens a new file, and writes the current data and the data previously stored in the memory to the file.
According to claim 4,
Conversion control unit,
After receiving the data including the storage start command and before receiving the storage end command, data is received from the control and recorded in the first file, and if the size of the first file exceeds the threshold, the first file An autonomous vehicle that ends recording for and records data by creating a new second file.
According to claim 5,
Conversion control unit,
When the data including the save end command is received, the size of the file being recorded is checked, and if the size of the file exceeds the threshold, the file is terminated, additional data is recorded in the file, and metadata for the file is created. Self-driving vehicle that transmits to the monitoring server.
According to claim 1,
The metadata includes information related to file size, creation time, and release of the autonomous driving mode.
According to claim 7,
The meta data is for configuring the display screen of the data list, and the arrangement position on the display screen is changed based on at least one of the file size, creation time, and information related to release of the autonomous driving mode. Characterized in that, self-driving vehicle.
According to claim 8,
Autonomous driving mode can be released based on dangerous situations,
The control unit calculates a probability value of a dangerous situation based on at least one of brake pressure, accelerator pedal pressure, steering angle, steering torque value, autonomous driving release button, autonomous driving release command, vehicle position, and vehicle speed, and the calculated A self-driving vehicle characterized in that a dangerous situation is determined based on a probability value.
According to claim 1,
An autonomous vehicle characterized in that the control unit transmits and stores data by local connection to the conversion transmission unit.
According to claim 1,
Characterized in that the conversion transmission unit stores data in synchronization with data storage in the control unit, an autonomous vehicle.
According to claim 11,
Characterized in that the conversion and transmission unit stores data in a manner of adding redundant data before and after the storage section of the control unit, the self-driving vehicle.
According to claim 1,
Data transmission in response to a user request is performed using a GHz band, and data transmission other than data transmission in response to a user request is performed by wire or using a mmWave band at a specific location.
According to claim 13,
Characterized in that the specific location is one of a gas station, a garage, and an electric vehicle charging station, an autonomous vehicle.
In a method for collecting large amounts of data from an autonomous vehicle,
Transmitting data related to autonomous driving from a controller to a conversion/transmission unit;
generating a file using the received data by the conversion transmission unit;
Generating meta data related to the created file and transmitting it to the monitoring server;
Including, a method for collecting data.
According to claim 14,
Data related to autonomous driving is data for checking the state of autonomous driving, a method for collecting data.
According to claim 14,
The conversion and transmission unit checks the size of the data received from the control unit, and if the size exceeds a preset threshold, deletes old data from the buffer, shifts the remaining data, and stores the received data in the buffer,
The conversion transmission unit checks the size of the data received from the control unit, and if the size is less than a preset threshold, shifts the remaining data without deleting the data recorded in the buffer and stores the received data in the buffer. way to collect.
According to claim 14,
A method for collecting data, wherein, when receiving data including a storage start command from the controller, the conversion/transmission unit creates and opens a new file, and records current data and data previously stored in a memory into the file.
According to claim 17,
Conversion control unit,
After receiving the data including the storage start command and before receiving the storage end command, data is received from the control and recorded in the first file, and if the size of the first file exceeds the threshold, the first file A method for collecting data, which ends recording for and records data by creating a new second file.
According to claim 18,
Conversion control unit,
When the data including the save end command is received, the size of the file being recorded is checked, and if the size of the file exceeds the threshold, the file is terminated, additional data is recorded in the file, and metadata for the file is created. A method for collecting data, which is transmitted to the monitoring server.
A computer system-readable recording medium in which a program for executing the method according to any one of claims 17 to 19 in a computer system is recorded.

Images