KR102601833B1 - Autonomous driving algorithm simulation method and system based on game engine - Google Patents

Abstract

The present invention relates to a method for simulating an autonomous driving algorithm based on a game engine performed in an autonomous driving algorithm simulation system including a simulation device and a lightweight embedded controller, comprising the steps of capturing a specific game screen performed in the game engine and collecting images; Analyzing the collected images and generating an autonomous driving algorithm for autonomous driving of the vehicle based on the analyzed results; Applying the generated self-driving algorithm to a virtual vehicle in a specific game to simulate autonomous driving according to the self-driving algorithm generated in the specific game; and controlling autonomous driving of a vehicle corresponding to the virtual vehicle based on a simulated autonomous driving algorithm. This makes it possible to test algorithms without applying actual physical laws or designing a simulator that represents the real physical world by configuring various obstacles, pedestrians, etc. and having the same characteristics as a real vehicle.

Description

Autonomous driving algorithm simulation method and system based on game engine {AUTONOMOUS DRIVING ALGORITHM SIMULATION METHOD AND SYSTEM BASED ON GAME ENGINE}

The present invention relates to a method and system for simulating an autonomous driving algorithm based on a game engine, and more specifically, to a method and system for simulating an autonomous driving algorithm based on a game engine that can test autonomous driving without directly designing a simulator similar to the real world. It's about.

Recently, as interest in autonomous driving has increased, various research on autonomous vehicles is being conducted. For autonomous driving of vehicles, objects such as people, animals, other vehicles, and traffic lights must be accurately identified, and after identification, the vehicle must be placed in a set state. This should work.

In addition, as it is an autonomous vehicle, the vehicle must automatically recognize all situations and drive, so in order to test this, the autonomous vehicle must be driven while checking whether it is operating normally with a person on board, but it has not received safety performance certification. Testing a self-driving vehicle with a human on board not only poses safety issues for the occupants, but also prevents a person from being able to respond individually during the test, which requires a long time, and also prevents accurate and objective test results from being obtained.

Accordingly, simulation is essential due to safety issues when implementing autonomous vehicles, but it is difficult to design a simulator that expresses the real physical world by applying actual physical laws, configuring various obstacles, pedestrians, etc., and having the same characteristics as a real vehicle. Not only is it high, but it also has economic limitations in that it costs a lot of money.

Korean Patent Publication No. 10-1832918

The present invention was created to solve the above problems, and the purpose of the present invention is to create a simulator that expresses the real physical world by applying actual physical laws or configuring various obstacles, pedestrians, etc. and having the same characteristics as a real vehicle. It provides a self-driving algorithm simulation method and system based on a game engine that allows testing the algorithm without designing it.

In order to achieve the above object, the autonomous driving algorithm simulation method based on a game engine performed in an autonomous driving algorithm simulation system including a simulation device and a lightweight embedded controller according to an embodiment of the present invention is performed by the simulation device in a game engine. Collecting images by capturing a specific game screen; Analyzing the collected images by the simulation device and generating an autonomous driving algorithm for autonomous driving of the vehicle based on the analyzed results; applying the self-driving algorithm generated by the simulation device to a virtual vehicle in a specific game to simulate autonomous driving according to the generated self-driving algorithm within the specific game; And a step of the lightweight embedded controller controlling autonomous driving of a vehicle corresponding to the virtual vehicle based on an autonomous driving algorithm that has been simulated in the simulation device.

And after controlling the autonomous driving of the vehicle corresponding to the virtual vehicle, the lightweight embedded controller transmits result data that is the result of the autonomous driving of the vehicle to the simulation device in real time; and a step of the simulation device modifying the autonomous driving algorithm in real time based on the result data, and when the step of modifying the autonomous driving algorithm in real time is performed, the autonomous driving algorithm is modified based on the modified autonomous driving algorithm. The step of simulating driving and the step of controlling autonomous driving of a vehicle corresponding to the virtual vehicle can be performed repeatedly.

Additionally, the autonomous driving algorithm generated in the step of generating the autonomous driving algorithm may include at least one of an algorithm for object detection and avoidance or an algorithm for lane detection and maintenance.

Here, if the generated autonomous driving algorithm is an algorithm for object detection and avoidance, the simulation step involves visualizing and displaying a bounding box on the detected object in the image collected while the virtual vehicle is performing autonomous driving. steps; distinguishing and classifying the detected objects; calculating a distance to an object based on the size of the bounding box; and controlling driving of the virtual vehicle based on the calculated distance.

Meanwhile, if the generated autonomous driving algorithm is an algorithm for lane detection and maintenance, the simulation step detects and masks a line in the image collected while the virtual vehicle performs autonomous driving, collecting line data; Calculating the slope of the line based on the collected line data and determining a driving lane, which is the lane in which the virtual vehicle is currently driving, according to preset conditions; It may include controlling the moving direction of the virtual vehicle based on the slope of the determined driving lane.

At this time, in the step of controlling the moving direction of the virtual vehicle, if the slope of the determined driving lane is positive, the moving direction of the virtual vehicle is controlled to the right, and if the slope of the determined driving lane is negative, the moving direction of the virtual vehicle is controlled to the left. It can be controlled with .

Meanwhile, an autonomous driving algorithm simulation system according to an embodiment of the present invention to achieve the above object includes a collection unit that collects images by capturing a specific game screen performed in a game engine, analyzes the collected images, and analyzes the results. Based on this, an algorithm generator generates an autonomous driving algorithm for autonomous driving of a vehicle, and the generated autonomous driving algorithm is applied to a virtual vehicle in a specific game to simulate autonomous driving according to the generated autonomous driving algorithm within the specific game. A simulation device including a simulation unit that does; and a lightweight embedded controller that controls autonomous driving of a vehicle corresponding to the virtual vehicle based on an autonomous driving algorithm simulated in the simulation device.

And the lightweight embedded controller transmits result data, which is the result of the autonomous driving of the vehicle, to the simulation device in real time, and the simulation device modifies the autonomous driving algorithm in real time based on the result data, and the simulation device If the autonomous driving algorithm is modified in real time, the simulation device simulates autonomous driving based on the modified autonomous driving algorithm, and the lightweight embedded controller repeatedly controls autonomous driving of the vehicle corresponding to the virtual vehicle. You can.

Additionally, the autonomous driving algorithm generated by the generator may include one of an algorithm for object detection and avoidance or an algorithm for lane detection and maintenance.

Here, the simulation unit includes an object detection unit that detects an object according to an algorithm for object detection and avoidance, and a driving control unit that controls driving of the virtual vehicle based on the detected object, and the object detection unit includes the virtual vehicle. A bounding box display unit that visualizes and displays a bounding box on an object detected within an image collected while the vehicle is performing autonomous driving; a classification unit that distinguishes and classifies the detected objects; and a calculation unit that calculates the distance to the object based on the size of the bounding box.

Meanwhile, the simulation unit includes a lane detection unit that detects a lane according to an algorithm for detecting and maintaining the lane; and a driving control unit that controls the moving direction of the virtual vehicle based on the detected lane, wherein the lane detection unit detects and masks a line in the image collected while the virtual vehicle is autonomously driving. and a line collection unit that collects line data; and a lane determination unit that calculates the slope of the line based on the collected line data and determines a driving lane, which is the lane in which the virtual vehicle is currently driving, according to preset conditions.

At this time, the driving control unit controls the moving direction of the virtual vehicle to the right if the slope of the driving lane determined by the lane determination unit is positive, and controls the moving direction of the virtual vehicle to the left if the slope of the determined driving lane is negative. can do.

According to one aspect of the present invention described above, by providing an autonomous driving algorithm simulation method and system based on a game engine, real physical laws can be applied or various obstacles, pedestrians, etc. can be configured to have the same characteristics as real vehicles. You can test algorithms without designing a simulator that represents the world.

1 is a block diagram for explaining a simulation system according to an embodiment of the present invention;
Figure 2 is a block diagram for explaining the configuration of a simulation device according to an embodiment of the present invention;
Figure 3 is a block diagram for explaining a simulation unit according to an embodiment of the present invention;
4 and 5 are diagrams for explaining an object detection unit according to an embodiment of the present invention.
6 and 7 are diagrams for explaining a lane detection unit according to an embodiment of the present invention;
Figure 8 is a diagram illustrating a process in which a lightweight embedded controller controls a vehicle by applying a simulated algorithm according to an embodiment of the present invention;
Figure 9 is a diagram for explaining an algorithmic feedback method according to the conventional sequential method;
10 is a diagram illustrating an algorithm feedback method through an autonomous driving algorithm simulation method based on a game engine according to the present invention;
11 and 12 are diagrams showing the results of comparing measurement time and memory usage when simulating through an autonomous driving algorithm simulation system according to an embodiment of the present invention;
13 is a flowchart illustrating a method for simulating an autonomous driving algorithm based on a game engine according to an embodiment of the present invention;
14 is a flowchart illustrating a method of simulating an object detection algorithm according to an embodiment of the present invention;
15 is a flowchart illustrating a method for simulating a lane detection algorithm according to an embodiment of the present invention, and
Figure 16 is a flowchart illustrating a method for simulating an autonomous driving algorithm based on a game engine according to another embodiment of the present invention.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein may be implemented in one embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description that follows is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, together with all equivalents to what those claims assert, if properly described. Similar reference numbers in the drawings refer to identical or similar functions across various aspects.

The components according to the present invention are components defined by functional division rather than physical division, and can be defined by the functions each performs. Each component may be implemented as hardware or program code and processing units that perform each function, and the functions of two or more components may be included and implemented in one component. Therefore, the names given to the components in the following embodiments are not intended to physically distinguish each component, but are given to suggest the representative function performed by each component, and the names of the components refer to the present invention. It should be noted that the technical idea is not limited.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a block diagram illustrating a simulation system 10 according to an embodiment of the present invention. The self-driving algorithm simulation system 10 (hereinafter referred to as system) based on the game engine of the present invention provides autonomous driving of an autonomous vehicle. Without creating a separate simulator to simulate the algorithm, you can create an autonomous driving algorithm based on a game engine or simulate the created autonomous driving algorithm.

In general, virtual reality simulation is an essential element in autonomous driving technology because it has complex issues such as stability and safety, but it is close to impossible to evaluate and implement all cases and objects in the real world when designing a simulation program. Accordingly, attempting to construct a simulation that is close to the actual phenomenon requires complex manual work, which inevitably results in significant cost and time consumption.

Many factors present in these simulations can be divided into deterministic factors and stochastic factors. The decisive factor is information that we can know in advance. Lane characteristics such as single dotted lanes or stop lines, road characteristics such as safety zones, curved roads, road width, intersections, crosswalks, and highways can be decisive factors. This information can be obtained in advance through GPS.

Meanwhile, stochastic factors are information that we cannot predict and may include the characteristics of objects such as traffic lights, pedestrians or animals, the distance between cars, the speed of other cars, lane departure, and the environment of surrounding vehicles such as special cars.

Therefore, deterministic factors can be easily processed because they can be predicted, but stochastic factors must be processed in real time, and in the real world, these factors are complexly combined. For example, on curved roads, it may be difficult to detect other vehicles in the next lane, and on merging roads such as entry and exit sections, it may be difficult to detect vehicles in the next lane. In addition, since it is difficult to detect adjacent vehicles on roads with different heights, such as roads with ramps, underground intersections, and 3D intersections, it is necessary to take into account deterministic factors and handle stochastic factors. To consider all factors and respond appropriately, you must run accurate simulations and minimize risks.

Therefore, the system 10 according to the present invention does not design a separate simulator that requires considerable cost and time to simulate the autonomous driving algorithm of a vehicle, but utilizes a game engine with a physics engine similar to the real world to develop an autonomous driving algorithm. By simulating, cost and time can be reduced because there is no need to design a sophisticated simulator yourself.

The system 10 according to the present invention may be prepared including a simulation device 100 and a lightweight embedded controller 200, as shown in FIG. 1, and this device 100 and controller 200 are game engines. It can be controlled based on software (application) to perform an autonomous driving algorithm simulation method based on .

The simulation device 100 (hereinafter referred to as device) can simulate an autonomous driving algorithm using a game engine, and this device 100 can transmit the simulated autonomous driving algorithm to the lightweight embedded controller 200 through cloud-based communication. and can receive data transmitted from the lightweight embedded controller 200.

In addition, the simulation device 100 can capture a specific game screen performed in a game engine, collect and analyze images, create or modify an autonomous driving algorithm, and apply the created or modified autonomous driving algorithm to a virtual vehicle in a specific game. Simulation can be conducted by allowing the virtual vehicle in the game to drive autonomously.

In addition, the simulation device 100 receives result data from the lightweight embedded controller 200, modifies the autonomous driving algorithm based on the received result data, and re-simulates autonomous driving according to the modified autonomous driving algorithm. can be performed repeatedly.

The lightweight embedded controller 200 (hereinafter referred to as the controller) can transmit and receive various information with the device 100 through cloud-based communication. Specifically, the controller 200 may control a vehicle performing actual autonomous driving corresponding to a virtual vehicle in the game based on an autonomous driving algorithm received from the device 100. And the controller 200 can transmit the actual vehicle control results to the device 100 as feedback data. Accordingly, the controller 200, although not shown in the drawing, includes a communication unit that communicates with the simulation device 100, a storage unit that stores various information, an input unit including sensors such as cameras, an output unit, and an autonomous driving algorithm simulation method. It may also include a control unit that controls the communication unit, storage unit, input unit, and output unit based on the software (application) to be executed.

Meanwhile, FIG. 2 is a block diagram for explaining the configuration of the simulation device 100 according to an embodiment of the present invention, and FIG. 3 is a block diagram for explaining the simulation unit 145 according to an embodiment of the present invention. 4 and 5 are diagrams for explaining the object detection unit 1451 according to an embodiment of the present invention, and FIGS. 6 and 7 are diagrams for explaining the lane detection unit 1453 according to an embodiment of the present invention. It is a drawing.

As shown in FIG. 2, the device 100 may be provided including a communication unit 110, an input unit 120, a storage unit 130, a control unit 140, and an output unit 150.

In addition, the device 100 of the present invention can be installed and executed with software (application) for performing an autonomous driving algorithm simulation method based on a game engine, and includes the above communication unit 110, input unit 120, and storage unit ( 130), the configuration of the control unit 140 and the output unit 150 may be controlled by software for performing an autonomous driving algorithm simulation executed in the device 100.

And the device 100 may be a separate terminal or a partial module of the terminal. Additionally, the communication unit 110, input unit 120, storage unit 130, control unit 140, and output unit 150 may be formed as an integrated module or may be comprised of one or more modules. However, on the contrary, each configuration may be made up of a separate module.

Device 100 may be mobile or fixed. The device 100 may be in the form of a server or engine, and may be a device, apparatus, terminal, user equipment (UE), mobile station (MS), wireless device, It may be called by other terms, such as handheld device.

Additionally, the device 100 can execute or produce various software based on an operating system (OS), that is, a system. The operating system is a system program that allows software to use the hardware of the device, and includes mobile computer operating systems such as Android OS, iOS, Windows Mobile OS, Bada OS, Symbian OS, Blackberry OS, Windows series, Linux series, Unix series, etc. It can include all computer operating systems such as MAC, AIX, and HP-UX.

The communication unit 110 may be provided to allow the device 100 to transmit and receive various information through a wired or wireless network. The communication unit 110 receives video or images to be analyzed by the control unit 140, or the control unit 140 receives images to be analyzed. In the process of analyzing, necessary data may be received and provided to the control unit 140. Additionally, the communication unit 110 may transmit data generated by the control unit 140 to the outside.

Meanwhile, the input unit 120 may be provided to receive data or user commands and transmit them to the control unit 140, and the output unit 150 may be provided to output the results processed by the control unit 140. The input unit 120 may include a user input means such as a keyboard, mouse, or touch panel, and the output unit 150 may include an output means such as a monitor or speaker.

The storage unit 130 can store various data or programs necessary for simulating the autonomous driving algorithm, and can be provided to store the results processed by the control unit 140. This storage unit 130 is stored in the game engine. Software (application) for performing a simulation method based on an autonomous driving algorithm may be stored.

The control unit 140 includes at least one processor, such as a CPU, and can control the overall operation and configuration of the device 100. The control unit 140 executes a program stored in the storage unit 130 or reads data to provide autonomous control. You can simulate the generated autonomous driving algorithm by creating a driving algorithm or applying the generated autonomous driving algorithm to a virtual vehicle in the game to control the virtual vehicle.

In the present invention, for convenience of explanation, the case of simulating an autonomous driving algorithm using GTA5 (Grand Theft Auto) as a specific game engine will be described as an example. GTA5 was selected as the game engine for this simulation because the movements of NPCs (Non-Player Characters), driving, and road conditions are realistic, and the autonomous driving algorithm can be sufficiently simulated without designing the movements of other cars and the paths of surrounding pedestrians. Because you can.

In addition, the game engine has a very realistic traffic environment such as traffic lights, crosswalks, and signs, and there are police officers who crack down on speeding and traffic violations, and it uses MOD (modification) files, which are user modification files created by GTA5 users to play various games. This game engine was selected because it allows for realistic vehicle customization by setting characteristics such as braking distance and maximum speed for each vehicle of the virtual vehicle in the game. As described above, the simulation using GTA5 is merely an example for convenience of explanation and is not limited thereto.

In actual autonomous driving situations, data collected from cameras is processed, so to identify the situation, the control unit 140 of the present invention captures the game screen played in the game engine, which is GTA5, extracts image data, and extracts the extracted game images in real time. It can be captured and stored as array data. You can also analyze the captured images, collect the analyzed data, classify the results, and simulate the autonomous driving algorithm by controlling the character riding the car in GTA5 using the sorted data.

For this purpose, the control unit 140 may include a collection unit 141, an algorithm generation unit 143, and a simulation unit 145.

In general, since data recognized by a camera mounted on a vehicle is processed in a driving situation through autonomous driving, the device 100 of the present invention also needs to extract image data from a game such as GTA5 to identify the situation.

To this end, the collection unit 141 may collect images by capturing a specific game screen being played in a game engine. These images may be collected through the communication unit 110 or the input unit 120 including an input means such as a camera. The game screen collected in this collection unit 141 is a screen output while the user directly operates the vehicle in the game and reaches a certain destination based on the map in the game, or is a recorded video uploaded to the network through other users. This may be the output screen, but is not limited to this.

And the collection unit 141 can capture and store game images as array data in real time using OpenCV (Open source computer vision) and NumPy (Numerical Python).

The algorithm generator 143 may analyze the images collected by the collection unit 141 and generate an autonomous driving algorithm for autonomous driving of the vehicle based on the analysis results.

The autonomous driving algorithm generated by the algorithm generator 143 may be one of an algorithm for object detection and avoidance or an algorithm for lane detection and maintenance.

Additionally, the algorithm generator 143 may modify or change the pre-generated autonomous driving algorithm based on feedback data received from the controller 200.

Meanwhile, the simulation unit 145 can simulate autonomous driving by applying the generated autonomous driving algorithm to a virtual vehicle in a specific game and controlling the virtual vehicle according to the autonomous driving algorithm generated in the specific game. In addition, the simulation unit 145 may store and use functions corresponding to operation keys that can control the vehicle in the game in advance in order to control the virtual vehicle in the game with code.

In the present invention, a function can be created in advance using the control keys 'W', 'D', 'A', and 'S' applied to GTA5, and using this, a virtual vehicle can be controlled within the game. This simulation unit 145 may include an object detection unit 1451, a lane detection unit 1453, and a driving control unit 1455.

If the algorithm generated by the algorithm generator 143 is an object detection algorithm for object detection and avoidance, the simulation unit 145 may perform simulation through the object detection unit 1451 and the driving control unit 1455.

The object detection unit 1451 can detect objects located in the game based on an object detection algorithm, and Figure 4 is an example of a rough process for detecting and avoiding such objects.

This object detection unit 1451 can analyze the image captured by the collection unit 141 using YOLO v3 and TensorFlow, collect the analyzed data, and classify the results. To this end, the object detection unit 1451 first You can use it by setting the classes and weights provided by YOLO v3.

Algorithm 1 below is the pseudocode of the algorithm for object detection performed in the object detection unit 1451, and the object detection unit 1451 detects objects through this, and the driving control unit 1455 ) can control the virtual vehicle according to the results detected by the object detection unit 1451.

[Algorithm 1]

This object detection unit 1451 may include a bounding box display unit 14511, a classification unit 14513, and a calculation unit 14515.

First, the bounding box display unit 14511 is provided to visualize and display a bounding box on an object detected in an image collected through the collection unit 141 while the virtual vehicle is autonomously driving in the game. Through this, users can visually check the process in which objects, including obstacles and objects, are detected according to the object detection algorithm during the simulation process. At this time, the bounding box may be displayed in a size corresponding to the number of pixels of the object detected in the collected image. An example of visualizing a bounding box on an object detected through the bounding box display unit 14511 is shown in FIG. 5.

As described above, the classification unit 14513 can distinguish and classify objects detected in the image as shown in FIG. 5 using YOLO v3 and TensorFlow. Specifically, YOLO and TensorFlow's models provide reliability for objects, so the corresponding values can be stored in an array, and the detected objects can be classified using the corresponding values. At this time, objects that need to be recognized during transportation, such as people and vehicles, can be classified using the object's unique array number stored in the class, as shown in Algorithm 1 above.

Meanwhile, in order to measure the distance between the vehicle and the obstacle, the calculation unit 14515 may calculate the distance between the object and the virtual vehicle based on the size of the bounding box whose size corresponds to the number of pixels occupied by the detected object.

The driving control unit 1455 may control driving of the virtual vehicle based on the distance calculated by the calculating unit 14515. Specifically, as in Algorithm 1 above, if the size of the box drawn on the object is larger than a certain value, the virtual vehicle in the game can be kept in a stopped state until the size of the box drawn on the object decreases below a certain value. Here, a certain value for determining whether to stop or run the vehicle may be stored in advance or may be modified by the user.

Meanwhile, if the algorithm generated by the algorithm generator 143 is a lane detection algorithm for lane detection and maintenance, the simulation unit 145 may perform simulation through the lane detection unit 1453 and the driving control unit 1455.

The lane detection unit 1453 can detect lanes located in the game based on a lane detection algorithm, and Figure 6 is an example of a rough process for detecting and maintaining such lanes.

The lane detection unit 1453 detects and masks a line in the image captured by the collection unit 141 using Canny edge detection, and can detect the lane as shown in FIG. 7. In this way, the driving control unit 1455 can control the driving of the vehicle based on the lane detected by the lane detection unit 1453.

Algorithms 2 and 3 below are pseudocodes of algorithms for lane detection performed by the lane detection unit 1453. The lane detection unit 1453 detects lanes through these, and the results detected by the lane detection unit 1453 are Accordingly, the driving control unit 1455 can control the virtual vehicle.

[Algorithm 2]

[Algorithm 3]

And this lane detection unit 1453 can detect lanes in the game, and for this purpose, it may include a line collection unit 14531 and a lane determination unit 14533.

The line collection unit 14531 detects and masks lines in the image collected through the collection unit 141 while the virtual vehicle is autonomously driving in the game, according to Algorithms 2 and 3 above, and collects line data. can be collected. Specifically, the 'lines' function allows you to mask, or draw a line, on a line detected within an image. And you can extract thick line data, such as slope and intercept, from the 'line_data' array and store it in the 'line_dict' array. And, if the extracted lines have similar slopes within an error range of 20%, they can be classified as one line. Accordingly, the last two lanes can be the lines with the most similar slope among the extracted lines.

At this time, in order to extract all lines of the image, which is input data, it is effective to convert the image to grayscale, so the 'img_process()' function makes it easier for the line collection unit 14531 to process the images collected in the line collection unit 14531. To do this, you can convert the image to grayscale.

The line collection unit 14531 can detect most outlines in the image using canny edge detection. You can also collect line data using the HoughLines function supported by OpenCV.

In addition, the line collection unit 14531 can mask the lanes by drawing lines so that the last two lanes described above can be easily recognized. Figure 7 is an example of a thick line masked to the selected lines.

The lane determination unit 14533 may calculate the slope of the line based on the collected line data and specify the driving lane, which is the lane in which the virtual vehicle is currently driving, according to preset conditions. Specifically, the lane determination unit 14533 may select a lane by calculating the slope of the two lanes masked with thick lines in the line collection unit 14531, and then continuously calculate and track the slope values. At this time, the lane determination unit 14533 may determine that the vehicle is tilted to the left if the slope values of both lanes are positive numbers. On the other hand, the lane determination unit 14533 may determine that the vehicle is tilted to the right if the slope values of both lanes are negative.

Additionally, the driving control unit 1455 may control the driving direction of the vehicle based on the inclination of the vehicle determined and determined by the lane determining unit 14533. If the lane determination unit 14533 determines that the vehicle is tilted to the left, the driving control unit 1455 can control the virtual vehicle to steer to the right to maintain the position and direction between lanes. Meanwhile, if the lane determination unit 14533 determines that the vehicle is tilted to the right, the driving control unit 1455 can steer the virtual vehicle to the left to prevent it from leaving the lane.

Meanwhile, as described above, the driving control unit 1455 may control the virtual vehicle in the game based on at least one of the object detected by the object detection unit 1451 or the lane detected by the lane detection unit 1453. Specifically, the driving control unit 1455 can drive, stop driving, and turn left or right the virtual vehicle so that the virtual vehicle in the game arrives at a preset destination through autonomous driving.

Additionally, when the driving control unit 1455 controls the virtual vehicle based on the lane detected by the lane detection unit 1453, the direction of movement of the virtual vehicle may be controlled based on the slope of the driving lane determined by the lane determining unit 14533. You can. Specifically, if the slope of the driving lane is positive, the virtual vehicle can be controlled to move to the right, and if the slope of the driving lane is negative, the virtual vehicle can be controlled to move to the left.

The autonomous driving algorithm can be simulated through a virtual vehicle in the game based on the game engine through the simulation unit 145, which includes the object detection unit 1451, the lane detection unit 1453, and the driving control unit 1455. Of course, an efficient digital twin can be realized by mounting the tested autonomous driving algorithm on an actual embedded controller and implementing, modifying, and supplementing it in reality through a model vehicle that corresponds to a virtual vehicle based on this.

When the simulation of the autonomous driving algorithm is completed through the above process, the simulated autonomous driving algorithm is stored in the lightweight embedded controller 200, and the lightweight embedded controller 200 provides information to the virtual vehicle through the simulated autonomous driving algorithm. You can implement autonomous driving in reality by controlling the autonomy of the corresponding vehicle.

Additionally, the device 100 of the present invention may include an object detection unit 1451 according to another embodiment. The object detection unit 1451 according to another embodiment of the present invention, like the object detection unit 1451 according to one embodiment, displays a bounding box on the detected object, classifies the class of the detected object, and classifies the detected object. In addition to calculating the distance between the vehicle and the vehicle, the number of detected people can also be calculated. To this end, the object detection unit 1451 may further include a people counting algorithm, such as Algorithm 4 below.

[Algorithm 4]

In addition to using a wide range of classes and weights in YOLO, the object detection unit 1451 according to one embodiment described above uses TensorFlow and Tensornet as in Algorithm 4 described above. This allows you to narrow the range to detect people. Here, Tensornet is a TensorFlow implementation that allows users to use the specified classes and weights they want, and is only compatible with TensorFlow v1.

And in order to use Tentsronet, the image size must be adjusted so that the resolution may be degraded, but a slight decrease in image quality is not a big problem in recognizing objects in real time, and the avoidance algorithm that detects and avoids an object allows the object to be identified as a specific object. Objects that are too small to be recognized are not associated with a slight decrease in image quality, as the vehicle stops running when close to within range.

This object detection unit 1451, like the object detection unit 1451 according to one embodiment, displays a bounding box around the detected person every time it detects a person, counts the number of displayed bounding boxes, and can count the number of bounding boxes in the corresponding area. Population density can be calculated.

Meanwhile, FIG. 8 is a diagram illustrating a process in which the lightweight embedded controller 200 controls a vehicle by applying a simulated algorithm according to an embodiment of the present invention.

The controller 200 may control autonomous driving of a vehicle corresponding to a virtual vehicle in the game based on an autonomous driving algorithm that has been simulated in the device 100. Hereinafter, for convenience of explanation, the vehicle corresponding to the virtual vehicle will be described as a model vehicle.

In addition, the controller 200 can transmit result data, which is the result of the autonomous driving of the model vehicle, to the device 100 in real time, and when the device 100 modifies the autonomous driving algorithm in real time based on the result data, the device 100 The autonomous driving of the virtual vehicle can be simulated again according to the modified autonomous driving algorithm, and the controller 200 to which the modified autonomous driving algorithm is applied can repeat the process of controlling the autonomous driving of the model vehicle.

The controller 200 according to the present invention can select a programmable car based on Python as a model vehicle and control such a vehicle. Algorithm 5 below shows the pseudocode of the control algorithm by which the controller 200 controls the vehicle, and Figure 8 is an example of the process of the shell script-based vehicle control algorithm.

[Algorithm 5]

First, the controller 200 can use commands to control the driver connected to the motor of the model vehicle to make the model vehicle go straight at a constant speed.

It then calls a Python script programmed to detect the object. If a Python script detects an object, it can return 0, otherwise it can return 1. And the controller 200 can control the model vehicle using the returned value. If the return value is 0, the speed is changed to 0 to stop driving, and if the return value is 1, driving can be maintained. Through this, the controller 200 can control the autonomous driving of a model vehicle, which is a vehicle in the real world, using the autonomous driving algorithm that has been simulated through the device 100.

Through this, the self-driving algorithm simulation system of the present invention not only allows testing the self-driving algorithm using a game engine without designing a simulator that represents the real physical world, but also allows the self-driving algorithm that has been simulated to be used as a model vehicle in reality. You can test the autonomous driving of a model vehicle by applying it to .

Hereinafter, experiments performed based on the autonomous driving algorithm simulation system based on the game engine of the present invention described above will be described. To implement the physical layer of digital twin technology, it is necessary to be able to program and control an actual car, so in this experiment, a model vehicle, DonkeyCar, was used as an embodiment of the physical layer. This Donkey Car is a programmable car based on Python, supports the IMX219 camera, and uses the Jetson Nano board as the main board. In addition, Donkey Car can be controlled manually with a joystick and provides its own simulation program based on Unity. You can train your Donkey Car on autopilot using Keras by controlling it multiple times on the same road. However, in real situations, self-driving cars encounter road environments and obstacles for which they have not been trained, so image data captured by cameras must be processed, so a simulation algorithm was applied in GTA5.

To operate this donkey car, a model vehicle was assembled and connected to the main board, Jetson Nano, as a controller (200). At this point, voltage controllers, servo drivers, and cameras can be connected directly to the Jetson Nano. A voltage converter was attached to the NiMH battery, and the ground (GND) of the servo driver was connected to the GND of the Jetson Nano. In addition, the servo driver's serial clock (SCL) is connected to the Jetson Nano's 5-pin SDI1 and I2C (Inter-integrated Circuit Communications) bus1, and the servo driver's serial data (SDA) is connected to the Jetson Nano's 3 pins (SDA1, I2C bus1). ) was connected. The VCC (Voltage Collector) of the servo driver was connected to the 3.3V pin (power) of the Jetson Nano. A 128GB microSD card was used to set up the software on the motherboard, Jetson Nano, and the Jetson Nano Developer Kit SD card image uploaded to Nvidia's official site was created on the microSD card. Jetson Nano was booted by inserting the microSD card containing the image and the plug barrel jack for the power supply, and after booting, Python, OpenCV, TensorFlow, and Donkika code were installed.

At this time, input image data was set using the camera mounted on the Donkey Car, and since the commands that control the speed and direction of the Donkey Car are shell-based, some code was added for smooth execution and compiled into a shell script. did.

And based on the above-mentioned Algorithm 5, a command to control the servo driver connected to the motor of the Donkey Car is used to make the Donkey Car go straight at a constant speed, and a Python script programmed to detect the object is called, and then the Python script detects the object. If detected, 0 is returned, otherwise, the Donkey Car is controlled based on the returned value by returning 1.

In order to implement a digital twin, the robustness and applicability of the fusion algorithm must be improved, and parallel computing must be applied to increase computing efficiency and handle large amounts of data. Additionally, for better deployment of cyber-physical convergence, connection and communication protocols must be standardized and predictable and reliable at the abstracted software level.

In order to implement a conventional digital twin, it was necessary to calculate and design models of all environments, but efficiency can be greatly increased by using games such as GTA5 with a physics engine as in the present invention.

In particular, in this experiment, by using Donkey Car to verify the algorithm, it is possible to check how predictable and well the designed algorithm and simulation work.

Figure 9 is a diagram to explain the algorithm feedback method according to the conventional sequential method. When using the conventional sequential method, simulation is performed with a simulator, code is written on a car that can actually be driven, firmware is written, and then the feedback method is used in the real environment. After executing autonomous driving, feedback must be provided manually.

Figure 10 is a diagram for explaining an algorithm feedback method through an autonomous driving algorithm simulation method based on a game engine according to the present invention. As shown, using a digital twin, a real car and a car in a virtual environment can be connected through cloud-based communication. Not only can a lot of time be saved because it changes organically, but it also has the advantage of being reasonable in implementing a digital twin because the development time is shortened, the number of linked codes by running the game engine is reduced, and the number of overhead repetitions is low.

Figures 11 and 12 are diagrams showing the results of comparing measurement time and memory usage when simulating using the autonomous driving algorithm simulation system 10 according to an embodiment of the present invention.

Specifically, Figure 11 (a) shows the execution time of the object detection, lane detection, and people counting algorithms. Code to record the time was added to measure the execution time of the algorithm. Total time refers to the total time spent on processing, which took 0.483 seconds for object detection, 0.197 seconds for lane detection, and 0.902 seconds for calculating the number of people.

Capture time is the time taken to capture image data from cameras and video inputs, which was 0.1333 seconds for object detection, lane detection, and people counting.

Detection time is the time required to recognize an object from the captured image information. It took 0.417 seconds to detect an object, 0.014 seconds to detect a lane, and 0.692 seconds to calculate the number of people.

Meanwhile, the reaction time is the time taken when GTA5's virtual car or model vehicle, that is, Donkey Car, reacts. It takes 0.002 seconds for object detection, 0.098 seconds for lane detection, and 0.078 seconds for calculating the number of people. This means that an object such as a driving car can be recognized within 1 second. In general, in GTA5 or real driving situations, objects such as objects or lanes do not disappear in 1 second, so it was judged that real-time execution is possible if the time required is more than 1 second, and these experimental results guarantee that it can be processed in real time. can do.

Meanwhile, Figure 11 (b) shows the results of measuring the memory usage of the object detection algorithm, and the working set may refer to the amount of physical memory currently being used by the program. 'Private bytes' refers to the amount of memory allocated to the program and can represent physical memory and swapped memory. 'Page Faults/sec' may refer to virtual memory usage. It can be seen that this memory usage remains the same regardless of the number of detected objects, as shown in Figure 11 (b).

Figure 12(a) is the result of measuring the memory usage of the lane detection algorithm, and Figure 12(b) shows the memory usage of the person counting algorithm. It can be seen that the memory usage is consistent despite changes in the number of people recognized. You can. This algorithm uses about 14MB of memory, so it can be seen that it is a very effective method.

The table below compares the performance of the simulation system according to the present invention.

[Table 1]

Among the contents shown in Table 1 above, venkatesan's research performed a state prediction simulation of the motor inside the car based on Matlab/simulink. A sensor was needed to measure the temperature and speed of the motor, and through this experiment, the motor's Although it is possible to predict the state and plan the execution time, there is a problem that it requires a lot of calculations.

Meanwhile, Liu's research combined GNSS information and Unity to simulate the response of surrounding vehicles when they change lanes. Two types of cameras were needed to recognize the surrounding environment and measure the distance between vehicles, with high accuracy and fast speed. Although it showed excellent response time, there is a problem when combining Unity and GNSS information that it becomes unstable when the GNSS information is delayed.

On the other hand, using GTA5 as a simulator as in the present invention has the advantage of not only requiring only one camera, but also enabling stable and easy simulation because an algorithm for measuring distance is written in the code.

FIG. 13 is a flowchart for explaining a self-driving algorithm simulation method based on a game engine according to an embodiment of the present invention. The self-driving algorithm simulation method based on a game engine according to an embodiment of the present invention is shown in FIG. Since it is carried out on substantially the same configuration as the autonomous driving algorithm simulation system, the same reference numerals will be assigned to the same components as the autonomous driving algorithm simulation system of Figure 1, and repeated descriptions will be omitted.

First, the simulation device 100 may collect images by capturing a specific game screen being played in a game engine (S110).

Then, the simulation device 100 can analyze the collected images and generate an autonomous driving algorithm for autonomous driving of the vehicle based on the analyzed results (S130). The autonomous driving algorithm generated in the step of generating the autonomous driving algorithm (S130) may include at least one of an algorithm for object detection and avoidance or an algorithm for lane detection and maintenance.

Additionally, as described above, an algorithm for calculating the number of people within an object, that is, the number of people among detected objects, may be further included.

Thereafter, the simulation device 100 can apply the generated autonomous driving algorithm to a virtual vehicle in a specific game to simulate autonomous driving according to the autonomous driving algorithm generated in the specific game (S150).

And the lightweight embedded controller 200 can control the autonomous driving of a vehicle corresponding to the virtual vehicle based on the autonomous driving algorithm that has been simulated in the simulation device 100 (S170).

Through this, the self-driving algorithm simulation method of the present invention can simulate the self-driving algorithm by using the game as a simulator without creating a separate simulator, which is costly and time-consuming.

Meanwhile, Figure 14 is a flowchart to explain a method of simulating an object detection algorithm according to an embodiment of the present invention. If the autonomous driving algorithm generated in the step of generating an autonomous driving algorithm (S130) is an object detection algorithm, autonomous driving algorithm This is a flowchart that explains the driving simulation step (S150) in more detail.

If the generated autonomous driving algorithm is an algorithm for object detection and avoidance, in the simulation step (S150), the object detection unit 1451 bounds to the object detected within the image collected while the virtual vehicle is performing autonomous driving. The box can be visualized and displayed (S210).

And the object detection unit 1451 can distinguish and classify the detected objects (S230).

Thereafter, the object detection unit 1451 may calculate the distance between the virtual vehicle and the object based on the size of the bounding box (S250).

Then, the driving control unit 1455 may control the driving of the virtual vehicle based on the distance calculated in the calculation step (S250).

Afterwards, the simulation unit 145 determines whether the virtual vehicle has finished driving based on the destination or driving route set in advance in the game (S290), and if driving has ended (S290 - yes), autonomous driving can be ended. there is.

Meanwhile, if the autonomous driving of the virtual vehicle has not ended (S290-No), the simulation unit 145 returns to the above-mentioned visualization and display step (S110) and repeats the autonomous driving until the autonomous driving of the virtual vehicle is terminated. It can be done.

Meanwhile, Figure 15 is a flowchart illustrating a method of simulating a lane detection algorithm according to an embodiment of the present invention. When the autonomous driving algorithm generated in the step of generating an autonomous driving algorithm (S130) is a lane detection algorithm, This is a flow chart that explains in more detail the step of simulating autonomous driving (S150).

If the generated autonomous driving algorithm is an algorithm for lane detection and maintenance, in the simulation step (S150), the lane detection unit 1453 detects a line in the image collected while the virtual vehicle is performing autonomous driving. You can do masking and collect line data (S310).

Then, the lane detection unit 1453 can calculate the slope of the line based on the collected line data and determine the driving lane, which is the lane in which the virtual vehicle is currently driving, according to preset conditions (S330).

Afterwards, the driving control unit 1455 can control the moving direction of the virtual vehicle based on the slope of the determined driving lane (S350). In the step of controlling the moving direction of the virtual vehicle (S350), if the slope of the designated driving lane is positive, the moving direction of the virtual vehicle is controlled to the right, and if the slope of the designated driving lane is negative, the moving direction of the virtual vehicle is controlled to the left. You can control it.

Afterwards, the simulation unit 145 determines whether the virtual vehicle has finished driving based on the destination or driving path preset in the game (S370), and if driving has ended (S370 - yes), autonomous driving can be ended. there is.

Meanwhile, if the autonomous driving of the virtual vehicle has not ended (S370-No), the simulation unit 145 returns to (S310) to collect the above-described line data until the autonomous driving of the virtual vehicle ends and repeats the autonomous driving. It can be done.

Figure 16 is a flowchart illustrating a method for simulating an autonomous driving algorithm based on a game engine according to another embodiment of the present invention. In a simulation method according to another embodiment of the present invention, first, the simulation device 100 can collect images by capturing a specific game screen played in a game engine (S110).

Then, the simulation device 100 can analyze the collected images and generate an autonomous driving algorithm for autonomous driving of the vehicle based on the analyzed results (S130). The autonomous driving algorithm generated in the step of generating the autonomous driving algorithm (S130) may include at least one of an algorithm for object detection and avoidance or an algorithm for lane detection and maintenance.

Additionally, as described above, an algorithm for calculating the number of people within an object, that is, the number of people among detected objects, may be further included.

Thereafter, the simulation device 100 can apply the generated autonomous driving algorithm to a virtual vehicle in a specific game to simulate autonomous driving according to the autonomous driving algorithm generated in the specific game (S150).

And the lightweight embedded controller 200 can control the autonomous driving of a vehicle corresponding to the virtual vehicle based on the autonomous driving algorithm that has been simulated in the simulation device 100 (S170). All of the above steps are the same as the simulation method according to an embodiment of the present invention shown in FIG. 13.

Thereafter, in the simulation method according to another embodiment of the present invention, after the step (S170) in which the lightweight embedded controller 200 controls autonomous driving of a vehicle corresponding to the virtual vehicle, the lightweight embedded controller 200 controls the autonomous driving result of the vehicle. It may further include a step of transmitting the result data to the simulation device 100 in real time (S210) and a step of the simulation device 100 modifying the autonomous driving algorithm in real time based on the result data (230).

More specifically, as shown in FIG. 16, the simulation method according to another embodiment of the present invention provides result data that is the result of the autonomous driving of the vehicle after the step (S170) in which the lightweight embedded controller 200 controls the autonomous driving of the vehicle. can be transmitted to the simulation device 100 in real time (S210).

Additionally, the simulation device 100 can determine whether modification of the previously simulated autonomous driving algorithm is necessary based on the result data received from the lightweight embedded controller 200 (S220).

If modification of the autonomous driving algorithm is not required (S220-No), the simulation method can be terminated.

On the other hand, if it is determined that modification of the autonomous driving algorithm is necessary (220-Yes), the simulation device 100 may modify the autonomous driving algorithm in real time based on the result data (S230).

And when the step of modifying the autonomous driving algorithm in real time (S230) is performed, the step of simulating autonomous driving based on the modified autonomous driving algorithm (S150) and the step of controlling autonomous driving of the vehicle corresponding to the virtual vehicle (S170) can be performed repeatedly. Here, as a standard for determining whether the self-driving algorithm needs to be modified, the conditions set by the user can be compared with the result data, and a decision can be made based on the comparison results, or the user can modify the self-driving algorithm through a separate input means. If you input that a correction is needed, you can make a decision based on that.

The self-driving algorithm simulation method of the present invention can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination.

The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or may be known and usable by those skilled in the computer software field.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and perform program instructions, such as ROM, RAM, flash memory, etc.

Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the invention and vice versa.

Although various embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and may be used in the technical field to which the invention pertains without departing from the gist of the invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

10: System 100: Simulation device
110: communication unit 120: input unit
130: storage unit 140: control unit
141: collection unit 143: algorithm generation unit
145: simulation unit 1451: object detection unit
14511: Bounding box display unit 14513: Classification unit
14515: Calculation unit 1453: Lane detection unit
14531: Line collection unit 14533: Lane decision unit
1455: Driving control unit 150: Output unit
200: Lightweight embedded controller

Claims (12)

In an autonomous driving algorithm simulation method based on a game engine performed in an autonomous driving algorithm simulation system including a simulation device and a lightweight embedded controller,
collecting images by capturing a screen of a specific game being played in a game engine by the simulation device;
Analyzing the collected images by the simulation device and generating an autonomous driving algorithm for autonomous driving of the vehicle based on the analyzed results;
applying the self-driving algorithm generated by the simulation device to a virtual vehicle in a specific game to simulate autonomous driving according to the generated self-driving algorithm within the specific game; and
Comprising the step of the lightweight embedded controller controlling autonomous driving of a vehicle corresponding to the virtual vehicle based on an autonomous driving algorithm that has been simulated in the simulation device,
The autonomous driving algorithm created in the step of generating the autonomous driving algorithm is,
It includes at least one of an algorithm for object detection and avoidance or an algorithm for lane detection and maintenance,
If the generated autonomous driving algorithm is an algorithm for lane detection and maintenance,
The simulation step is,
Detecting and masking a line in an image collected while the virtual vehicle is autonomously driving, and collecting line data;
Calculating the slope of the line based on the collected line data and determining a driving lane, which is the lane in which the virtual vehicle is currently driving, according to preset conditions;
It includes controlling the direction of movement of the virtual vehicle based on the slope of the determined driving lane,
In the step of controlling the moving direction of the virtual vehicle,
An autonomous driving algorithm simulation method based on a game engine that controls the moving direction of the virtual vehicle to the right when the slope of the determined driving lane is positive, and controls the moving direction of the virtual vehicle to the left when the slope of the determined driving lane is negative. According to paragraph 1,
After controlling the autonomous driving of a vehicle corresponding to the virtual vehicle, the lightweight embedded controller transmits result data, which is a result of the autonomous driving of the vehicle, to the simulation device in real time; and
Further comprising the step of the simulation device modifying the autonomous driving algorithm in real time based on the result data,
When the step of modifying the autonomous driving algorithm in real time is performed,
An autonomous driving algorithm simulation method based on a game engine, wherein the steps of simulating autonomous driving based on a modified autonomous driving algorithm and controlling autonomous driving of a vehicle corresponding to the virtual vehicle are repeatedly performed. delete According to paragraph 1,
If the generated autonomous driving algorithm is an algorithm for object detection and avoidance, the simulation step is,
Visualizing and displaying a bounding box on an object detected within an image collected while the virtual vehicle is performing autonomous driving;
distinguishing and classifying the detected objects;
calculating a distance to an object based on the size of the bounding box; and
An autonomous driving algorithm simulation method based on a game engine, comprising the step of controlling driving of the virtual vehicle based on the calculated distance. delete delete A collection unit that collects images by capturing a specific game screen played in a game engine, an algorithm generation unit that analyzes the collected images and generates an autonomous driving algorithm for autonomous driving of the vehicle based on the analyzed results, and the generated autonomy A simulation device including a simulation unit that applies a driving algorithm to a virtual vehicle in a specific game to simulate autonomous driving according to the generated autonomous driving algorithm in the specific game; and
A lightweight embedded controller that controls autonomous driving of a vehicle corresponding to the virtual vehicle based on an autonomous driving algorithm simulated in the simulation device,
The autonomous driving algorithm generated by the generator is,
It includes at least one of an algorithm for object detection and avoidance or an algorithm for lane detection and maintenance,
The simulation unit,
a lane detection unit that detects a lane according to the algorithm for detecting and maintaining the lane; and
A driving control unit that controls the direction of movement of the virtual vehicle based on the detected lane,
The lane detection unit,
A line collection unit that detects and masks lines in images collected while the virtual vehicle performs autonomous driving and collects line data; and
A lane determination unit that calculates the slope of the line based on the collected line data and determines a driving lane, which is the lane in which the virtual vehicle is currently driving, according to preset conditions,
The driving control unit,
An autonomous driving algorithm simulation system that controls the moving direction of the virtual vehicle to the right when the slope of the driving lane determined by the lane determination unit is positive, and controls the moving direction of the virtual vehicle to the left when the slope of the determined driving lane is negative. . In clause 7,
The lightweight embedded controller,
Transmitting result data, which is the result of the autonomous driving of the vehicle, to the simulation device in real time,
The simulation device is,
The autonomous driving algorithm is modified in real time based on the result data,
When the simulation device modifies the autonomous driving algorithm in real time,
An autonomous driving algorithm simulation system, wherein the simulation device simulates autonomous driving based on a modified autonomous driving algorithm, and the lightweight embedded controller repeatedly controls autonomous driving of a vehicle corresponding to the virtual vehicle. delete In clause 7,
The simulation unit,
An object detection unit that detects an object according to the algorithm for object detection and avoidance, and
A driving control unit that controls driving of the virtual vehicle based on the detected object,
The object detection unit,
a bounding box display unit that visualizes and displays a bounding box on an object detected within an image collected while the virtual vehicle performs autonomous driving;
a classification unit that distinguishes and classifies the detected objects; and
An autonomous driving algorithm simulation system comprising a calculation unit that calculates the distance to an object based on the size of the bounding box. delete delete