KR20230162865A - Tele-operated vehicle, controlling method of tele-operated vehicle, tele-operated center and controlling method of tele-operated center - Google Patents

Abstract

The present invention includes an image and data transmission unit that transmits images and data of an autonomous vehicle through a network; a control signal receiving unit that receives an integrated control signal through the network; a control signal analysis unit that analyzes the integrated control signal; and an autonomous driving control signal generator that generates an autonomous driving control signal for controlling the autonomous vehicle, wherein the control signal analysis unit analyzes additional information and a control signal included in the integrated control signal, and the additional information and generating the autonomous driving control signal when the control signal indicates the same autonomous vehicle.

Description

Self-driving vehicle, control method of self-driving vehicle, self-driving center, and control method of self-driving center {TELE-OPERATED VEHICLE, CONTROLLING METHOD OF TELE-OPERATED VEHICLE, TELE-OPERATED CENTER AND CONTROLLING METHOD OF TELE-OPERATED CENTER}

The present invention relates to an autonomous vehicle, a control method of an autonomous vehicle, an autonomous center, and a control method of the autonomous center.

An autonomous vehicle is a vehicle that can operate on its own without driver or passenger intervention. Additionally, as communication technology develops, high-speed and large-capacity data transmission becomes possible, making it possible to provide more diverse services through wireless communication systems.

In particular, the autonomous vehicle can receive a control signal from the autonomous driving center through a network and generate an autonomous driving control signal based on the received control signal to operate the autonomous vehicle.

In this case, because the autonomous vehicle receives control signals through the network, there is a problem that the autonomous vehicle may generate abnormal behavior when exposed to errors or malicious attacks on the network, which may lead to casualties.

The present invention relates to an autonomous vehicle, a control method of an autonomous vehicle, an autonomous driving center, and a control method of an autonomous driving center, and provides stable autonomous driving by analyzing and applying the normal or abnormal state of control signals transmitted to the autonomous vehicle. The purpose is to provide

According to an embodiment of the present invention, an image and data transmission unit that transmits images and data of an autonomous vehicle through a network; a control signal receiving unit that receives an integrated control signal through the network; a control signal analysis unit that analyzes the integrated control signal; and an autonomous driving control signal generator that generates an autonomous driving control signal for controlling the autonomous vehicle, wherein the control signal analysis unit analyzes additional information and a control signal included in the integrated control signal, and the additional information and generating the autonomous driving control signal when the control signal indicates the same autonomous vehicle.

The autonomous driving control signal generator is characterized in that it generates a control signal for controlling an emergency process to operate when the additional information and the control signal do not represent the same autonomous vehicle.

The additional information is characterized as information excluding signals received from the driving device of the autonomous driving center.

The images and data of the self-driving vehicle include image information from a front camera and a rear camera installed on the self-driving vehicle.

According to another embodiment of the present invention, transmitting images and data of an autonomous vehicle through a network; Receiving an integrated control signal through the network; analyzing the integrated control signal; Generating an autonomous driving control signal to control the autonomous vehicle; Analyzing additional information and control signals included in the integrated control signal, respectively; and generating the autonomous driving control signal when the additional information and the control signal indicate the same autonomous vehicle.

If the additional information and the control signal do not represent the same autonomous vehicle, the method further includes generating a control signal to control an emergency process to operate.

The additional information is characterized as information excluding signals received from the driving device of the autonomous driving center.

The images and data of the self-driving vehicle include image information from a front camera and a rear camera installed on the self-driving vehicle.

According to another embodiment of the present invention, a driving device; An image and data receiving unit that receives images and data of an autonomous vehicle through a network; Control signal generator; and a control signal transmitting unit, wherein the control signal generating unit includes an additional information generating unit generating additional information. and a control command generator that generates a control signal based on the signal received from the driving device, generates an integrated control signal including the additional information and the control signal, and transmits the integrated control signal through the control signal transmitter. An autonomous driving center is provided, characterized in that it transmits information to an autonomous vehicle.

The additional information generator generates additional information based on the received images and data, and the images and data include image information from a front camera and a rear camera installed in the autonomous vehicle.

The additional information generator generates additional information by preprocessing the received image and data, and detects a lane based on the generated additional information.

The additional information generator generates a trajectory based on the detected lane, and information about the generated trajectory is included in the additional information.

The control signal generator further includes a position estimation unit, wherein the position estimation unit estimates the current location of the autonomous vehicle based on the generated trajectory, and information about the estimated location is included in the additional information. Do it as

The location estimation unit outputs a plurality of first way points based on latitude and longitude from the estimated current location, and information about the output first way point is included in the additional information. do.

The position estimation unit sets the estimated current position as the origin and outputs a plurality of second waypoints based on the distance from the origin coordinate system and the azimuth, and information about the output second waypoints is included in the additional information. It is characterized by being

The control signal generator is characterized in that it individually messages and packetizes the additional information and the control signal.

The control signal may include at least one of a steering angle of the driving device, brake pressure, a forward signal, and a reverse signal.

According to another embodiment of the present invention, receiving images and data of an autonomous vehicle through a network; Generating additional information; generating a control signal based on a signal received from the driving device; generating an integrated control signal including the additional information and the control signal; and transmitting the integrated control signal to an autonomous vehicle.

The method further includes generating additional information based on the received images and data, wherein the images and data include image information from a front camera and a rear camera installed in the autonomous vehicle.

Preprocessing the received images and data to generate additional information; and

It may further include detecting a lane based on the generated additional information.

The method further includes generating a trajectory based on the detected lane, wherein information about the generated trajectory is included in the additional information.

The method further includes estimating the current location of the autonomous vehicle based on the generated trajectory, wherein information about the estimated location is included in the additional information.

Further comprising outputting a plurality of first way points based on latitude and longitude from the estimated current location, wherein information about the output first way points is included in the additional information. Do it as

Setting the estimated current position as the origin and outputting a plurality of second waypoints based on the distance from the origin coordinate system and the azimuth, wherein information about the output second waypoints is included in the additional information. It is characterized by being

It may further include the step of individually messaging and packetizing the additional information and the control signal.

The control signal may include at least one of a steering angle of the driving device, brake pressure, a forward signal, and a reverse signal.

According to an embodiment of the present invention, it is possible to prevent sudden operation of an autonomous vehicle and prevent casualties by receiving information generated and transmitted from the autonomous driving center and received from the autonomous vehicle, and comparing and analyzing it from different perspectives. There is an advantage.

In addition, according to an embodiment of the present invention, when an autonomous driving center makes a sharp change of direction to prevent an accident, the autonomous vehicle can also detect this, and the autonomous driving center can determine whether the control signal is an abnormal signal generated in the network. It has the advantage of being able to discern with a high probability whether the signal is intended.

However, those skilled in the art can understand technical effects that are not explicitly mentioned here through the meaning of the entire specification.

1 is a diagram illustrating an autonomous driving system according to an embodiment of the present invention.
Figure 2 is a diagram explaining the configuration of an autonomous driving system according to an embodiment of the present invention.
Figure 3 is a diagram explaining the configuration of another autonomous driving system according to an embodiment of the present invention.
Figure 4 is a diagram explaining the configuration of a control signal generator according to an embodiment of the present invention.
Figure 5 is a diagram explaining the configuration of an additional information generator according to an embodiment of the present invention.
Figure 6 is a diagram showing a screen output from an autonomous vehicle according to an embodiment of the present invention.
Figure 7 is a diagram explaining the configuration of a control signal analysis unit according to an embodiment of the present invention.
Figure 8 is a flowchart explaining a control method of an autonomous driving system according to an embodiment of the present invention.
Figure 9 is a flowchart explaining a control method of an autonomous vehicle according to an embodiment of the present invention.
Figure 10 is a flowchart explaining a control method of an autonomous driving center according to an embodiment of the present invention.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted. The suffixes “part” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves.

Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be.

On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

1 is a diagram illustrating an autonomous driving system according to an embodiment of the present invention.

When a problem occurs in the vehicle 110 equipped with the autonomous driving system 100, the autonomous driving (Tele-operated driving, ToD) system 100 controls the autonomous driving vehicle 110 through remote control (tele-operated) 120. ) is a technology that directly or indirectly manipulates (driving) and has recently been researched and developed in various fields.

The autonomous driving system 100 is a technology for operating the autonomous vehicle 110 from a remote location 120 and basically consists of three components.

More specifically, the self-driving system 100 may be composed of a self-driving vehicle (Tele-operated vehicle, ToV, 110), a network (network, 130), and an autonomous driving center (Tele-operation center, ToC, 120). there is.

Here, the autonomous vehicle 110 refers to a movable device that receives driving control from the autonomous driving center 120. The autonomous driving center 120 refers to a remote driving center that generates control signals for the autonomous vehicle 110. The network 130 refers to a network including a mobile communication network connecting the autonomous vehicle 110 and the autonomous driving center 120. Additionally, the network 130 may use 5G communications.

In addition, the autonomous vehicle 110 may include autonomous driving system 100 devices such as an encoder, and the autonomous driving center 120 may include autonomous driving system 100 devices such as a decoder and a display. ) may include a device.

Additionally, the self-driving vehicle 110 is capable of autonomous driving and is capable of environmental model (E/M) encoding. In addition, the autonomous driving center 120 is capable of decoding the environmental model (E/M), and is capable of display and remote operation. Here, the environmental model corresponds to modeling surrounding environmental data using vehicle sensors (speed, location, direction, vehicle status) and autonomous driving sensors (V2X, cameras, etc.).

In particular, in order to implement the autonomous driving system 100, network adaptation technology is necessary to overcome the communication environment, and includes uplink and downlink technology and autonomous driving technology. Among them, uplink technology is responsible for transmitting image and sensor data, and downlink technology is responsible for generating and transmitting control data of the vehicle 110 from a remote location.

In more detail, uplink technology will be described.

The autonomous vehicle (ToV, 110) can encode at least two environment models and transmit them to the autonomous driving center (ToC, 120). At this time, the autonomous vehicle 110 may encode an environmental model including sensing data using an encoder and transmit it to the autonomous driving center through the network 130 (eg, 5G). Likewise, the autonomous driving center 120 can decode the received environment model through a decoder and output it through a display.

At this time, the two environment models may include driver vision data and vehicle sensor data. At this time, the driver's field of view data can be transmitted as an image outside the vehicle (4 channels or 2 channels), and the vehicle sensor data can transmit the vehicle location and status.

To this end, low-latency video communication technology, fast and stable network technology, and low-latency display technology are required. Through this, it is possible to achieve accurate and fast data communication by reducing video and network delay and latency as much as possible.

Additionally, downlink technology is explained.

The autonomous driving center (ToC, 120) can determine the status of the autonomous vehicle 110 and transmit direct or indirect control signals to the autonomous vehicle 110. Here, the direct control signal may include vehicle driving device control data. Accordingly, the autonomous driving center (ToC, 120) can generate and transmit vehicle driving device control data. Additionally, indirect control signals may include driver guide data. Accordingly, the autonomous driving center 120 may generate driver guide data and transmit it to the autonomous vehicle 110.

The autonomous driving system 100 can remotely drive the autonomous vehicle 110 through control of the autonomous driving center 120, and the devices of the autonomous driving center 120 include a steering wheel, transmission, controller, and a plurality of displays. can do. That is, the autonomous driving center 120 generates a control signal for the autonomous vehicle 110 and transmits it to the autonomous vehicle 110, and the autonomous vehicle 110 receives the control signal and transmits it to the autonomous vehicle 110 as is. The actuator can be manipulated.

Figure 2 is a diagram explaining the configuration of an autonomous driving system according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Referring to FIG. 2 , the autonomous driving system 200 may include an autonomous vehicle 210, an autonomous driving center 220, and a network 230. However, here, the fact that the autonomous driving system 200 includes all of the above-described components does not mean that they are physically included, but rather means that they can be connected and operated within the autonomous driving system 200.

The self-driving vehicle 210 includes an autonomous driving sensor processing unit 211, an autonomous driving algorithm unit 212, an autonomous driving control signal generating unit 213, an image and data transmitting unit 214, and a control signal receiving unit 215. can do. Here, the image and data transmitting unit 214 and the control signal receiving unit 215 may correspond to the autonomous driving system 200 device installed in the autonomous vehicle 210.

The autonomous driving sensor processing unit 211 can process data sensed from various sensors included in the autonomous vehicle 210.

The autonomous driving algorithm unit 212 may generate and process an algorithm using data processed by the autonomous driving sensor processing unit 211.

The autonomous driving control signal generator 213 generates an autonomous driving control signal for controlling the autonomous vehicle 210 based on the algorithm received from the autonomous driving algorithm unit 212 or the control signal received from the control signal receiving unit 215. can be created. Here, the control signal received from the control signal receiver 215 may include a steering control signal, a brake control signal, a transmission control signal, and an acceleration control signal. That is, the control signal may include a plurality of control signals for controlling the autonomous vehicle 210. Accordingly, the autonomous driving control signal generator 2130 may generate a steering control signal, a brake control signal, a transmission control signal, and an acceleration control signal for controlling the autonomous vehicle 210.

The image and data transmission unit 214 may transmit images and data collected from the self-driving vehicle 210 to the self-driving center 220 through the network 230.

The control signal receiver 215 may receive a control signal from the autonomous driving center 220 through the network 230.

The autonomous driving center 220 may include a display 221, an image and data receiving unit 222, and a control signal transmitting unit 223. Here, the image and data receiving unit 222 and the control signal transmitting unit 223 may correspond to the autonomous driving system 200 device installed in the autonomous driving center 220.

The display 221 can output images and data received from the image and data receiver 222.

The image and data receiver 222 may receive images and data collected from the autonomous vehicle 210 through the network 230.

The control signal transmission unit 223 transmits control signals generated by various driving devices present in the autonomous driving center 220 (which can be installed in the autonomous driving center 220 for remote driving mode) to the network 230. ) can be transmitted to the autonomous vehicle 210.

In particular, the autonomous vehicle 210 may operate in an autonomous driving mode or a remote driving mode within the autonomous driving system 200. At this time, when the autonomous vehicle 210 operates in the autonomous driving mode, the autonomous vehicle 210 includes the autonomous driving sensor processing unit 211, the autonomous driving algorithm unit 212, and Based on the autonomous driving control signal generator 213, steering (steering handle), brakes, transmission, and engine output can be adjusted.

On the other hand, the case where the autonomous vehicle 210 operates in remote driving mode is explained as follows.

When the autonomous vehicle 210 operates in a remote driving mode, the autonomous vehicle 210 includes an autonomous driving sensor processing unit 211, an autonomous driving algorithm unit 212, and an autonomous driving control unit mounted on the autonomous vehicle 210. A control signal transmitted by the autonomous driving center 220 instead of the signal output from the signal generator 213 (here, the control signal is a steering control signal, a brake control signal, a transmission control signal, and an acceleration control signal as in the autonomous driving mode) (may include) can be used to move the autonomous vehicle 210.

At this time, the control signal transmitted from the autonomous driving center 220 through the control signal transmitter 223 is called the first control signal, and the control signal received from the autonomous vehicle 210 through the control signal receiver 215 is called the first control signal. It may be referred to as a second control signal, and the control signal ultimately used to move the autonomous vehicle 210 may be referred to as a third control signal.

In this case, the third control signal and the second control signal may be the same or may be changed to the final command depending on the autonomous driving system 200. For example, if the rotation angle of the steering wheel is transmitted in the first control signal, the vehicle ECU (electronic control unit, which controls the status of the car's engine, automatic transmission, and ABS) can understand this in the third control signal. It can be changed with the command.

However, in the case of the second control signal, since it is a control signal transmitted through the network 230, an error may be included depending on the environment, or the command may be changed due to a malicious impulse attack such as a jamming attack. This is a problem that occurs due to the use of the network 230, and algorithms such as error correction may be included to compensate for this, but realistically, there is no method that can perfectly compensate for all network environments.

That is, in general data communication, such errors can be minimized (for example, a certain amount of error can be replaced by retransmitting the packet or ignoring the error and accepting it), but the control signal of the autonomous vehicle 210 does not allow the packet to be retransmitted in time. Accidents that occur when errors cannot be accepted and are ignored can result in casualties, so a solution is needed.

Therefore, in one embodiment of the present invention, when operating an autonomous driving system, the normal or abnormal state of various autonomous driving control signals transmitted to the autonomous vehicle is analyzed and applied to enable stable autonomous driving of the autonomous vehicle and remote location (autonomous driving system). We would like to propose a remote driving control signal processing unit installed in the center.

More specifically, the autonomous driving center can generate vehicle progress information using other signals excluding the autonomous vehicle signal. Additionally, the autonomous driving center can generate progress information of the autonomous vehicle in parallel using signals from the autonomous vehicle. At this time, other signals excluding the autonomous vehicle signal and the autonomous vehicle signal may be encoded and transmitted to the autonomous vehicle.

Afterwards, self-driving vehicles can analyze each received signal and compare them. Accordingly, the autonomous vehicle can analyze the normal or abnormal state of the current signal and use it for autonomous driving control. At this time, it is not important whether the problem is the network of the received signal, the device in the autonomous driving center, or whether there was a malicious jamming attack, but only the reliability of the received control signal can be known.

Hereinafter, it will be described in detail through FIGS. 3 to 10.

Figure 3 is a diagram explaining the configuration of another autonomous driving system according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Referring to FIG. 3 , the autonomous driving system 300 may include an autonomous vehicle 310, an autonomous driving center 320, and a network 330. At this time, redundant description of the configuration described above in FIG. 2 will be omitted. Additionally, the embodiment of FIG. 3 assumes that the autonomous vehicle 310 operates in a remote driving mode using the network 330.

The autonomous vehicle 310 may include an image and data transmission unit 311, a control signal reception unit 312, a control signal analysis unit 313, and an autonomous driving control signal generation unit 314.

The image and data transmission unit 311 may transmit images and data within the self-driving vehicle 310 to the self-driving center 320 through the network 330.

The control signal receiver 312 may receive an integrated control signal transmitted from the autonomous driving center 320 through the network 330. At this time, the integrated control signal may include additional information and control signals. The control signal receiver 312 may transmit the received integrated control signal to the control signal analysis unit 313.

The control signal analysis unit 313 may analyze the received integrated control signal. The control signal analysis unit 313 may analyze the received integrated control signal, process it into a second control signal, and then transmit the second control signal to the autonomous driving control signal generation unit 314. At this time, the control signal analysis unit 313 will be described in detail with reference to FIG. 7 below.

The autonomous driving control signal generator 314 may generate a third control signal for controlling the autonomous vehicle 310 based on the second signal received from the control signal analysis unit 313.

The autonomous driving center 320 may include an image and data receiving unit 321, a control signal transmitting unit 322, a control signal generating unit 323, a display 324, and a driving device 325.

The video and data receiving unit 321 may receive video and data of the autonomous vehicle 310 through the network 330. The image and data receiver 321 may transmit the received image and data to the display 324. In one embodiment, the image and data receiver 321 may transmit the received additional information to the control signal generator 323.

The control signal transmitter 322 may transmit the integrated control signal generated through the control signal generator 323 to the autonomous vehicle 310 through the network 330. Here, the integrated control signal includes additional information and a control signal, and can be transmitted in separate packets.

The control signal generator 323 may generate an integrated control signal for controlling the autonomous vehicle 310 based on the received additional information and the control signal received from the driving device 325. Here, the control signal received from the driving device 325 may include a steering control signal, a brake control signal, a transmission control signal, and an acceleration control signal. At this time, the control signal generator 323 will be described in detail with reference to FIG. 4 below.

The display 324 can output images and data received through the image and data receiver 321.

The driving device 325 is installed in the autonomous driving center 320 for remote driving, and the user can directly control the autonomous vehicle 310 for remote driving through the driving device 325.

Accordingly, the autonomous driving center 320 generates a plurality of different information and transmits it through the network 330, and the autonomous vehicle 310 receives it, compares and analyzes it from different perspectives, and signals the vehicle control signal. Because it is used, it is possible to prevent sudden movement of the autonomous vehicle 310 and prevent casualties.

In addition, when the autonomous driving center 320 engages in sudden braking or sharp steering to prevent an accident, this can also be detected by the autonomous vehicle 310, so that autonomous driving can determine whether the control signal is an abnormal signal generated by the network 330. There is an advantage that the center 320 can discern with a high probability whether the signal is intended.

Figure 4 is a diagram explaining the configuration of a control signal generator according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Referring to FIG. 4 , the control signal generation unit 400 may include an additional information generation unit 410, a control command generation unit 420, and a message structuring unit 430.

The additional information generator 410 may generate additional information based on additional information (eg, trajectory) received through an image and data receiver (not shown). Here, the additional information may include image information from a camera installed in the autonomous vehicle. At this time, the additional information generator 410 will be described in detail with reference to FIG. 5 below.

The control command generator 420 may receive a steering angle, brake pressure, forward or reverse signal, etc. from a driving device (not shown). Accordingly, the control command generator 420 may generate a control signal based on the steering device sensor signal generated from the driving device. Thereafter, the control command generation unit 420 may transmit the generated control signal to the message structuring unit 430. Additionally, in another embodiment, the control signal may be directly generated by the CPU within the driving device. In this case, the control command generator 420 can directly transmit the control signal received from the driving device to the message structuring unit 430.

The message structuring unit 430 may structure the additional information generated by the additional information generating unit 410 and the control signal generated by the control command generating unit 420 into one message and transmit it to the control signal transmitting unit (not shown). there is.

Figure 5 is a diagram explaining the configuration of an additional information generator according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Referring to FIG. 5 , the additional information generation unit 500 may include a preprocessing unit 510, a detection unit 520, a visual trajectory generating unit (530), and a position estimation unit (540).

The preprocessing unit 510 can preprocess additional information received from the image and data receiving unit (e.g., front, front, and rear camera images captured through cameras installed in the autonomous vehicle, image information about the current location, etc.) there is.

The detection unit 520 may detect a road or lane based on additional information preprocessed through the preprocessor 510.

The trajectory generator 530 may generate a visually visible trajectory based on road or lane information detected through the additional information and the detection unit 520. That is, the trajectory generator 530 may generate a visually visible trajectory based on the received front and rear camera images and detected road or lane information. Through this, as shown in Figure 6, the driver in the autonomous vehicle can predict the driving intention of the driver in the autonomous driving center.

In one embodiment, the trajectory generator 530 may generate first additional information about whether the autonomous vehicle is proceeding within a lane or on a road. Thereafter, the trajectory generator 530 may include the generated first additional information in the integrated control signal and transmit it to the autonomous vehicle through the network.

The location estimation unit 540 may receive additional information received from the image and data reception unit. That is, the location estimation unit 540 can estimate the current location based on additional information before passing through the preprocessor 510. In one embodiment, the location estimation unit 540 may generate second additional information including a current location estimation message based on the received additional information and the first additional information received through the trajectory generator 530. . Thereafter, the location estimation unit 540 may include the generated second additional information in the integrated control signal and transmit it to the autonomous vehicle through the network.

Figure 6 is a diagram showing a screen output from an autonomous vehicle according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Based on the method described above in FIG. 5, the autonomous vehicle can receive additional information including a visually visible trajectory.

Accordingly, as shown in (a) of FIG. 6, the autonomous vehicle can see that the vehicle is moving straight within the lane, and as shown in (b) of FIG. 6, the autonomous vehicle can know that the vehicle is trying to change lanes to the right lane.

At this time, an important point is that the sensor output of the driving device is not used to predict the driver's driving path. In other words, the intention of the driver of the autonomous driving center who drives purely by looking at the screen must be encoded into information. At this time, the output outputs a plurality of way points based on latitude and longitude from the current location using the visual trajectory generated in Figure 5, or sets the current location as the origin and calculates the azimuth and distance using the origin coordinate system. Multiple waypoints can be output based on .

In addition, as shown in Figures 6 (a) and (b), two pieces of information can be generated simultaneously using position estimation such as Kalman Filter or Particle Filter based on the current position. Here, the Kalman filter corresponds to a recursive filter that estimates the state of a linear dynamical system based on measurements containing noise. Likewise, the particle filter is a prediction technology based on simulation and corresponds to a filter that infers the information of the system by applying several randomly generated inputs with an appropriately proposed probability distribution to the system and combining them. However, since the Kalman filter and particle filter are already known technologies, detailed descriptions will be omitted. Accordingly, the position estimation unit described above in FIG. 5 may include filters such as a Kalman filter and a particle filter.

Afterwards, this additional information and the control signals of the driving device at the autonomous driving center can be individually messaged and packetized and transmitted to the autonomous vehicle.

In one embodiment, the additional information may be transmitted through a safe channel such as TCP (Transmission Control Protocol) because it also conveys the direction and situation of the future autonomous vehicle.

Accordingly, the driver of the autonomous vehicle can predict the driving intention of the driver of the autonomous driving center.

Figure 7 is a diagram explaining the configuration of a control signal analysis unit according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Self-driving vehicles can receive integrated control signals from the self-driving center through the network. At this time, the integrated control signal includes additional information and control signals, and in an autonomous vehicle, these need to be analyzed separately. At this time, the integrated control signal may be packetized and transmitted in the form of a message. Accordingly, the autonomous vehicle can analyze the additional information and control signal of the integrated control signal received through the control signal analysis unit, respectively.

Referring to FIG. 7 , the control signal analysis unit 700 may include an additional information analysis unit 710, a control command analysis unit 720, a comparison unit 730, and a control signal generation unit 740.

The additional information analysis unit 710 may analyze the received additional information. In one embodiment, the additional information analysis unit 710 may analyze a driving situation constructed from data excluding the driving device of the autonomous driving center. For example, the additional information analysis unit 710 can analyze whether the autonomous vehicle is currently crossing the lane or is in progress.

The control command analysis unit 720 may analyze the received control signal. In one embodiment, the control command analysis unit 720 may analyze control signals received from the driving device of the autonomous driving center.

The comparison unit 730 may compare the additional information analyzed through the additional information analysis unit 710 and the control signal analyzed through the control command analysis unit 720. More specifically, the comparison unit 730 may transmit a control signal to the control signal generator 740 only when the analyzed additional information and the control signal represent the same vehicle.

The autonomous driving control signal generator 740 may generate a control signal for controlling the autonomous vehicle based on the two pieces of information received through the comparison unit 730. In one embodiment, when the additional information and control signals represent the same vehicle through the comparison unit 730, a control signal for normally controlling the autonomous vehicle is generated, but otherwise, the autonomous vehicle is normally controlled. An autonomous vehicle can generate a control signal to operate a designated emergency process (for example, emergency roadside parking, etc.) without generating a control signal for the autonomous vehicle.

Figure 8 is a flowchart explaining a control method of an autonomous driving system according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Referring to FIG. 8, in step S810, the autonomous vehicle may transmit images and data to the autonomous driving center through the network. Here, the images and data may include image information from the front camera and rear camera installed in the autonomous vehicle.

In step S820, the autonomous driving center may generate additional information by preprocessing the received images and data. At this time, the autonomous driving center can detect the lane based on the generated additional information. Additionally, the autonomous driving center can generate a trajectory based on the detected lane. Additionally, the autonomous driving center can estimate the current location of the autonomous vehicle based on the generated trajectory.

In step S830, the autonomous driving center may receive a control signal from the driving device. Here, the control signal may include at least one of the steering angle of the driving device, brake pressure, forward signal, and reverse signal.

In step S840, the autonomous driving center may generate an integrated control signal based on the additional information and the control signal. At this time, the autonomous driving center can individually message and package additional information and control signals.

In step S850, the autonomous driving center may transmit an integrated control signal to the autonomous vehicle through the network.

In step S860, the autonomous vehicle can analyze each of the additional information and control signals included in the integrated control signal.

In step S870, the autonomous vehicle may generate an autonomous driving control signal to control the vehicle. In one embodiment, the autonomous vehicle may generate an autonomous driving control signal only when the additional information and control signals included in the integrated control signal represent the same autonomous vehicle. In another embodiment, the autonomous vehicle may generate a control signal to control an emergency process to operate when the additional information and control signal included in the integrated control signal do not represent the same autonomous vehicle.

Figure 9 is a flowchart explaining a control method of an autonomous vehicle according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Referring to FIG. 9, in step S910, the self-driving vehicle can transmit images and data to the self-driving center through a network.

In step S920, the autonomous vehicle may receive an integrated control signal from the autonomous driving center through the network.

In step S930, the autonomous vehicle may analyze the integrated control signal. Here, the integrated control signal may include additional information and control signals.

In step S940, the autonomous vehicle may determine whether the additional information and the control signal represent the same autonomous vehicle.

If the additional information and the control signal represent the same autonomous vehicle, in step S950, the autonomous vehicle may generate an autonomous driving control signal for controlling the autonomous vehicle.

If the additional information and the control signal do not represent the same autonomous vehicle, in step S960, the autonomous vehicle may generate a control signal that controls the emergency process to operate.

Figure 10 is a flowchart explaining a control method of an autonomous driving center according to an embodiment of the present invention. Hereinafter, descriptions that overlap with the above-described content will be omitted.

Referring to FIG. 10, in step S1010, the autonomous driving center can receive images and data from the autonomous vehicle through the network.

In step S1020, the autonomous driving center may generate additional information based on the received images and data. Here, the images and data may include image information from the front camera and rear camera installed in the autonomous vehicle. At this time, the autonomous driving center can generate additional information by preprocessing the received images and data, and detect the lane based on the generated additional information. In one embodiment, the autonomous driving center may generate a trajectory based on the detected lane and estimate the current location of the autonomous vehicle based on the generated trajectory.

In step S1030, the autonomous driving center may generate a control signal based on a signal received from a driving device installed in the autonomous driving center. Here, the control signal may include at least one of the steering angle of the driving device, brake pressure, forward signal, and reverse signal.

In step S1040, the autonomous driving center may generate an integrated control signal including additional information and control signals. At this time, the autonomous driving center can individually message and packetize additional information and control signals.

In step S1050, the autonomous driving center may transmit an integrated control signal to the autonomous vehicle.

Additionally, of course, the embodiments of FIGS. 1 to 7 can be performed in the same manner as FIGS. 8 to 10.

The above-described present invention can be implemented as computer-readable code on a program-recorded medium. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. It also includes those implemented in the form of carrier waves (e.g., transmission via the Internet).

The video transmission and reception system and method of the terminal, cloud server, and cloud AR platform according to the embodiment of the present invention have been described above as specific embodiments, but this is only an example and the present invention is not limited thereto, and the basic idea disclosed herein It should be interpreted as having the widest scope in accordance with . A person skilled in the art may combine and substitute the disclosed embodiments to implement embodiments not specified, but this also does not deviate from the scope of the present invention. In addition, a person skilled in the art can easily change or modify the embodiments disclosed based on the present specification, and it is clear that such changes or modifications also fall within the scope of the present invention.

300: Autonomous driving system
310: Self-driving vehicle
311: Video and data transmission unit
312: Control signal receiver
313: Control signal analysis unit
314: Self-driving control signal generation unit
320: Autonomous driving center
321: Video and data receiving unit
322: Control signal transmission unit
323: Control signal generator
324: display
325: Driving device
330: network

Claims (26)

A video and data transmission unit that transmits video and data of the autonomous vehicle through a network;
a control signal receiving unit that receives an integrated control signal through the network;
a control signal analysis unit that analyzes the integrated control signal; and
Comprising an autonomous driving control signal generator that generates an autonomous driving control signal for controlling the autonomous vehicle,
The control signal analysis unit
Analyzing each of the additional information and control signals included in the integrated control signal,
An autonomous vehicle, characterized in that generating the autonomous driving control signal when the additional information and the control signal indicate the same autonomous vehicle. According to claim 1,
The autonomous driving control signal generator
When the additional information and the control signal do not represent the same autonomous vehicle, generating a control signal to control an emergency process to operate. According to claim 1,
An autonomous vehicle, characterized in that the additional information is information excluding signals received from the driving device of the autonomous driving center. According to claim 1,
An autonomous vehicle, characterized in that the images and data of the autonomous vehicle include image information from a front camera and a rear camera installed on the autonomous vehicle. Transmitting images and data of the autonomous vehicle through a network;
Receiving an integrated control signal through the network;
analyzing the integrated control signal;
Generating an autonomous driving control signal to control the autonomous vehicle;
Analyzing additional information and control signals included in the integrated control signal, respectively; and
When the additional information and the control signal indicate the same autonomous vehicle, generating the autonomous driving control signal. According to claim 5,
If the additional information and the control signal do not represent the same autonomous vehicle, generating a control signal to control an emergency process to operate. According to claim 5,
A method of controlling an autonomous vehicle, characterized in that the additional information is information excluding signals received from the driving device of the autonomous driving center. According to claim 5,
A method of controlling an autonomous vehicle, characterized in that the images and data of the autonomous vehicle include image information from a front camera and a rear camera installed on the autonomous vehicle.
driving device;
An image and data receiving unit that receives images and data of an autonomous vehicle through a network;
Control signal generator; and
Includes a control signal transmission unit,
The control signal generator
an additional information generation unit that generates additional information; and
It includes a control command generator that generates a control signal based on a signal received from the driving device,
Generate an integrated control signal including the additional information and control signal,
An autonomous driving center, characterized in that the integrated control signal is transmitted to the autonomous vehicle through the control signal transmission unit. According to clause 9,
The additional information generator
An autonomous driving center that generates additional information based on the received images and data, wherein the images and data include image information from a front camera and a rear camera installed in the autonomous vehicle. According to claim 10,
The additional information generator
Preprocessing the received images and data to generate additional information,
An autonomous driving center characterized in that it detects a lane based on the generated additional information. According to claim 11,
The additional information generator
An autonomous driving center, wherein a trajectory is generated based on the detected lane, and information about the generated trajectory is included in the additional information. According to claim 12,
The control signal generator
Further comprising a location estimation unit,
The location estimation unit
An autonomous driving center, wherein the current location of the autonomous vehicle is estimated based on the generated trajectory, and information about the estimated location is included in the additional information. According to claim 13,
The location estimation unit
Autonomous driving, wherein a plurality of first way points are output based on latitude and longitude from the estimated current location, and information about the output first way point is included in the additional information. center. According to claim 13,
The location estimation unit
The estimated current position is set as the origin, and a plurality of second waypoints are output based on the distance from the origin coordinate system and the azimuth, wherein information about the output second waypoints is included in the additional information. Autonomous driving center. According to clause 9,
The control signal generator
An autonomous driving center, characterized in that the additional information and the control signal are individually messaged and packetized. According to clause 9,
The control signal includes at least one of a steering angle of the driving device, brake pressure, a forward signal, and a reverse signal. Receiving images and data of an autonomous vehicle through a network;
Generating additional information;
generating a control signal based on a signal received from the driving device;
generating an integrated control signal including the additional information and the control signal; and
A control method of an autonomous driving center, comprising transmitting the integrated control signal to an autonomous vehicle. According to claim 18,
Further comprising generating additional information based on the received images and data, wherein the images and data include image information from a front camera and a rear camera installed in the autonomous vehicle. Control method of the center. According to claim 19,
Preprocessing the received images and data to generate additional information; and
A control method for an autonomous driving center, further comprising detecting a lane based on the generated additional information. According to claim 20,
A control method for an autonomous driving center, further comprising generating a trajectory based on the detected lane, wherein information about the generated trajectory is included in the additional information. According to claim 21,
A method of controlling an autonomous driving center further comprising estimating the current location of the autonomous vehicle based on the generated trajectory, wherein information about the estimated location is included in the additional information. According to claim 22,
Further comprising outputting a plurality of first way points based on latitude and longitude from the estimated current location, wherein information about the output first way points is included in the additional information. A control method for an autonomous driving center. According to claim 23,
Setting the estimated current position as the origin and outputting a plurality of second waypoints based on the distance from the origin coordinate system and the azimuth, wherein information about the output second waypoints is included in the additional information. A control method for an autonomous driving center, characterized in that: According to claim 18,
A control method for an autonomous driving center, further comprising the step of individually messaging and packetizing the additional information and the control signal. According to claim 18,
The control signal is characterized in that it includes at least one of a steering angle of the driving device, a brake pressure, a forward signal, and a reverse signal.