KR20210070700A - Method of ai learning data inheritance in autonomous driving system - Google Patents

Abstract

A method of inheriting artificial intelligence training data of a vehicle in an autonomous driving system comprises: a step of transmitting sensing information associated with a stationary object to a server based on sensing information generated through a sensor; a step of receiving a first model trained by using the sensing information associated with the stationary object from the server; a step of transmitting sensing information associated with a moving object generated by using the first model to the server; and a step of receiving a second model trained by using the sensing information associated with the moving object from the server. Accordingly, the vehicle can inherit artificial intelligence training data. In addition, one or more among an autonomous vehicle, a user terminal, and a server of the present invention can be linked to an artificial intelligence module, an unmanned aerial vehicle (UAV) robot, an augmented reality (AR) device, a virtual reality (VR) device, a 5G service-related device, etc.

Description

How to inherit artificial intelligence learning data from autonomous driving system {METHOD OF AI LEARNING DATA INHERITANCE IN AUTONOMOUS DRIVING SYSTEM}

This specification relates to an autonomous driving system and to a method of inheriting artificial intelligence learning data.

An automobile may be classified into an internal combustion engine automobile, an external combustion engine automobile, a gas turbine automobile, an electric vehicle, or the like, according to a type of a prime mover used.

Autonomous vehicle refers to a vehicle that can drive itself without driver or passenger manipulation, and Automated Vehicle & Highway Systems is a system that monitors and controls such autonomous vehicles so that they can operate on their own. say

An object of the present specification is to propose a method of inheriting learning data for learning an artificial intelligence model in an autonomous driving system.

In addition, an object of the present specification is to propose a method of inheriting learning data for learning a specialized artificial intelligence model according to the characteristics of an object sensed by a vehicle in an autonomous driving system.

The technical problems to be achieved by this specification are not limited to the technical problems mentioned above, and other technical problems not mentioned are clear to those of ordinary skill in the art to which this specification belongs from the detailed description of the invention below. will be able to be understood

An aspect of the present specification provides a method for inheriting artificial intelligence learning data of a vehicle in an autonomous driving system, the method comprising: transmitting, to a server, sensing information related to a fixed object based on sensing information generated through a sensor; receiving, from the server, a first model learned using sensing information related to the fixed object; transmitting, to the server, sensing information related to a floating object generated using the first model; and receiving, from the server, a second model learned using sensing information related to the moving object. The first model may determine a fixed object included in the sensing information generated by the sensor, and the second model may determine a moving object included in the sensing information generated by the sensor.

In addition, transmitting, to the server, a request message for requesting information on a fixed object located in a driving section of the vehicle or a section to be driven; and receiving, from the server, information on the fixed object. may further include.

In addition, transmitting the specification information of the vehicle to another vehicle; receiving, from the other vehicle, sensing information related to a fixed object of another vehicle, based on the specification information of the vehicle; and learning the first model by using sensing information related to a stationary object of the other vehicle. may further include.

The method may further include: receiving, from the other vehicle, a first model of the other vehicle based on the specification information of the vehicle; may further include.

In addition, receiving a first V2X message from another vehicle; and setting, based on a result of decoding the first V2X message, the sensing information generated through the sensor as the sensing information related to the moving object; may further include.

In addition, receiving a second V2X message from the other vehicle; and based on a result of decoding the second V2X message, releasing the setting of the sensing information related to the floating object; may further include.

Another aspect of the present specification provides a method for inheriting artificial intelligence learning data of a server in an autonomous driving system, the method comprising: receiving, from a first vehicle, sensing information related to a fixed object of the first vehicle; receiving, from a second vehicle, sensing information related to a stationary object of the second vehicle; learning a first model by using the sensing information related to the fixed object of the first vehicle and the sensing information related to the fixed object of the second vehicle; and transmitting the first model for determining the fixed object included in the sensing information to the first vehicle or the second vehicle. may include.

In addition, receiving, from the first vehicle, sensing information related to the moving object of the first vehicle; receiving, from the second vehicle, sensing information related to a moving object of the second vehicle; learning a second model by using the sensing information related to the moving object of the first vehicle and the sensing information related to the moving object of the second vehicle; and transmitting, to the first vehicle or the second vehicle, the first model for determining a moving object included in the sensing information. may further include.

In addition, the sensing information related to the moving object of the first vehicle is generated using a first model of the first vehicle, and the sensing information related to the moving object of the second vehicle is generated using the first model of the second vehicle. can be created by

The method may further include: receiving, from the first vehicle, a request message for requesting information on a fixed object located in a driving section of the first vehicle or a scheduled driving section; and transmitting, to the first vehicle, the information of the fixed object as a response to the request message. may further include.

In addition, the information on the fixed object may include specification information of a vehicle that has generated the information on the fixed object.

In addition, the request message may include the specification information of the first vehicle, and the information of the fixed object may be transmitted based on the specification information of the first vehicle.

The method may further include: generating information on a fixed object by using the sensing information related to the fixed object of the first vehicle and the sensing information related to the fixed object of the second vehicle; may further include.

Another aspect of the present specification provides a vehicle for performing an AI learning data inheritance method in an autonomous driving system, comprising: a sensor; transceiver; Memory; and a processor for controlling the sensor, the transceiver, and the memory. and, the processor transmits sensing information related to a fixed object to the server through the transceiver, based on the sensing information generated through the sensor, and from the server through the transceiver, sensing information related to the fixed object. Receives a first model learned using , transmits sensing information related to a floating object generated using the first model to the server through the transceiver, and from the server through the transceiver to the floating object Receives a second model learned using sensing information related to , the first model determines a stationary object included in the sensing information generated through the sensor, and the second model is a sensing device generated through the sensor It is possible to judge the moving objects included in the information.

The present specification may inherit the learning data for learning the artificial intelligence model in the autonomous driving system.

In addition, the present specification may inherit learning data for learning a specialized artificial intelligence model according to the characteristics of an object sensed by the vehicle in the autonomous driving system.

The effects obtainable in the present specification are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which this specification belongs from the description below. .

1 illustrates a block diagram of a wireless communication system to which the methods proposed in the present specification can be applied.
2 is a diagram illustrating an example of a signal transmission/reception method in a wireless communication system.
3 shows an example of basic operations of a user terminal and a 5G network in a 5G communication system.
4 is a diagram illustrating a vehicle according to an embodiment of the present specification.
5 is a block diagram of an AI device according to an embodiment of the present specification.
6 is a diagram for describing a system in which an autonomous driving vehicle and an AI device are linked according to an embodiment of the present specification.
7 is an example of a DNN model to which this specification can be applied.
8 is an example of V2X communication to which this specification can be applied.
9 illustrates a resource allocation method in a sidelink in which V2X is used.
10 is a diagram illustrating a procedure for a broadcast mode of V2X communication using PC5.
11 to 14 are exemplary embodiments to which the present specification can be applied.
15 is an embodiment of a vehicle to which this specification can be applied.
16 is an embodiment of a server to which this specification can be applied.
17 is an example of an object detection and tracking method to which the present specification can be applied.
18 is an embodiment to which the present specification can be applied.
19 is an embodiment to which the present specification can be applied.
20 is a block diagram of a general apparatus to which the present specification can be applied.
BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to help the understanding of the present specification, provide embodiments of the present specification, and together with the detailed description, explain the technical features of the present specification.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for the components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have a meaning or role distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present specification , should be understood to include equivalents or substitutes.

Terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as "comprises" or "have" are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, 5G communication required by an autonomous driving device and/or AI processor requiring AI-processed information will be described through paragraphs A to G.

A. Example UE and 5G network block diagram

1 illustrates a block diagram of a wireless communication system to which the methods proposed in the present specification can be applied.

Referring to FIG. 1 , a device (AI device) including an AI module may be defined as a first communication device ( 910 in FIG. 1 ), and a processor 911 may perform detailed AI operations.

A second communication device ( 920 in FIG. 1 ) may perform a 5G network including another device (AI server) that communicates with the AI device, and the processor 921 may perform detailed AI operations.

The 5G network may be represented as the first communication device, and the AI device may be represented as the second communication device.

For example, the first communication device or the second communication device may include a base station, a network node, a transmitting terminal, a receiving terminal, a wireless device, a wireless communication device, a vehicle, a vehicle equipped with an autonomous driving function, and a connected car. ), drone (Unmanned Aerial Vehicle, UAV), AI (Artificial Intelligence) module, robot, AR (Augmented Reality) device, VR (Virtual Reality) device, MR (Mixed Reality) device, hologram device, public safety device, MTC device , IoT devices, medical devices, fintech devices (or financial devices), security devices, climate/environmental devices, devices related to 5G services, or other devices related to the 4th industrial revolution field.

For example, a terminal or user equipment (UE) includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, personal digital assistants (PDA), a portable multimedia player (PMP), a navigation system, and a slate PC. (slate PC), tablet PC (tablet PC), ultrabook (ultrabook), wearable device (e.g., watch-type terminal (smartwatch), glass-type terminal (smart glass), HMD (head mounted display)) and the like. For example, the HMD may be a display device worn on the head. For example, an HMD may be used to implement VR, AR or MR. For example, the drone may be a flying vehicle that does not ride by a person and flies by a wireless control signal. For example, the VR device may include a device that implements an object or a background of a virtual world. For example, the AR device may include a device that implements by connecting an object or background in the virtual world to an object or background in the real world. For example, the MR device may include a device that implements a virtual world object or background by fusion with a real world object or background. For example, the hologram device may include a device for realizing a 360-degree stereoscopic image by recording and reproducing stereoscopic information by utilizing an interference phenomenon of light generated by the meeting of two laser beams called holography. For example, the public safety device may include an image relay device or an image device that can be worn on a user's body. For example, the MTC device and the IoT device may be devices that do not require direct human intervention or manipulation. For example, the MTC device and the IoT device may include a smart meter, a bending machine, a thermometer, a smart light bulb, a door lock, or various sensors. For example, a medical device may be a device used for the purpose of diagnosing, treating, alleviating, treating, or preventing a disease. For example, a medical device may be a device used for the purpose of diagnosing, treating, alleviating or correcting an injury or disorder. For example, a medical device may be a device used for the purpose of examining, replacing, or modifying structure or function. For example, the medical device may be a device used for the purpose of controlling pregnancy. For example, the medical device may include a medical device, a surgical device, an (ex vivo) diagnostic device, a hearing aid, or a device for a procedure. For example, the security device may be a device installed to prevent a risk that may occur and maintain safety. For example, the security device may be a camera, CCTV, recorder or black box. For example, the fintech device may be a device capable of providing financial services such as mobile payment.

Referring to FIG. 1 , a first communication device 910 and a second communication device 920 include a processor 911,921, a memory 914,924, and one or more Tx/Rx RF modules (radio frequency module, 915,925). , including Tx processors 912 and 922 , Rx processors 913 and 923 , and antennas 916 and 926 . Tx/Rx modules are also called transceivers. Each Tx/Rx module 915 transmits a signal via a respective antenna 926 . The processor implements the functions, processes and/or methods salpinned above. The processor 921 may be associated with a memory 924 that stores program code and data. Memory may be referred to as a computer-readable medium. More specifically, in DL (communication from a first communication device to a second communication device), the transmit (TX) processor 912 implements various signal processing functions for the L1 layer (ie, the physical layer). The receive (RX) processor implements the various signal processing functions of L1 (ie, the physical layer).

The UL (second communication device to first communication device) is handled in the first communication device 910 in a manner similar to that described with respect to the receiver function in the second communication device 920 . Each Tx/Rx module 925 receives a signal via a respective antenna 926 . Each Tx/Rx module provides an RF carrier and information to the RX processor 923 . The processor 921 may be associated with a memory 924 that stores program code and data. Memory may be referred to as a computer-readable medium.

According to an embodiment of the present specification, the first communication device may be a vehicle, and the second communication device may be a 5G network.

B. Signal transmission/reception method in wireless communication system

2 is a diagram illustrating an example of a signal transmission/reception method in a wireless communication system.

Referring to FIG. 2 , the UE performs an initial cell search operation such as synchronizing with the BS when the power is turned on or a new cell is entered ( S201 ). To this end, the UE receives a primary synchronization channel (P-SCH) and a secondary synchronization channel (S-SCH) from the BS, synchronizes with the BS, and acquires information such as cell ID can do. In the LTE system and the NR system, the P-SCH and the S-SCH are called a primary synchronization signal (PSS) and a secondary synchronization signal (SSS), respectively. After the initial cell discovery, the UE may receive a physical broadcast channel (PBCH) from the BS to obtain broadcast information in the cell. Meanwhile, the UE may check the downlink channel state by receiving a downlink reference signal (DL RS) in the initial cell search step. After the initial cell search, the UE receives a physical downlink control channel (PDCCH) and a physical downlink shared channel (PDSCH) according to information carried on the PDCCH to obtain more specific system information. It can be done (S202).

On the other hand, when there is no radio resource for the first access to the BS or signal transmission, the UE may perform a random access procedure (RACH) to the BS (steps S203 to S206). To this end, the UE transmits a specific sequence as a preamble through a physical random access channel (PRACH) (S203 and S205), and a random access response to the preamble through the PDCCH and the corresponding PDSCH (random access response, RAR) message may be received (S204 and S206). In the case of contention-based RACH, a contention resolution procedure may be additionally performed.

After performing the process as described above, the UE receives PDCCH/PDSCH (S207) and a physical uplink shared channel (PUSCH)/physical uplink control channel as a general uplink/downlink signal transmission process. Uplink control channel, PUCCH) transmission (S208) may be performed. In particular, the UE receives downlink control information (DCI) through the PDCCH. The UE monitors a set of PDCCH candidates in monitoring opportunities set in one or more control element sets (CORESETs) on a serving cell according to corresponding search space configurations. The set of PDCCH candidates to be monitored by the UE is defined in terms of search space sets, which may be a common search space set or a UE-specific search space set. CORESET consists of a set of (physical) resource blocks with a time duration of 1 to 3 OFDM symbols. The network may configure the UE to have multiple CORESETs. The UE monitors PDCCH candidates in one or more search space sets. Here, monitoring means trying to decode PDCCH candidate(s) in the search space. If the UE succeeds in decoding one of the PDCCH candidates in the search space, the UE determines that the PDCCH is detected in the corresponding PDCCH candidate, and performs PDSCH reception or PUSCH transmission based on the DCI in the detected PDCCH. The PDCCH may be used to schedule DL transmissions on PDSCH and UL transmissions on PUSCH. Here, the DCI on the PDCCH is a downlink assignment (ie, downlink grant; DL grant), or uplink, including at least modulation and coding format and resource allocation information, related to the downlink shared channel. It includes an uplink grant (UL grant) including a modulation and coding format and resource allocation information related to a shared channel.

Referring to FIG. 2 , an initial access (IA) procedure in a 5G communication system will be additionally described.

The UE may perform cell search, system information acquisition, beam alignment for initial access, DL measurement, and the like based on the SSB. The SSB is mixed with an SS/PBCH (Synchronization Signal/Physical Broadcast channel) block.

SSB consists of PSS, SSS and PBCH. The SSB is configured in four consecutive OFDM symbols, and PSS, PBCH, SSS/PBCH or PBCH are transmitted for each OFDM symbol. PSS and SSS consist of 1 OFDM symbol and 127 subcarriers, respectively, and PBCH consists of 3 OFDM symbols and 576 subcarriers.

Cell discovery refers to a process in which the UE acquires time/frequency synchronization of a cell, and detects a cell ID (Identifier) (eg, Physical layer Cell ID, PCI) of the cell. PSS is used to detect a cell ID within a cell ID group, and SSS is used to detect a cell ID group. PBCH is used for SSB (time) index detection and half-frame detection.

There are 336 cell ID groups, and there are 3 cell IDs for each cell ID group. There are a total of 1008 cell IDs. Information on the cell ID group to which the cell ID of the cell belongs is provided/obtained through the SSS of the cell, and information on the cell ID among 336 cells in the cell ID is provided/obtained through the PSS

The SSB is transmitted periodically according to the SSB period (periodicity). The SSB basic period assumed by the UE during initial cell discovery is defined as 20 ms. After cell access, the SSB period may be set to one of {5ms, 10ms, 20ms, 40ms, 80ms, 160ms} by the network (eg, BS).

Next, the acquisition of system information (SI) will be described.

The SI is divided into a master information block (MIB) and a plurality of system information blocks (SIB). SI other than MIB may be referred to as Remaining Minimum System Information (RMSI). The MIB includes information/parameters for monitoring the PDCCH scheduling the PDSCH carrying the System Information Block1 (SIB1) and is transmitted by the BS through the PBCH of the SSB. SIB1 includes information related to availability and scheduling (eg, transmission period, SI-window size) of the remaining SIBs (hereinafter, SIBx, where x is an integer of 2 or more). SIBx is included in the SI message and transmitted through the PDSCH. Each SI message is transmitted within a periodically occurring time window (ie, an SI-window).

Referring to FIG. 2 , a random access (RA) process in a 5G communication system will be additionally described.

The random access process is used for various purposes. For example, the random access procedure may be used for network initial access, handover, and UE-triggered UL data transmission. The UE may acquire UL synchronization and UL transmission resources through a random access procedure. The random access process is divided into a contention-based random access process and a contention free random access process. The detailed procedure for the contention-based random access process is as follows.

The UE may transmit the random access preamble through the PRACH as Msg1 of the random access procedure in the UL. Random access preamble sequences having two different lengths are supported. The long sequence length 839 applies for subcarrier spacings of 1.25 and 5 kHz, and the short sequence length 139 applies for subcarrier spacings of 15, 30, 60 and 120 kHz.

When the BS receives the random access preamble from the UE, the BS sends a random access response (RAR) message (Msg2) to the UE. The PDCCH scheduling the PDSCH carrying the RAR is CRC-masked and transmitted with a random access (RA) radio network temporary identifier (RNTI) (RA-RNTI). The UE detecting the PDCCH masked by the RA-RNTI may receive the RAR from the PDSCH scheduled by the DCI carried by the PDCCH. The UE checks whether the random access response information for the preamble it has transmitted, that is, Msg1, is in the RAR. Whether or not random access information for Msg1 transmitted by itself exists may be determined by whether a random access preamble ID for the preamble transmitted by the UE exists. If there is no response to Msg1, the UE may retransmit the RACH preamble within a predetermined number of times while performing power ramping. The UE calculates the PRACH transmit power for the retransmission of the preamble based on the most recent path loss and power ramping counter.

The UE may transmit UL transmission on the uplink shared channel as Msg3 of the random access procedure based on the random access response information. Msg3 may include the RRC connection request and UE identifier. As a response to Msg3, the network may send Msg4, which may be treated as a contention resolution message on DL. By receiving Msg4, the UE can enter the RRC connected state.

C. Beam Management (BM) Procedure of 5G Communication System

The BM process may be divided into (1) a DL BM process using SSB or CSI-RS, and (2) a UL BM process using a sounding reference signal (SRS). In addition, each BM process may include Tx beam sweeping to determine a Tx beam and Rx beam sweeping to determine an Rx beam.

Let's look at the DL BM process using SSB.

A configuration for a beam report using the SSB is performed during channel state information (CSI)/beam configuration in RRC_CONNECTED.

- The UE receives from the BS a CSI-ResourceConfig IE including a CSI-SSB-ResourceSetList for SSB resources used for BM. The RRC parameter csi-SSB-ResourceSetList indicates a list of SSB resources used for beam management and reporting in one resource set. Here, the SSB resource set may be set to {SSBx1, SSBx2, SSBx3, SSBx4, ??}. The SSB index may be defined from 0 to 63.

- UE receives signals on SSB resources from the BS based on the CSI-SSB-ResourceSetList.

- When the CSI-RS reportConfig related to reporting on SSBRI and reference signal received power (RSRP) is configured, the UE reports the best SSBRI and RSRP corresponding thereto to the BS. For example, when the reportQuantity of the CSI-RS reportConfig IE is set to 'ssb-Index-RSRP', the UE reports the best SSBRI and the corresponding RSRP to the BS.

If the CSI-RS resource is configured in the same OFDM symbol(s) as the SSB, and 'QCL-TypeD' is applicable, the UE has the CSI-RS and the SSB similarly located in the 'QCL-TypeD' point of view ( quasi co-located, QCL). Here, QCL-TypeD may mean QCL between antenna ports in terms of spatial Rx parameters. When the UE receives signals of a plurality of DL antenna ports in a QCL-TypeD relationship, the same reception beam may be applied.

Next, a DL BM process using CSI-RS will be described.

The Rx beam determination (or refinement) process of the UE using the CSI-RS and the Tx beam sweeping process of the BS will be described in turn. In the UE Rx beam determination process, the repetition parameter is set to 'ON', and in the BS Tx beam sweeping process, the repetition parameter is set to 'OFF'.

First, a process of determining the Rx beam of the UE will be described.

- The UE receives the NZP CSI-RS resource set IE including the RRC parameter for 'repetition' from the BS through RRC signaling. Here, the RRC parameter 'repetition' is set to 'ON'.

- The UE repeats signals on the resource(s) in the CSI-RS resource set in which the RRC parameter 'repetition' is set to 'ON' in different OFDM symbols through the same Tx beam (or DL spatial domain transmission filter) of the BS receive

- The UE determines its own Rx beam.

- The UE omits CSI reporting. That is, the UE may omit the CSI report when the multi-RRC parameter 'repetition' is set to 'ON'.

Next, the Tx beam determination process of the BS will be described.

- The UE receives the NZP CSI-RS resource set IE including the RRC parameter for 'repetition' from the BS through RRC signaling. Here, the RRC parameter 'repetition' is set to 'OFF' and is related to the Tx beam sweeping process of the BS.

- The UE receives signals on resources in the CSI-RS resource set in which the RRC parameter 'repetition' is set to 'OFF' through different Tx beams (DL spatial domain transmission filter) of the BS.

- The UE selects (or determines) the best beam.

- The UE reports the ID (eg, CRI) and related quality information (eg, RSRP) for the selected beam to the BS. That is, when the CSI-RS is transmitted for the BM, the UE reports the CRI and the RSRP to the BS.

Next, a UL BM process using SRS will be described.

- The UE receives the RRC signaling (eg, SRS-Config IE) including the (RRC parameter) usage parameter set to 'beam management' from the BS. SRS-Config IE is used for SRS transmission configuration. The SRS-Config IE includes a list of SRS-Resources and a list of SRS-ResourceSets. Each SRS resource set means a set of SRS-resources.

- The UE determines Tx beamforming for the SRS resource to be transmitted based on the SRS-SpatialRelation Info included in the SRS-Config IE. Here, the SRS-SpatialRelation Info is set for each SRS resource and indicates whether to apply the same beamforming as that used in SSB, CSI-RS, or SRS for each SRS resource.

- If SRS-SpatialRelationInfo is configured in the SRS resource, the same beamforming as that used in SSB, CSI-RS or SRS is applied and transmitted. However, if SRS-SpatialRelationInfo is not configured in the SRS resource, the UE arbitrarily determines Tx beamforming and transmits the SRS through the determined Tx beamforming.

Next, a beam failure recovery (BFR) process will be described.

In a beamformed system, Radio Link Failure (RLF) may frequently occur due to rotation, movement, or beamforming blockage of the UE. Therefore, BFR is supported in NR to prevent frequent RLF from occurring. BFR is similar to the radio link failure recovery process, and can be supported when the UE knows new candidate beam(s). For beam failure detection, the BS sets beam failure detection reference signals to the UE, and the UE determines that the number of beam failure indications from the physical layer of the UE is within a period set by the RRC signaling of the BS. When a threshold set by RRC signaling is reached (reach), a beam failure is declared (declare). after beam failure is detected, the UE triggers beam failure recovery by initiating a random access procedure on the PCell; Beam failure recovery is performed by selecting a suitable beam (if the BS provides dedicated random access resources for certain beams, these are prioritized by the UE). Upon completion of the random access procedure, it is considered that beam failure recovery has been completed.

D. URLLC (Ultra-Reliable and Low Latency Communication)

URLLC transmission defined in NR is (1) a relatively low traffic size, (2) a relatively low arrival rate (low arrival rate), (3) extremely low latency requirements (eg, 0.5, 1ms), (4) a relatively short transmission duration (eg, 2 OFDM symbols), and (5) transmission for an urgent service/message. In the case of UL, transmission for a specific type of traffic (eg, URLLC) is multiplexed with other previously scheduled transmission (eg, eMBB) in order to satisfy a more stringent latency requirement. Needs to be. In this regard, as one method, information to be preempted for a specific resource is given to the previously scheduled UE, and the resource is used by the URLLC UE for UL transmission.

For NR, dynamic resource sharing between eMBB and URLLC is supported. eMBB and URLLC services may be scheduled on non-overlapping time/frequency resources, and URLLC transmission may occur on resources scheduled for ongoing eMBB traffic. The eMBB UE may not know whether the PDSCH transmission of the corresponding UE is partially punctured, and the UE may not be able to decode the PDSCH due to corrupted coded bits. In consideration of this, NR provides a preemption indication. The preemption indication may be referred to as an interrupted transmission indication.

With respect to the preemption indication, the UE receives the DownlinkPreemption IE through RRC signaling from the BS. When the UE is provided with the DownlinkPreemption IE, for monitoring the PDCCH carrying DCI format 2_1, the UE is configured with the INT-RNTI provided by the parameter int-RNTI in the DownlinkPreemption IE. The UE is additionally configured with a set of serving cells by INT-ConfigurationPerServing Cell including a set of serving cell indices provided by servingCellID and a corresponding set of positions for fields in DCI format 2_1 by positionInDCI, dci-PayloadSize It is established with the information payload size for DCI format 2_1 by , and is set with the indicated granularity of time-frequency resources by timeFrequencySect.

The UE receives DCI format 2_1 from the BS based on the DownlinkPreemption IE.

When the UE detects DCI format 2_1 for a serving cell in the configured set of serving cells, the UE determines that the DCI format of the set of PRBs and the set of symbols of the monitoring period immediately preceding the monitoring period to which the DCI format 2_1 belongs. It can be assumed that there is no transmission to the UE in the PRBs and symbols indicated by 2_1. For example, the UE sees that the signal in the time-frequency resource indicated by the preemption is not the scheduled DL transmission for itself and decodes data based on the signals received in the remaining resource region.

E. mMTC (massive MTC)

mMTC (massive machine type communication) is one of the scenarios of 5G to support hyper-connectivity service that communicates simultaneously with a large number of UEs. In this environment, the UE communicates intermittently with a very low transmission rate and mobility. Therefore, mMTC is primarily aimed at how long the UE can run at a low cost. In relation to mMTC technology, 3GPP deals with MTC and NB (NarrowBand)-IoT.

The mMTC technology has features such as repeated transmission of PDCCH, PUCCH, physical downlink shared channel (PDSCH), PUSCH, and the like, frequency hopping, retuning, and guard period.

That is, a PUSCH (or PUCCH (particularly, long PUCCH) or PRACH) including specific information and a PDSCH (or PDCCH) including a response to specific information are repeatedly transmitted. Repeated transmission is performed through frequency hopping, and for repeated transmission, (RF) retuning is performed in a guard period from a first frequency resource to a second frequency resource, and specific information And a response to specific information may be transmitted/received through a narrowband (ex. 6 RB (resource block) or 1 RB).

F. Basic AI operation using 5G communication

3 shows an example of basic operations of a user terminal and a 5G network in a 5G communication system.

The UE transmits the specific information transmission to the 5G network (S1). The 5G network performs 5G processing on the specific information (S2). Here, the 5G processing may include AI processing. Then, the 5G network transmits a response including the AI processing result to the UE (S3).

G. Application operation between user terminal and 5G network in 5G communication system

Hereinafter, the AI operation using 5G communication will be described in more detail with reference to FIGS. 1 and 2 and salpin wireless communication technology (BM procedure, URLLC, Mmtc, etc.).

First, the method proposed in this specification, which will be described later, and the basic procedure of the application operation to which the eMBB technology of 5G communication is applied will be described.

As in step S1 and step S3 of FIG. 3, in order for the UE to transmit/receive signals, information, etc. with the 5G network, the UE has an initial access procedure and random access (initial access) with the 5G network before step S1 of FIG. random access) procedure.

More specifically, the UE performs an initial connection procedure with the 5G network based on the SSB to obtain DL synchronization and system information. A beam management (BM) process and a beam failure recovery process may be added to the initial access procedure, and in the process of the UE receiving a signal from the 5G network, a QCL (quasi-co location) relationship can be added.

In addition, the UE performs a random access procedure with the 5G network for UL synchronization acquisition and/or UL transmission. In addition, the 5G network may transmit a UL grant for scheduling transmission of specific information to the UE. Accordingly, the UE transmits specific information to the 5G network based on the UL grant. In addition, the 5G network transmits a DL grant for scheduling transmission of a 5G processing result for the specific information to the UE. Accordingly, the 5G network may transmit a response including the AI processing result to the UE based on the DL grant.

Next, the method proposed in the present specification, which will be described later, and the basic procedure of the application operation to which the URLLC technology of 5G communication is applied will be described.

As described above, after the UE performs an initial access procedure and/or a random access procedure with the 5G network, the UE may receive a DownlinkPreemption IE from the 5G network. Then, the UE receives DCI format 2_1 including a pre-emption indication from the 5G network based on the DownlinkPreemption IE. And, the UE does not perform (or expect or assume) the reception of eMBB data in the resource (PRB and/or OFDM symbol) indicated by the pre-emption indication. Thereafter, the UE may receive a UL grant from the 5G network when it is necessary to transmit specific information.

Next, the method proposed in the present specification, which will be described later, and the basic procedure of the application operation to which the mMTC technology of 5G communication is applied will be described.

Among the steps of FIG. 3, the parts that are changed by the application of mMTC technology will be mainly described.

In step S1 of FIG. 3 , the UE receives a UL grant from the 5G network to transmit specific information to the 5G network. Here, the UL grant includes information on the number of repetitions for the transmission of the specific information, and the specific information may be repeatedly transmitted based on the information on the number of repetitions. That is, the UE transmits specific information to the 5G network based on the UL grant. In addition, repeated transmission of specific information may be performed through frequency hopping, transmission of the first specific information may be transmitted in a first frequency resource, and transmission of the second specific information may be transmitted in a second frequency resource. The specific information may be transmitted through a narrowband of 6RB (Resource Block) or 1RB (Resource Block).

The above salpin 5G communication technology may be applied in combination with the methods proposed in the present specification to be described later, or may be supplemented to specify or clarify the technical characteristics of the methods proposed in the present specification.

4 is a diagram illustrating a vehicle according to an embodiment of the present specification.

Referring to FIG. 4 , a vehicle 10 according to the embodiment of the present specification is defined as a transportation means traveling on a road or track. The vehicle 10 is a concept including a car, a train, and a motorcycle. The vehicle 10 may be a concept including all of an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source. The vehicle 10 may be a vehicle owned by an individual. The vehicle 10 may be a shared vehicle. The vehicle 10 may be an autonomous vehicle.

5 is a block diagram of an AI device according to an embodiment of the present specification.

The AI device 20 may include an electronic device including an AI module capable of performing AI processing, or a server including the AI module. Also, the AI device 20 may be included in at least a part of the vehicle 10 shown in FIG. 1 to perform at least a part of AI processing together.

The AI processing may include all operations related to driving of the vehicle 10 illustrated in FIG. 4 . For example, an autonomous vehicle may process sensing data or driver data by AI processing to perform processing/judgment and control signal generation operations. Also, for example, the autonomous driving vehicle may perform autonomous driving control by AI-processing data obtained through interaction with other electronic devices provided in the vehicle.

The AI device 20 may include an AI processor 21 , a memory 25 , and/or a communication unit 27 .

The AI device 20 is a computing device capable of learning a neural network, and may be implemented in various electronic devices such as a server, a desktop PC, a notebook PC, and a tablet PC.

The AI processor 21 may learn the neural network using a program stored in the memory 25 . In particular, the AI processor 21 may learn a neural network for recognizing vehicle-related data. Here, the neural network for recognizing vehicle-related data may be designed to simulate a human brain structure on a computer, and may include a plurality of network nodes having weights that simulate neurons of the human neural network. The plurality of network modes may transmit and receive data according to a connection relationship, respectively, so as to simulate a synaptic activity of a neuron in which a neuron sends and receives a signal through a synapse. Here, the neural network may include a deep learning model developed from a neural network model. In a deep learning model, a plurality of network nodes can exchange data according to a convolutional connection relationship while being located in different layers. Examples of neural network models include deep neural networks (DNN), convolutional deep neural networks (CNN), Recurrent Boltzmann Machine (RNN), Restricted Boltzmann Machine (RBM), deep trust It includes various deep learning techniques such as neural networks (DBN, deep belief networks) and deep Q-networks, and can be applied to fields such as computer vision, speech recognition, natural language processing, and voice/signal processing.

Meanwhile, the processor performing the above-described function may be a general-purpose processor (eg, CPU), but may be an AI-only processor (eg, GPU) for artificial intelligence learning.

The memory 25 may store various programs and data necessary for the operation of the AI device 20 . The memory 25 may be implemented as a non-volatile memory, a volatile memory, a flash-memory, a hard disk drive (HDD), or a solid state drive (SDD). The memory 25 is accessed by the AI processor 21 , and reading/writing/modification/deletion/update of data by the AI processor 21 may be performed. In addition, the memory 25 may store a neural network model (eg, deep learning model 26) generated through a learning algorithm for data classification/recognition according to an embodiment of the present specification.

Meanwhile, the AI processor 21 may include a data learning unit 22 that learns a neural network for data classification/recognition. The data learning unit 22 may learn a criterion regarding which training data to use to determine data classification/recognition and how to classify and recognize data using the training data. The data learning unit 22 may learn the deep learning model by acquiring learning data to be used for learning and applying the acquired learning data to the deep learning model.

The data learning unit 22 may be manufactured in the form of at least one hardware chip and mounted on the AI device 20 . For example, the data learning unit 22 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics-only processor (GPU) to the AI device 20 . may be mounted. Also, the data learning unit 22 may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a computer-readable non-transitory computer readable medium. In this case, the at least one software module may be provided by an operating system (OS) or may be provided by an application.

The data learning unit 22 may include a training data acquiring unit 23 and a model learning unit 24 .

The training data acquisition unit 23 may acquire training data required for a neural network model for classifying and recognizing data. For example, the training data acquisition unit 23 may acquire vehicle data and/or sample data to be input to the neural network model as training data.

The model learning unit 24 may use the acquired training data to learn so that the neural network model has a criterion for determining how to classify predetermined data. In this case, the model learning unit 24 may train the neural network model through supervised learning using at least a portion of the training data as a criterion for determination. Alternatively, the model learning unit 24 may learn the neural network model through unsupervised learning for discovering a judgment criterion by self-learning using learning data without guidance. Also, the model learning unit 24 may train the neural network model through reinforcement learning using feedback on whether the result of the situation determination according to the learning is correct. Also, the model learning unit 24 may train the neural network model by using a learning algorithm including an error back-propagation method or a gradient decent method.

When the neural network model is trained, the model learning unit 24 may store the learned neural network model in a memory. The model learning unit 24 may store the learned neural network model in the memory of the server connected to the AI device 20 through a wired or wireless network.

The data learning unit 22 further includes a training data preprocessing unit (not shown) and a training data selection unit (not shown) to improve the analysis result of the recognition model or to save resources or time required for generating the recognition model You may.

The learning data preprocessor may preprocess the acquired data so that the acquired data can be used for learning for situation determination. For example, the training data preprocessor may process the acquired data into a preset format so that the model learning unit 24 may use the acquired training data for image recognition learning.

In addition, the learning data selection unit may select data required for learning from among the learning data acquired by the learning data acquiring unit 23 or the training data preprocessed by the preprocessing unit. The selected training data may be provided to the model learning unit 24 . For example, the learning data selector may select only data about an object included in the specific region as the learning data by detecting a specific region among images acquired through a vehicle camera.

In addition, the data learning unit 22 may further include a model evaluation unit (not shown) in order to improve the analysis result of the neural network model.

The model evaluator may input evaluation data to the neural network model and, when an analysis result output from the evaluation data does not satisfy a predetermined criterion, may cause the model learning unit 22 to learn again. In this case, the evaluation data may be predefined data for evaluating the recognition model. As an example, the model evaluator may evaluate as not satisfying a predetermined criterion when, among the analysis results of the learned recognition model for the evaluation data, the number or ratio of evaluation data for which the analysis result is not accurate exceeds a preset threshold. have.

The communication unit 27 may transmit the AI processing result by the AI processor 21 to an external electronic device.

Here, the external electronic device may be defined as an autonomous vehicle. Also, the AI device 20 may be defined as another vehicle or a 5G network communicating with the autonomous driving module vehicle. Meanwhile, the AI device 20 may be implemented by being functionally embedded in an autonomous driving module provided in a vehicle. In addition, the 5G network may include a server or module that performs autonomous driving-related control.

On the other hand, the AI device 20 shown in FIG. 5 has been described as functionally divided into the AI processor 21, the memory 25, the communication unit 27, and the like, but the above-described components are integrated into one module and the AI module Note that it may also be called

6 is a diagram for explaining a system in which an autonomous driving vehicle and an AI device are linked according to an embodiment of the present specification.

Referring to FIG. 6 , the autonomous vehicle 10 may transmit data requiring AI processing to the AI device 20 through the communication unit, and the AI device 20 including the deep learning model 26 is the deep learning AI processing results using the model 26 may be transmitted to the autonomous vehicle 10 . The AI device 20 may refer to the contents described in FIG. 2 .

The autonomous driving vehicle 10 may include a memory 140 , a processor 170 , and a power supply unit 190 , and the processor 170 may further include an autonomous driving module 260 and an AI processor 261 . can Also, the autonomous driving vehicle 10 may include an interface unit that is connected to at least one electronic device provided in the vehicle by wire or wirelessly to exchange data required for autonomous driving control. At least one electronic device connected through the interface unit may include an object detection unit 210 , a communication unit 220 , a driving control unit 230 , a main ECU 240 , a vehicle driving unit 250 , a sensing unit 270 , and location data generation. part 280 may be included.

The interface unit may be composed of at least one of a communication module, a terminal, a pin, a cable, a port, a circuit, an element, and a device.

The memory 140 is electrically connected to the processor 170 . The memory 140 may store basic data for the unit, control data for operation control of the unit, and input/output data. The memory 140 may store data processed by the processor 170 . The memory 140 may be configured as at least one of ROM, RAM, EPROM, flash drive, and hard drive in terms of hardware. The memory 140 may store various data for the overall operation of the autonomous vehicle 10 , such as a program for processing or controlling the processor 170 . The memory 140 may be implemented integrally with the processor 170 . According to an embodiment, the memory 140 may be classified into a sub-configuration of the processor 170 .

The power supply unit 190 may supply power to the autonomous driving device 10 . The power supply unit 190 may receive power from a power source (eg, a battery) included in the autonomous vehicle 10 to supply power to each unit of the autonomous vehicle 10 . The power supply unit 190 may be operated according to a control signal provided from the main ECU 240 . The power supply unit 190 may include a switched-mode power supply (SMPS).

The processor 170 may be electrically connected to the memory 140 , the interface unit 280 , and the power supply unit 190 to exchange signals. The processor 170 is, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors (processors), controller It may be implemented using at least one of controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

The processor 170 may be driven by power provided from the power supply 190 . The processor 170 may receive data, process data, generate a signal, and provide a signal while power is supplied by the power supply unit 190 .

The processor 170 may receive information from another electronic device in the autonomous vehicle 10 through the interface unit. The processor 170 may provide a control signal to another electronic device in the autonomous vehicle 10 through the interface unit.

The autonomous vehicle 10 may include at least one printed circuit board (PCB). The memory 140 , the interface unit, the power supply unit 190 , and the processor 170 may be electrically connected to the printed circuit board.

Hereinafter, other electronic devices in the vehicle connected to the interface unit, the AI processor 261, and the autonomous driving module 260 will be described in more detail. Hereinafter, for convenience of description, the autonomous vehicle 10 will be referred to as a vehicle 10 .

First, the object detector 210 may generate information about an object outside the vehicle 10 . The AI processor 261 applies a neural network model to the data obtained through the object detector 210, so that at least one of the presence or absence of an object, location information of the object, distance information between the vehicle and the object, and relative speed information between the vehicle and the object. You can create one.

The object detection unit 210 may include at least one sensor capable of detecting an object outside the vehicle 10 . The sensor may include at least one of a camera, a radar, a lidar, an ultrasonic sensor, and an infrared sensor. The object detector 210 may provide data on an object generated based on a sensing signal generated by the sensor to at least one electronic device included in the vehicle.

Meanwhile, the vehicle 10 transmits the data acquired through the at least one sensor to the AI device 20 through the communication unit 220 , and the AI device 20 adds a neural network model 26 to the transmitted data. AI processing data generated by applying may be transmitted to the vehicle 10 . The vehicle 10 may recognize information on the detected object based on the received AI processing data, and the autonomous driving module 260 may perform an autonomous driving control operation using the recognized information.

The communication unit 220 may exchange signals with a device located outside the vehicle 10 . The communication unit 220 may exchange signals with at least one of an infrastructure (eg, a server, a broadcasting station), another vehicle, and a terminal. The communication unit 220 may include at least one of a transmit antenna, a receive antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

By applying the neural network model to the data obtained through the object detector 210, at least one of the existence of an object, location information of the object, distance information between the vehicle and the object, and relative speed information between the vehicle and the object can be generated. .

The driving operation unit 230 is a device for receiving a user input for driving. In the manual mode, the vehicle 10 may be driven based on a signal provided by the driving control unit 230 . The driving manipulation unit 230 may include a steering input device (eg, a steering wheel), an acceleration input device (eg, an accelerator pedal), and a brake input device (eg, a brake pedal).

Meanwhile, in the autonomous driving mode, the AI processor 261 may generate an input signal of the driving manipulation unit 230 according to a signal for controlling the movement of the vehicle according to the driving plan generated through the autonomous driving module 260 . have.

Meanwhile, the vehicle 10 transmits data required for control of the driver manipulation unit 230 to the AI device 20 through the communication unit 220 , and the AI device 20 transmits the neural network model 26 to the transmitted data. AI processing data generated by applying may be transmitted to the vehicle 10 . The vehicle 10 may use the input signal of the driver manipulation unit 230 to control the movement of the vehicle based on the received AI processing data.

The main ECU 240 may control the overall operation of at least one electronic device included in the vehicle 10 .

The vehicle driving unit 250 is a device for electrically controlling various vehicle driving devices in the vehicle 10 . The vehicle driving unit 250 may include a power train drive control device, a chassis drive control device, a door/window drive control device, a safety device drive control device, a lamp drive control device, and an air conditioning drive control device. The power train drive control device may include a power source drive control device and a transmission drive control device. The chassis drive control device may include a steering drive control device, a brake drive control device, and a suspension drive control device. Meanwhile, the safety device drive control device may include a safety belt drive control device for seat belt control.

The vehicle driving unit 250 includes at least one electronic control device (eg, a control electronic control unit (ECU)).

The vehicle driving unit 250 may control a power train, a steering device, and a brake device based on a signal received from the autonomous driving module 260 . The signal received from the autonomous driving module 260 may be a driving control signal generated by applying a neural network model to vehicle-related data in the AI processor 261 . The driving control signal may be a signal received from the external AI device 20 through the communication unit 220 .

The sensing unit 270 may sense the state of the vehicle. The sensing unit 270 may include an inertial measurement unit (IMU) sensor, a collision sensor, a wheel sensor, a speed sensor, an inclination sensor, a weight sensor, a heading sensor, a position module, and a vehicle. It may include at least one of a forward/reverse sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor, a temperature sensor, a humidity sensor, an ultrasonic sensor, an illuminance sensor, and a pedal position sensor. Meanwhile, an inertial measurement unit (IMU) sensor may include at least one of an acceleration sensor, a gyro sensor, and a magnetic sensor.

The AI processor 261 may generate vehicle state data by applying a neural network model to sensing data generated by at least one sensor. AI processing data generated by applying the neural network model includes vehicle attitude data, vehicle motion data, vehicle yaw data, vehicle roll data, vehicle pitch data, vehicle crash data, vehicle direction data, Vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward/reverse data, vehicle weight data, battery data, fuel data, tire pressure data, vehicle interior temperature data, vehicle interior humidity data, steering wheel rotation It may include angle data, external illumination data of the vehicle, pressure data applied to the accelerator pedal, pressure data applied to the brake pedal, and the like.

The autonomous driving module 260 may generate a driving control signal based on the AI-processed state data of the vehicle.

On the other hand, the vehicle 10 transmits the sensing data acquired through the at least one sensor to the AI device 20 through the communication unit 22, and the AI device 20 uses the transmitted sensing data as a neural network model 26 ), the generated AI processing data may be transmitted to the vehicle 10 .

The location data generator 280 may generate location data of the vehicle 10 . The location data generator 280 may include at least one of a Global Positioning System (GPS) and a Differential Global Positioning System (DGPS).

The AI processor 261 may generate more accurate vehicle location data by applying a neural network model to location data generated by at least one location data generating device.

According to an embodiment, the AI processor 261 performs a deep learning operation based on at least one of an Inertial Measurement Unit (IMU) of the sensing unit 270 and a camera image of the object detection device 210 , and the generated Position data may be corrected based on AI processing data.

On the other hand, the vehicle 10 transmits the location data obtained from the location data generating unit 280 to the AI device 20 through the communication unit 220, and the AI device 20 adds the received location data to the neural network model ( 26), the generated AI processing data may be transmitted to the vehicle 10 .

The vehicle 10 may include an internal communication system 50 . A plurality of electronic devices included in the vehicle 10 may exchange signals via the internal communication system 50 . Signals may contain data. The internal communication system 50 may use at least one communication protocol (eg, CAN, LIN, FlexRay, MOST, Ethernet).

The autonomous driving module 260 may generate a path for autonomous driving based on the obtained data, and may generate a driving plan for driving along the generated path.

The autonomous driving module 260 may implement at least one Advanced Driver Assistance System (ADAS) function. ADAS includes Adaptive Cruise Control (ACC), Autonomous Emergency Braking (AEB), Forward Collision Warning (FCW), Lane Keeping Assist (LKA), ), Lane Change Assist (LCA), Target Following Assist (TFA), Blind Spot Detection (BSD), Adaptive High Beam Control (HBA) , Auto Parking System (APS), Pedestrian Collision Warning System (PD Collision Warning System), Traffic Sign Recognition (TSR), Trafffic Sign Assist (TSA), Night Vision System At least one of a Night Vision (NV), a Driver Status Monitoring (DSM), and a Traffic Jam Assist (TJA) may be implemented.

The AI processor 261 applies at least one sensor provided in the vehicle, traffic-related information received from an external device, and information received from another vehicle communicating with the vehicle to the neural network model, thereby performing the at least one ADAS function described above. A control signal capable of performing these functions may be transmitted to the autonomous driving module 260 .

In addition, the vehicle 10 transmits at least one data for performing ADAS functions to the AI device 20 through the communication unit 220 , and the AI device 20 adds a neural network model 260 to the received data. By applying, a control signal capable of performing an ADAS function may be transmitted to the vehicle 10 .

The autonomous driving module 260 acquires the driver's state information and/or the vehicle's state information through the AI processor 261 , and based on this, a switching operation from the autonomous driving mode to the manual driving mode or autonomous in the manual driving mode A switching operation to the driving mode may be performed.

Meanwhile, the vehicle 10 may use AI processing data for passenger support for driving control. For example, as described above, the state of the driver and the passenger may be checked through at least one sensor provided inside the vehicle.

Alternatively, the vehicle 10 may recognize a driver's or passenger's voice signal through the AI processor 261 , perform a voice processing operation, and perform a voice synthesis operation.

Deep Neural Network (DNN) Model

7 is an example of a DNN model to which this specification can be applied.

A deep neural network (DNN) is an artificial neural network (ANN) composed of several hidden layers between an input layer and an output layer. Deep neural networks can model complex non-linear relationships like general artificial neural networks.

For example, in a deep neural network structure for an object identification model, each object may be expressed as a hierarchical configuration of image basic elements. In this case, the additional layers may aggregate the characteristics of the gradually gathered lower layers. This feature of deep neural networks enables modeling of complex data with fewer units (units, nodes) compared to similarly performed artificial neural networks.

As the number of hidden layers increases, the artificial neural network is called 'deep', and the machine learning paradigm that uses the sufficiently deep artificial neural network as a learning model is called deep learning. In addition, a sufficiently deep artificial neural network used for such deep learning is collectively referred to as a deep neural network (DNN).

In this specification, sensing data of the vehicle 10 or data required for autonomous driving may be input to the input layer of the DNN, and meaningful data that can be used for autonomous driving may be generated through the output layer while passing through the hidden layers. can

In the specification of the present specification, an artificial neural network used for such a deep learning method is collectively referred to as a DNN, but if meaningful data can be output in a similar manner, of course, other deep learning methods may be applied.

V2X (Vehicle-to-Everything)

8 is an example of V2X communication to which this specification can be applied.

V2X communication is V2V (Vehicle-to-Vehicle), which refers to communication between vehicles, V2I (Vehicle to Infrastructure), which refers to communication between a vehicle and an eNB or RSU (Road Side Unit), vehicle and individual It includes communication between the vehicle and all entities, such as V2P (Vehicle-to-Pedestrian) and V2N (vehicle-to-network), which refers to communication between UEs possessed by (pedestrian, cyclist, vehicle driver, or passenger).

V2X communication may represent the same meaning as V2X sidelink or NR V2X, or may represent a broader meaning including V2X sidelink or NR V2X.

V2X communication is, for example, forward collision warning, automatic parking system, cooperative adaptive cruise control (CACC), loss of control warning, traffic queue warning, traffic vulnerable safety warning, emergency vehicle warning, when driving on a curved road. It can be applied to various services such as speed warning and traffic flow control.

V2X communication may be provided through a PC5 interface and/or a Uu interface. In this case, in a wireless communication system supporting V2X communication, specific network entities for supporting communication between the vehicle and all entities may exist. For example, the network entity may be a BS (eNB), a road side unit (RSU), a UE, or an application server (eg, a traffic safety server).

In addition, the UE performing V2X communication, as well as a general handheld UE (handheld UE), vehicle UE (V-UE (Vehicle UE)), pedestrian UE (pedestrian UE), BS type (eNB type) RSU, or UE It may mean an RSU of a UE type, a robot equipped with a communication module, and the like.

V2X communication may be performed directly between UEs, or may be performed through the network entity(s). A V2X operation mode may be divided according to a method of performing such V2X communication.

V2X communication is required to support the anonymity and privacy of the UE when using the V2X application so that an operator or a third party cannot track the UE identifier within the region where V2X is supported. do.

Terms frequently used in V2X communication are defined as follows.

- RSU (Road Side Unit): RSU is a V2X service capable device that can transmit/receive with a mobile vehicle using V2I service. In addition, RSU is a fixed infrastructure entity that supports V2X applications, and can exchange messages with other entities that support V2X applications. RSU is a term frequently used in the existing ITS specification, and the reason for introducing this term to the 3GPP specification is to make the document easier to read in the ITS industry. The RSU is a logical entity that combines the V2X application logic with the function of a BS (referred to as BS-type RSU) or UE (referred to as UE-type RSU).

- V2I service: A type of V2X service, in which one side is a vehicle and the other side is an entity belonging to the infrastructure.

- V2P service: A type of V2X service where one side is a vehicle and the other side is a device carried by an individual (eg, a portable UE device carried by a pedestrian, cyclist, driver or passenger).

- V2X service: A 3GPP communication service type involving a vehicle transmitting or receiving device.

-V2X enabled (enabled) UE: UE supporting the V2X service.

- V2V service: A type of V2X service, where both sides of the communication are vehicles.

- V2V communication range: Direct communication range between two vehicles participating in V2V service.

V2X applications, called Vehicle-to-Everything (V2X), are (1) vehicle-to-vehicle (V2V), (2) vehicle-to-infrastructure (V2I), (3) vehicle-to-network (V2N), (4) vehicle There are 4 types of pedestrians (V2P).

9 illustrates a resource allocation method in a sidelink in which V2X is used.

In the sidelink, different sidelink control channels (physical sidelink control channels, PSCCHs) may be allocated spaced apart in the frequency domain, and different physical sidelink shared channels (PSSCHs) may be allocated spaced apart from each other. Alternatively, different PSCCHs may be consecutively allocated in the frequency domain, and PSSCHs may also be allocated consecutively in the frequency domain.

NR V2X

To extend the 3GPP platform to the automotive industry during 3GPP Releases 14 and 15, support for V2V and V2X services in LTE was introduced.

The requirements for support for the enhanced (enhanced) V2X use case are largely organized into four use case groups.

(1) Vehicle Platooning allows vehicles to dynamically form platoons that move together. All vehicles in the Platoon get information from the lead vehicle to manage this Platoon. This information allows vehicles to drive more harmoniously than normal, go in the same direction and drive together.

(2) extended sensors are collected through a local sensor or a live video image in a vehicle, a road site unit, a pedestrian device, and a V2X application server raw (raw) ) or to exchange processed data. Vehicles can increase their environmental awareness beyond what their sensors can detect, and provide a broader and holistic picture of local conditions. A high data rate is one of the main characteristics.

(3) Advanced driving enables semi-automatic or fully-automatic driving. Each vehicle and/or RSU shares self-awareness data obtained from local sensors with nearby vehicles, allowing the vehicle to synchronize and coordinate its trajectory or maneuver. Each vehicle shares driving intent with the proximity-driving vehicle.

(4) Remote driving enables remote drivers or V2X applications to drive remote vehicles by themselves or for passengers who cannot drive with remote vehicles in hazardous environments. When variability is limited and routes can be predicted, such as in public transport, driving based on cloud computing can be used. High reliability and low latency are key requirements.

Identifier for V2X communication through PC5

Each terminal has a Layer-2 identifier for V2 communication through one or more PC5. This includes a source Layer-2 ID and a destination Layer-2 ID.

The source and destination Layer-2 IDs are included in the Layer-2 frame, and the Layer-2 frame is transmitted over the layer-2 link of PC5 that identifies the source and destination of Layer-2 on the frame.

Source and destination Layer-2 ID selection of the terminal is based on the communication mode of V2X communication of PC5 of the layer-2 link. The source Layer-2 ID may be different between different communication modes.

If IP-based V2X communication is allowed, the terminal sets the link-local IPv6 address to be used as the source IP address. The UE may use this IP address for V2X communication of PC5 without sending a Neighbor Solicitation and Neighbor Advertisement message for duplicate address discovery.

If one terminal has an activated V2X application that requires privacy protection supported in the current geographic area, the source terminal (eg, vehicle) is tracked or identified from other terminals only for a specific time, source Layer- 2 IDs change over time and can be randomized. In the case of IP-based V2X communication, the source IP address must also change over time and must be randomized.

The change of identifiers of the source terminal should be synchronized in the layer used for PC5. That is, if the application layer identifier is changed, the change of the source Layer-2 ID and the source IP address is also required.

Broadcast mode

10 is a diagram illustrating a procedure for a broadcast mode of V2X communication using PC5.

One. The receiving terminal determines a destination Layer-2 ID for broadcast reception. The destination Layer-2 ID is transmitted to the AS layer of the receiving terminal for reception.

2. The V2X application layer of the transmitting terminal may provide a data unit, and may provide V2X application requirements (Application Requirements).

3. The transmitting terminal determines the destination Layer-2 ID for the broadcast. The transmitting terminal itself allocates a source Layer-2 ID.

4. One broadcast message transmitted by the transmitting terminal transmits V2X service data using a source Layer-2 ID and a destination Layer-2 ID.

In the above, 5G communication necessary for implementing the vehicle control method according to an embodiment of the present specification and AI processing by applying the 5G communication have been described, and schematic contents for transmitting and receiving AI processing results have been described.

The above salpin 5G communication technology may be applied in combination with the methods proposed in the present specification to be described later, or may be supplemented to specify or clarify the technical characteristics of the methods proposed in the present specification.

Hereinafter, various embodiments of the present specification will be described in detail with reference to the accompanying drawings.

Each autonomous vehicle learns through the AI processor 261 and maintains the learning data, or prepares a learner included in the AI device 20 in the server and trains unique data input for each autonomous vehicle each time It takes a lot of time until a deep learning logic of a sufficient level (level) is formed. In addition, in the case of learning by data of each vehicle, CPU resources of the server and the autonomous vehicle are wasted severely.

In order to solve the above problem, the present specification exemplifies the following method.

Among the sensing information generated by each vehicle 10 through a sensor, sensing information on a fixed object that can be used in common with other vehicles is divided and input to the fixed object learner for training, and other vehicles also sense fixed objects When performing recognition, judgment, and control operations using information, the logic of the fixed object learner formed by training the vehicle 10 may be used.

In addition, the logic of the floating object learner trained by the other vehicle 10 can be inherited (utilized) between vehicles that can share sensing information on floating objects by reflecting the characteristics of each vehicle 10 . The vehicle 10 may additionally input its own sensing information and train it with its own floating object learner to generate logic to meet the driving purpose.

In addition, the vehicle 10 may share the fixed object data and the fixed object learner between vehicles traveling in the same section as V2X.

For example, the vehicle 10 may exchange the difference with V2X when there is a difference with other vehicles. More preferably, the vehicle 10 may compare the specification information of another vehicle with the specification information of the vehicle 10 through the specification information. As a result of the comparison, if the difference is small (ex: when the vehicle model and sensor installation location are the same but the sensor sampling rate is different), the fixed object learner built by another vehicle is received, and the vehicle 10 recognizes and judges using it. work can be done Alternatively, in order to learn the fixed object learner of the vehicle 10 , a fixed object learner of another vehicle may be used.

Through this, the present specification can obtain the following effects.

The process of inputting sensing information to the learner for each vehicle 10 and training can be reduced. In addition, the vehicle 10 can quickly recognize and determine the fixed terrain and features of the road with an existing fixed object learner. Also, the vehicle 10 may put more resources into a task for learning the floating object learner. In addition, the vehicle 10 may reduce the time required to generate the logic for autonomous driving and deepen the depth and level of the logic for autonomous driving within a short time.

fixed object data

The stationary object refers to topography, features, sign boards, lanes, locations, intersection locations, and the like on a fixed road that the vehicle 10 collects from the road. The vehicle 10 may determine whether the detected object is a fixed object. For example, the vehicle 10 may determine whether it is a fixed object by analyzing the sensing information through the processor 170 . Also, the vehicle 10 may determine whether the detected object is a fixed object by using map information including information on the fixed object.

The server may select information that is not affected by the height of the vehicle and sensor/camera performance among the fixed object sensing information received from the vehicles 10 . For example, since the video image of a road sign changes the height of the sign according to the height of the camera, the video image itself is flexible sensing information for each vehicle 10, but the server recognizes and reads the camera video image of the road sign. Sign information that is information (eg: turn right, go straight, turn left) may be fixed object data.

Accordingly, even when the specifications of the vehicle 10 are different, the fixed object data may be equally used for the vehicles 10 . Such fixed object data may be combined with map information and stored in a server. The vehicle 10 may travel by using the fixed object data.

Fixed object sensing information learning technology

The vehicle 10 transmits fixed object sensing information, for example, road sign information, and road lane information to the server, and the server inputs the fixed object sensing information to the AI device 20 to train the fixed object learner. have. To this end, the AI device 20 may be included in the server.

In more detail, the vehicle 10 inputs the front/rear/left/right camera image data of the vehicle 10, lidar/rider data, etc., obtained through the sensor, into the stationary object learner received from the server, and the degree of similarity calculated If the value is greater than or equal to a specific value, it is possible to determine sign information, road lane information, and the like, from the fixed object sensing information.

As another embodiment, the vehicle 10 inputs fixed object sensing information similar to the fixed object sensing information of other vehicles to the fixed object learner of the server trained through the existing fixed object sensing information of other vehicles, can train a fixed-object learner of

In more detail, the vehicle 10 may be trained by inputting sensing information sensing a guard rail, a topography on a road, a feature, etc. to a stationary object learner of a server through a lidar or a radar.

For example, when the vehicle 10 travels on the 4th street of Balsan Station at 4 pm, the fixed object learning machine (or pre-learned fixed object sensing information such as lanes, guard rails, and signs) in the AI device 20 of the server (or Logic) input and outputted fixed object data can be used to select a path.

Using the stationary object learner

The vehicle 10 may check whether the fixed object data of the section being driven or scheduled to be driven exists in the server. The vehicle 10 includes specification information (eg, vehicle type, sensor/camera attachment location/height, and sensor/camera performance) of other vehicles that have traveled in the same time range in the section being driven or scheduled to be driven in the same time range. ), if it is in the same range as the specification information, you can use the fixed object data generated by other vehicles. To this end, the fixed object data stored in the server may include specification information of vehicles that have generated the fixed object data. Through this, the vehicle 10 may determine the stationary object in the section being driven or scheduled to be driven without additional calculation.

Moving object sensing information

A moving object means an object whose position is not fixed on the road. The moving object sensing information means sensing information related to an object with variable movement in addition to the above-described fixed object sensing information among sensing information of the vehicle 10 .

In relation to the moving object detected by the vehicle 10 , the server should generate a learning logic for determining the moving object. Accordingly, when the vehicle 10 detects a new moving object, it should be able to train the existing floating object learner using the related moving object sensing information.

Moving object sensing information learning technology

The vehicle 10 may generate sensing information related to a ball and a child jumping into the road through the sensor. Since there is no fixed object data related to such sensing information, and no fixed object learning logic (learning machine) for this purpose, the vehicle 10 cannot identify a child jumping into a ball or road. Accordingly, the vehicle 10 can learn the floating object learner of the server by determining it as the floating object sensing information. That is, the server may learn the floating object learner by inputting the sensing information of the children of the lesson, the vehicle 10 may receive the floating object learner, and identify these objects using the floating object learner. In more detail, the vehicle 10 may use a fixed object learner to determine whether it is a moving object. That is, the vehicle 10 may determine an object that is not determined as a floating object through the stationary object learner.

As another example, the vehicle 10 may learn the floating object learner by using the sensing information received through the V2X message from the front vehicle located in a short distance.

As another example, the vehicle 10 may learn the moving object learner by using sensing information related to the vehicle, pedestrian, or bicycle of the front/rear/side measured by the vehicle's lidar/radar.

As another example, when another vehicle is located 100 m ahead of the vehicle 10 (that is, when no other object exists within the sensing range of the vehicle 10), the fixed object learner can be used as it is without a separate training process. can However, when the distance to the vehicle in front is short-distance (for example, within 10m), the vehicle 10 flows the generated sensing information to analyze the change in the front image that occurs according to the movement of the vehicle in front. It can be classified as object sensing information, and inputted into a floating object learner to learn. Thereafter, the vehicle 10 may drive by using the floating object learner in an environment similar to that at the time when the floating object learner was taught.

As another example, the vehicle 10 can be trained by inputting sensing information about the amount of sunlight, the direction of light, the length of shadows of other vehicles according to the altitude of the sun, the depth of the shadow color and the shape of the vehicle to the floating object learner. have.

As another example, in the vehicle 10, the installed sensor/camera position and the performance of the sensor and camera may be different depending on the specification, so the vehicle 10 inputs the specification information of the vehicle 10 to the floating object learner for training can do it

Learning technology to add sensing information of moving objects

The server can train the existing floating object learner by adding floating object sensing information when vehicles traveling in the same section have the same vehicle model, same sensor/camera height, installation location, and sensor/camera performance.

Also, the server may train the floating object learner by inputting change information of vehicles around the vehicle 10 into the floating object learner.

The change information may mean, for example, an image in which the vehicle 10 captures the rear view of the vehicle in front and the lane at the same time. In more detail, the rear view of the vehicle in front or the distance between lanes, the direction of the shadow generated by sunlight shining on the vehicle in front, and the image in which the shadow covers the lane may be variable information.

Short-distance determination criteria for classification of sensing information of moving objects

The vehicle 10 may classify the generated sensing information as moving object sensing information when the distance to the vehicle in front or the pedestrian is short distance. To this end, the short distance may be determined as follows.

1. When the vehicle in front is an autonomous vehicle

The short distance means a distance at which the vehicle 10 can receive the V2X message from the vehicle in front, and decode the received V2X message without loss. That is, if the vehicle 10 normally decodes the V2X message received from the vehicle in front, the vehicle 10 may classify the sensing information as floating object sensing information. For example, if the vehicle 10 receives and decodes the header of the V2X message periodically broadcast by the vehicle in front to obtain the header information without error, the vehicle 10 transmits the sensing information to the floating object sensing information. can be classified as

2. If the vehicle in front is a normal vehicle or there is a pedestrian in front

The short-distance means a distance at which the vehicle 10 can decode a message received through a terminal of a pedestrian or a driver of a general vehicle without loss. That is, if the vehicle 10 normally decodes a message received through a terminal of a pedestrian or a driver of a general vehicle, the vehicle 10 may classify the sensing information as sensing information of moving objects.

driving action

1. Fixed object

Since there is no change in the fixed object unless there is a special circumstance, the vehicle 10 may drive using the fixed object data. In more detail, the fixed object sensing information changes image values through driving of the vehicle 10, but the vehicle 10 uses the map information to determine the driving environment (eg, location information, speed, angle of the wheel) can be determined, so that the processor 170 of the vehicle 10 can determine the stationary object.

2. When fixed object sensing information is changed to floating object sensing information

When the vehicle in front of the vehicle 10 is located at a long distance, the vehicle 10 may drive using the fixed object data. However, when the distance between the vehicle 10 and the vehicle in front becomes a short distance, the vehicle 10 may set the space between the vehicle 10 and the vehicle 10 in front as the moving object area. In addition, the vehicle 10 may classify the sensing information into the moving object sensing information, and detect the object through the AI model learned through the moving object learner.

3. When moving object sensing information is changed to fixed object sensing information

When the V2X message received from the vehicle in front is not normally decoded, the vehicle 10 may determine that the vehicle in front is not in a short distance, and may drive using the fixed object data.

11 to 14 are exemplary embodiments to which the present specification can be applied.

11 to 14 , the server includes an AI device ( ), and the AI processor 170 includes a fixed object learner (which may be referred to as a first model in this specification) and a floating object learner (the first model in this specification). 2 can be referred to as a model). In addition, the memory 25 of the server may store the fixed object data generated through the AI processor 170 . The fixed object data includes map information, specification information of the generated vehicle 10, and generated section information. In addition, the processor 170 of the vehicle 10 may include a stationary object learner (first model) and a floating object learner (second model). The vehicle 10 includes a first vehicle and a second vehicle.

The operations of FIGS. 11 to 14 of the present specification may be performed sequentially or may be performed individually, and the order is not limited.

Referring to FIG. 11 , the vehicle 10 transmits a request message for requesting fixed object data to the server (S1100). In more detail, the request message may include specification information of the vehicle 10 and information on the driving section or scheduled driving section of the vehicle 10 .

As a response to the request message, the server may transmit the fixed object data and/or the first model for the vehicle 10 to the vehicle 10 (S1110). In more detail, in the server, based on the request message received from the vehicle 10 , there is fixed object data regarding the section in which the vehicle 10 is driving or scheduled to be driven, and the fixed object data is the specification information of the vehicle 10 . Based on the , it can be determined that the vehicle 10 is similar enough to be used. In this case, the server may transmit the stationary object data and/or the first model to the vehicle 10 .

The vehicle 10 determines sensing information by using the received first model (S1120). Since the first model is a model trained to detect a fixed object, the vehicle 10 may more efficiently determine the fixed object sensing information.

The vehicle 10 transmits the fixed object sensing information to the server (S1130). For example, when the fixed object determined through the first model is not included in the received fixed object data, the vehicle 10 may transmit the fixed object sensing information to the server.

The server learns the first model based on the fixed object sensing information (S1140).

The server generates fixed object data based on the fixed object sensing information (S1150). For example, when the server has fixed object data corresponding to the fixed object sensing information, the server may update the fixed object data using the fixed object sensing information.

The server transmits the fixed object data and/or the first model to the vehicle 10 (S1160). In more detail, when the fixed object data is updated or the first model is learned, the server may transmit the fixed object data and/or the first model to the vehicle 10 .

The vehicle 10 performs a control operation using the fixed object data (S1170).

Referring to FIG. 12 , the processor 170 of the vehicle 10 includes a first model.

The vehicle 10 determines sensing information by using the first model (S1200). In more detail, since the first model is a model trained to determine a stationary object, the vehicle 10 may determine, through the first model, sensing information including an undetermined object as floating object sensing information.

The vehicle 10 transmits the moving object sensing information to the server (S1210).

The server uses the received floating object sensing information to learn the second model (S1220).

The server transmits the second model to the vehicle 10 (S1230).

The vehicle 10 performs a control operation using the received second model (S1240).

13 is an embodiment to which the present specification can be applied.

Referring to FIG. 13 , the memory 140 of the second vehicle stores the first model or fixed object data.

The first vehicle transmits the specification information and the request message of the first vehicle to the second vehicle (S1300). The request message is a message for requesting the first model of the second vehicle or the fixed object data of the second vehicle from the second vehicle.

The second vehicle compares the specification information of the first vehicle with the specification information of the second vehicle (S1310). More specifically, when the specification information of the first vehicle and the second vehicle is in the same or similar range, the first vehicle may use the first model or fixed object data of the second vehicle. Accordingly, the second vehicle compares the specification information of the first vehicle with the specification information of the second vehicle and, if the same or within a preset similarity range, transmits the first model or fixed object data of the second vehicle to the first vehicle may decide to

The second vehicle transmits the first model or fixed object data of the second vehicle to the first vehicle (S1320).

The second vehicle transmits sensing information of the second vehicle to the first vehicle (S1330). For example, when the specification information of the first vehicle and the second vehicle is in the same or similar range, the second vehicle may transmit sensing information of the second vehicle to the first vehicle.

The first vehicle learns the first model of the first vehicle by using the first model of the second vehicle or sensing information (S1340).

The first vehicle performs a control operation using the stationary object data of the second vehicle (S1350).

14 is an embodiment to which the present specification can be applied.

The first vehicle determines sensing information by using the first model (S1400). In more detail, the first vehicle may determine the stationary object using the first model through the sensing information. Alternatively, sensing information including an object that is not determined using the first model of the first vehicle may be determined as floating object sensing information.

The first vehicle receives the first V2X message from the second vehicle (S1410). For example, the first V2X message may be a Basic Safety Message (BSM) including driving information of the second vehicle.

When the first V2X message is normally decoded, the first vehicle classifies the sensing information as floating object sensing information (S1420). In more detail, when the first V2X message is received and normal decoding is confirmed, the first vehicle may determine that the second vehicle is located in a short distance. The first vehicle may set all of the generated sensing information as floating object sensing information, and may perform a control operation using the second model.

The first vehicle receives the second V2X message from the second vehicle (S1430).

When the first V2X message is not normally decoded, the first vehicle releases the floating object sensing information setting (S1440). In more detail, when the second V2X message is received, but normal decoding is not performed, the first vehicle may determine that the second vehicle is located in a remote location. Accordingly, the first vehicle may release that all of the generated sensing information is set as the floating object sensing information.

15 is an embodiment of a vehicle to which this specification can be applied.

The vehicle 10 transmits the sensing information related to the fixed object to the server (S1500). In more detail, the processor 170 may determine whether the object included in the sensing information is a fixed object, based on the sensing information generated through the sensor. The vehicle 10 may transmit sensing information related to a fixed object to the server in order to learn the first model of the server.

The vehicle 10 receives the first model from the server (S1510). In more detail, the first model may be learned using sensing information related to a stationary object received from a plurality of vehicles. For example, a plurality of vehicles may each have similar specifications.

The vehicle 10 transmits sensing information related to the moving object to the server (S1520). In more detail, the vehicle 10 may determine sensing information related to a moving object by using the received first model. The vehicle 10 may transmit sensing information related to a moving object to the server in order to learn the second model of the server.

The vehicle receives the second model from the server (S1530). In more detail, the second model may be learned using sensing information related to a moving object received from a plurality of vehicles.

16 is an embodiment of a server to which this specification can be applied.

The server receives, from the first vehicle, sensing information related to the fixed object of the first vehicle (S1600).

The server receives sensing information related to a fixed object of the second vehicle from the second vehicle (S1610). For example, the first vehicle and the second vehicle may have similar specifications.

The server learns the first model (S1620). In more detail, the server may learn the first model by using the sensing information related to the fixed object of the first vehicle and the sensing information related to the fixed object of the second vehicle.

The server transmits the first model to the first vehicle or the second vehicle (S1630). For example, the server may receive a request to transmit the first model, and transmit the first model in response thereto. The request message for requesting transmission of the first model may be received together with sensing information related to the fixed object.

Object detection and tracking method

The purpose of object detection is to segment a region of interest from a video image and to continuously track and detect situations such as motion and positioning.

17 is an example of an object detection and tracking method to which the present specification can be applied.

The first thing to be done in detecting an object is to recognize a region of interest. Although it is desirable to detect objects using the most well-known Object Detection Algorithm for recognizing objects, the detection process is inevitably difficult because there are many variables such as unknown objects, colors, and shapes. Therefore, most of the object detection and tracking methods perform the detection process using a fixed camera sensor.

Object detection is the process of identifying an object of interest from cluster pixels of an object and a video sequence. This can be performed using various techniques such as frame differencing, optical flow, and background subtraction (S1710).

Objects can be classified as cars, birds, clouds, trees, or other moving objects. Methods for classifying such objects include Shape-based classification, Motion-based classification, Color based classification and Texture based classification (S1720).

The tracking method can be said to be a problem to approximate the path of an object on the image plane in a moving scene. That is, if the path of the object of interest in the image is similar to the previous frame and recognized as the same object, it can be said that the object is continuously tracked. Methods for tracking an object include Point Tracking, Kernel Tracking, and Silhouette (S1730).

The first model may not perform an object tracking step. In more detail, the first model is an AI model specialized for detecting a stationary object, and since the stationary object does not move, the first model may omit the object tracking step. Through this, the first model can quickly classify the stationary object.

Also, the first model may use fixed object data for object detection and object classification. The first model may acquire image information or location information related to a fixed object located on a road on which the vehicle 10 travels through the fixed object data, and may perform object detection and object classification using the fixed object data.

The second model may not perform an object detection step. In more detail, the second model may receive information on the marked object through the first model. The marked object information includes information on the object detected by the first model. Accordingly, the second model may omit the object detection step. Through this, the second model can quickly classify and track moving objects.

The first model and the second model are not limited to the algorithm described above, and include algorithms having similar purposes.

18 is an embodiment to which the present specification can be applied.

Referring to FIG. 18 , the vehicle 10 detects an object using the first model ( S1800 ). In more detail, since the first model is an AI model trained to detect a stationary object, the processor 170 may detect the stationary object using the first model.

The vehicle 10 marks an object that is not classified by using the first model (S1810). More specifically, the first model cannot classify objects in motion. Accordingly, the processor 170 may mark an unclassified object as a floating object by using the first model. For example, the processor 170 may mark data related to a bounding box including an unclassified object by using the first model.

The vehicle 10 transmits sensing information related to an object that is not classified using the first model to the server (S1820). In more detail, the processor 170 may perform object detection through the fixed object data or the first model. If the processor 170 finds an object that is not classified using the first model, the processor 170 may determine the object that is not classified using the first model as a floating object. For example, the processor 170 may transmit sensing information including the marked bounding box to the server.

The server learns the second model by using sensing information related to an object that is not classified as the first model (S1830). In more detail, the AI device 20 may learn the second model by using sensing information related to an object included in the marked bounding box as an input value. Accordingly, the second model may be specialized for detection of an object not classified as the first model and may be trained.

19 is an embodiment to which the present specification can be applied.

Referring to FIG. 19 , the vehicle 10 detects an object using the first model ( S1900 ).

When there is an object that is not classified using the first model, the vehicle 10 classifies the object using the second model ( S1910 ). In more detail, the processor 170 may input information on an object that is not classified through the first model into the second model. For example, the information on the object that is not classified through the first model may be sensing information related to the object included in the marked bounding box. The second model may acquire information on the detected object without the object detection step by using the information on the object that is not classified through the received first model. Accordingly, the second model can immediately perform object classification. Additionally, the processor 170 may input information on the object classified through the second model into the first model. For example, the information of the object classified through the second model may include marking information indicating "a moving object". The first model may use this to learn to detect an object designated as a “floating object”. In more detail, the first model may detect and define moving objects such as cars and people as “moving objects” in the object detection and classification process. Also, similarly to S1820 , when the first model defines an object as a “moving object”, the processor 170 may transmit sensing information related to the object defined as a “moving object” to the second model. Through this, the processor 170 may classify and track an object defined as a “moving object” using the second model.

The vehicle 10 tracks the object using the second model (S1920).

The vehicle 10 may switch the use of the first model and the second model through the above operation. Through this, the vehicle 10 can accurately determine an object using a specialized model, and efficient driving is possible.

General devices to which this specification can be applied

Referring to FIG. 20 , the server X200 according to the proposed embodiment may be an MEC server or a cloud server, and may include a communication module X210, a processor X220, and a memory X230. The communication module X210 is also referred to as a radio frequency (RF) unit. The communication module X210 may be configured to transmit various signals, data and information to an external device and to receive various signals, data and information to an external device. The server X200 may be connected to an external device by wire and/or wirelessly. The communication module X210 may be implemented by being separated into a transmitter and a receiver. The processor X220 may control the overall operation of the server X200, and the server X200 may be configured to perform a function of calculating and processing information to be transmitted and received with an external device. In addition, the processor X220 may be configured to perform the server operation proposed in this specification. The processor X220 may control the communication module X210 to transmit data or a message to the UE, another vehicle, or another server according to the proposal of the present specification. The memory X230 may store arithmetic-processed information and the like for a predetermined time, and may be replaced with a component such as a buffer.

In addition, the specific configuration of the terminal device X100 and the server X200 as described above may be implemented such that the matters described in the various embodiments of the present specification are independently applied or two or more embodiments are simultaneously applied, and overlapping Descriptions are omitted for clarity.

The above-described specification can be implemented as computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that is implemented in the form of. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of this specification should be determined by a reasonable interpretation of the appended claims, and all modifications within the scope of equivalents of this specification are included in the scope of this specification.

In addition, although the above description has been focused on services and embodiments, these are merely examples and do not limit the present specification, and those of ordinary skill in the art to which this specification pertains will not depart from the essential characteristics of the present service and embodiments. It can be seen that various modifications and applications not exemplified above are possible. For example, each component specifically shown in the embodiments can be implemented by modification. And differences related to such modifications and applications should be construed as being included in the scope of the present specification defined in the appended claims.

Although the present specification has been mainly described as an example applied to an autonomous driving system (Automated Vehicle & Highway Systems) based on a 5G (5 generation) system, it can be applied to various wireless communication systems and autonomous driving devices.

Claims (20)

In a method of inheriting artificial intelligence learning data of a vehicle in an autonomous driving system,
receiving, from a server, a first model learned using sensing information related to a fixed object;
using the first model, first marking an object that is not classified;
transmitting, to the server, sensing information related to the first marked object; and
receiving, from the server, a second model learned using sensing information related to the first marked object;
includes
The first model classifies fixed objects, and the second model classifies and tracks moving objects.
According to claim 1,
transmitting, to a server, a request message for requesting information on a fixed object located in a driving section or scheduled driving section of the vehicle; and
receiving, from the server, information on the fixed object;
Inheritance method further comprising.
According to claim 1,
transmitting, to another vehicle, specification information of the vehicle;
receiving, from the other vehicle, sensing information related to a fixed object of another vehicle, based on the specification information of the vehicle; and
learning the first model by using sensing information related to the fixed object of the other vehicle;
Inheritance method further comprising.
4. The method of claim 3,
receiving, from the other vehicle, a first model of the other vehicle, based on the specification information of the vehicle;
Inheritance method further comprising.
According to claim 1,
Receiving a first V2X message from another vehicle; and
setting, based on a result of decoding the first V2X message, sensing information generated through a sensor as sensing information related to the moving object;
Inheritance method further comprising.
6. The method of claim 5,
Receiving a second V2X message from the other vehicle; and
releasing the setting of the sensing information related to the floating object based on a result of decoding the second V2X message;
Inheritance method further comprising.
According to claim 1,
classifying the first marked object through the second model based on sensing information related to the marked object; and
tracking the first marked object through the second model;
Inheritance method further comprising.
8. The method of claim 7,
second marking the classified object through the second model; and
learning the first model by using the second marked object;
Inheritance method further comprising.
In a method of inheriting artificial intelligence learning data of a server in an autonomous driving system,
receiving, from a first vehicle, sensing information related to a fixed object of the first vehicle;
receiving, from a second vehicle, sensing information related to a stationary object of the second vehicle;
learning a first model by using the sensing information related to the fixed object of the first vehicle and the sensing information related to the fixed object of the second vehicle; and
transmitting the first model for determining the fixed object included in the sensing information to the first vehicle or the second vehicle;
Inheritance method including
10. The method of claim 9,
receiving, from the first vehicle, sensing information related to an object that is not classified using a first model of the first vehicle;
receiving, from the second vehicle, sensing information related to an object that is not classified using a first model of the second vehicle;
First for determining a moving object using sensing information related to an object that is not classified using the first model of the first vehicle or sensing information related to an object that is not classified using the first model of the second vehicle 2 training the model; and
transmitting the second model to the first vehicle or the second vehicle;
Inheritance method further comprising.
11. The method of claim 10,
The second model is
Based on sensing information related to an object that is not classified using the first model of the first vehicle or sensing information related to an object that is not classified using the first model of the second vehicle, sensing information related to a marked object Inheritance method to learn to detect moving objects using
10. The method of claim 9,
receiving, from the first vehicle, a request message for requesting information on a fixed object located in a driving section of the first vehicle or a section to be driven; and
transmitting, to the first vehicle, the information of the fixed object as a response to the request message;
Inheritance method further comprising.
13. The method of claim 12,
The information on the fixed object is
Inheritance method including the specification information of the vehicle that generated the information of the fixed object.
14. The method of claim 13,
The request message includes the specification information of the first vehicle, and the information of the fixed object is transmitted based on the specification information of the first vehicle.
10. The method of claim 9,
generating fixed object information by using the sensing information related to the fixed object of the first vehicle and the sensing information related to the fixed object of the second vehicle;
Inheritance method further comprising.
In a vehicle performing an artificial intelligence learning data inheritance method in an autonomous driving system,
sensor;
transceiver;
Memory; and
a processor for controlling the sensor, the transceiver, and the memory; including,
the processor
Receive a first model learned using sensing information related to a fixed object from the server through the transceiver,
Using the first model, a first marking object that is not classified,
Transmitting sensing information related to the first marked object to the server through the transceiver,
Receive a second model learned using sensing information related to the first marked object from the server through the transceiver,
The first model classifies a stationary object, and the second model classifies and tracks a moving object.
17. The method of claim 16,
the processor
Transmitting a request message to the server through the transceiver to request information about a fixed object located in a driving section of the vehicle or a scheduled driving section,
A vehicle receiving the information of the fixed object from the server.
17. The method of claim 16,
the processor
Transmitting the specification information of the vehicle to another vehicle through the transceiver,
Receive sensing information related to a fixed object of another vehicle from the other vehicle through the transceiver, based on the specification information of the vehicle,
A vehicle for learning the first model by using sensing information related to the fixed object of the other vehicle.
19. The method of claim 18,
the processor
A vehicle that receives a first model of another vehicle from the other vehicle through the transceiver, based on the specification information of the vehicle.
17. The method of claim 16,
the processor
Receive a first V2X message from another vehicle through the transceiver,
Based on a result of decoding the first V2X message, a vehicle for setting the sensing information generated through the sensor as the sensing information related to the moving object.

Images