WO2020004767A1

WO2020004767A1 - Telematics system provided in vehicle and method for controlling same

Info

Publication number: WO2020004767A1
Application number: PCT/KR2019/001975
Authority: WO
Inventors: 신정은; 조장형; 최성하
Original assignee: 엘지전자 주식회사
Priority date: 2018-06-25
Filing date: 2019-02-19
Publication date: 2020-01-02

Abstract

Proposed is a method for controlling a telematics system provided in a vehicle. Specifically, proposed is a method for controlling a telematics system provided in a vehicle, the method comprising the steps of: receiving a request for content output; requesting, from an external server, information associated with the content through a base station connected to the telematics system; and receiving, from the external server, the information through the base station.

Description

Telematics system provided in a vehicle and a method of controlling the same

The present invention relates to a telematics system provided in a vehicle and a method for controlling the same, and more particularly, to a method for transmitting and receiving a signal to and from an external server through a base station.

The vehicle traditionally functions as a user's means of transportation, but for the convenience of the user, the vehicle is provided with various sensors, electronic devices, and the like, to provide driving convenience for the user. In particular, the development of the Advanced Driver Assistance System (ADAS) and autonomous vehicles (Autonomous Vehicle) for the user's driving convenience is being actively made.

Recently, the driver assistance system and the autonomous vehicle provide various display devices for the convenience of the passenger as well as the driving convenience of the user. This is also known as in-car entertainment (ICE) or in-vehicle infotainment (IVI). ICE or IVI are automotive hardware and software that provide audio or video entertainment.

On the other hand, recently, researches are being actively conducted on mobile communication technologies, particularly technologies that provide ICE or IVI services through 5G NR (New Radio Access Technology). In particular, development of a 5G use case that provides communication with external devices through a telematics application, an ECU (Electronic Control Unit), a 5G modem, and the like is in progress. 5G use case includes Intersection Movement Assistant, Real-time Situational Awareness & HD Maps, Cooperative Lane Change of Automated Vehicles, See-Through, Vulnerable Road User Discovery, Shared Vision, Collective Sensory, Hybrid Intelligence, Journey Studio, Media Center on Wheels, etc. There is this.

In order to provide the above-described 5G use cases, a vehicle may be provided with a plurality of display devices. However, according to the related art, there is a problem in that content cannot be efficiently provided to a plurality of display devices in a vehicle through a telematics control unit (TCU).

In order to solve the above problems, the present invention is to provide a telematics system provided in a vehicle and a method of controlling the same.

The technical problem to be achieved in the present invention is to solve this problem of the prior art. Technical problems to be achieved in the present invention are not limited to the above technical problems, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention for achieving the above technical problem, receiving a request for content output, requesting information related to the content to an external server through a base station connected to the telematics system and the external through the base station A method of controlling a telematics system provided in a vehicle comprising receiving the information from a server is proposed. The received information may be delivered to a plurality of target devices connected to the telematics system, respectively.

The information is received from the external server via a predetermined protocol, and the predetermined protocol may include a transmission protocol, a streaming control protocol, and a quality of service (QoS) protocol.

If the received information is media content provided by the external server, the telematics system generates a URL (Universal Resource Locator) to which each of the plurality of target devices can access, and converts the generated URLs into the plurality of targets. Each can be delivered to the device.

The transmitted information may be output in synchronization with each of the plurality of target devices based on reference time information included in the QoS protocol and time information included in a packet constituting information received from the external server.

The plurality of target devices may include a center information display (CID), an audio video navigation (AVN), rear seat entertainment (RSE), and a consumer electronic (CE) device provided inside or outside the vehicle. Can be.

The content received from the external server may correspond to media content or a camera image of another vehicle.

According to an aspect of the present invention, the above-described problem of the related art, that is, the problem of failing to efficiently provide content to a plurality of display devices in a vehicle through a TCU (Telematics Control Unit) can be solved. Specifically, the telematics system according to an aspect of the present invention may provide audio / video synchronized content in a plurality of display devices in a vehicle.

Effects obtained in the present invention are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description in order to provide a thorough understanding of the present invention, provide an embodiment of the present invention and together with the description, illustrate the technical idea of the present invention.

1 is a view showing the appearance of a vehicle according to an aspect of the present invention.

2 is a view of the vehicle according to an aspect of the present invention from various angles from the outside.

3 to 4 are views illustrating the interior of a vehicle according to an aspect of the present invention.

5 to 6 are views referred to for explaining an object according to an aspect of the present invention.

7 is a block diagram referred to describe a vehicle according to an aspect of the present invention.

8 to 10 are diagrams for explaining the structure of a telematics system according to an aspect of the present invention.

11 to 14 are diagrams for explaining an implementation example of a telematics system according to an aspect of the present invention.

15 is a view for explaining the characteristics of the telematics system architecture according to an aspect of the present invention.

16 to 17 are diagrams for explaining the SW platform of the telematics system according to an aspect of the present invention.

18 is a view for explaining a telematics system according to an aspect of the present invention.

FIG. 19 illustrates a relationship between a telematics system and a cloud according to an aspect of the present invention.

20 is a view for explaining the relationship between the framework and the cloud of the telematics system according to an aspect of the present invention.

21 is a diagram for describing a media cast center of a telematics system according to an aspect of the present invention.

FIG. 22 illustrates a positioning service provided by a telematics system according to an aspect of the present invention.

FIG. 23 is a diagram for describing an electronic horizon provided by a telematics system according to an aspect of the present disclosure.

24 to 31 are views for explaining an implementation example of a telematics system according to an aspect of the present invention.

32 to 34 are views for explaining a technical effect of a telematics system according to an aspect of the present invention.

35 to 36 are diagrams for explaining a signal transmission and reception sequence between a TCU and a cloud server in a telematics system according to an aspect of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description, which will be given below with reference to the accompanying drawings, is intended to explain exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, one of ordinary skill in the art appreciates that the present invention may be practiced without these specific details.

1 to 7, the vehicle 100 may include a wheel that rotates by a power source and a steering input device 510 for adjusting a traveling direction of the vehicle 100.

The vehicle 100 may be an autonomous vehicle. The vehicle 100 may be switched to an autonomous driving mode or a manual mode based on a user input. For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the received user input through the user interface device 200.

The vehicle 100 may be switched to the autonomous driving mode or the manual mode based on the driving situation information. The driving situation information may include at least one of object information, navigation information, and vehicle state information outside the vehicle.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the driving situation information generated by the object detecting apparatus 300. For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the driving situation information received through the communication device 400.

The vehicle 100 may switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on information, data, and signals provided from an external device.

When the vehicle 100 is driven in the autonomous driving mode, the autonomous vehicle 100 may be driven based on the driving system 700. For example, the autonomous vehicle 100 may be driven based on information, data, or signals generated by the driving system 710, the parking system 740, and the parking system 750.

When the vehicle 100 is driven in the manual mode, the autonomous vehicle 100 may receive a user input for driving through the driving manipulation apparatus 500. Based on a user input received through the driving manipulation apparatus 500, the vehicle 100 may be driven.

The overall length is the length from the front to the rear of the vehicle 100, the width is the width of the vehicle 100, and the height is the length from the bottom of the wheel to the roof. In the following description, the full length direction L is a direction in which the full length measurement of the vehicle 100 is a reference, the full width direction W is a direction in which the full width measurement of the vehicle 100 is a reference, and the total height direction H is a vehicle. It may mean the direction which is the reference of the height measurement of (100).

As illustrated in FIG. 7, the vehicle 100 includes a user interface device 200, an object detecting device 300, a communication device 400, a driving manipulation device 500, a vehicle driving device 600, and a traveling system. 700, a navigation system 770, a sensing unit 120, an interface unit 130, a memory 140, a control unit 170, and a power supply unit 190 may be included.

According to an exemplary embodiment, the vehicle 100 may further include other components in addition to the components described herein, or may not include some of the described components. The sensing unit 120 may include a state of the vehicle. Can sense. The sensing unit 120 may include an attitude sensor (for example, a yaw sensor, a roll sensor, a pitch sensor), a collision sensor, a wheel sensor, a speed sensor, and an inclination. Sensor, Weight Sensor, Heading Sensor, Gyro Sensor, Position Module, Vehicle Forward / Reverse Sensor, Battery Sensor, Fuel Sensor, Tire Sensor, Steering Sensor by Steering Wheel, Vehicle And an internal temperature sensor, an in-vehicle humidity sensor, an ultrasonic sensor, an illuminance sensor, an accelerator pedal position sensor, a brake pedal position sensor, and the like.

The sensing unit 120 includes vehicle attitude information, vehicle collision information, vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery Acquire sensing signals for information, fuel information, tire information, vehicle lamp information, vehicle internal temperature information, vehicle internal humidity information, steering wheel rotation angle, vehicle external illumination, pressure applied to the accelerator pedal, pressure applied to the brake pedal, and the like. can do.

The sensing unit 120 may further include an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake air temperature sensor (ATS), a water temperature sensor (WTS), and a throttle position sensor. (TPS), TDC sensor, crank angle sensor (CAS), and the like.

The sensing unit 120 may generate vehicle state information based on the sensing data. The vehicle state information may be information generated based on data sensed by various sensors provided in the vehicle.

For example, the vehicle state information includes vehicle attitude information, vehicle speed information, vehicle tilt information, vehicle weight information, vehicle direction information, vehicle battery information, vehicle fuel information, vehicle tire pressure information, The vehicle may include steering information of the vehicle, vehicle indoor temperature information, vehicle indoor humidity information, pedal position information, vehicle engine temperature information, and the like.

The interface unit 130 may serve as a path to various types of external devices connected to the vehicle 100. For example, the interface unit 130 may include a port connectable with the mobile terminal, and may connect with the mobile terminal through the port. In this case, the interface unit 130 may exchange data with the mobile terminal.

Meanwhile, the interface unit 130 may serve as a path for supplying electrical energy to the connected mobile terminal. When the mobile terminal is electrically connected to the interface unit 130, under the control of the controller 170, the interface unit 130 may provide the mobile terminal with electrical energy supplied from the power supply unit 190.

The memory 140 is electrically connected to the controller 170. The memory 140 may store basic data for the unit, control data for controlling the operation of the unit, and input / output data. The memory 140 may be various storage devices such as a ROM, a RAM, an EPROM, a flash drive, a hard drive, and the like, in hardware. The memory 140 may store various data for overall operation of the vehicle 100, such as a program for processing or controlling the controller 170. According to an embodiment, the memory 140 may be integrally formed with the controller 170 or may be implemented as a subcomponent of the controller 170.

The controller 170 may control the overall operation of each unit in the vehicle 100. The controller 170 may be referred to as an electronic control unit (ECU). The power supply unit 190 may supply power required for the operation of each component under the control of the controller 170. In particular, the power supply unit 190 may receive power from a battery inside the vehicle.

One or more processors and controllers 170 included in vehicle 100 may include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable (FPGAs). Gate arrays, processors, controllers, micro-controllers, microprocessors, and other electrical units for performing other functions may be implemented.

In addition, the sensing unit 120, the interface unit 130, the memory 140, the power supply unit 190, the user interface device 200, the object detection device 300, the communication device 400, and the driving operation device 500. The vehicle driving apparatus 600, the driving system 700, and the navigation system 770 may have separate processors or may be integrated into the controller 170.

The user interface device 200 is a device for communicating with the vehicle 100 and a user. The user interface device 200 may receive a user input and provide the user with information generated in the vehicle 100. The vehicle 100 may implement user interfaces (UI) or user experience (UX) through the user interface device 200.

The user interface device 200 may include an input unit 210, an internal camera 220, a biometric detector 230, an output unit 250, and a processor 270. Each component of the user interface device 200 may be structurally and functionally separated from or integrated with the interface unit 130 described above.

According to an embodiment, the user interface device 200 may further include other components in addition to the described components, or may not include some of the described components.

The input unit 210 is for receiving information from a user, and the data collected by the input unit 210 may be analyzed by the processor 270 and processed as a user's control command.

The input unit 210 may be disposed in the vehicle. For example, the input unit 210 may include one area of a steering wheel, one area of an instrument panel, one area of a seat, one area of each pillar, and a door. one area of the door, one area of the center console, one area of the head lining, one area of the sun visor, one area of the windshield or of the window It may be disposed in one area or the like.

The input unit 210 may include a voice input unit 211, a gesture input unit 212, a touch input unit 213, and a mechanical input unit 214.

The voice input unit 211 may convert a user's voice input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170. The voice input unit 211 may include one or more microphones.

The gesture input unit 212 may convert a user's gesture input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170. The gesture input unit 212 may include at least one of an infrared sensor and an image sensor for detecting a user's gesture input.

According to an embodiment, the gesture input unit 212 may detect a 3D gesture input of the user. To this end, the gesture input unit 212 may include a light output unit or a plurality of image sensors for outputting a plurality of infrared light. The gesture input unit 212 may detect a user's 3D gesture input through a time of flight (TOF) method, a structured light method, or a disparity method.

The touch input unit 213 may convert a user's touch input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170. The touch input unit 213 may include a touch sensor for detecting a user's touch input. According to an embodiment, the touch input unit 213 may be integrally formed with the display unit 251 to implement a touch screen. Such a touch screen may provide an input interface and an output interface between the vehicle 100 and the user.

The mechanical input unit 214 may include at least one of a button, a dome switch, a jog wheel, and a jog switch. The electrical signal generated by the mechanical input unit 214 may be provided to the processor 270 or the controller 170. The mechanical input unit 214 may be disposed on a steering wheel, a center fascia, a center console, a cockpit module, a door, or the like.

The processor 270 starts a learning mode of the vehicle 100 in response to user inputs to at least one of the voice input unit 211, the gesture input unit 212, the touch input unit 213, and the mechanical input unit 214 described above. can do. In the learning mode, the vehicle 100 may perform driving path learning and surrounding environment learning of the vehicle 100. The learning mode will be described in detail later with reference to the object detecting apparatus 300 and the driving system 700.

The internal camera 220 may acquire a vehicle interior image. The processor 270 may detect a state of the user based on the vehicle interior image. The processor 270 may acquire the gaze information of the user from the vehicle interior image. The processor 270 may detect a gesture of the user in the vehicle interior image.

The biometric detector 230 may acquire biometric information of the user. The biometric detector 230 may include a sensor for acquiring biometric information of the user, and may acquire fingerprint information, heartbeat information, etc. of the user using the sensor. Biometric information may be used for user authentication.

The output unit 250 is for generating output related to visual, auditory or tactile. The output unit 250 may include at least one of the display unit 251, the audio output unit 252, and the haptic output unit 253.

The display unit 251 may display graphic objects corresponding to various pieces of information. The display unit 251 is a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display (flexible) display, a 3D display, or an e-ink display.

The display unit 251 forms a layer structure or is integrally formed with the touch input unit 213 to implement a touch screen. The display unit 251 may be implemented as a head up display (HUD). When the display unit 251 is implemented as a HUD, the display unit 251 may include a projection module to output information through an image projected on a wind shield or a window. The display unit 251 may include a transparent display. The transparent display can be attached to the wind shield or window.

The transparent display may display a predetermined screen while having a predetermined transparency. In order to have transparency, a transparent display is a transparent thin film electroluminescent (TFEL), a transparent organic light-emitting diode (OLED), a transparent liquid crystal display (LCD), a transmissive transparent display, a transparent light emitting diode (LED) display It may include at least one of. The transparency of the transparent display can be adjusted.

The user interface device 200 may include a plurality of display units 251a to 251g.

The display unit 251 may include one region of the steering wheel, one

region

251a, 251b, and 251e of the instrument panel, one region 251d of the seat, one region 251f of each pillar, and one region of the door ( 251g), one area of the center console, one area of the head lining, one area of the sun visor, or may be implemented in one area 251c of the windshield and one area 251h of the window.

The sound output unit 252 converts an electrical signal provided from the processor 270 or the controller 170 into an audio signal and outputs the audio signal. To this end, the sound output unit 252 may include one or more speakers.

The haptic output unit 253 generates a tactile output. For example, the haptic output unit 253 may vibrate the steering wheel, the seat belt, and the seats 110FL, 110FR, 110RL, and 110RR so that the user may recognize the output.

The processor 270 may control the overall operation of each unit of the user interface device 200. According to an embodiment, the user interface device 200 may include a plurality of processors 270 or may not include the processor 270.

When the processor 270 is not included in the user interface device 200, the user interface device 200 may be operated under the control of the processor or the controller 170 of another device in the vehicle 100. The user interface device 200 may be referred to as a vehicle display device. The user interface device 200 may be operated under the control of the controller 170.

The object detecting apparatus 300 is a device for detecting an object located outside the vehicle 100. The object detecting apparatus 300 may generate object information based on the sensing data.

The object information may include information about the presence or absence of the object, location information of the object, distance information between the vehicle 100 and the object, and relative speed information between the vehicle 100 and the object. The object may be various objects related to the driving of the vehicle 100.

5 to 6, the object O includes a lane OB10, another vehicle OB11, a pedestrian OB12, a two-wheeled vehicle OB13, traffic signals OB14, OB15, light, a road, a structure, Speed bumps, features, animals and the like can be included.

The lane OB10 may be a driving lane, a lane next to the driving lane, and a lane in which an opposite vehicle travels. The lane OB10 may be a concept including left and right lines forming a lane.

The other vehicle OB11 may be a vehicle that is driving around the vehicle 100. The other vehicle may be a vehicle located within a predetermined distance from the vehicle 100. For example, the other vehicle OB11 may be a vehicle that precedes or follows the vehicle 100.

The pedestrian OB12 may be a person located near the vehicle 100. The pedestrian OB12 may be a person located within a predetermined distance from the vehicle 100. For example, the pedestrian OB12 may be a person located on a sidewalk or a roadway.

The two-wheeled vehicle OB13 may be a vehicle that is positioned around the vehicle 100 and moves using two wheels. The motorcycle OB13 may be a vehicle having two wheels located within a predetermined distance from the vehicle 100. For example, the motorcycle OB13 may be a motorcycle or a bicycle located on sidewalks or roadways.

The traffic signal may include a traffic light OB15, a traffic sign OB14, and a pattern or text drawn on a road surface. The light may be light generated by a lamp provided in another vehicle. The light, can be light generated from the street light. The light may be sunlight. The road may include a road surface, a curve, an uphill slope, a slope downhill, or the like. The structure may be an object located around a road and fixed to the ground. For example, the structure may include a street lamp, a roadside tree, a building, a power pole, a traffic light, a bridge. The features may include mountains, hills, and the like.

On the other hand, the object may be classified into a moving object and a fixed object. For example, the moving object may be a concept including another vehicle and a pedestrian. For example, the fixed object may be a concept including a traffic signal, a road, and a structure.

The object detecting apparatus 300 may include a camera 310, a radar 320, a lidar 330, an ultrasonic sensor 340, an infrared sensor 350, and a processor 370. Each component of the object detecting apparatus 300 may be structurally and functionally separated or integrated with the sensing unit 120 described above.

According to an embodiment, the object detecting apparatus 300 may further include other components in addition to the described components, or may not include some of the described components.

The camera 310 may be located at a suitable place outside the vehicle to acquire an image outside the vehicle. The camera 310 may be a mono camera, a stereo camera 310a, an around view monitoring (AVM) camera 310b, or a 360 degree camera.

The camera 310 may acquire location information of the object, distance information with respect to the object, or relative speed information with the object by using various image processing algorithms.

For example, the camera 310 may obtain distance information and relative speed information with respect to the object based on the change in the object size over time in the acquired image.

For example, the camera 310 may acquire distance information and relative velocity information with respect to an object through a pin hole model, road surface profiling, or the like.

For example, the camera 310 may obtain distance information and relative speed information with respect to the object based on the disparity information in the stereo image acquired by the stereo camera 310a.

For example, the camera 310 may be disposed in close proximity to the front windshield in the interior of the vehicle in order to acquire an image in front of the vehicle. Alternatively, the camera 310 may be disposed around the front bumper or the radiator grille.

For example, the camera 310 may be disposed in close proximity to the rear glass in the interior of the vehicle to acquire an image of the rear of the vehicle. Alternatively, the camera 310 may be disposed around the rear bumper, the trunk, or the tail gate.

For example, the camera 310 may be disposed in close proximity to at least one of the side windows in the interior of the vehicle to acquire an image of the vehicle side. Alternatively, the camera 310 may be arranged around the side mirror, fender or door.

The camera 310 may provide the obtained image to the processor 370.

The radar 320 may include an electromagnetic wave transmitter and a receiver. The radar 320 may be implemented in a pulse radar method or a continuous wave radar method in terms of radio wave firing principle. The radar 320 may be implemented by a frequency modulated continuous wave (FSCW) method or a frequency shift keying (FSK) method according to a signal waveform among continuous wave radar methods.

The radar 320 detects an object based on a time of flight (TOF) method or a phase-shift method based on electromagnetic waves, and detects the position of the detected object, distance to the detected object, and relative velocity. Can be detected.

The radar 320 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The lidar 330 may include a laser transmitter and a receiver. The lidar 330 may be implemented in a time of flight (TOF) method or a phase-shift method.

The lidar 330 may be implemented as driven or non-driven. When implemented in a driving manner, the lidar 330 may be rotated by a motor and detect an object around the vehicle 100. When implemented in a non-driven manner, the lidar 330 may detect an object located within a predetermined range with respect to the vehicle 100 by optical steering. The vehicle 100 may include a plurality of non-driven lidars 330.

The lidar 330 detects an object based on a time of flight (TOF) method or a phase-shift method using laser light, and detects an object, a position of the detected object, a distance from the detected object, and Relative speed can be detected. The lidar 330 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The ultrasonic sensor 340 may include an ultrasonic transmitter and a receiver. The ultrasonic sensor 340 may detect an object based on the ultrasonic wave, and detect a position of the detected object, a distance to the detected object, and a relative speed. The ultrasonic sensor 340 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The infrared sensor 350 may include an infrared transmitter and a receiver. The infrared sensor 340 may detect an object based on infrared light, and detect a position of the detected object, a distance to the detected object, and a relative speed. The infrared sensor 350 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The processor 370 may control overall operations of each unit of the object detecting apparatus 300. The processor 370 compares the data sensed by the camera 310, the radar 320, the lidar 330, the ultrasonic sensor 340, and the infrared sensor 350 with previously stored data to detect or classify an object. can do.

The processor 370 may detect and track the object based on the obtained image. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with the object through an image processing algorithm.

For example, the processor 370 may acquire distance information and relative speed information with respect to the object based on the change in the object size over time in the obtained image.

For example, the processor 370 may acquire distance information and relative velocity information with respect to an object through a pin hole model, road surface profiling, or the like.

For example, the processor 370 may obtain distance information and relative speed information with the object based on the disparity information in the stereo image acquired by the stereo camera 310a.

The processor 370 may detect and track the object based on the reflected electromagnetic wave reflected by the transmitted electromagnetic wave to the object. The processor 370 may perform an operation such as calculating a distance from the object, calculating a relative speed with the object, and the like based on the electromagnetic waves.

The processor 370 may detect and track the object based on the reflected laser light reflected by the transmitted laser back to the object. The processor 370 may perform an operation such as calculating a distance from the object, calculating a relative speed with the object, and the like based on the laser light.

The processor 370 may detect and track the object based on the reflected ultrasound, in which the transmitted ultrasound is reflected by the object and returned. The processor 370 may perform an operation such as calculating a distance from the object, calculating a relative speed with the object, and the like based on the ultrasound.

The processor 370 may detect and track the object based on the reflected infrared light from which the transmitted infrared light is reflected back to the object. The processor 370 may perform an operation such as calculating a distance to the object, calculating a relative speed with the object, and the like based on the infrared light.

As described above, when the learning mode of the vehicle 100 is started in response to a user input to the input unit 210, the processor 370 may include a camera 310, a radar 320, a lidar 330, and an ultrasonic sensor. The data sensed by the 340 and the infrared sensor 350 may be stored in the memory 140.

Each step of the learning mode based on the analysis of the stored data and an operation mode following the learning mode will be described in detail with reference to the operation system 700. According to an embodiment, the object detecting apparatus 300 In addition, the processor 370 may or may not include the processor 370. For example, each of the camera 310, the radar 320, the lidar 330, the ultrasonic sensor 340, and the infrared sensor 350 may individually include a processor.

When the processor 370 is not included in the object detecting apparatus 300, the object detecting apparatus 300 may be operated under the control of the processor or the controller 170 of the apparatus in the vehicle 100. The object detecting apparatus 300 may be operated under the control of the controller 170.

The communication device 400 is a device for performing communication with an external device. Here, the external device may be another vehicle, a mobile terminal or a server. The communication device 400 may include at least one of a transmit antenna, a receive antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

The communication device 400 includes a short range communication unit 410, a location information unit 420, a V2X communication unit 430, an optical communication unit 440, a broadcast transmission / reception unit 450, an ITS (Intelligent Transport Systems) communication unit 460, and a processor. 470 may include. According to an embodiment, the communication device 400 may further include other components in addition to the described components, or may not include some of the described components.

The short range communication unit 410 is a unit for short range communication. The local area communication unit 410 may include Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), and Wi-Fi (Wireless). Local area communication may be supported using at least one of Fidelity, Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technologies. The short range communication unit 410 may form short range wireless networks to perform short range communication between the vehicle 100 and at least one external device.

The location information unit 420 is a unit for obtaining location information of the vehicle 100. For example, the location information unit 420 may include a global positioning system (GPS) module or a differential global positioning system (DGPS) module.

The V2X communication unit 430 is a unit for performing wireless communication with a server (V2I: Vehicle to Infra), another vehicle (V2V: Vehicle to Vehicle), or a pedestrian (V2P: Vehicle to Pedestrian). The V2X communication unit 430 may include an RF circuit that can implement a communication with the infrastructure (V2I), an inter-vehicle communication (V2V), and a communication with the pedestrian (V2P).

The optical communication unit 440 is a unit for performing communication with an external device via light. The optical communication unit 440 may include an optical transmitter that converts an electrical signal into an optical signal and transmits the external signal to the outside, and an optical receiver that converts the received optical signal into an electrical signal. According to an embodiment, the light emitting unit may be formed to be integrated with the lamp included in the vehicle 100.

The broadcast transceiver 450 is a unit for receiving a broadcast signal from an external broadcast management server or transmitting a broadcast signal to a broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal.

The ITS communication unit 460 may exchange information, data, or signals with the traffic system. The ITS communication unit 460 may provide the obtained information and data to the transportation system. The ITS communication unit 460 may receive information, data, or a signal from a traffic system. For example, the ITS communication unit 460 may receive road traffic information from the traffic system and provide the road traffic information to the control unit 170. For example, the ITS communication unit 460 may receive a control signal from a traffic system and provide the control signal to a processor provided in the controller 170 or the vehicle 100.

The processor 470 may control the overall operation of each unit of the communication device 400. According to an embodiment, the communication device 400 may include a plurality of processors 470 or may not include the processor 470. When the processor 470 is not included in the communication device 400, the communication device 400 may be operated under the control of the processor or the controller 170 of another device in the vehicle 100.

Meanwhile, the communication device 400 may implement a vehicle display device together with the user interface device 200. In this case, the vehicle display device may be called a telematics device or an audio video navigation (AVN) device. The communication device 400 may be operated under the control of the controller 170.

The driving operation apparatus 500 is a device that receives a user input for driving. In the manual mode, the vehicle 100 may be driven based on a signal provided by the driving manipulation apparatus 500. The driving manipulation apparatus 500 may include a steering input apparatus 510, an acceleration input apparatus 530, and a brake input apparatus 570.

The steering input device 510 may receive a driving direction input of the vehicle 100 from the user. The steering input device 510 is preferably formed in a wheel shape to enable steering input by rotation. According to an embodiment, the steering input device may be formed in the form of a touch screen, a touch pad, or a button.

The acceleration input device 530 may receive an input for accelerating the vehicle 100 from a user. The brake input device 570 may receive an input for deceleration of the vehicle 100 from a user. The acceleration input device 530 and the brake input device 570 are preferably formed in the form of a pedal. According to an embodiment, the acceleration input device or the brake input device may be formed in the form of a touch screen, a touch pad, or a button.

The driving manipulation apparatus 500 may be operated under the control of the controller 170.

The vehicle drive device 600 is a device that electrically controls the driving of various devices in the vehicle 100. The vehicle driving apparatus 600 may include a power train driver 610, a chassis driver 620, a door / window driver 630, a safety device driver 640, a lamp driver 650, and an air conditioning driver 660. Can be. According to an embodiment, the vehicle driving apparatus 600 may further include other components in addition to the described components, or may not include some of the described components. On the other hand, the vehicle driving device 600 may include a processor. Each unit of the vehicle drive apparatus 600 may each include a processor individually.

The power train driver 610 may control the operation of the power train device. The power train driver 610 may include a power source driver 611 and a transmission driver 612.

The power source driver 611 may control the power source of the vehicle 100. For example, when the fossil fuel-based engine is a power source, the power source driver 610 may perform electronic control of the engine. Thereby, the output torque of an engine, etc. can be controlled. The power source drive unit 611 can adjust the engine output torque under the control of the control unit 170.

For example, when the electric energy based motor is a power source, the power source driver 610 may control the motor. The power source driver 610 may adjust the rotational speed, torque, and the like of the motor under the control of the controller 170.

The transmission driver 612 may control the transmission. The transmission driver 612 can adjust the state of the transmission. The transmission drive part 612 can adjust the state of a transmission to forward D, backward R, neutral N, or parking P. FIG. On the other hand, when the engine is a power source, the transmission drive unit 612 can adjust the bite state of the gear in the forward D state.

The chassis driver 620 may control the operation of the chassis device. The chassis driver 620 may include a steering driver 621, a brake driver 622, and a suspension driver 623.

The steering driver 621 may perform electronic control of a steering apparatus in the vehicle 100. The steering driver 621 may change the traveling direction of the vehicle.

The brake driver 622 may perform electronic control of a brake apparatus in the vehicle 100. For example, the speed of the vehicle 100 may be reduced by controlling the operation of the brake disposed on the wheel.

On the other hand, the brake drive unit 622 can individually control each of the plurality of brakes. The brake driver 622 may control the braking force applied to the plurality of wheels differently.

The suspension driver 623 may perform electronic control of a suspension apparatus in the vehicle 100. For example, when there is a curvature on the road surface, the suspension driver 623 may control the suspension device to control the vibration of the vehicle 100 to be reduced. Meanwhile, the suspension driver 623 may individually control each of the plurality of suspensions.

The door / window driver 630 may perform electronic control of a door apparatus or a window apparatus in the vehicle 100. The door / window driver 630 may include a door driver 631 and a window driver 632.

The door driver 631 may control the door apparatus. The door driver 631 may control opening and closing of the plurality of doors included in the vehicle 100. The door driver 631 may control the opening or closing of a trunk or a tail gate. The door driver 631 may control the opening or closing of the sunroof.

The window driver 632 may perform electronic control of the window apparatus. The opening or closing of the plurality of windows included in the vehicle 100 may be controlled.

The safety device driver 640 may perform electronic control of various safety apparatuses in the vehicle 100. The safety device driver 640 may include an airbag driver 641, a seat belt driver 642, and a pedestrian protection device driver 643.

The airbag driver 641 may perform electronic control of an airbag apparatus in the vehicle 100. For example, the airbag driver 641 may control the airbag to be deployed when the danger is detected.

The seat belt driver 642 may perform electronic control of a seatbelt apparatus in the vehicle 100. For example, the seat belt driver 642 may control the passenger to be fixed to the seats 110FL, 110FR, 110RL, and 110RR by using the seat belt when detecting a danger.

The pedestrian protection device driver 643 may perform electronic control of the hood lift and the pedestrian airbag. For example, the pedestrian protection device driver 643 may control the hood lift up and the pedestrian air bag to be deployed when the collision with the pedestrian is detected.

The lamp driver 650 may perform electronic control of various lamp apparatuses in the vehicle 100.

The air conditioning driver 660 may perform electronic control of an air conditioner in the vehicle 100. For example, when the temperature inside the vehicle is high, the air conditioning driver 660 may control the air conditioning apparatus to operate to supply cool air to the inside of the vehicle.

The vehicle driving apparatus 600 may include a processor. Each unit of the vehicle drive apparatus 600 may each include a processor individually. The vehicle driving apparatus 600 may be operated under the control of the controller 170.

The travel system 700 is a system for controlling various travels of the vehicle 100. The navigation system 700 can be operated in an autonomous driving mode.

The travel system 700 can include a travel system 710, a parking system 740, and a parking system 750. In some embodiments, the navigation system 700 may further include other components in addition to the described components, or may not include some of the described components. Meanwhile, the driving system 700 may include a processor. Each unit of the navigation system 700 may each include a processor individually.

On the other hand, the driving system 700 may control the driving of the autonomous driving mode based on the learning. In this case, the learning mode and the operation mode on the premise that the learning is completed may be performed. A method of the processor of the driving system 700 to perform a learning mode and an operating mode will be described below.

The learning mode may be performed in the manual mode described above. In the learning mode, the processor of the driving system 700 may perform driving path learning and surrounding environment learning of the vehicle 100.

The driving route learning may include generating map data on a route on which the vehicle 100 travels. In particular, the processor of the driving system 700 may generate map data based on information detected by the object detecting apparatus 300 while the vehicle 100 travels from the starting point to the destination.

The surrounding environment learning may include storing and analyzing information about the surrounding environment of the vehicle 100 in the driving process and the parking process of the vehicle 100. In particular, the processor of the driving system 700 may detect information detected by the object detecting apparatus 300 during the parking process of the vehicle 100, for example, location information of the parking space, size information, fixed (or not fixed). Information about the surrounding environment of the vehicle 100 may be stored and analyzed based on information such as obstacle information.

The operation mode may be performed in the autonomous driving mode described above. The operation mode will be described on the premise that the driving route learning or the surrounding environment learning is completed through the learning mode.

The operation mode may be performed in response to a user input through the input unit 210, or may be automatically performed when the vehicle 100 reaches a driving path and a parking space where learning is completed.

The operating mode is a semi autonomous operating mode that requires some user's manipulation of the drive manipulator 500 and a full-autonomous operation requiring no user's manipulation of the drive manipulator 500. May include a fully autonomous operating mode.

Meanwhile, according to an exemplary embodiment, the processor of the driving system 700 may control the driving system 710 in the operation mode to drive the vehicle 100 along the driving path where learning is completed.

Meanwhile, according to an exemplary embodiment, the processor of the driving system 700 may control the parking system 740 in the operation mode to release the parked vehicle 100 from the parking space where the learning is completed.

Meanwhile, according to an exemplary embodiment, the processor of the driving system 700 may control the parking system 750 in the operation mode to park the vehicle 100 from the current position to the parking space where the learning is completed. When the driving system 700 is implemented in software, the driving system 700 may be a lower concept of the controller 170.

In some embodiments, the driving system 700 may include a user interface device 270, an object detecting device 300, a communication device 400, a driving manipulation device 500, a vehicle driving device 600, and a navigation system. 770, the sensing unit 120, and the control unit 170 may include a concept including at least one.

The traveling system 710 may perform driving of the vehicle 100. The driving system 710 may receive navigation information from the navigation system 770, provide a control signal to the vehicle driving apparatus 600, and perform driving of the vehicle 100.

The driving system 710 may receive object information from the object detecting apparatus 300 and provide a control signal to the vehicle driving apparatus 600 to perform driving of the vehicle 100. The driving system 710 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving device 600, and perform driving of the vehicle 100.

The driving system 710 may include a user interface device 270, an object detection device 300, a communication device 400, a driving manipulation device 500, a vehicle driving device 600, a navigation system 770, and a sensing unit ( At least one of the 120 and the controller 170 may be a system concept for driving the vehicle 100. The driving system 710 may be referred to as a vehicle driving control device.

The taking-out system 740 may perform taking out of the vehicle 100. The taking-out system 740 may receive navigation information from the navigation system 770, provide a control signal to the vehicle driving apparatus 600, and perform take-out of the vehicle 100.

The taking-out system 740 may receive the object information from the object detecting apparatus 300, provide a control signal to the vehicle driving apparatus 600, and perform take-out of the vehicle 100.

The taking-off system 740 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving apparatus 600, and perform take-out of the vehicle 100.

The car leaving system 740 includes a user interface device 270, an object detecting device 300 and a communication device 400, a driving control device 500, a vehicle driving device 600, a navigation system 770, and a sensing unit ( Including at least one of the controller 120 and the controller 170, the concept of a system that performs the taking out of the vehicle 100 may be performed.

Such a car leaving system 740 may be referred to as a vehicle parking control device.

The parking system 750 may perform parking of the vehicle 100. The parking system 750 may receive navigation information from the navigation system 770, provide a control signal to the vehicle driving apparatus 600, and perform parking of the vehicle 100.

The parking system 750 may receive the object information from the object detecting apparatus 300, provide a control signal to the vehicle driving apparatus 600, and perform parking of the vehicle 100.

The parking system 750 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving device 600, and perform parking of the vehicle 100.

The parking system 750 includes a user interface device 270, an object detection device 300 and a communication device 400, a driving operation device 500, a vehicle driving device 600, a navigation system 770, and a sensing unit ( At least one of the 120 and the controller 170 may be a system concept for parking the vehicle 100.

Such a parking system 750 may be referred to as a vehicle parking control device.

The navigation system 770 can provide navigation information. The navigation information may include at least one of map information, set destination information, route information according to the destination setting, information on various objects on the route, lane information, and current location information of the vehicle.

The navigation system 770 may include a memory and a processor. The memory may store navigation information. The processor may control the operation of the navigation system 770.

According to an embodiment, the navigation system 770 may receive information from an external device through the communication device 400 and update the pre-stored information. According to an embodiment, the navigation system 770 may be classified as a subcomponent of the user interface device 200.

8 is a view for explaining a telematics system according to an aspect of the present invention. The telematics system according to an aspect of the present invention may provide communication with an external device through a telematics application, an ECU (Electronic Control Unit), a 5G modem, or the like.

The telematics system (or 5G telematics system) according to an aspect of the present invention may be provided in a vehicle together with an AVN (Audio / Video / Navigation) and an ECU. In addition, the CE device (Consumer Electronics Device) of the occupant may exist together in the vehicle. The telematics system can be connected to the AVN and ECU via Ethernet. In addition, telematics systems can be connected to CE devices through the Open Connectivity Foundation (OCF), which is known as one of the IoT protocols. An internal configuration of the telematics system according to an aspect of the present invention, for example, a service framework, a framework, a platform service, an OS layer, and an HW, will be described in detail below.

9 to 10 are diagrams for explaining a telematics system according to an aspect of the present invention. Telematics system architecture should be considered in terms of E / E (Electrical / Electronic) architecture. In addition, the software in the telematics system must be designed to be compatible or applicable to heterogeneous architectures and all ECUs.

Referring to FIG. 9, a telematic control unit (TCU) of a telematics system may be implemented in a structure separate from a modem (case 1). In other words, case 1 is a method in which the modem and telematics are separated in the TCU to communicate using only the interface. Unlike case 1, case 2 is how a modem and an AP are implemented in one SoC and the entire TCU on top of it. Meanwhile, as shown in the right side of FIG. 9, a method of separately implementing a modem and a telematics system may be considered.

Telematics Control Unit (TCU) can be considered in two types. First, Modem and Application Processor are implemented as physically different chipsets, so two chipsets can be considered as a form of TCU. Second, Modem and Application Processor are physically composed of one Chipset and can be considered as a form that does not require special I / F configuration. On the other hand, in addition to the two types, Modem and Application Processor are composed of two chipsets, but there is a form that is not one board, but this is similar in terms of the first configuration and S / W configuration.

In the first configuration described above, Modem only plays a role as a communication module and may be implemented to provide all data to the AP (Application Processor) as PCIe. The AP can pass data to other connected ECUs. In the TCU, a service having a special function may be configured according to its purpose. For example, there may be a device driver configuration for high-speed data processing related to data routing and a configuration having priority in S / W according to characteristics of ECUs to be linked.

In the second configuration described above, the modem and the AP may be configured as one module. Data can be transferred through internal memory without PCIe I / F. On the other hand, due to the priority and performance distribution of Modem for data processing, the role of Application Processor can be reduced.

On the other hand, there may be a number of ways to implement the software supporting the first and second configuration. In order to effectively manage post-development and perform horizontal development and improvement of development results, S / W should be implemented to increase reusability. For this purpose, removing the platform dependency may be the basic method. Consideration should also be given to methods for enabling communication between software in different physical locations.

Meanwhile, a protocol oriented to Service Oriented Architecture (SOA) should be used so that implemented SW can operate regardless of location. This should allow implementations of SW (eg SW5 and SW6) to work regardless of their physical location. As a protocol oriented to Service Oriented Architecture, SOME / IP and DDS can be provided as Framework, and it must be designed to run in various E / E Architectures. Hereinafter, the function of each component of the telematics system architecture will be described.

Service Oriented Architecture (SOA) is i) a fully integrated system with complete control, ii) rapid response to rapidly changing business conditions or vendor-specific platforms, and iii) price advantage.

As an SOA-based architecture, SOME / IP (Scalable Service-Oriented Middleware over IP) or DDS (Data Distribution Service) can be used. Both SOME / IP and DDS are SOA-based middleware solutions. In SOME / IP and DDS, service discovery is used to find service instances and to find out if a service instance is running. SOME / IP was implemented in vehicles before DDS. DDS was first implemented in the aviation and military sectors and could benefit V2X.

Platform based on AGL (Automotive Grade Linux) can be used. AGL is not much different from other Linux platforms such as GENIVI, and it is a standard that can be expanded not only for expansion but also for revision and improvement.

TSN (Time Sensitive Network) is a standard that can be used as an upversion of eAVB, but the standard is in progress. TSN is an efficient technology that needs to be applied to the entire network, and it takes a lot of time to support commercial backbone networks, so it can be secured first through in-vehicle applications.

MultiLink Controller is a module for data transmission management for various networks. MultiLink Controller can support management of connected radio channel such as Modem communication (LTE / 5G) and Wi-Fi. Modem communication can be used for data that needs to be transmitted in real time, and monitoring and backup data can be controlled to be transmitted only via Wi-Fi depending on the memory setting. Each communication must be considered in terms of mobility and charging, and can be flexibly modified according to the service provided in the vehicle.

Media cast center represents a multimedia service received or transmitted through the TCU. The media cast center can perform processing on various multimedia sources as well as basic streaming protocol and data compression / encoding / decoding. If 5G network is used, various multimedia services are expected to increase, which is expected to be important service not only in TCU but also in AVN specification.

IoT2V is an IoT service gateway that can interoperate with various IoT services and corresponds to a gateway for interworking AI services of service providers (eg, Amazon, Google, Naver).

Cloud Service Manager is a service manager to provide services provided by interworking network servers as if they are in-vehicle services. When functions that cannot be embedded are provided through the network according to various TCU capabilities / support ranges, a service supporting SOA may be considered as a proxy service to provide the same functions in a vehicle.

In addition, the server on the network marked Cloud Server can provide the same function as the embedded service when supporting Mobile Edge Computing. To do this, Cloud Server (Network Server) must also be provided as a framework supporting SOA. Meanwhile, service interworking between SOA frameworks may be affected by the protocol transmission method of the framework rather than the physical location.

Referring to FIG. 10, it is necessary to consider a telematics AP system, a modem, SoC integration into IVI, and TCU location considering harnessing with an antenna from an E / E architecture perspective. Specifically, in terms of consolidation, integration of TCU functions into other ECUs, integration of other functions into TCUs, and integration of TCU APs and modems can be considered. In addition, in terms of wiring harnessing, it is necessary to consider the optimal location in the vehicle in consideration of harnessing such as RF cable and Ethernet in connection with consolidation.

11 is a view for explaining an implementation example of a telematics system according to an aspect of the present invention.

According to the telematics system according to an aspect of the present invention, with respect to 5G throughput through camera image up-streaming, camera-based applications through up-streaming and cloud computing of camera images are possible.

On the other hand, according to the telematics system according to an aspect of the present invention it is possible to improve Cloud Computing usability (feasibility) according to the enhancement of cloud computing performance. In addition, low-cost SoC design is possible by comparing TCU's CPU / GPU share with embedded computing.

On the other hand, according to the telematics system according to an aspect of the present invention it is possible to dynamically switch from embedded computing to cloud computing through a dynamic configuration based on SOA (Service Oriented Architecture).

12 is a view for explaining an implementation example of a telematics system according to an aspect of the present invention. According to one aspect of the invention, it is possible to recognize the road situation in real time by searching for the image of the vehicle running in front of the vehicle equipped with a telematics system.

FIG. 13 is a diagram for describing an implementation example of a telematics system according to an aspect of the present disclosure. According to an aspect of the present invention, synchronized playback of content for display devices (e.g., mobile, CID (Center Information Display), RSE (Rear Seat Entertainment)) connected to in-vehicle Ethernet is possible. Specifically, i) down-streaming a high resolution image and re-transmitting it to a multi display in a vehicle, thereby enabling synchronized playback. ii) It is possible to integrate CE devices with telematics system using IoT protocol.

14 is a view for explaining an implementation example of a telematics system according to an aspect of the present invention. According to an aspect of the present invention, i) HD-Positioning based on Visual-inertial Odometry (VIO) and ii) Here LiveSight AR may be provided. VIO can be implemented through monocular camera systems and inertial sensors (e.g. accelerometer, gyroscope). The combination of VIO and smartphone-grade GPS can provide centimeter positioning on the map at a lower cost than RTK GPS systems.

Specifically, for VIO-based HD-Positioning, i) TCU can perform VIO-based HD-Positioning by combining external camera image and inertial sensor GPS. In addition, ii) the TCU transmits the positioning information to the AVN to compare accuracy with general GPS information. To provide LiveSight AR, AVN can compare the mapping accuracy of the AR through HD-Positioning or General Positioning.

15 is a view for explaining the characteristics of the telematics system architecture according to an aspect of the present invention. Specifically, the telematics system according to an aspect of the present invention may provide iii) embedded computing and distributed processing using i) cloud or ii) computing power of a mobile device through a service-oriented communication framework.

First, Telematics AP of telematics system can translate different service-oriented communication methods. Specifically, the services provided by the cloud and the mobile device may be provided in the TCU through IoTivity. In addition, services found in the TCU may be provided via SOME / IP or DDS in the in-vehicle network. Secondly, according to the telematics system, the camera-based application service can be verified based on a comparison through cloud, mobile or embedded computing. Third, the telematics system can be utilized as a protocol of media multicast service through OCF and external interface.

16 to 17 are diagrams for explaining the SW platform of the telematics system according to an aspect of the present invention. SW platform of the telematics system according to an aspect of the present invention can be designed from the SOA perspective, the main features may be as follows.

i) Plug / Play: E / E Architecture is based on manufacturer's requirements, TCU can be integrated or separated, and SW components of telematics system according to the present invention can be used in any ECU regardless of manufacturer's E / E Architecture. It can operate flexibly.

ii) Loosely Coupling: Through the publish / subscribe model from the service point of view, the dependency between ECUs can be reduced through abstraction of communication part. Telematics can be designed to publish all data from outside as a data hub and subscribe to all data coming from inside.

In terms of data management, consideration should be given to scheduling and QoS management of data to be processed first, data queue management, and ensuring real-time data. In terms of data storage, consideration should be given to buffer management of large data, lifetime management of flash storage as data storage, and data log management. On the other hand, from a 5G telematics perspective, SOA based SW design, flexible E / E architecture design with TCU and heterogeneous ECUs (eg PnP support, 10Gbps network, etc.), real-time data flow management and extended flash storage life Should be considered.

18 is a view for explaining a telematics system according to an aspect of the present invention. Referring to FIG. 18, it can be seen that a cloud and an in-vehicle network must be supported to implement a time sensitive network (TSN) through 5G.

TSN is a deterministic network that always targets time sensitive applications. TSNs were developed to provide a way to deliver information from source to destination in i) fixed, ii) predictable time. TSN focuses on control data. TSN features include: i) time synchronization (e.g., less than 1us) and guaranteed end-to-end latency, ii) resource reservation, iii) extraordinarily low packet loss ratios (10 ^-6 to 10 ^-9 ), iv) There may be convergence all data streams.

FIG. 19 illustrates a relationship between a telematics system and a cloud according to an aspect of the present invention. According to the cloud system architecture shown in FIG. 19, the cloud service may be provided like an in-vehicle service based on the SOA framework. In the related art, development of SW embedded in a vehicle has been focused, but according to the present invention, 5G-based cloud computing is provided. That is, according to the present invention, it is possible to process through the cloud a service that is difficult to process in the embedded. Basically, the framework of the cloud must be the same as embedded in the vehicle to enable communication.

Services and applications that can be provided according to the present invention may include a vehicle location determination service, a camera video relay service, an IoT service, and a latest DSM engine. In the database (service DB), all services that the vehicle or the cloud can provide may be registered in advance. API Gateway can provide interoperability between vehicles and devices by means of RESTful APIs.

20 is a view for explaining the relationship between the framework and the cloud of the telematics system according to an aspect of the present invention. As shown in FIG. 20, the Cloud and the Telematics AP may be connected through an Open Connectivity Foundation (OCF) for IoT communication. For this purpose, the telematics of the vehicle must have i) an OCF framework for IoT communication and ii) an SOA protocol for embedded communication (eg SOME / IP or DDS).

Through the structure shown in FIG. 20, interoperability and seamless connectivity between devices can be reconsidered. Through this, i) secure and reliable device discovery, ii) connectivity across multiple Oss and platforms may be provided.

21 is a diagram for describing a media cast center of a telematics system according to an aspect of the present invention. Specifically, efficient sharing of media data via broadband Ethernet will be described.

The basic functions of media processing performed by the Media Cast Center are as follows. i) Relay (e.g. unicast relay or multicast relay), ii) De-multiplex (separate to 2 media stream for video and audio from 1 stream), iii) Media data transport, iv) Media streaming control, v) Interface between services included out-vehicle OCF Interface, SOME / IP Interface, vi) Encoder / Decoder.

Use cases that may be provided through the Media Cast Center may include i) See through and ii) Synchronized playing at multiple CE devices. With See through, it is possible to share in-vehicle media data (e.g., front camera video) to external vehicles or A / V devices connected to the vehicle via the cloud.

On the other hand, through synchronized playback, it is possible to provide a shared screen from multiple CE devices connected to the vehicle. The media source may include i) a media file of cloud or web, ii) a media stream URL (Universal Resource Locator) provided by a device connected to a streaming server and a vehicle.

FIG. 22 illustrates a positioning service provided by a telematics system according to an aspect of the present invention. High precision positioning of the vehicle can be provided through the integration of GPS / GNSS and Visual-inertial Odometry (VIO). Main features of the positioning service provided by the telematics system according to an aspect of the present invention may be as follows.

i) Sensor collection: Sensor data is collected from GPS, accelerometer, gyroscope and camera, and this data should be time synchronized

ii) VIO operation: VIO algorithm estimates the relative position and orientation of a vehicle using a camera and inertial sensors. The camera processing block detects features and tracks them. The inertial data processing block samples the sensors at a very high frequency (100 Hz or Higher)

iii) GPS + VIO fusion: Tightly couple GPS / GNSS measurements and local / relative coordinate measurements from VIO to achieve highly accurate global position

iv) Position handler: The SOA framework-based protocol is used to transfer the global position to the Vehicle Position Manager in the cloud.

Meanwhile, DSDA (Dual SIM Dual Active) means installing two SIMs in one TCU and using two carrier services at the same time. For example, SIM # 1 is used by vehicle manufacturers for OTA, V2X, big data, and maintenance, while SIM # 2 allows users to select a carrier for wi-fi hotspot, web access, and video steaming. From a 5G telematics point of view, DSDA has a low 5G dependency on the technical side, and the interests of carriers and vehicle manufacturers are key to commercializing the technology. To date, no prototype of 5G Modem vendor has been published.

FIG. 23 is a diagram for describing an electronic horizon provided by a telematics system according to an aspect of the present disclosure. The telematics system can provide geometry information in front of the vehicle. This can be called Electronic Horizon and can be provided through the ADASIS Protocol. A telematics system can act as a horizon provider. Details are as follows.

i) Electronic Horizon

Electronic Horizon provides geometry information to other ECUs. (ADAS, Navigation, ...). Electronic Horizon requires vehicle's position (GPS) and a digital map (from cloud service). Horizon's Information is Position, Path, Lane, Road (tunnel, bridge), Speed Limit, Traffic sign, Warning (under construction). The ADASIS forum makes ADASIS protocol to standardize the interface between a digital map and ADAS applications.

ii) ADASIS v2

ADASIS v2 is designed for CAN bus communication restricted to 8 byte messages. ADASIS v2 consists of Horizon Provider, Horizon Reconstructor, ADASIS Protocol and ADAS Application.

iii) ADASIS v3

ADASIS v3 is designed for higher bandwidth communication. ADASIS v3 provides additional data with more detailed content. (Lane level information). ADASIS v3 supports detailed information (e.g., HD-GPS, HD-maps, sensors, and V2X).

24 to 31 are views for explaining an implementation example of a telematics system according to an aspect of the present invention. More specifically, FIGS. 24 to 25 are diagrams for explaining route forecasting, FIGS. 26 to 27 are diagrams for explaining crowd-eye sourcing, and FIGS. 28 to 29 are diagrams for explaining take-out cinema 30 and 31 are diagrams for explaining adaptive guidance.

Referring to FIGS. 24 to 25, route forecasting refers to a 5G navigation service capable of predicting road conditions in real time by searching for images of vehicles in front of the vehicle.

Key values for implementing route forecasting may be real-time video of a desired road, linkage with a route, and video information search. Through route forecasting, real-time video can be shared between vehicles and the shared video can be analyzed. Route forecasting can be used for cloud computing, HD-Positioning, and AR. As a specific implementation, i) it may be possible to predict road conditions by searching the images of the vehicles ahead of the road when the road is blocked, and ii) may be used in a briefing section of a difficult road before the vehicle starts, and iii) the surroundings. It can be used for route guidance using the image of the feature. Referring to FIG. 25, technical elements required for implementing route forecasting may include the aforementioned TSN, DPDK, SOA framework, Advanced Positioning, Cloud server, Media streaming, and AR engine.

Referring to FIGS. 26 to 27, crowd-eye sourcing refers to a service used by various vehicles to secure an image of photographing my vehicle at various angles.

Key values for implementing crowd-eye sourcing include automatic collection of relevant images in specific situations (eg accidents), real-time editing of multi-view / multi-channel accident images, insurance companies / car rental companies, etc. There may be. Crowd-eye sourcing enables real-time video sharing and editing between vehicles. Crowd-eye sourcing can be used for cloud computing and IoT managers. As a specific implementation example, i) a system that obtains a black box image of surrounding vehicles from various angles and immediately combines them to provide a driver and an insurance company in case of an accident, and ii) receives and utilizes images of my vehicle taken by the surrounding vehicles while traveling. It is possible. Referring to FIG. 27, technical elements required to implement crowd-eye sourcing may include the aforementioned TSN, DPDK, Advanced Positioning, Cloud server, and media streaming.

Referring to FIGS. 28 to 29, the take-out cinema refers to a solution capable of viewing ultra-high definition streaming content from outside the vehicle by connecting a 5G infotainment system to a peripheral device.

Key values for implementing take-out cinema can be extended to the role of car HMI to an outdoor media center, utilization of IVI interface on tethered devices, and easy and efficient tethering compared to smartphones. A take-out cinema can be used to build an integrated connectivity system that can utilize peripheral multimedia devices. Take-out cinema can be used for Media Multicast Manager and AR. As a specific example of implementation, i) connecting a media device such as a projector or a speaker to enjoy high-definition content outdoors, ii) using a high-quality and high-definition device in a vehicle to use as a media room, iii) connecting a VR device It is possible to provide the surrounding video viewing service while boarding. Referring to FIG. 29, technical elements required to implement take-out cinema may include the above-described media streaming and cloud server.

Referring to FIGS. 30 to 31, Adaptive Guidance refers to an autonomous driving system that receives a road situation from a front vehicle, sets a driving pattern bonded to an occupant activity, and provides attention notification.

Key values for implementing Adaptive Guidance may be path / speed control based on occupant activity, preliminary notice and warning of precautions, DMS utilization and major activity analysis. Adaptive Guidance enables sharing of real-time road situation information and analysis of occupant activity. Adaptive Guidance can be used for cloud computing, data managers, and E-horizon. As a specific implementation example, i) monitoring the driver's forward direction in manual driving to provide traffic light change notification, ii) in automatic driving, monitoring the objects related to personal work situation inside the vehicle to warn of starting / stopping / lane changing, iii) If the user is sleeping in automatic driving, there may be a route change so as to avoid uneven section or sudden start / stop. Referring to FIG. 31, technical elements for implementing adaptive guidance may include TSN, DPDK, Advanced Positioning, Cloud server, DMS, Road forecasting, and IoTivity.

Media Cast Center and cloud server of the telematics system shown in FIG. 32 are characterized by high throughput. In particular, the Media Cast Center features Relay, Media data transport, Media streaming control, Encoder / Decoder. As a result, there may be i) System for increased bandwidth efficiency, ii) Media service scalability, and iii) High quality of media services. Restrictions due to high throughput may include i) Non consideration about secure channel between content server, and ii) Non consideration about copyright of content.

The TCU and cloud server of the telematics system shown in FIG. 33 are characterized by low latency. In particular, MMI (Multi Media Interface) and 360 Media processing are possible through SOA framework (SOME / IP or IoTivity) of TCU and cloud server. On the other hand, low latency constraints may include i) need a edge computing by MNO, ii) 360 or VR camera performance, iii) 360 or VR player (Software / Required HW), and iv) Plyer Interface.

The telematics system shown in FIG. 34 is characterized by a vehicle to network (V2N) see-through. As described above in FIG. 21, the telematics system according to an aspect of the present invention may provide a see-through function through an SOA framework, a cloud server, and a media cast center.

35 to 36 illustrate a signal transmission and reception sequence between a TCU and a cloud server (or a network server) in a telematics system according to an aspect of the present invention. First, the signal transmission and reception sequence of the network server and the TCU will be described with reference to FIG. 35.

1. 카메라 영상 원격 1. Camera video remote

모니터링monitoring 시 5G 5G telematicstelematics system (이하 5G TS)와 cloud server (이하 서버)간의 연동 방법 Interworking method between system (hereafter 5G TS) and cloud server (hereafter server)

1.1 Upload Camera Video

A. A remote monitoring start event occurs on the server. B. Check and open the port number where the Camera Cast Service of the server receives the video from the Media Cast Service (hereinafter referred to as MCS) of the 5G TS (initialization of the server). C. Message Queuing Telemetry Transport (MQTT) Broker of server requests camera video from Vehicle Event Manager of 5G TS (Request parameters: protocol type (UDP / TCP), port number, camera type (front, 360, cabin)). D. Vehicle Event Manager of 5G TS forwards C request information to MCS. E. MCS prepares to turn on the camera and deliver the image to the server based on the information delivered by the Vehicle Event Manager of 5G TS. F. MCS starts to acquire camera image. G. If necessary, re-packetize the camera image and send it to the server.

1.2 Stop uploading camera video

H. A remote monitoring stop event occurs on the server. I. MQTT Broke on the server asks the 5G TS Vehicle Event Manager to stop sending the camera. J. The Vehicle Event Manager of the 5G TS sends an ICS stop request to the MCS. K. MCS stops transmitting J's camera video. L. MCS turns off the camera.

2. 미디어 동기 재생 시 Media Sharing Service 활용 방법2. How to Use Media Sharing Service for Media Sync Play

A. Pass the media streaming URL (e.g., server URL, in-vehicle streaming URL) to AVG or RSE to the Media Sharing Service (MSS) of 5G TS for synchronous playback. B. MSS validates A's media streaming URL. C. MSS constructs a streaming pipeline that is associated with A's media streaming URL. D. MSS generates a URL to connect to C's pipeline (RTSP). E. Pass the URL to the target device. F. Play the media streaming on the target device's URL. G. For synchronous playback on the target device, A / V is output in synchronous form by using the reference time information (in RTCP) transmitted periodically and the time stamp of the RTP packet header.

Meanwhile, in the C, a real time transport protocol (RTP) / UDP (user datagram protocol) may be used as a transport protocol, a real time streaming protocol (RTSP) may be used as a streaming control protocol, and a QoS protocol may be used. Real time transport control protocol (RTCP) may be used. On the other hand, the definition and role of each component shown in Figure 35 may be as shown in Table 1 below.

36 is a flowchart illustrating an interworking scenario for using a media cast center between a telematics system and a cloud server according to an aspect of the present invention.

When the media information request is input to the vehicle, the 5G TS of the vehicle prepares to transmit the request to the cloud server. The vehicle controls the frequency output through an antenna unit connected by a 5G TS and a coaxial cable. The vehicle transmits signals to the cloud server through the gNB connected by Ethernet cable. The vehicle receives the content of the media cast center of the cloud server from the cloud server. The vehicle transmits the content received through the antenna to the 5G TS through the cable. Information is transmitted to AVN connected by 5G TS and Ethernet cable.

Embodiments of the present invention described above may be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

For implementation in hardware, a method according to embodiments of the present invention may include one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), and Programmable Logic Devices (PLDs). It may be implemented by field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, a procedure, or a function that performs the functions or operations described above. The software code may be stored in a memory unit and driven by a processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various known means.

The detailed description of the preferred embodiments of the invention disclosed as described above is provided to enable any person skilled in the art to make and practice the invention. Although the above has been described with reference to the preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention without departing from the spirit and scope of the invention as set forth in the claims below. I can understand that you can. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, while the preferred embodiments of the present specification have been shown and described, the present specification is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present specification claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present specification.

In the present specification, both the object invention and the method invention are described, and the description of both inventions may be supplementarily applied as necessary.

Various aspects for carrying out the invention have been described in the best mode for the practice of the invention.

The description is not to be construed as limiting in all respects, but should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

The present invention described above can be embodied as computer readable codes on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. This also includes implementations in the form of carrier waves (eg, transmission over the Internet). In addition, the computer may include the controller 180 of the terminal. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

Claims

In the method for controlling a telematics system provided in a vehicle,

Receiving a request for outputting content; Requesting information related to the content from an external server through a base station connected to the telematics system; And receiving the information from the external server via the base station,

And the received information is transmitted to a plurality of target devices respectively connected to the telematics system.
The method of claim 1,

Wherein said information is received from said external server via a predetermined protocol, said predetermined protocol comprising a transmission protocol, a streaming control protocol, and a quality of service protocol.
The method of claim 1,

If the received information is media content provided by the external server, the telematics system generates a URL (Universal Resource Locator) to which each of the plurality of target devices can access, and converts the generated URLs into the plurality of targets. A method for controlling a telematics system provided in a vehicle, each delivered to a device.
The method of claim 2,

The transmitted information is synchronized to each of the plurality of target devices based on reference time information included in the QoS protocol and time information included in a packet constituting information received from the external server. How to control your telematics system.
The method of claim 1,

The plurality of target devices include a center information display (CID), audio video navigation (AVN), rear seat entertainment (RSE), and consumer electronics (CE) device provided inside or outside the vehicle. , A method for controlling a telematics system provided in a vehicle.
The method of claim 1,

The content received from the external server corresponds to the media content or the camera image of another vehicle, the method of controlling a telematics system provided in the vehicle.
In the telematics system provided in a vehicle,

RF (Radio Frequency) unit; And

Receiving a request for content output, controlling the RF unit to request information related to the content to an external server through a base station connected to the telematics system, and controlling the RF unit to control the RF unit from the external server through the base station; A processor for receiving information,

And the received information is respectively delivered to a plurality of target devices connected to the telematics system.
The method of claim 7, wherein

Wherein the information is received from the external server via a predetermined protocol, wherein the predetermined protocol comprises a transport protocol, a streaming control protocol, and a quality of service (QoS) protocol.
The method of claim 7, wherein

When the received information is media content provided by the external server, the processor generates a universal resource locator (URL) to which the plurality of target devices can connect, and converts the generated URLs into the plurality of target devices. Each telematics system provided to the vehicle, to transmit to.
The method of claim 8,

The transmitted information is synchronized to each of the plurality of target devices based on reference time information included in the QoS protocol and time information included in a packet constituting information received from the external server. Telematics system.
The method of claim 7, wherein

The plurality of target devices include a center information display (CID), audio video navigation (AVN), rear seat entertainment (RSE), and consumer electronics (CE) device provided inside or outside the vehicle. , Telematics system provided in the vehicle.
The method of claim 7, wherein

And the content received from the external server corresponds to media content or a camera image of another vehicle.