WO2020149433A1 - Image output device and method for controlling same - Google Patents

Abstract

The present invention relates to an image output device which is mounted in a vehicle and implements augmented reality, and a method for controlling same. The image output device comprises: an image output unit for outputting visual information for implementing the augmented reality; a communication unit for receiving a front side image obtained by photographing a front side of the vehicle; and a processor for searching for one or more vehicle lanes, on which the vehicle is scheduled to run, in the front side image, and controlling the image output unit to output a main carpet image for guiding the discovered one or more lanes in lane units, wherein the processor searches for an object corresponding to a preconfigured condition, and controls the image output unit to output a sub carpet image for guiding a route to a destination corresponding to the object in response to a preconfigured user input, together with the main carpet image.

Description

Video output device and control method thereof

The present invention relates to an image output device mounted on a vehicle to implement augmented reality and a control method thereof.

Vehicle means a means of transportation that can move people or luggage using kinetic energy. Examples of vehicles are automobiles and motorcycles.

For the safety and convenience of users of the vehicle, various sensors and devices are provided in the vehicle, and the functions of the vehicle are diversified.

The function of the vehicle may be divided into a convenience function for promoting the convenience of the driver, and a safety function for promoting the safety of the driver and/or pedestrian.

First, the convenience function has a development motivation related to driver convenience, such as giving the vehicle an infotainment (information + entertainment) function, supporting a partial autonomous driving function, or helping to secure a driver's view such as a night vision or a blind spot. . For example, active cruise control (ACC), smart parking assist system (SPAS), night vision (NV), head up display (HUD), around view monitor (around view monitor, AVM), and adaptive headlight system (AHS).

The safety function is a technology that ensures driver safety and/or pedestrian safety, lane departure warning system (LDWS), lane keeping assist system (LKAS), and autonomous emergency braking, AEB) functions.

In order to further improve the above-mentioned convenience functions and safety functions, communication technologies specialized for vehicles have been developed. For example, V2I (Vehicle to Infrastructure) that enables communication between the vehicle and the infrastructure, V2V (Vehicle to Vehicle) that enables communication between the vehicle and the vehicle, and V2X (Vehicle to Everything) that enables communication between the vehicle and the vehicle. There is this.

The vehicle is equipped with an image output device for visually providing various information to passengers. The image output device includes a head shield display (HUD) for outputting information through a windshield of a vehicle or a transparent screen provided separately and/or various displays for outputting information through a panel.

The video output device provides information on route guidance to a destination and information on a point of interest (POI), and is developing in a direction to effectively provide various information. In particular, research into an image output device that directly and effectively provides necessary information within a range that does not interfere with driving to a driver who needs to focus on driving continues.

The present invention aims to solve the above and other problems.

One object of the present invention is to provide a video output device capable of providing various information to a passenger in augmented reality and a control method thereof.

One object of the present invention is to provide an image output device and a control method thereof that enable a passenger to select a driving direction of a vehicle using augmented reality.

The present invention relates to an image output device provided or mounted on a vehicle and a control method thereof.

The image output device includes an image output unit that outputs visual information for realizing the augmented reality; A communication unit that receives a front image photographing the front of the vehicle; And a processor that searches for one or more lanes scheduled for driving of the vehicle from the front image, and controls the image output unit to output a main carpet image for guiding the searched one or more lanes in lane units. , The processor searches for an object corresponding to a preset condition, and outputs the image so that a sub carpet image guiding a path to a destination corresponding to the object is output together with the main carpet image in response to a preset user input. Control wealth.

According to an embodiment of the present disclosure, when there are a plurality of objects corresponding to the preset condition, the processor may control the image output unit such that a plurality of sub carpet images are output together with the main carpet image.

According to an embodiment of the present disclosure, the processor highlights any one of the sub-carpet images, and any one of the sub-carpet images subjected to highlight processing may be changed according to a user input.

According to an embodiment of the present disclosure, when the one sub-carpet image is selected by the user, the processor controls the communication unit to change the driving direction of the vehicle to a path set in the one sub-carpet image. can do.

According to an embodiment of the present disclosure, when the one of the sub-carpet images is selected by the user, the processor may control the image output unit so that the main carpet image disappears.

According to an embodiment, the processor sets a priority for the plurality of objects, controls the video output unit so that each sub-carpet image has a color corresponding to the priority, and the priority is changed If possible, the color of at least one sub-carpet image may be changed.

According to an embodiment, when the number of objects corresponding to the preset condition is greater than the reference number, the processor filters the reference number based on a preset criterion, and the sub carpet images of the reference number are the main carpet image The image output unit may be controlled to be output together.

According to an embodiment of the present disclosure, the processor may filter objects of the reference number based on the gaze of the passengers in the vehicle.

According to an embodiment of the present disclosure, the processor searches for another vehicle located in a predetermined range in front of the vehicle as the object, and at least one of the size, shape, and location of the sub carpet image according to the location of the other vehicle The image output unit may be controlled to be variable.

According to an embodiment of the present disclosure, the processor may calculate the fuel efficiency when the vehicle follows the other vehicle, and control the image output unit to output the fuel efficiency together with the main carpet image and the sub carpet image. .

According to an embodiment of the present disclosure, the processor may control the image output unit so that the sub carpet images have different colors according to the fuel efficiency.

According to an embodiment of the present disclosure, when the object is located outside a predetermined range based on the vehicle while outputting the sub carpet image, the processor may control the image output unit to disappear the sub carpet image. .

According to an embodiment, the processor may collect traffic congestion for a route to a destination corresponding to the object, and control the video output unit such that the sub-carpet images have different colors according to the traffic congestion. have.

According to an embodiment, the processor may control the image output unit to output an object image guiding the object in response to the object being searched.

According to an embodiment, the processor may collect object information corresponding to the object and control the image output unit to output the object information.

According to one embodiment, the main carpet image is a transparent image of a first color, the sub carpet image is a transparent image of a second color, and the processor is configured to display a portion where the main carpet image and the sub carpet image overlap. The image output unit may be controlled to search and have a third color in which the first color and the second color are mixed.

According to the present invention, a control method of the above-described image output device is provided. The control method includes receiving a front image photographing the front of the vehicle; Searching for one or more lanes scheduled for driving of the vehicle from the front image; And outputting a main carpet image for guiding the searched one or more lanes in lane units through an image output unit, and searching for an object corresponding to a preset condition; And outputting a sub carpet image for guiding a path to a destination corresponding to the object along with the main carpet image in response to a preset user input.

According to an embodiment, when there are a plurality of objects corresponding to the preset condition, a plurality of sub-carpet images may be output together with the main carpet image.

According to an embodiment, when a subcarpet image of one of the plurality of subcarpet images is selected by a user, changing a driving direction of the vehicle to a path set in the one of the subcarpet images It may further include.

According to one embodiment, when the object is located outside a predetermined range based on the vehicle while outputting the sub-carpet image, the method may further include controlling the image output unit to disappear the sub-carpet image. have.

When explaining the effects of the video output device and its control method according to the present invention are as follows.

Through the main carpet image, the passenger may be provided with route information in which the vehicle is to drive autonomously or to be driven by the driver in lane units.

1 is a view showing the appearance of a vehicle according to an embodiment of the present invention

2 is a view of a vehicle according to an embodiment of the present invention as viewed from various angles outside

3 to 4 are views showing the interior of a vehicle according to an embodiment of the present invention

5 to 6 are diagrams referred to for describing an object according to an embodiment of the present invention

7 is a block diagram referred to for describing a vehicle according to an embodiment of the present invention

8 is a conceptual diagram illustrating an image output device according to an embodiment of the present invention

9 is a flowchart for explaining a control method of a video output device

10A and 10B are conceptual views illustrating the control method of FIG. 9.

11 is a flowchart illustrating a control method of a video output device outputting a sub carpet image

12, 13 and 14 are exemplary views for explaining the control method of FIG.

15 is a flowchart for explaining the operation of the video output device based on user input.

16 is an exemplary view for explaining an embodiment in which the color of a sub carpet image is changed according to priority.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are assigned the same reference numbers regardless of the reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "modules" and "parts" for components used in the following description are given or mixed only considering the ease of writing the specification, and do not have meanings or roles distinguished from each other in themselves. In addition, in the description of the embodiments disclosed herein, when it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed herein, detailed descriptions thereof are omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed in the specification is not limited by the accompanying drawings, and all modifications included in the spirit and technical scope of the present invention , It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

When an element is said to be "connected" or "connected" to another component, it is understood that other components may be directly connected to or connected to the other component, but there may be other components in between. It should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, terms such as “comprises” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and that one or more other features are present. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

The vehicle described herein may be a concept including an automobile and a motorcycle. Hereinafter, a vehicle is mainly described for a vehicle.

The vehicle described in this specification may be a concept including both an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.

In the following description, the left side of the vehicle means the left side of the driving direction of the vehicle, and the right side of the vehicle means the right side of the driving direction of the vehicle.

1 is a view showing the appearance of a vehicle according to an embodiment of the present invention.

2 is a view of a vehicle according to an embodiment of the present invention viewed from various angles outside.

3 to 4 are views showing the interior of a vehicle according to an embodiment of the present invention.

5 to 6 are views referred to for describing an object according to an embodiment of the present invention.

7 is a block diagram referred to for describing a vehicle according to an embodiment of the present invention.

1 to 7, the vehicle 100 may include a wheel rotated by a power source and a steering input device 510 for adjusting the traveling direction of the vehicle 100.

The vehicle 100 may be an autonomous vehicle.

Here, autonomous driving is defined as controlling at least one of acceleration, deceleration, and driving directions based on a preset algorithm. In other words, even if a user input is not input to the driving manipulation apparatus, it means that the driving manipulation apparatus is automatically operated.

The vehicle 100 may be switched to an autonomous driving mode or a manual mode based on a user input.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or the autonomous driving mode to the manual mode based on the received user input through the user interface device 200.

The vehicle 100 may be switched to an autonomous driving mode or a manual mode based on driving situation information. The driving situation information may be generated based on object information provided by the object detection device 300.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the driving situation information generated by the object detection device 300.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode, or may be switched from the autonomous driving mode to the manual mode based on the driving situation information received through the communication device 400.

The vehicle 100 may be switched from a manual mode to an autonomous driving mode based on information, data, and signals provided from an external device, or may be switched from an autonomous driving mode to a manual mode.

When the vehicle 100 is operated in an autonomous driving mode, the autonomous driving vehicle 100 may be operated based on the driving system 700.

For example, the autonomous vehicle 100 may be driven based on information, data, or signals generated from the driving system 710, the exit system 740, and the parking system 750.

When the vehicle 100 is driven in the manual mode, the autonomous vehicle 100 may receive a user input for driving through the driving manipulation apparatus 500. The vehicle 100 may be driven based on a user input received through the driving manipulation apparatus 500.

Overall length is the length from the front to the rear of the vehicle 100, width is the width of the vehicle 100, and height is the length from the bottom of the wheel to the roof. In the following description, the full-length direction L is a direction that is a reference for measuring the full-length of the vehicle 100, the full-width direction W is a direction that is a reference for the full-width measurement of the vehicle 100, and the full-height direction H is the vehicle It may mean a direction that is a reference for measuring the height of the (100).

As illustrated in FIG. 7, the vehicle 100 includes a user interface device 200, an object detection device 300, a communication device 400, a driving operation device 500, a vehicle driving device 600, and a driving system 700, a navigation system 770, a sensing unit 120, an interface unit 130, a memory 140, a control unit 170, and a power supply unit 190.

Depending on the embodiment, the vehicle 100 may further include other components in addition to the components described herein, or may not include some of the components described.

The user interface device 200 is a device for communication between the vehicle 100 and a user. The user interface device 200 may receive user input and provide information generated in the vehicle 100 to the user. The vehicle 100 may implement User Interfaces (UI) or User Experience (UX) through the user interface device 200.

The user interface device 200 may include an input unit 210, an internal camera 220, a biometric sensing unit 230, an output unit 250, and a processor 270.

According to an embodiment, the user interface device 200 may further include other components in addition to the components described, or may not include some of the components described.

The input unit 200 is for receiving information from a user, and data collected by the input unit 200 may be analyzed by the processor 270 and processed by a user's control command.

The input unit 200 may be disposed inside the vehicle. For example, the input unit 200 includes a region of a steering wheel, a region of an instrument panel, a region of a seat, a region of each pillar, and a door One area of the door, one area of the center console, one area of the head lining, one area of the sun visor, one area of the windshield or one of the windows It can be arranged in one area or the like.

The input unit 200 may include a voice input unit 211, a gesture input unit 212, a touch input unit 213, and a mechanical input unit 214.

The voice input unit 211 may convert a user's voice input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the control unit 170.

The voice input unit 211 may include one or more microphones.

The gesture input unit 212 may convert a user's gesture input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the control unit 170.

The gesture input unit 212 may include at least one of an infrared sensor and an image sensor for sensing a user's gesture input.

According to an embodiment, the gesture input unit 212 may detect a user's 3D gesture input. To this end, the gesture input unit 212 may include a light output unit outputting a plurality of infrared light or a plurality of image sensors.

The gesture input unit 212 may detect a user's 3D gesture input through a time of flight (TOF) method, a structured light method, or a disparity method.

The touch input unit 213 may convert a user's touch input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170.

The touch input unit 213 may include a touch sensor for detecting a user's touch input.

According to an embodiment, the touch input unit 213 may be integrally formed with the display unit 251 to implement a touch screen. The touch screen may provide an input interface and an output interface between the vehicle 100 and a user.

The mechanical input unit 214 may include at least one of a button, a dome switch, a jog wheel, and a jog switch. The electrical signal generated by the mechanical input unit 214 may be provided to the processor 270 or the control unit 170.

The mechanical input unit 214 may be disposed on a steering wheel, center fascia, center console, cock module, door, or the like.

The internal camera 220 may acquire an image inside the vehicle. The processor 270 may detect a user's state based on an image inside the vehicle. The processor 270 may acquire the user's gaze information from the image inside the vehicle. The processor 270 may detect a gesture of the user from the image inside the vehicle.

The biometric sensing unit 230 may acquire biometric information of the user. The biometric sensing unit 230 includes a sensor capable of acquiring the user's biometric information, and may acquire the user's fingerprint information, heartbeat information, and the like using the sensor. Biometric information may be used for user authentication.

The output unit 250 is for generating output related to vision, hearing, or tactile sense.

The output unit 250 may include at least one of a display unit 251, an audio output unit 252, and a haptic output unit 253.

The display unit 251 may display graphic objects corresponding to various information.

The display unit 251 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible) display), a three-dimensional display (3D display), an electronic ink display (e-ink display).

The display unit 251 forms a mutual layer structure with the touch input unit 213 or is integrally formed, thereby realizing a touch screen.

The display unit 251 may be implemented as a head up display (HUD). When the display unit 251 is implemented as a HUD, the display unit 251 may include a projection module to output information through a wind shield or an image projected on the window.

The display unit 251 may include a transparent display. The transparent display can be attached to a wind shield or window.

The transparent display can display a predetermined screen while having a predetermined transparency. Transparent display, to have transparency, the transparent display is transparent Thin Film Elecroluminescent (TFEL), transparent OLED (Organic Light-Emitting Diode), transparent LCD (Liquid Crystal Display), transmissive transparent display, transparent LED (Light Emitting Diode) display It may include at least one of. The transparency of the transparent display can be adjusted.

Meanwhile, the user interface device 200 may include a plurality of display units 251a to 251g.

The display unit 251 includes one region of the steering wheel, one region 521a, 251b, and 251e of the instrument panel, one region 251d of the seat, one region 251f of each filler, and one region of the door ( 251g), one area of the center console, one area of the headlining, one area of the sun visor, or one area 251c of the wind shield or one area 251h of the window.

The sound output unit 252 converts and outputs an electrical signal provided from the processor 270 or the control unit 170 into an audio signal. To this end, the sound output unit 252 may include one or more speakers.

The haptic output unit 253 generates a tactile output. For example, the haptic output unit 253 may operate by vibrating the steering wheel, seat belt, and seats 110FL, 110FR, 110RL, and 110RR, so that the user can recognize the output.

The processor 270 may control the overall operation of each unit of the user interface device 200.

According to an embodiment, the user interface device 200 may include a plurality of processors 270 or may not include a processor 270.

When the processor 270 is not included in the user interface device 200, the user interface device 200 may be operated under the control of the processor or control unit 170 of another device in the vehicle 100.

Meanwhile, the user interface device 200 may be referred to as a vehicle display device.

The user interface device 200 may be operated under the control of the control unit 170.

The object detection device 300 is a device for detecting an object located outside the vehicle 100.

The object may be various objects related to the operation of the vehicle 100.

5 to 6, the object (O) is a lane (OB10), another vehicle (OB11), pedestrian (OB12), two-wheeled vehicle (OB13), traffic signals (OB14, OB15), light, road, structure, It may include a speed bump, terrain, and animals.

The lane OB10 may be a driving lane, a side lane next to the driving lane, or a lane through which an opposed vehicle travels. The lane OB10 may be a concept including left and right lines forming a lane.

The other vehicle OB11 may be a vehicle driving around the vehicle 100. The other vehicle may be a vehicle located within a predetermined distance from the vehicle 100. For example, the other vehicle OB11 may be a vehicle preceding or following the vehicle 100.

The pedestrian OB12 may be a person located around the vehicle 100. The pedestrian OB12 may be a person located within a predetermined distance from the vehicle 100. For example, the pedestrian OB12 may be a person located on a sidewalk or a road.

The two-wheeled vehicle OB12 may mean a vehicle that is located around the vehicle 100 and moves using two wheels. The two-wheeled vehicle OB12 may be a vehicle having two wheels positioned within a predetermined distance from the vehicle 100. For example, the two-wheeled vehicle OB13 may be a motorcycle or a bicycle located on a sidewalk or a driveway.

The traffic signal may include a traffic light OB15, a traffic sign OB14, a pattern or text drawn on the road surface.

The light may be light generated from a lamp provided in another vehicle. Light can be light generated from street lights. The light can be sunlight.

Roads may include road surfaces, curves, slopes such as uphills, downhills, and the like.

The structure may be an object located around the road and fixed to the ground. For example, the structure may include street lights, street trees, buildings, power poles, traffic lights, and bridges.

Terrain can include mountains, hills, and the like.

Meanwhile, the object may be classified into a moving object and a fixed object. For example, the moving object may be a concept including other vehicles and pedestrians. For example, the fixed object may be a concept including traffic signals, roads, and structures.

The object detection device 300 may include a camera 310, a radar 320, a lidar 330, an ultrasonic sensor 340, an infrared sensor 350, and a processor 370.

According to an embodiment, the object detection device 300 may further include other components in addition to the components described, or may not include some of the components described.

The camera 310 may be located at an appropriate location outside the vehicle in order to acquire an image outside the vehicle. The camera 310 may be a mono camera, a stereo camera 310a, an AVM (Around View Monitoring) camera 310b, or a 360 degree camera.

For example, the camera 310 may be disposed close to the front windshield, in the interior of the vehicle, to obtain an image in front of the vehicle. Alternatively, the camera 310 may be disposed around the front bumper or radiator grille.

For example, the camera 310 may be disposed close to the rear glass, in the interior of the vehicle, in order to acquire an image behind the vehicle. Alternatively, the camera 310 may be disposed around the rear bumper, trunk, or tail gate.

For example, the camera 310 may be disposed close to at least one of the side windows in the interior of the vehicle in order to acquire an image on the side of the vehicle. Alternatively, the camera 310 may be disposed around a side mirror, fender, or door.

The camera 310 may provide the obtained image to the processor 370.

The radar 320 may include an electromagnetic wave transmitting unit and a receiving unit. The radar 320 may be implemented in a pulse radar method or a continuous wave radar method in accordance with the principle of radio wave launch. The radar 320 may be implemented by a FMCW (Frequency Modulated Continuous Wave) method or a FSK (Frequency Shift Keyong) method according to a signal waveform among continuous wave radar methods.

The radar 320 detects an object based on a time-of-flight (TOF) method or a phase-shift method via electromagnetic waves, the position of the detected object, the distance from the detected object, and a relative speed Can be detected.

The radar 320 may be disposed at an appropriate location outside the vehicle to detect objects located in the front, rear, or side of the vehicle.

The lidar 330 may include a laser transmitter and a receiver. The lidar 330 may be implemented by a time of flight (TOF) method or a phase-shift method.

The lidar 330 may be implemented in a driving type or a non-driving type.

When implemented in a driving type, the lidar 330 is rotated by a motor and can detect objects around the vehicle 100.

When implemented in a non-driven manner, the rider 330 may detect an object located within a predetermined range with respect to the vehicle 100 by optical steering. The vehicle 100 may include a plurality of non-driven lidars 330.

The lidar 330 detects an object based on a time of flight (TOF) method or a phase-shift method using laser light, and the position of the detected object, the distance to the detected object, and Relative speed can be detected.

The lidar 330 may be disposed at an appropriate location outside the vehicle in order to detect objects located in the front, rear, or side of the vehicle.

The ultrasonic sensor 340 may include an ultrasonic transmitter and a receiver. The ultrasonic sensor 340 may detect an object based on ultrasonic waves and detect a position of the detected object, a distance from the detected object, and a relative speed.

The ultrasonic sensor 340 may be disposed at an appropriate location outside the vehicle in order to sense an object located in front, rear, or side of the vehicle.

The infrared sensor 350 may include an infrared transmitter and a receiver. The infrared sensor 340 may detect an object based on infrared light, and detect a position of the detected object, a distance from the detected object, and a relative speed.

The infrared sensor 350 may be disposed at an appropriate location outside the vehicle to sense an object located in front, rear, or side of the vehicle.

The processor 370 may control the overall operation of each unit of the object detection device 300.

The processor 370 may detect and track an object based on the acquired image. The processor 370 may perform operations such as calculating the distance to the object and calculating the relative speed with the object through an image processing algorithm.

The processor 370 may detect and track the object based on the reflected electromagnetic wave from which the transmitted electromagnetic wave is reflected and returned. The processor 370 may perform operations such as calculating a distance from the object and calculating a relative speed with the object based on electromagnetic waves.

The processor 370 may detect and track an object based on reflected laser light from which the transmitted laser is reflected and returned. The processor 370 may perform operations such as calculating the distance to the object and calculating the relative speed with the object, based on the laser light.

The processor 370 may detect and track the object based on the reflected ultrasonic waves from which the transmitted ultrasonic waves are reflected and returned. The processor 370 may perform operations such as calculating the distance to the object and calculating the relative speed with the object based on ultrasound.

The processor 370 may detect and track the object based on the reflected infrared light from which the transmitted infrared light is reflected and returned. The processor 370 may perform operations such as calculating the distance to the object and calculating the relative speed with the object, based on the infrared light.

According to an embodiment, the object detection apparatus 300 may include a plurality of processors 370 or may not include a processor 370. For example, each of the camera 310, the radar 320, the lidar 330, the ultrasonic sensor 340, and the infrared sensor 350 may individually include a processor.

When the processor 370 is not included in the object detection device 300, the object detection device 300 may be operated under the control of the processor or control unit 170 of the device in the vehicle 100.

The object detection device 300 may be operated under the control of the control unit 170.

The communication device 400 is a device for performing communication with an external device. Here, the external device may be another vehicle, a mobile terminal, or a server. The communication device 400 may be referred to as a “wireless communication unit”.

The communication device 400 may include at least one of a transmitting antenna, a receiving antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

The communication device 400 may include a short-range communication unit 410, a location information unit 420, a V2X communication unit 430, an optical communication unit 440, a broadcast transmission/reception unit 450, and a processor 470.

According to an embodiment, the communication device 400 may further include other components in addition to the components described, or may not include some of the components described.

The short-range communication unit 410 is a unit for short-range communication. The short-range communication unit 410 includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), and Wireless Wi-Fi -Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology can be used to support short-range communication.

The short-range communication unit 410 may form short-range wireless communication networks (Wireless Area Networks) to perform short-range communication between the vehicle 100 and at least one external device.

The location information unit 420 is a unit for obtaining location information of the vehicle 100. For example, the location information unit 420 may include a global positioning system (GPS) module or a differential global positioning system (DGPS) module.

The V2X communication unit 430 is a unit for performing wireless communication with a server (V2I: Vehicle to Infra), another vehicle (V2V: Vehicle to Vehicle), or a pedestrian (V2P: Vehicle to Pedestrian). The V2X communication unit 430 may include an RF circuit capable of implementing communication with infrastructure (V2I), communication between vehicles (V2V), and communication with pedestrians (V2P).

The optical communication unit 440 is a unit for performing communication with an external device via light. The optical communication unit 440 may include an optical transmitter that converts an electrical signal into an optical signal and transmits it to the outside, and an optical receiver that converts the received optical signal into an electrical signal.

According to an embodiment, the light emitting unit may be formed integrally with a lamp included in the vehicle 100.

The broadcast transmission/reception unit 450 is a unit for receiving a broadcast signal from an external broadcast management server through a broadcast channel or transmitting a broadcast signal to the broadcast management server. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal.

The processor 470 may control the overall operation of each unit of the communication device 400.

According to an embodiment, the communication device 400 may include a plurality of processors 470 or may not include a processor 470.

When the processor 470 is not included in the communication device 400, the communication device 400 may be operated under the control of the processor or control unit 170 of another device in the vehicle 100.

Meanwhile, the communication device 400 may implement a vehicle display device together with the user interface device 200. In this case, the vehicle display device may be referred to as a telematics device or an audio video navigation (AVN) device.

The communication device 400 may be operated under the control of the control unit 170.

The driving operation device 500 is a device that receives a user input for driving.

In the manual mode, the vehicle 100 may be driven based on a signal provided by the driving manipulation device 500.

The driving manipulation device 500 may include a steering input device 510, an acceleration input device 530, and a brake input device 570.

The steering input device 510 may receive an input of a traveling direction of the vehicle 100 from a user. The steering input device 510 is preferably formed in a wheel shape to enable steering input by rotation. According to an embodiment, the steering input device may be formed in the form of a touch screen, a touch pad, or a button.

The acceleration input device 530 may receive an input for acceleration of the vehicle 100 from a user. The brake input device 570 may receive an input for deceleration of the vehicle 100 from a user. The acceleration input device 530 and the brake input device 570 are preferably formed in the form of a pedal. According to an embodiment, the acceleration input device or the brake input device may be formed in the form of a touch screen, a touch pad or a button.

The driving operation apparatus 500 may be operated under the control of the control unit 170.

The vehicle driving device 600 is a device that electrically controls driving of various devices in the vehicle 100.

The vehicle driving device 600 includes a power train driving part 610, a chassis driving part 620, a door/window driving part 630, a safety device driving part 640, a lamp driving part 650 and an air conditioning driving part 660. Can.

According to an embodiment, the vehicle driving apparatus 600 may further include other components in addition to the components described, or may not include some of the components described.

Meanwhile, the vehicle driving apparatus 600 may include a processor. Each unit of the vehicle driving apparatus 600 may individually include a processor.

The power train driver 610 may control the operation of the power train device.

The power train driving unit 610 may include a power source driving unit 611 and a transmission driving unit 612.

The power source driving unit 611 may control the power source of the vehicle 100.

For example, when the fossil fuel-based engine is a power source, the power source driving unit 610 may perform electronic control of the engine. Thus, the output torque of the engine and the like can be controlled. The power source driving unit 611 can adjust the engine output torque under the control of the control unit 170.

For example, when the electric energy-based motor is a power source, the power source driving unit 610 may perform control for the motor. The power source driving unit 610 may adjust the rotational speed, torque, and the like of the motor under the control of the control unit 170.

The transmission driver 612 may perform control of the transmission.

The transmission drive unit 612 can adjust the state of the transmission. The transmission drive unit 612 can adjust the state of the transmission to forward (D), reverse (R), neutral (N), or parking (P).

On the other hand, when the engine is a power source, the transmission drive unit 612 can adjust the gear engagement state in the forward (D) state.

The chassis driver 620 may control the operation of the chassis device.

The chassis driving unit 620 may include a steering driving unit 621, a brake driving unit 622, and a suspension driving unit 623.

The steering driving unit 621 may perform electronic control of a steering apparatus in the vehicle 100. The steering driving unit 621 may change the traveling direction of the vehicle.

The brake driving unit 622 may perform electronic control of a brake apparatus in the vehicle 100. For example, by controlling the operation of the brake disposed on the wheel, the speed of the vehicle 100 can be reduced.

Meanwhile, the brake driving unit 622 can individually control each of the plurality of brakes. The brake driving unit 622 may control braking forces applied to the plurality of wheels differently.

The suspension driver 623 may perform electronic control of a suspension apparatus in the vehicle 100. For example, the suspension driving unit 623 may control the suspension device to control vibration of the vehicle 100 when the road surface is curved, by controlling the suspension device.

Meanwhile, the suspension driving unit 623 may individually control each of the plurality of suspensions.

The door/window driver 630 may perform electronic control of a door apparatus or a window apparatus in the vehicle 100.

The door/window driving unit 630 may include a door driving unit 631 and a window driving unit 632.

The door driver 631 may perform control of the door device. The door driver 631 may control opening and closing of a plurality of doors included in the vehicle 100. The door driver 631 may control opening or closing of a trunk or tail gate. The door driving unit 631 may control opening or closing of a sunroof.

The window driver 632 may perform electronic control of a window apparatus. The opening or closing of a plurality of windows included in the vehicle 100 may be controlled.

The safety device driver 640 may perform electronic control of various safety devices in the vehicle 100.

The safety device driving unit 640 may include an airbag driving unit 641, a seat belt driving unit 642, and a pedestrian protection device driving unit 643.

The airbag driving unit 641 may perform electronic control of an airbag apparatus in the vehicle 100. For example, the airbag driving unit 641 may control the airbag to be deployed when a danger is detected.

The seatbelt driving unit 642 may perform electronic control of a seatbelt appartus in the vehicle 100. For example, the seat belt driving unit 642 may control the passenger to be fixed to the seats 110FL, 110FR, 110RL, and 110RR using the seat belt when the danger is detected.

The pedestrian protection device driver 643 may perform electronic control of the hood lift and the pedestrian airbag. For example, the pedestrian protection device driving unit 643 may control the hood lift-up and the pedestrian airbag deployment when a collision with the pedestrian is detected.

The lamp driving unit 650 may perform electronic control of various lamp apparatuses in the vehicle 100.

The air conditioning driving unit 660 may perform electronic control of an air cinditioner in the vehicle 100. For example, when the temperature inside the vehicle is high, the air conditioning driving unit 660 may control the air conditioning device to operate so that cold air is supplied into the vehicle.

The vehicle driving apparatus 600 may include a processor. Each unit of the vehicle driving apparatus 600 may individually include a processor.

The vehicle driving apparatus 600 may be operated under the control of the control unit 170.

The operation system 700 is a system that controls various operations of the vehicle 100. The driving system 700 may be operated in an autonomous driving mode.

The driving system 700 may include a driving system 710, an exit system 740 and a parking system 750.

Depending on the embodiment, the driving system 700 may further include other components in addition to the components described, or may not include some of the components described.

Meanwhile, the driving system 700 may include a processor. Each unit of the driving system 700 may individually include a processor.

Meanwhile, according to an embodiment, when the driving system 700 is implemented in software, it may be a sub-concept of the control unit 170.

Meanwhile, according to an embodiment, the driving system 700 may include at least one of the user interface device 200, the object detection device 300, the communication device 400, the vehicle driving device 600, and the control unit 170. It can be an inclusive concept.

The driving system 710 may perform driving of the vehicle 100.

The driving system 710 may receive navigation information from the navigation system 770 and provide a control signal to the vehicle driving apparatus 600 to perform driving of the vehicle 100.

The driving system 710 may receive object information from the object detection device 300 and provide a control signal to the vehicle driving device 600 to perform driving of the vehicle 100.

The driving system 710 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving apparatus 600, and perform driving of the vehicle 100.

The unloading system 740 may perform unloading of the vehicle 100.

The unloading system 740 may receive navigation information from the navigation system 770 and provide a control signal to the vehicle driving apparatus 600 to perform unloading of the vehicle 100.

The unloading system 740 may receive object information from the object detection device 300 and provide a control signal to the vehicle driving device 600 to perform the unloading of the vehicle 100.

The exit system 740 may receive a signal from an external device through the communication device 400 and provide a control signal to the vehicle driving apparatus 600 to perform the exit of the vehicle 100.

The parking system 750 may perform parking of the vehicle 100.

The parking system 750 may receive the navigation information from the navigation system 770 and provide a control signal to the vehicle driving apparatus 600 to perform parking of the vehicle 100.

The parking system 750 may receive object information from the object detection device 300, provide a control signal to the vehicle driving device 600, and perform parking of the vehicle 100.

The parking system 750 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving apparatus 600, and perform parking of the vehicle 100.

The navigation system 770 may provide navigation information. The navigation information may include at least one of map information, set destination information, route information according to the destination setting, information on various objects on the route, lane information, and current location information of the vehicle.

The navigation system 770 may include a memory and a processor. The memory can store navigation information. The processor can control the operation of the navigation system 770.

According to an embodiment, the navigation system 770 may receive information from an external device through the communication device 400 and update pre-stored information.

According to an embodiment, the navigation system 770 may be classified as a sub-component of the user interface device 200.

The sensing unit 120 may sense the state of the vehicle. The sensing unit 120 includes a posture sensor (for example, a yaw sensor, a roll sensor, a pitch sensor), a collision sensor, a wheel sensor, a speed sensor, and an inclination Sensor, weight sensor, heading sensor, yaw sensor, gyro sensor, position module, vehicle forward/reverse sensor, battery sensor, fuel sensor, tire sensor, handle It may include a steering sensor by rotation, a temperature sensor inside the vehicle, a humidity sensor inside the vehicle, an ultrasonic sensor, an illuminance sensor, an accelerator pedal position sensor, a brake pedal position sensor, and the like.

The sensing unit 120 includes vehicle attitude information, vehicle collision information, vehicle direction information, vehicle location information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward/reverse information, battery Acquire sensing signals for information, fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, steering wheel rotation angle, vehicle exterior illumination, pressure applied to the accelerator pedal, and pressure applied to the brake pedal. can do.

The sensing unit 120 includes, in addition, an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake air temperature sensor (ATS), a water temperature sensor (WTS), and a throttle position sensor (TPS), a TDC sensor, a crank angle sensor (CAS), and the like.

The interface unit 130 may serve as a passage with various types of external devices connected to the vehicle 100. For example, the interface unit 130 may have a port connectable to the mobile terminal, and may connect to the mobile terminal through the port. In this case, the interface unit 130 may exchange data with the mobile terminal.

Meanwhile, the interface unit 130 may function as a passage for supplying electrical energy to the connected mobile terminal. When the mobile terminal is electrically connected to the interface unit 130, under the control of the control unit 170, the interface unit 130 may provide the mobile terminal with electric energy supplied from the power supply unit 190.

The memory 140 is electrically connected to the control unit 170. The memory 140 may store basic data for the unit, control data for controlling the operation of the unit, and input/output data. The memory 140 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, and the like in hardware. The memory 140 may store various data for the overall operation of the vehicle 100, such as a program for processing or controlling the control unit 170.

According to an embodiment, the memory 140 may be integrally formed with the control unit 170 or may be implemented as a lower component of the control unit 170.

The control unit 170 may control the overall operation of each unit in the vehicle 100. The control unit 170 may be referred to as an electronic controller unit (ECU).

The power supply unit 190 may supply power required for the operation of each component under the control of the control unit 170. In particular, the power supply unit 190 may receive power from a battery or the like inside the vehicle.

One or more processors and controllers 170 included in the vehicle 100 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs ( Field programmable gate arrays, processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions can be implemented.

Hereinafter, the image output device 800 provided in the vehicle 100 will be described in detail.

The image output device 800 is provided in the vehicle 100, may be made of an independent device that can be detachably attached to the vehicle 100, or may be integrally installed in the vehicle 100 and be a part of the vehicle 100. have.

The operation and control method of the image output device 800 described in this specification may be performed by the control unit 170 of the vehicle 100. That is, the operation and/or control method performed by the processor 870 of the image output device 800 may be performed by the control unit 170 of the vehicle 800.

Referring to FIG. 8, the image output device 800 includes a communication unit 810, an image output unit 850, and a processor 830.

The communication unit 810 is configured to perform communication with various components described in FIG. 7. For example, the communication unit 810 may receive various information provided through a controller (CAN). As another example, the communication unit 810 may perform communication with all devices capable of communication such as a vehicle, a mobile terminal and a server, and other vehicles. This may be called V2X (Vehicle to everything) communication. V2X communication can be defined as a technology that exchanges or shares information such as traffic conditions while communicating with road infrastructure and other vehicles while driving.

The communication unit 810 is configured to perform communication with one or more devices provided in the vehicle 100.

Furthermore, the communication unit 810 may receive information related to driving of the vehicle from most devices provided in the vehicle 100. Information transmitted from the vehicle 100 to the display device 800 is referred to as “vehicle driving information”.

The vehicle driving information includes vehicle information and surrounding information of the vehicle. Based on the frame of the vehicle 100, information related to the inside of the vehicle may be defined as vehicle information and information related to the outside of the vehicle as surrounding information.

Vehicle information refers to information about the vehicle itself. For example, vehicle information includes the vehicle's driving speed, driving direction, acceleration, angular velocity, position (GPS), weight, vehicle occupancy, vehicle braking force, vehicle maximum braking force, air pressure of each wheel, and centrifugal force applied to the vehicle. , Vehicle driving mode (whether autonomous driving mode or manual driving), vehicle parking mode (autonomous parking mode, automatic parking mode, manual parking mode), whether the user is in the vehicle, and information related to the user It can contain.

The surrounding information refers to information related to other objects located within a predetermined range around the vehicle and information related to the outside of the vehicle. For example, the condition of the road surface where the vehicle is driving (friction force), weather, distance from the vehicle in front (or rear), the relative speed of the vehicle in front (or rear), and the curvature of the curve when the driving lane is a curve, vehicle Ambient brightness, information related to an object existing in a reference area (constant area) based on a vehicle, whether an object enters/departs from the predetermined area, whether a user exists around the vehicle, and information related to the user (eg For example, whether the user is an authenticated user).

In addition, the surrounding information, ambient brightness, temperature, sun location, object information (people, other vehicles, signs, etc.) located in the vicinity, the type of road surface being driven, terrain features, lane (Line) information, driving lane (Lane ) Information, information necessary for autonomous driving/autonomous parking/automatic parking/manual parking mode.

Further, the surrounding information includes an object (object) existing around the vehicle and a distance to the vehicle 100, a possibility of collision, the type of the object, a parking space in which the vehicle can park, and an object for identifying the parking space (for example , Parking lines, strings, other vehicles, walls, etc.).

The vehicle driving information is not limited to the example described above, and may include all information generated from components provided in the vehicle 100.

The image output unit 850 outputs various visual information under the control of the processor 870. The image output unit 850 may output visual information to a windshield of a vehicle or a screen provided separately, or to output visual information through a panel. The image output unit 850 may correspond to the display unit 251 described with reference to FIGS. 1 to 7.

For example, the visual information output from the image output unit 850 is reflected from the windshield or the screen, and the visual information is displayed on the windshield or the screen. The passenger simultaneously checks the real world located outside the vehicle 100 and the virtual object displayed on the windshield or the screen, and augmented reality is implemented by the image output unit 850.

The processor 830 performs various operations to be described later, and controls the communication unit 810 and the image output unit 850.

The processor 830 may be configured to control one or more devices provided in the vehicle 100 using the communication unit 810.

Specifically, the processor 830 may determine whether at least one condition is satisfied among a plurality of preset conditions based on vehicle driving information received through the communication unit 810. According to the satisfied condition, the processor 807 may control the one or more displays in different ways.

In relation to a preset condition, the processor 870 may detect that an event has occurred in an electronic device and/or application provided in the vehicle 100, and determine whether the detected event satisfies the preset condition. At this time, the processor 870 may detect that an event has occurred from information received through the communication unit 810.

The application is a concept including a widget or a home launcher, and means all types of programs that can be driven in the vehicle 100. Accordingly, the application may be a web browser, a video playback, message transmission and reception, schedule management, a program that performs the functions of the application update.

Furthermore, the applications include Forward Collision Warning (FCW), Blind Spot Detection (BSD), Lane Departure Warning (LDW), Pedestrian Detection (PD), Curve Speed Warning (Curve Speed Warning, CSW) and turn by turn navigation (TBT).

For example, when an event occurs, when there is a missed call, when there is an application to be updated, when a message arrives, start on, start off, autonomous driving on/off, display activation key is pressed (LCD awake key), alarm (alarm), call connection (Incoming call), and may be missed notification (missed notification).

As another example, the event generation may be a case in which an alert set by the ADAS (advanced driver assistance system) is generated and a function set by the ADAS is performed. For example, when a forward collision warning occurs, when a blind spot detection occurs, when a lane departure warning occurs, a lane keeping warning When assist warning occurs, an event may be considered to occur when an emergency braking function is performed.

As another example, when changing from a forward gear to a reverse gear, when an acceleration greater than a predetermined value occurs, when a deceleration greater than a predetermined value occurs, when a power unit is changed from an internal combustion engine to a motor, or at a motor Even if it is changed to an internal combustion engine, it can be considered that the event has occurred.

In addition, it can be considered that an event has occurred even when various ECUs provided in the vehicle 100 perform a specific function.

When the generated event is satisfied with a preset condition, the processor 870 controls the communication unit 810 such that information corresponding to the satisfied condition is displayed on the one or more displays.

The processor 870 may transmit an autonomous driving message to at least one of a plurality of devices provided in the vehicle 100 so that autonomous driving is performed in the vehicle 100. For example, the autonomous driving message may be transmitted to the brake so that the deceleration is achieved, or the autonomous driving message may be transmitted to the steering device so that the driving direction is changed.

9 is a flowchart illustrating a control method of an image output device, and FIGS. 10A and 10B are conceptual diagrams for describing a control method of FIG. 9.

The processor 870 receives a front image photographing the front of the vehicle 100 (S910).

The front image is received through the communication unit 810, and may be made of one or more images.

Next, the processor 870 searches for one or more lanes in which the vehicle 100 is scheduled to travel from the front image (S930).

For convenience of description, one or more lanes for which the vehicle 100 is scheduled to be driven are referred to as “scheduled lanes”.

The driving scheduled lane refers to a lane in which the vehicle 100 is scheduled to travel until a point t that is a positive real number based on the current time. The t may vary depending on the speed of the vehicle 100, the characteristics of the road on which the vehicle 100 is driving, and the speed limit set on the road on which the vehicle 100 is driving.

When the vehicle 100 is autonomous driving, the driving scheduled lane refers to a lane scheduled for autonomous driving. When the vehicle 100 is manually driven, the planned driving lane refers to a lane recommended to the driver.

In order to search for the driving lane, the processor 870 may receive a high-precision map from a route providing device and/or a server, and receive vehicle driving information capable of specifying the driving lane.

More specifically, the processor 870 may receive front path information that guides a road located in front of the vehicle 100 in units of lanes.

The forward route information is provided to provide a driving route to a destination for each lane marked on the road, and may be route information that conforms to ADASIS standards.

The forward route information may be provided by subdividing a route in which a vehicle should travel or a travelable route in lane units. The forward route information may be information for guiding a driving route to a destination in units of lanes. When the front path information is displayed on a display mounted on the vehicle 100, a guide line for guiding a lane capable of driving may be displayed on a map. Furthermore, a graphic object indicating the location of the vehicle 100 may be included on at least one lane in which the vehicle 100 is located among a plurality of lanes included in the map.

For example, a road located in front of the vehicle 100 may be 8 lanes, and the planned driving lane may be 2 lanes. In this case, the processor 870 may search for the second lane in the front image.

For another example, the road located in front of the vehicle 100 may be 8 lanes, driving from the current point to the front 50m in the second lane is planned, and the lane change from the front 50m to the third lane may be scheduled. have. In this case, the processor 870 may search for 2 lanes up to 50 m ahead and 3 lanes after 50 m ahead from the front image.

Here, searching for a lane means searching for a part of the entire area of the front image, which includes the planned driving lane. This is to enable the passenger of the vehicle 100 to intuitively recognize the planned driving lane by displaying the carpet image guiding the driving planned lane so as to overlap with the searched partial area.

Next, the processor 870 outputs a main carpet image for guiding the searched one or more lanes in lane units through an image output unit in operation S950.

As illustrated in FIG. 10A, the processor 870 sets an image display area 1010 to output visual information based on the occupant's eye position and/or gaze.

Furthermore, the processor 870 determines at least one of the position, size, and shape of the main carpet image 1020 based on the occupant's eye position and/or gaze. At least one of the position, size, and shape of the main carpet image displayed on the windshield or the screen may be changed according to the eye position and/or gaze of the passenger. This is to provide an augmented reality in which the real world and the virtual image are exactly matched.

The main carpet image guides the lane to be driven and may be a transparent image overlapping the lane to be driven and having a predetermined color.

The predetermined color may vary depending on the reference conditions. For example, in the case of a general road, it is the first color, but when snow is accumulated on the road, it may be a second color different from the first color.

Through the main carpet image 1020, a passenger may receive route information on a planned lane in which the vehicle 100 is to travel in an autonomous driving or a driver should drive in lane units.

Meanwhile, the processor 870 may provide not only the main carpet image but also one or more sub-carpet images selectable by the passenger together with the main carpet image.

11 is a flowchart illustrating a control method of an image output device outputting a sub-carpet image, and FIGS. 12, 13, and 14 are exemplary views for explaining the control method of FIG. 11.

The processor 870 searches for an object corresponding to a preset condition (S1110).

Here, the preset condition may include various conditions. In other words, it can be variously modified according to the embodiment.

For example, as illustrated in FIG. 12, a bypass road that can reach a destination in a shorter time than the main route guided by the main carpet image may be searched for as the object. A bypass road cheaper than the main route or a bypass road that a driver can easily drive compared to the main route may be searched for as the object.

For another example, as shown in FIG. 13, the processor 870 may search for a point of interest (POI) set by the passenger as the object. The point of interest may be limited to being located in the front within a predetermined range based on the vehicle 100. Examples of points of interest include parking lots, movie theaters, and restaurants, and the points of interest may vary depending on the passengers who board the vehicle 100.

For another example, as shown in FIG. 14, the processor 870 may search for another vehicle located in a predetermined range in front of the vehicle as the object. In this case, the other vehicle may be limited to being located in the front within a predetermined range based on the vehicle 100. Furthermore, the other vehicle is a vehicle that the vehicle 100 can follow and may be limited to that communication with the vehicle 100 is approved. In other words, a vehicle for which communication is not authorized is not searched for the other vehicle.

The processor 870 may control the image output unit 850 so that an object image guiding the object is output in response to the object being searched.

For example, when the object is an object, a boundary line surrounding the edge of the object may be output as the object image. Through the object image, the passenger can visually check what the searched object is.

Furthermore, the processor 870 may collect object information corresponding to the object and control the image output unit 850 to output the object information.

For example, as illustrated in FIG. 12, when the object is a bypass path, when the vehicle travels on the bypass path, the time required to reach a destination, the expected time of arrival at the destination, and the amount of fuel required are the objects. Information may be collected.

For another example, as illustrated in FIG. 13, when the object is a watchpoint 1210, an event provided by the point of interest 1210, an address, a contact number, a trade name, or a trademark among the points of interest 1212 At least one may be collected as the object information. The processor 870 may request object information related to the point of interest 1210 to a server, and output object information received from the server through the image output unit 850.

For another example, as illustrated in FIG. 14, when the object is another vehicle 1410 or 1412, at least one of a type, speed, and destination of another vehicle may be collected. In this case, the processor 870 may perform V2X communication with another vehicle and receive other vehicle information from the other vehicle. Furthermore, the processor 870 may transmit an image of the other vehicle to a server and receive the other vehicle information from the server.

Furthermore, the processor 870 may calculate fuel efficiency when the vehicle 100 follows the other vehicle and collect the fuel efficiency as the object information. The processor 870 may calculate the fuel efficiency when the vehicle follows the other vehicle, and control the image output unit 850 such that the fuel efficiency is output together with the main carpet image and the sub carpet image. have.

Next, the processor 870 may output a sub carpet image that guides a path to a destination corresponding to the object along with the main carpet image in response to a preset user input (S1130).

The preset user input may include various user inputs.

For example, it may be a voice command input through a microphone provided in the vehicle 100. The preset natural language may be input through a natural language in which the artificial intelligence interacts. For example, when a natural language such as “Is the left turn fast” is input, the processor 870 searches for a bypass route through the left turn, and the sub carpet image that guides the searched bypass route in lane units with the main carpet image? You can print them together.

For another example, the preset user input may be a user input for operating a jog diamond provided in the vehicle 100 in a predetermined manner, a predetermined gesture, or a predetermined touch input.

The preset user input requires a path different from the main path guided by the main carpet image, and may be variously modified according to embodiments.

The processor 870 searches for a path to a destination corresponding to the object. In order to distinguish it from the main path guided by the main carpet image, a path guided by the sub carpet image is referred to as a “sub path”. That is, the processor 870 searches for a sub-path to a destination corresponding to the object, and the sub-carpet image guides the sub-path in lane units.

As illustrated in FIG. 12, when the object is a bypass path, a sub carpet image 1220 for guiding the bypass path in lane units may be output.

As shown in FIG. 13, when the object is the point of interest 1310, the processor 870 searches for a sub-path in a lane unit, the sub-path of which is the point where the vehicle 100 can be parked at the point of interest. Then, a sub carpet image 1320 corresponding thereto may be output.

As illustrated in FIG. 14, when the object is another vehicle 1410 and 1412, the processor 870 searches a sub-path following the other vehicle in a lane unit, and a sub carpet image 1420 corresponding thereto , 1422).

When the object is another vehicle, the processor 870 controls the image output unit 850 such that at least one of the size, shape, and location of the sub-carpet image varies according to the location of the other vehicle. For example, when the other vehicle changes the lane, the sub-carpet image may be updated to guide the vehicle 100 to move to the lane changed by the other vehicle.

The processor 870 may control the image output unit 850 so that a plurality of sub carpet images are output together with the main carpet image when there are a plurality of objects corresponding to the preset condition. When a plurality of sub carpet images are output, each sub carpet image may have a different color according to an object corresponding to each sub carpet image.

Meanwhile, when the number of objects corresponding to the preset condition is greater than the reference number, the processor 870 may filter the reference number based on the preset number. For example, when the number of searched objects is 10 and the reference number is 3, the processor 870 selects (or filters) 3 objects out of 10 objects. When filtering is performed, sub-carpet images corresponding to the reference number are output together with the main carpet image.

The filtering criterion may be the gaze of a passenger who boards the vehicle 100. For example, a partial area of a predetermined size that the passenger is looking at may be extracted from the front image, and an object located in the partial area may be selected. Alternatively, the point that the passenger is looking at may be extracted, and the objects of the reference number may be selected in the order located closest to the point.

The processor 870 may collect traffic congestion for a route to a destination corresponding to the object, and control the image output unit 850 such that the sub-carpet images have different colors according to the traffic congestion. have.

For example, traffic congestion may be calculated as a number, and may have a first color when the traffic congestion is within a first range, and a second color when the traffic congestion is within a second range.

The main carpet image may be a first color transparent image, and the sub carpet image may be a second color transparent image. In this case, the processor 870 searches for a portion where the main carpet image and the sub carpet image overlap, and outputs the image so that the portion has a third color in which the first color and the second color are mixed. We can control wealth. As the vehicle 100 moves, the overlapping portion is variable.

When the object is another vehicle, the processor 870 calculates fuel efficiency when following the other vehicle, and sets the image output unit 850 so that the sub carpet images have different colors according to the fuel efficiency. Can be controlled.

The processor 870 may output the sub-carpet image in response to an object corresponding to the preset condition being searched, even without the preset user input.

Next, the processor 870 may control the image output unit 850 so that the sub carpet image disappears when the object is located outside a predetermined range based on the vehicle while outputting the sub carpet image. It can be (S1150).

By the movement of the vehicle 100 and / or by the movement of the object, the object may be located within a predetermined range based on the vehicle 100 and then moved outward. When located within the predetermined range, the sub-carpet image is output, but when located outside the predetermined range, output of the sub-carpet image is limited. That is, the sub-carpet image disappears from the image display area.

15 is a flowchart for explaining the operation of the video output device based on the user input.

The processor 870 may output a plurality of sub-carpet images together with the main carpet image (S1510).

The processor 870 controls the image output unit 850 so that a plurality of sub-carpet images are output together with the main carpet image when there are a plurality of objects corresponding to the preset condition.

Next, the processor 870 may highlight any one according to a user input (S1530).

The processor 870 highlights any one of the sub-carpet images, and the one sub-carpet image that is the highlight processing is changed according to a user input.

The user input may be input by at least one of voice input, touch input, gesture input and button input. For example, when first, second, and third sub carpet images are output from the left end of the video display area, the first, second, and third subs are generated by a user input that rotates the jog dial clockwise. Each carpet image may be sequentially selected and highlighted.

Here, the highlight processing means that at least one of the transparency, color, and brightness of any one of the sub-carpet images is processed differently from the other. Furthermore, a boundary line representing the edge of the one sub-carpet image may be generated.

Next, the processor 870 may change the path of the vehicle when any one of the above is selected (S1550).

The processor 870, when the one of the sub-carpet image is selected by the user, the communication unit 850 so that the driving direction of the vehicle 100 is changed to a path set in the one of the sub-carpet images ) Control.

Next, the processor 870 may control the image output unit so that the main carpet image disappears when either one is selected (S1570).

When one of the sub-carpet images is selected, since the driving path of the vehicle 100 is changed, the main carpet image disappears from the image display area.

16 is an exemplary view for explaining an embodiment in which colors of subcarpet images are changed according to priority.

The processor 870 may search for a plurality of objects that satisfy a preset condition. In this case, the processor 870 sets priority for each of the plurality of objects. The processor 870 controls the image output unit 850 such that each sub-carpet image has a color corresponding to the priority.

When the priority is changed, the color of at least one sub-carpet image may be changed. For example, if the priority of the first sub-carpet image is the first priority, the first sub-carpet image has a first color. Thereafter, when the priority of the first sub-carpet image is changed to the second priority, the first color may be changed to the second color.

The above-described present invention can be embodied as computer readable codes (or applications or software) on a medium on which a program is recorded. The above-described control method of an autonomous vehicle can be realized by a code stored in a memory or the like.

The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system are stored. Examples of computer-readable media include a hard disk drive (HDD), solid state disk (SSD), silicon disk drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. This includes, and is also implemented in the form of a carrier wave (eg, transmission over the Internet). Also, the computer may include a processor or a control unit. Accordingly, the above detailed description should not be construed as limiting in all respects, but should be considered illustrative. The scope of the invention should be determined by rational interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

Claims (20)

1. A video output device mounted on a vehicle to realize augmented reality,

   An image output unit outputting visual information for realizing the augmented reality;

   A communication unit that receives a front image photographing the front of the vehicle; And

   And a processor for controlling the image output unit to search for one or more lanes scheduled to drive the vehicle from the front image and output a main carpet image for guiding the searched one or more lanes in lane units,

   The processor,

   Searching for an object corresponding to a preset condition, and controlling the video output unit to output a sub carpet image that guides a path to a destination corresponding to the object in response to a preset user input, along with the main carpet image. Video output device.
2. According to claim 1,

   The processor,

   When there are a plurality of objects corresponding to the preset condition, the video output device, characterized in that for controlling the video output unit to output a plurality of sub-carpet images together with the main carpet image.
3. According to claim 2,

   The processor highlights any one of the sub carpet images,

   The image output device, characterized in that the one of the sub-carpet image that is the highlight processing is changed according to the user input.
4. According to claim 3,

   The processor,

   When the one of the sub-carpet image is selected by the user, the image output device, characterized in that for controlling the communication unit so that the driving direction of the vehicle is changed to a path set in the one of the sub-carpet image.
5. According to claim 4,

   The processor,

   When the one of the sub-carpet image is selected by the user, the video output device, characterized in that for controlling the video output unit to disappear the main carpet image.
6. According to claim 2,

   The processor,

   Priority is set for the plurality of objects, and the image output unit is controlled so that each sub-carpet image has a color corresponding to the priority,

   When the priority is changed, the video output device, characterized in that the color of the at least one sub-carpet image is changed.
7. According to claim 2,

   The processor,

   When the number of objects corresponding to the preset condition is greater than the reference number, filtering the reference number based on the preset criterion, and controlling the image output unit to output the sub-carpet images of the reference number together with the main carpet image Video output device characterized in that.
8. The method of claim 7,

   The processor,

   The image output device, characterized in that for filtering the objects of the reference number based on the gaze of the passengers in the vehicle.
9. According to claim 1,

   The processor,

   Based on the vehicle, other vehicles located in a predetermined range are searched as the object,

   The image output device, characterized in that for controlling the image output unit so that at least one of the size, shape and position of the sub-carpet image is variable according to the position of the other vehicle.
10. The method of claim 9,

    The processor,

    A video output device comprising calculating the fuel efficiency when the vehicle follows the other vehicle and controlling the image output unit to output the fuel efficiency together with the main carpet image and the sub carpet image.
11. The method of claim 10,

    The processor,

    And controlling the image output unit so that the sub carpet images have different colors according to the fuel efficiency.
12. According to claim 1,

    The processor,

    When outputting the sub-carpet image, when the object is located outside a predetermined range based on the vehicle, the image output device, characterized in that for controlling the image output unit to disappear the sub-carpet image.
13. According to claim 1,

    The processor,

    A video output device comprising collecting traffic congestion for a route to a destination corresponding to the object and controlling the video output unit so that the sub-carpet images have different colors according to the traffic congestion.
14. According to claim 1,

    The processor,

    And an image output unit controlling the image output unit to output an object image guiding the object in response to the object being searched.
15. According to claim 1,

    The processor,

    The image output device, characterized in that for collecting the object information corresponding to the object, and controlling the image output unit to output the object information.
16. According to claim 1,

    The main carpet image is a first color transparent image, and the sub carpet image is a second color transparent image,

    The processor,

    Searching for a portion where the main carpet image and the sub carpet image overlap, and controlling the image output unit to control the image output unit such that the portion has a third color in which the first color and the second color are mixed. Device.
17. Receiving a front image photographing the front of the vehicle;

    Searching for one or more lanes scheduled for driving of the vehicle from the front image; And

    And outputting a main carpet image for guiding the searched one or more lanes in lane units through an image output unit,

    Searching for an object corresponding to a preset condition; And

    And outputting a sub-carpet image for guiding a path to a destination corresponding to the object together with the main carpet image in response to a preset user input.
18. The method of claim 17,

    When there are a plurality of objects corresponding to the preset condition, a plurality of sub-carpet images are output together with the main carpet image, the control method of the video output device.
19. The method of claim 18,

    And when a sub-carpet image of one of the plurality of sub-carpet images is selected by a user, further comprising changing a driving direction of the vehicle to a path set in the one of the sub-carpet images. Control method of video output device.
20. The method of claim 16,

    And while outputting the sub-carpet image, when the object is located outside a predetermined range based on the vehicle, controlling the image output unit to disappear the sub-carpet image. Control method.

Images