KR102578679B1 - Head-up display apparatus and control method for the same - Google Patents

Abstract

The present invention relates to a head-up display device and a control method thereof. The head-up display device according to an embodiment of the present invention provides the vehicle with a virtual screen including at least one indicator that guides driving information of the vehicle. It includes a display unit that displays information on at least a portion of the projected projection surface, and a processor that changes at least one of the position and size of the virtual screen based on movement information of the vehicle.

Description

Head-up display device and its control method {HEAD-UP DISPLAY APPARATUS AND CONTROL METHOD FOR THE SAME}

The present invention relates to a head-up display device and a control method thereof. More specifically, the present invention relates to a head-up display device and a control method thereof. More specifically, the present invention relates to a head-up display device and a control method thereof. More specifically, the present invention relates to a head-up display device and a control method thereof, by controlling a virtual screen based on vehicle movement information to ensure consistency between the virtual screen displayed on the projection surface and the foreground of the vehicle visible to the driver. It relates to an improved vehicle head-up display device and its control method.

A vehicle is a device that drives wheels to transport people or cargo from one place to another. For example, vehicles include two-wheeled vehicles such as motorcycles, four-wheeled vehicles such as sedans, and trains.

In order to increase the safety and convenience of vehicle users, the development of technology to integrate various sensors and electronic devices into vehicles is accelerating. In particular, systems that provide various functions (eg, smart cruise control, lane keeping assistance) developed for the user's driving convenience are being installed in vehicles. Accordingly, so-called autonomous driving, in which a vehicle drives on the road by itself considering the external environment without driver intervention, has become possible.

Meanwhile, vehicles may be equipped with various types and types of displays at various locations inside the vehicle to provide various vehicle information and convenience information to passengers.

Among these, the head-up display outputs images indicating various information on a projection surface such as the vehicle's windshield or a self-equipped combiner, allowing the driver to recognize various information related to the vehicle while looking ahead. This can help reduce the incidence of accidents caused by neglecting to look ahead.

Recently, augmented reality technology has been applied to head-up displays. The Augmented Reality-based head-up display provides the driver with useful information about various objects (e.g., other vehicles, pedestrians, traffic lights) around the vehicle by overlaying virtual graphic objects on the real world visible beyond the windshield. This is of great help in easily recognizing and taking appropriate action.

The image produced by a conventional head-up display is recognized by the driver as if it were floating in front of the vehicle, but in this case, there is a limitation in that the image is displayed without considering the movement of the vehicle. For example, when a vehicle is driving on a curved road, there is a problem in that the image projected by the head-up display is displayed in a position beyond the curve, thereby interfering with the driver's field of view.

The present invention was developed to solve the above-mentioned problems, and in order to improve the consistency between the virtual screen displayed on the projection surface and the foreground of the vehicle shown to the driver, the head-up device controls the virtual screen based on vehicle movement information. The purpose is to provide a display device and its control method.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

In order to achieve the above or other objects, according to one aspect of the present invention, there is provided a display unit that displays a virtual screen including at least one indicator guiding driving information of the vehicle on at least a portion of a projection surface provided in the vehicle, and the A head-up display device is provided, including a processor that changes at least one of the position and size of the virtual screen based on vehicle movement information.

Additionally, the display unit may include a display panel that outputs display light corresponding to the driving information; and a reflection unit including at least one mirror that reflects the display light onto the projection surface. In this case, the virtual screen is an area in which a virtual image of the display light reflected by the reflector is displayed among the entire area of the projection surface.

In addition, it may further include a posture adjustment unit that adjusts the rotation angle or position of the reflector according to a control signal provided from the processor.

Additionally, the processor may change the position of the virtual screen in the vertical direction based on the forward and backward tilt included in the motion information of the vehicle.

Additionally, the processor may change the position of the virtual screen in the left and right directions based on the steering angle included in the motion information of the vehicle.

Additionally, the processor may determine a lane in which the vehicle is scheduled to travel based on the steering angle and road information, and change the position of the virtual screen in the left and right directions based on the location of the lane in which the vehicle is scheduled to travel. there is.

Additionally, the processor may change the size of at least a portion of the virtual screen based on the driving speed included in the motion information of the vehicle.

Additionally, the processor may enlarge the size of an indicator guiding the driving speed among at least one indicator included in the virtual screen, based on an increase in the driving speed included in the motion information of the vehicle.

Additionally, it may further include a light detection sensor. In this case, when the headlights of the oncoming vehicle are detected by the light detection sensor, the processor sets the target position of the virtual screen based on the position of the headlights with respect to the vehicle, and sets the target position of the virtual screen. The position can be changed up to the target position.

Additionally, the processor sets a target position of the virtual screen based on the slope of the ramp closest to the vehicle, and gradually adjusts the position of the virtual screen to the target position as the vehicle approaches the ramp. You can change it.

Specific details of other embodiments are included in the detailed description and drawings.

The effects of the head-up display device and its control method according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, by adjusting at least one of the position and size of the virtual screen displayed on the projection surface based on the movement information of the vehicle, consistency between the virtual screen and the foreground of the vehicle visible to the driver is improved. It can be improved.

In addition, according to at least one of the embodiments of the present invention, based on vehicle driving information (e.g., steering, speed, vehicle body tilt) and external information (e.g., road information, path information, object information), the target of the virtual screen is By setting the location or modifying or resetting the preset target location, information appropriate to the vehicle's driving situation can be provided to the driver through a virtual screen.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 shows a block diagram of a vehicle according to one embodiment of the present invention.
Figure 2 shows an exemplary exterior of the vehicle shown in Figure 1.
FIG. 3 shows an example of images generated by a plurality of cameras shown in FIG. 2.
Figure 4 shows a block diagram of a head-up display device according to an embodiment of the present invention.
Figure 5 shows an example of an internal block diagram of the processor shown in Figure 4.
FIGS. 6A and 6B are diagrams referenced in explaining the operation of the processor shown in FIG. 5.
7A and 7B are conceptual diagrams for explaining the structure and operation of a display unit included in a head-up display device according to an embodiment of the present invention.
Figure 8 shows an exemplary process performed by a head-up display device according to an embodiment of the present invention.
FIGS. 9A and 9B are diagrams referenced to explain an operation of a head-up display device according to an embodiment of the present invention to change the position of a virtual screen based on the forward-backward tilt of the vehicle.
Figure 10 shows an exemplary process performed by a head-up display device according to an embodiment of the present invention.
Figure 11 shows an example of an operation in which a head-up display device changes the position of a virtual screen in the left and right directions according to an embodiment of the present invention.
Figure 12 shows another example of an operation in which a head-up display device changes the position of a virtual screen in the left and right directions according to an embodiment of the present invention.
Figure 13 shows an exemplary process performed by a head-up display device according to an embodiment of the present invention.
Figures 14a to 14c show an operation of a head-up display device according to an embodiment of the present invention to control a virtual screen according to the driving speed of the vehicle.
Figure 15 shows the operation of a head-up display device according to an embodiment of the present invention to control a virtual screen according to the driving speed of the vehicle.
Figure 16 shows an operation of a head-up display device according to an embodiment of the present invention to control the position of a virtual screen based on the headlights of an oncoming vehicle.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between. In addition, the fact that a component "controls" another component should be understood to encompass not only direct control of another component by a component, but also control through the mediation of a third component. something to do. In addition, the fact that a component "provides" information or signals to another component means that a component provides information or signals directly to another component as well as providing it through the mediation of a third component. It should be understood as

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

The vehicle described in this specification may be a concept that includes all internal combustion engine vehicles having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, and an electric vehicle having an electric motor as a power source.

Figure 1 shows a block diagram of a vehicle 100 according to one embodiment of the present invention.

Referring to FIG. 1, the vehicle 100 includes a communication unit 110, an input unit 120, a memory 130, an output unit 140, a vehicle driving unit 150, a sensing unit 160, a control unit 170, and an interface. It may include a unit 180 and a power unit 190.

The communication unit 110 may include one or more modules that enable wireless communication between the vehicle 100 and external devices (eg, mobile terminals, external servers, other vehicles). Additionally, the communication unit 110 may include one or more modules that connect the vehicle 100 to one or more networks.

The communication unit 110 may include a broadcast reception module 111, a wireless Internet module 112, a short-range communication module 113, a location information module 114, and an optical communication module 115.

The broadcast reception module 111 receives broadcast signals or broadcast-related information from an external broadcast management server through a broadcast channel. Here, broadcasting includes radio broadcasting or TV broadcasting.

The wireless Internet module 112 refers to a module for wireless Internet access and may be built into or external to the vehicle 100. The wireless Internet module 112 is configured to transmit and receive wireless signals in a communication network based on wireless Internet technologies.

Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), and WiMAX (Worldwide). Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc., and the wireless Internet module ( 112) transmits and receives data according to at least one wireless Internet technology, including Internet technologies not listed above. For example, the wireless Internet module 112 can wirelessly exchange data with an external server. The wireless Internet module 112 may receive weather information and road traffic situation information (eg, Transport Protocol Expert Group (TPEG)) information from an external server.

The short-range communication module 113 is for short-range communication, including Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC. Short-distance communication can be supported using at least one of (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

The short-range communication module 113 forms a wireless area network and can perform short-range communication between the vehicle 100 and at least one external device. For example, the short-range communication module 113 can wirelessly exchange data with the passenger's mobile terminal. The short-range communication module 113 may receive weather information and road traffic situation information (eg, Transport Protocol Expert Group (TPEG)) from a mobile terminal or an external server. For example, when a user boards the vehicle 100, the user's portable terminal and the vehicle 100 may pair with each other automatically or by executing the user's application.

The location information module 114 is a module for acquiring the location of the vehicle 100, and a representative example thereof is a Global Positioning System (GPS) module. For example, if a vehicle uses a GPS module, it can obtain the vehicle's location using signals sent from GPS satellites.

The optical communication module 115 may include a light emitting unit and a light receiving unit.

The light receiver can convert a light signal into an electric signal and receive information. The light receiver may include a photo diode (PD) for receiving light. Photodiodes can convert light into electrical signals. For example, the light receiver may receive information about the vehicle ahead through light emitted from a light source included in the vehicle ahead.

The light emitting unit may include at least one light emitting element for converting an electrical signal into an optical signal. Here, the light emitting device is preferably an LED (Light Emitting Diode). The optical transmitter converts the electrical signal into an optical signal and transmits it to the outside. For example, the light emitting unit may emit an optical signal to the outside through blinking of a light emitting element corresponding to a predetermined frequency. Depending on the embodiment, the light emitting unit may include a plurality of light emitting element arrays. Depending on the embodiment, the light emitting unit may be integrated with a lamp provided in the vehicle 100. For example, the light emitting unit may be at least one of a headlight, a tail light, a brake light, a turn signal light, and a side light. For example, the optical communication module 115 can exchange data with another vehicle through optical communication.

The input unit 120 may include a driving control unit 121, a microphone 123, and a user input unit 124.

The driving operation means 121 receives user input for driving the vehicle 100. The driving operation means 121 may include a steering input means 121a, a shift input means 121b, an acceleration input means 121c, and a brake input means 121d.

The steering input means 121a receives input of the moving direction of the vehicle 100 from the user. The steering input means 121a may include a steering wheel. Depending on the embodiment, the steering input means 121a may be formed as a touch screen, touch pad, or button.

The shift input means 121b receives inputs for parking (P), forward (D), neutral (N), and reverse (R) of the vehicle 100 from the user. The shift input means 121b is preferably formed in the shape of a lever. Depending on the embodiment, the shift input means 121b may be formed as a touch screen, touch pad, or button.

The acceleration input means 121c receives an input for acceleration of the vehicle 100 from the user. The brake input means 121d receives an input for decelerating the vehicle 100 from the user. The acceleration input means 121c and the brake input means 121d are preferably formed in the form of pedals. Depending on the embodiment, the acceleration input means 121c or the brake input means 121d may be formed as a touch screen, a touch pad, or a button.

The camera 122 is placed on one side of the interior of the vehicle 100 and generates an indoor image of the vehicle 100. For example, the camera 122 may be placed at various locations of the vehicle 100, such as the dashboard surface, roof surface, and rear view mirror, to capture occupants of the vehicle 100. In this case, the camera 122 may generate an indoor image of an area including the driver's seat of the vehicle 100. Additionally, the camera 122 may generate an indoor image of an area including the driver's seat and the passenger seat of the vehicle 100. The indoor image generated by the camera 122 may be a two-dimensional image and/or a three-dimensional image. To generate a 3D image, the camera 122 may include at least one of a stereo camera, a depth camera, and a 3D laser scanner. The camera 122 may provide indoor images it generates to the control unit 170 that is functionally coupled thereto.

The control unit 170 can analyze indoor images provided from the camera 122 and detect various objects. For example, the control unit 170 may detect the driver's gaze and/or gesture from a portion of the indoor image corresponding to the driver's seat area. As another example, the control unit 170 may detect the passenger's gaze and/or gesture from a portion of the indoor image corresponding to the indoor area excluding the driver's seat area. Of course, the driver and passenger's gaze and/or gestures may be detected simultaneously.

The microphone 123 can process external acoustic signals into electrical data. The processed data can be used in various ways depending on the function being performed by the vehicle 100. The microphone 123 can convert the user's voice command into electrical data. The converted electrical data may be transmitted to the control unit 170.

Meanwhile, depending on the embodiment, the camera 122 or the microphone 123 may be a component included in the sensing unit 160 rather than a component included in the input unit 120.

The user input unit 124 is for receiving information from the user. When information is input through the user input unit 124, the control unit 170 can control the operation of the vehicle 100 to correspond to the input information. The user input unit 124 may include a touch input means or a mechanical input means. Depending on the embodiment, the user input unit 124 may be placed in one area of the steering wheel. In this case, the driver can manipulate the user input unit 124 with his or her fingers while holding the steering wheel.

The input unit 120 may include a plurality of buttons or touch sensors. It is also possible to perform various input operations through a plurality of buttons or touch sensors.

The memory 130 is electrically connected to the control unit 170. The memory 170 can store basic data for the unit, control data for controlling the operation of the unit, and input/output data. The memory 190 may be a variety of hardware storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. The memory 130 may store various data for the overall operation of the vehicle 100, such as programs for processing or controlling the control unit 170.

The output unit 140 is for outputting information processed by the control unit 170 and may include a display unit 141, an audio output unit 142, and a haptic output unit 143.

The display unit 141 may display information processed by the control unit 170. For example, the display unit 141 may display vehicle-related information. Here, the vehicle-related information may include vehicle control information for direct control of the vehicle, or vehicle driving assistance information for driving guidance to the vehicle driver. Additionally, vehicle-related information may include vehicle status information indicating the current status of the vehicle or vehicle operation information related to the operation of the vehicle.

The display unit 141 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), or a flexible display. It may include at least one of a display, a 3D display, and an e-ink display.

The display unit 141 can implement a touch screen by forming a layered structure or being integrated with the touch sensor. This touch screen functions as a user input unit 124 that provides an input interface between the vehicle 100 and the user, and can simultaneously provide an output interface between the vehicle 100 and the user. In this case, the display unit 141 may include a touch sensor that detects a touch on the display unit 141 so that a control command can be input by a touch method. Using this, when a touch is made to the display unit 141, the touch sensor can detect the touch, and the control unit 170 can generate a control command corresponding to the touch based on this. Content input by the touch method may be letters or numbers, instructions in various modes, or designable menu items.

Meanwhile, the display unit 141 may include a cluster so that the driver can check vehicle status information or vehicle operation information while driving. Clusters can be located on dashboards. In this case, the driver can check the information displayed on the cluster while keeping his gaze in front of the vehicle.

Meanwhile, depending on the embodiment, the display unit 141 may be implemented as a Head Up Display (HUD). When the display unit 141 is implemented as a HUD, information can be output through a transparent display provided on the windshield.

The audio output unit 142 converts the electrical signal from the control unit 170 into an audio signal and outputs it. For this purpose, the sound output unit 142 may be equipped with a speaker, etc. The sound output unit 142 can also output sound corresponding to the operation of the user input unit 124.

The haptic output unit 143 generates a tactile output. For example, the haptic output unit 143 may operate to vibrate the steering wheel, seat belt, and seat so that the user can perceive the output.

The vehicle driving unit 150 can control the operation of various vehicle devices. The vehicle driving unit 150 includes a power source driving unit 151, a steering driving unit 152, a brake driving unit 153, a lamp driving unit 154, an air conditioning driving unit 155, a window driving unit 156, an airbag driving unit 157, and a sunroof. It may include at least one of the driving unit 158 and the wiper driving unit 159.

The power source driver 151 may perform electronic control of the power source within the vehicle 100. The power source driving unit 151 may include an acceleration device that increases the speed of the vehicle 100 and a deceleration device that reduces the speed of the vehicle 100.

For example, when a fossil fuel-based engine (not shown) is the power source, the power source driver 151 may perform electronic control of the engine. Thereby, the output torque of the engine, etc. can be controlled. When the power source driving unit 151 is an engine, the speed of the vehicle can be limited by limiting the engine output torque under the control of the control unit 170.

As another example, when an electric motor (not shown) is the power source, the power source driver 151 may control the motor. Thereby, the rotation speed and torque of the motor can be controlled.

The steering drive unit 152 may include a steering apparatus. Accordingly, the steering drive unit 152 may perform electronic control of the steering apparatus within the vehicle 100. For example, the steering drive unit 152 may be provided with a steering torque sensor, a steering angle sensor, and a steering motor, and the steering torque applied by the driver to the steering wheel may be detected by the steering torque sensor. The steering driver 152 can control the steering force and steering angle by changing the magnitude and direction of the current applied to the steering motor based on the speed and steering torque of the vehicle 100. Additionally, the steering driver 152 may determine whether the driving direction of the vehicle 100 is properly adjusted based on the steering angle information acquired by the steering angle sensor. Thereby, the driving direction of the vehicle can be changed. In addition, the steering drive unit 152 increases the steering force of the steering motor when the vehicle 100 is traveling at low speeds to reduce the weight of the steering wheel, and when the vehicle 100 is traveling at high speeds, it reduces the steering force of the steering motor to reduce the weight of the steering wheel. Weight can be increased. In addition, when the autonomous driving function of the vehicle 100 is executed, the steering drive unit 152 provides Based on a sensing signal or a control signal provided by the controller 170, the steering motor may be controlled to generate an appropriate steering force.

The brake driver 153 may perform electronic control of a brake apparatus (not shown) in the vehicle 100. For example, the speed of the vehicle 100 can be reduced by controlling the operation of the brakes disposed on the wheels. As another example, the moving direction of the vehicle 100 can be adjusted to the left or right by varying the operation of the brakes disposed on the left and right wheels, respectively.

The lamp driver 154 can control the turn on/off of at least one lamp placed inside or outside the vehicle. The lamp driver 154 may include a lighting device. Additionally, the lamp driver 154 can control the intensity and direction of light output from each lamp included in the lighting device. For example, control of turn signal lamps, headlamps, brake lamps, etc. can be performed.

The air conditioning driver 155 may perform electronic control of an air conditioning device (not shown) in the vehicle 100. For example, when the temperature inside the vehicle is high, the air conditioning device operates to control the supply of cold air into the vehicle interior.

The window driver 156 may perform electronic control of a window apparatus within the vehicle 100. For example, the opening or closing of the left and right windows on the side of the vehicle 100 can be controlled.

The airbag driving unit 157 may perform electronic control of the airbag apparatus within the vehicle 100. For example, in case of danger, the airbag can be controlled to explode.

The sunroof driver 158 may perform electronic control of a sunroof apparatus (not shown) in the vehicle 100. For example, you can control the opening or closing of the sunroof.

The wiper driving unit 159 may control the wipers 14a and 14b provided in the vehicle 100. For example, when the wiper driving unit 159 receives a user input commanding to drive the wiper through the user input unit 124, it electronically controls the number of times the wipers 14a and 14b are driven, the driving speed, etc., according to the user input. can be performed. For another example, the wiper driving unit 159 determines the amount or intensity of rainwater based on a sensing signal from a rain sensor included in the sensing unit 160, and wipes the wipers 14a and 14b without user input. can be operated automatically.

Meanwhile, the vehicle driving unit 150 may further include a suspension driving unit (not shown). The suspension drive unit may perform electronic control of a suspension apparatus (not shown) within the vehicle 100. For example, when the road surface is curved, the suspension device can be controlled to reduce vibration of the vehicle 100.

The sensing unit 160 senses signals related to the driving of the vehicle 100. For this purpose, the sensing unit 160 includes a collision sensor, a steering sensor, a speed sensor, an inclination sensor, a weight sensor, a heading sensor, a yaw sensor, a gyro sensor, Position module, vehicle forward/backward sensor, battery sensor, fuel sensor, tire sensor, steering sensor by steering wheel rotation, vehicle interior temperature sensor, vehicle interior humidity sensor, ultrasonic sensor, infrared sensor, radar, lidar, etc. may include.

As a result, the sensing unit 160 provides vehicle collision information, vehicle direction information, vehicle location information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward/backward information, battery information, Sensing signals for fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, and steering wheel rotation angle can be obtained. In addition, the control unit 170 accelerates, decelerates, and accelerates the vehicle 100 based on external environmental information acquired by at least one of a camera, an ultrasonic sensor, an infrared sensor, a radar, and a lidar provided in the vehicle 100. Control signals for direction changes, etc. can be generated. Here, the external environment information may be information related to various objects located within a predetermined distance range from the running vehicle 100. For example, the external environment information may include information about the number of obstacles located within 100 m from the vehicle 100, the distance to the obstacle, the size of the obstacle, the type of obstacle, etc.

Meanwhile, the sensing unit 160 includes an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake temperature sensor (ATS), a water temperature sensor (WTS), and a throttle position sensor. It may further include a sensor (TPS), TDC sensor, crank angle sensor (CAS), etc.

The sensing unit 160 may include a biometric information sensing unit. The biometric information detection unit detects and obtains the passenger's biometric information. Biometric information includes fingerprint recognition information, iris-scan information, retina-scan information, hand geometry information, facial recognition information, and voice recognition ( Voice recognition) information may be included. The biometric information detection unit may include a sensor that senses the passenger's biometric information. Here, the camera 122 and microphone 123 may operate as sensors. The biometric information detection unit may acquire hand shape information and facial recognition information through the camera 122.

The sensing unit 160 may include at least one camera 161 that photographs the exterior of the vehicle 100. Camera 161 may be referred to as an external camera. For example, the sensing unit 160 may include a plurality of cameras 161 disposed at different locations on the exterior of the vehicle. This camera 161 may include an image sensor and an image processing module. The camera 161 can process still images or moving images obtained by an image sensor (eg, CMOS or CCD). The image processing module may process still images or moving images obtained through an image sensor, extract necessary information, and transmit the extracted information to the control unit 170.

The camera 161 may include an image sensor (eg, CMOS or CCD) and an image processing module. Additionally, the camera 161 can process still images or moving images obtained by an image sensor. The image processing module can process still images or moving images obtained through an image sensor. Additionally, the camera 161 may acquire images including at least one of traffic lights, traffic signs, pedestrians, other vehicles, and the road surface.

The sensing unit 160 may include a radar 162, lidar 163, or ultrasonic sensor 164 for measuring the surrounding environment. The radar 162 is mounted on one side of the vehicle 100 and can emit electromagnetic waves toward the surroundings of the vehicle 100 and receive electromagnetic waves reflected from various objects existing around the vehicle 100. For example, the radar 162 can measure the time of an electromagnetic wave reflected by an object and obtain information related to the distance, direction, altitude, etc. of the object.

Lidar 163 is mounted on one side of the vehicle 100 and can fire a laser toward the surroundings of the vehicle 100. The laser fired by the LiDAR 163 may be scattered or reflected and return to the vehicle 100, and the LiDAR 163 may be based on the laser return time, intensity, change in frequency, and change in polarization state. , information on physical characteristics such as distance, speed, and shape of targets located around the vehicle 100 can be obtained.

The ultrasonic sensor 164 is mounted on one side of the vehicle 100 and generates ultrasonic waves toward the surroundings of the vehicle 100. Ultrasound generated by the ultrasonic sensor 164 has a high frequency (about 20 KHz or more) and a short wavelength. This ultrasonic sensor 164 can be mainly used to recognize obstacles close to the vehicle 100.

The interface unit 180 may serve as a passageway for various types of external devices connected to the vehicle 100. For example, the interface unit 180 may have a port that can be connected to a mobile terminal, and can be connected to a mobile terminal through the port. In this case, the interface unit 180 can exchange data with the portable terminal.

The interface unit 180 may receive turn signal information. Here, the turn signal information may be a turn on signal of a turn signal for turning left or right input by the user. When a left or right turn indicator turn-on input is received through the vehicle's user input unit (724 in FIG. 6), the interface unit 180 may receive left or right turn signal information.

The interface unit 180 may receive vehicle speed information, steering wheel rotation angle information, or gear shift information. The interface unit 180 may receive vehicle speed information, steering wheel rotation angle information, or gear shift information sensed through the vehicle's sensing unit 160. Alternatively, the interface unit 180 may receive vehicle speed information, steering wheel rotation angle information, or gear shift information from the vehicle control unit 170. Meanwhile, here, the gear shift information may be information about what state the vehicle's shift lever is in. For example, gear shift information may be information about which of the following states the shift lever is in: park (P), reverse (R), neutral (N), drive (D), and first to multi-gear states. .

The interface unit 180 may receive user input received through the user input unit 124 of the vehicle 100. The interface unit 180 may receive user input from the input unit 120 of the vehicle 100 or may receive it through the control unit 170.

The interface unit 180 may receive information obtained from an external device. For example, when traffic light change information is received from an external server through the communication unit 110 of the vehicle 100, the interface unit 180 may receive the traffic light change information from the control unit 170.

The control unit 170 may control the overall operation of each unit within the vehicle 100. The control unit 170 may be named ECU (Electronic Control Unit).

The control unit 170 is hardware-wise, including application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and processors. ), controllers, micro-controllers, microprocessors, and other electrical units to perform functions.

The power unit 190 may supply power required for the operation of each component under the control of the control unit 170. In particular, the power supply unit 170 may receive power from a battery (not shown) inside the vehicle.

An Audio Video Navigation (AVN) device can exchange data with the control unit 170. The control unit 170 may receive navigation information from an AVN device or a separate navigation device (not shown). Here, navigation information may include set destination information, route information according to the destination, map information or vehicle location information related to vehicle driving.

Meanwhile, some of the components shown in FIG. 1 may not be essential in implementing the vehicle 100. Accordingly, the vehicle 100 described herein may have more or fewer components than those listed above. For example, the vehicle 100 may further include a head-up display device 400, which will be described later.

FIG. 2 shows an exemplary exterior of the vehicle 100 shown in FIG. 1 .

Referring to FIG. 2, four cameras 161a, 161b, 161c, and 161d may be mounted at different positions on the exterior of the vehicle 100. Each of the four cameras 161a, 161b, 161c, and 161d may be the same as the camera 161 described above. A plurality of cameras 161a, 161b, 161c, and 161d may be placed in the front, left, right, and rear of the vehicle 100, respectively. Each of the plurality of cameras 161a, 161b, 161c, and 161d may be included in the camera 161 shown in FIG. 1.

The front camera 161a may be placed near the windshield, near the emblem, or near the radiator grill.

The left camera 161b may be placed in a case surrounding the left side mirror. Alternatively, the left camera 161b may be placed outside the case surrounding the left side mirror. Alternatively, the left camera 161b may be placed in an area outside the left front door, left rear door, or left fender.

The right camera 161c may be placed in a case surrounding the right side mirror. Alternatively, the right camera 161c may be placed outside the case surrounding the right side mirror. Alternatively, the right camera 161c may be placed in an area outside the right front door, right rear door, or right fender.

Meanwhile, the rear camera 161d may be placed near the rear license plate or trunk switch.

Each image captured by the plurality of cameras 161a, 161b, 161c, and 161d is transmitted to the control unit 170, and the control unit 170 can synthesize each image to generate an external image for each direction of the vehicle. . Alternatively, each image captured by the plurality of cameras 161a, 161b, 161c, and 161d is transmitted to the processor 470 of the head-up display device 400, which will be described later, and the processor 470 synthesizes each image , it is also possible to generate external images for each direction of the vehicle.

FIG. 3 shows an example of images generated by a plurality of cameras 161a, 161b, 161c, and 161d shown in FIG. 2.

Referring to FIG. 3 , the vehicle 100 can generate a composite image 300. The composite image 300 includes a first image area 301 corresponding to an external image captured by the front camera 161a, a second image area 302 corresponding to an external image captured by the left camera 161b, It may include a third image area 303 corresponding to an external image captured by the right camera 161c and a fourth image area 304 corresponding to an external image captured by the rear camera 161d. The composite image 300 may be named an Around View Monitoring (AVM) image.

When generating the composite image 300, boundary lines 311, 312, 313, and 314 are generated between any two external images included in the composite image 300. The vehicle 100 can be displayed naturally by performing image blending on the border lines 311, 312, 313, and 314.

Additionally, the center of the composite image 300 may include an image preset to indicate the vehicle 100. Additionally, the composite image 300 may be displayed on the head-up display device 400 mounted on the vehicle 100.

Figure 4 shows a block diagram of a head-up display device 400 according to an embodiment of the present invention.

Referring to FIG. 4, the head-up display device 400 may be mounted in a fixed or separate manner inside the vehicle 100. The fixed head-up display device 400 may be coupled and fixed to a certain space (eg, center fascia) allocated within the dashboard of the vehicle 100. The detachable head-up display device 400 may be mounted detachably on the dashboard of the vehicle 100 and, in some cases, may be detached from the vehicle 100 by the user.

As shown in FIG. 4, the head-up display device 400 includes a communication unit 410, an input unit 420, a memory 430, a display unit 440, an interface unit 450, a sensing unit 460, and a processor. It may include (470) and a power supply unit (480).

The communication unit 410 may include one or more modules that enable wireless communication with the vehicle 100 and other external devices (eg, mobile terminals, external servers). Additionally, the communication unit 410 may include one or more modules that connect the vehicle 100 to one or more networks. For example, the communication unit 410 can form a wireless link with a mobile terminal (eg, a smartphone) inside the vehicle 100 using various data communication methods such as Bluetooth and WiFi.

In addition, the communication unit 410 provides video and video according to various standards such as traffic information such as TPEG format, terrestrial wave, satellite digital multimedia broadcasting (DMB), digital audio broadcasting (DAB), and digital video broadcasting (DVB-T, DVB-H). Audio data can be received. For example, traffic information received by the communication unit 410 may include lane information, driving speed limit information, turn-by-turn information, traffic safety information, traffic guidance information, route finding information, etc.

Additionally, the communication unit 410 may generate location data of the vehicle 100 based on a global positioning system (GPS) signal received from a satellite. For example, the GPS signal is 802.11, a wireless network standard for wireless LAN including wireless LAN and some infrared communications proposed by IEEE (Institute of Electrical and Electronics Engineers), and wireless network including Bluetooth, UWB, ZigBee, etc. 802.15, a standard specification for PAN (Personal Area Network), and 802.16, a standard specification for wireless MAN (Metropolitan Area Network) (Broadband Wireless Access: BWA), including urban broadband networks (Fixed Wireless Access: FWA), etc. It may be received according to a wireless communication method such as 802.20, which is a standard for mobile Internet for wireless MAN (MBWA: Mobile Broadband Wireless Access) including Wibro, WiMAX, etc.

The input unit 420 may receive information or select at least one of a plurality of functions executable by the head-up display device 400 from the user. This input unit 420 may include various types of input means such as a keypad, touch screen, jog shuttle, button, and microphone.

The memory 430 can store map data and data and programs necessary for the head-up display device 400 to operate. The memory 430 may be a variety of hardware storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. The memory 430 may store various data for the overall operation of the head-up display device 400, such as programs for processing or controlling the processor 470.

Map data stored in the memory 430 may include geographic coordinates (geographic coordinates or longitude-latitude coordinates) indicating latitude and longitude in degree/minute/second units (DMS units: Degree/Minute/Second). At this time, the map data may use UTM (Universal Transverse Mercator) coordinates, UPS (Universal Polar System) coordinates, TM (Transverse Mercator) coordinates, etc. in addition to the geographical coordinates described above.

In addition, the memory 430 stores various menu screens, points of interest (POI: Point Of Interest, hereinafter referred to as 'POI'), and location-specific feature information (e.g., intersection, number of lanes, slope, road curvature) of map data. You can. Additionally, the memory 430 stores user interfaces related to various user inputs, vehicle 100 information, etc. Additionally, the memory 430 stores destination information input by the user through the input unit 420.

As will be described later, the display unit 440 may display a virtual screen including at least one indicator guiding driving information of the vehicle 100 on at least a portion of the projection surface provided in the vehicle 100.

The display unit 440 may project display light representing various types of information processed by the processor 470 onto a projection surface (eg, windshield, reflective film, combiner) provided in the vehicle 100. For example, the display unit 440 may output display light corresponding to driving information of the vehicle 100.

The display light that the display unit 440 outputs to implement a virtual screen on at least a portion of the projection surface is a liquid crystal display (LCD) or a thin film transistor-liquid crystal display (TFT LCD). , it may be formed by at least one of an organic light-emitting diode (OLED), a flexible display, a 3D display, and an electronic ink display.

The display unit 440 can display various contents, such as various screens and route information, using the user interface stored in the memory 430. Here, content displayed on the display unit 440 may include various text or image data (including map data and various information data), icons, menu lists, etc.

The interface unit 450 may receive data related to the vehicle 100 or provide signals processed or generated by the processor 470 to the outside. For example, the interface unit 450 may perform data communication with the sensing unit 160 and the control unit 170 of the vehicle 100 through wired or wireless communication.

The sensing unit 460 may include an indoor camera 461, an outdoor camera 462, a posture detection sensor 463, and a light detection sensor 464.

Images captured by the indoor camera 461 or the outdoor camera 462 may be provided to the processor 470. The indoor camera 461 may be placed inside the vehicle 100 to face the driver's seat and generate an image of the driver. The driver image may refer to an image including the face of the driver riding in the vehicle 100. Additionally, the outdoor camera 462 may be disposed to face the front of the vehicle 100 at various locations, such as one top side of the windshield, and generate a front image. In the front image provided from the outdoor camera 462, at least one object such as a pedestrian, traffic light, lane, or other vehicle existing in front of the vehicle 100 may appear.

In one embodiment, the sensing unit 460 detects any object that is close to the display unit 440 and displays a message corresponding to the position or movement (e.g., movement direction, movement speed, movement distance, type) of the detected object. Sensing information may be generated and the generated sensing information may be provided to the processor 470.

The posture detection sensor 463 can measure the posture of the vehicle 100. Specifically, the posture detection sensor 463 is one or more of a steering sensor, a speed sensor, a tilt sensor, a heading sensor, a yaw sensor, and a gyro sensor. The posture of the vehicle 100 can be measured based on the sensing data obtained by the sensor. For example, the posture sensor 463 may measure the forward/backward inclination, left/right inclination, steering angle, driving direction, etc. of the vehicle 100 and provide corresponding sensing data to the processor 470.

The light detection sensor 464 can detect light outside the vehicle 100. In particular, the light detection sensor 464 can detect the headlights of an oncoming vehicle. In this case, the opposing vehicle may be another vehicle approaching from the opposite side of the vehicle 100. The light detection sensor 464 may provide sensing data including the location of the detected headlight to the processor 470.

The processor 470 controls the overall operation of the head-up display device 400. The processor 470 may match the location data of the vehicle 100 to map data stored in the memory 430 and output an image corresponding to the matched map information through the display unit 440. Specifically, the processor 470 obtains the estimated location of the vehicle 100 based on the location data of the vehicle 100, and reads map data corresponding to the estimated location and driving path of the vehicle 100 from the memory 430. It can be displayed on the display unit 440. That is, the processor 470 can generate route guidance information based on map information and output the generated route guidance information using the display unit 440.

Additionally, the processor 470 is connected to a base station near the vehicle 100 through the communication unit 410 and can transmit and receive traffic information and accident information around the vehicle 100.

Additionally, the processor 470 may display external images received from the cameras 161 and 462 placed outside the vehicle 100 on the display unit 440 through the interface unit 450.

When the POI search menu is selected by the user, the processor 470 can search for POIs located on a path from the current location to the destination and display the searched POI-related information on the display unit 440.

The processor 470 may change at least one of the position and size of the virtual screen displayed by the display unit 440 based on the movement information of the vehicle 100.

The power supply unit 480 may supply power required for the operation of each component included in the head-up display device 400 under the control of the processor 470. For example, the power supply unit 480 may use power stored in a battery built in the head-up display device 400 to supply power necessary for the operation of each component included in the head-up display device 400. Alternatively, the power unit 480 is connected to a USB port provided in the vehicle 100 through a connector, and includes power supplied from a battery (not shown) inside the vehicle 100 in the head-up display device 400. The power required for the operation of each component can be supplied.

Meanwhile, some of the components shown in FIG. 4 may not be essential in implementing the head-up display device 400. Accordingly, the head-up display device 400 described in this specification may have more or fewer components than those listed above.

FIG. 5 shows an example of an internal block diagram of the processor 470 shown in FIG. 4.

Referring to FIG. 5, the processor 470 may include an image preprocessor 510, a disparity calculation unit 520, an object detection unit 534, an object tracking unit 540, and an application unit 550. .

The image preprocessor 510 may receive images provided from the cameras 122, 161, 461, and 462 shown in FIG. 1 and/or 4 and perform preprocessing.

Specifically, the image preprocessor 510 performs noise reduction, rectification, calibration, color enhancement, and color space conversion (CSC) on the image. , interpolation, camera gain control, etc. can be performed. Accordingly, it is possible to obtain clearer images than stereo images captured by the cameras 122, 161, 461, and 462.

The disparity calculator 520 receives the image signal processed in the image preprocessor 510, performs stereo matching on the received images, and calculates the disparity according to the stereo matching. You can obtain a map (dispartiy map). In other words, disparity information about the stereo image in front of the vehicle can be obtained.

At this time, stereo matching may be performed on a per-pixel basis or a predetermined block basis of stereo images. Meanwhile, the disparity map may refer to a map that numerically represents binocular parallax information of a stereo image, i.e., left and right images acquired at the same time point.

The segmentation unit 532 may perform segmentation and clustering on at least one of the images based on disparity information from the disparity calculation unit 520.

Specifically, the segmentation unit 532 may separate the background and foreground for at least one stereo image based on disparity information.

For example, an area in the disparity map where disparity information is below a predetermined value can be calculated as the background and the corresponding part can be excluded. By this, the foreground can be relatively separated.

As another example, an area in the disparity map where disparity information is greater than a predetermined value can be calculated as the foreground and the corresponding portion can be extracted. By this, the foreground can be separated.

In this way, by separating the foreground and background based on the disparity information extracted based on the stereo image, it is possible to shorten the signal processing speed and amount of signal processing when detecting an object later.

Next, the object detector 534 may detect the object based on the image segment from the segmentation unit 532.

That is, the object detector 534 may detect an object for at least one of the images based on disparity information.

Specifically, the object detector 534 may detect an object for at least one of the images. For example, objects can be detected from the foreground separated by image segments.

Next, the object verification unit 536 can classify and verify the separated objects.

For this purpose, the object confirmation unit 536 uses an identification method using a neural network, a Support Vector Machine (SVM) technique, an identification technique by AdaBoost using Haar-like features, or a Histograms of Oriented Gradients (HOG) technique. etc. can be used.

Meanwhile, the object confirmation unit 536 may check the object by comparing the detected object with the objects stored in the memory 130.

For example, the object confirmation unit 536 may check surrounding vehicles, lanes, road surfaces, signs, dangerous areas, tunnels, etc. located around the vehicle.

The object tracking unit 540 may perform tracking on the identified object. For example, sequentially, the object in the acquired stereo images is identified, the motion or motion vector of the identified object is calculated, and the movement of the object is tracked based on the calculated motion or motion vector. You can. Accordingly, it is possible to track surrounding vehicles, lanes, road surfaces, signs, dangerous areas, tunnels, etc. located around the vehicle.

Next, the application unit 550 may calculate the risk level of the vehicle 100 based on various objects (e.g., other vehicles, lanes, road surfaces, signs, etc.) located around the vehicle 100. . In addition, it is possible to calculate the possibility of a collision with the car in front and whether the vehicle will slip.

Additionally, the application unit 550 may output a message to inform the user of such information, as vehicle driving assistance information, based on the calculated risk, possibility of collision, or slippage. Alternatively, a control signal for posture control or driving control of the vehicle 100 may be generated as vehicle control information.

Meanwhile, depending on the embodiment, the processor 470 includes an image preprocessor 510, a disparity calculation unit 520, a segmentation unit 532, an object detection unit 534, an object confirmation unit 536, and an object tracking unit ( It may include only part of the 540) and the application unit 550. For example, if the cameras 122, 161, 461, and 462 are cameras that provide only two-dimensional images, the disparity calculation unit 520 may be excluded.

FIGS. 6A and 6B are diagrams referenced in explaining the operation of the processor 470 shown in FIG. 5 .

FIGS. 6A and 6B are diagrams referenced for explaining the operation method of the processor 470 of FIG. 5 based on stereo images obtained in the first and second frame sections, respectively.

First, referring to FIG. 6A, when the camera 161 is a stereo camera, the camera 161 acquires a stereo image during the first frame period.

The disparity calculation unit 520 in the processor 470 receives the stereo images FR1a and FR1b signal-processed by the image preprocessor 510 and performs stereo matching on the received stereo images FR1a and FR1b. , obtain a disparity map (620).

The disparity map 620 levels the disparity between the stereo images FR1a and FR1b. The larger the disparity level, the closer the distance to the vehicle, and the smaller the disparity level, the closer to the vehicle. It can be calculated based on the distance.

Meanwhile, when displaying such a disparity map, the larger the disparity level, the higher the luminance, and the smaller the disparity level, the lower the luminance.

In the drawing, in the disparity map 620, the first to fourth lanes 628a, 628b, 628c, and 628d each have corresponding disparity levels, and the construction area 622 and the first vehicle ahead 624 ), illustrating that the second front vehicle 626 each has a corresponding disparity level.

The segmentation unit 532, the object detection unit 534, and the object confirmation unit 536 perform segment, object detection, and object detection for at least one of the stereo images FR1a and FR1b based on the disparity map 620. Perform verification.

The drawing illustrates that object detection and confirmation are performed on the second stereo image FR1b using the disparity map 620.

That is, in the image 630, the first to fourth lanes 638a, 638b, 638c, and 638d, the construction area 632, the first vehicle 634, and the second vehicle 636 are detected as objects. and verification can be performed.

Next, referring to FIG. 6B, during the second frame period, the stereo camera 161 acquires a stereo image.

The disparity calculation unit 520 in the processor 470 receives stereo images (FR2a, FR2b) that have been signal processed in the image pre-processor 510, and performs stereo matching on the received stereo images (FR2a, FR2b). , obtain a disparity map (640).

In the drawing, in the disparity map 640, the first to fourth lanes 648a, 6 48b, 648c, and 648d each have corresponding disparity levels, and the construction area 642 and the first vehicle ahead ( 644), illustrating that each of the second front vehicles 646 has a corresponding disparity level.

The segmentation unit 532, the object detection unit 534, and the object confirmation unit 536 perform segment, object detection, and object detection for at least one of the stereo images FR2a and FR2b based on the disparity map 640. Perform verification.

The drawing illustrates that object detection and confirmation are performed on the second stereo image FR2b using the disparity map 640.

That is, in the image 650, the first to fourth lanes 658a, 658b, 658c, and 658d, the construction area 652, the first vehicle ahead 654, and the second vehicle ahead 656 are used for object detection and Verification can be performed.

Meanwhile, the object tracking unit 540 may compare the stereo image of FIG. 6A and the stereo image of FIG. 6B and perform tracking on the identified object.

Specifically, the object tracking unit 540 may track the movement of the corresponding object based on the movement or motion vector of each object identified in FIGS. 6A and 6B. Accordingly, it is possible to perform individual tracking of objects located around the vehicle, such as lanes, construction areas, the first vehicle ahead, and the second vehicle ahead.

FIGS. 7A and 7B are conceptual diagrams for explaining the structure and operation of the display unit 440 included in the head-up display device 400 according to an embodiment of the present invention.

First, referring to FIG. 7A, the display unit 440 may include a display panel 442, a reflector 444, and a posture adjustment unit 446.

The display panel 442 is equipped with a back light unit and can output display light corresponding to driving information of the vehicle 100 under the control of the processor 470.

In one embodiment, the display panel 442 may project display light corresponding to driving information to the reflector 444 based on image data provided from the processor 470.

The reflector 444 may include at least one mirror. For example, as shown, the reflector 444 may include a first mirror 444a and a second mirror 444b.

The first mirror 444a may reflect display light output from the display panel 442 to the second mirror 444b. That is, the display light output from the display panel 442 may be refracted by the first mirror 444a and provided to the second mirror 444b. At this time, the first mirror 444a may be a flat mirror.

The second mirror 444b re-reflects the display light reflected from the first mirror 444a onto the projection surface 100, thereby displaying the virtual screen 720 on the projection surface 101. For example, as shown, the projection surface 101 may be a windshield, which is the windshield of the vehicle 100. At this time, the second mirror 444b may be a concave mirror with a predetermined curvature.

At this time, the virtual screen 720 may be an area in which the virtual image 730 of the display light reflected by the reflector 444 is displayed among the entire area of the projection surface 101. In some cases, the virtual screen 720 and the virtual image 730 may be named the same.

Meanwhile, a reflective film may be attached to the projection surface 101 so that the virtual screen 720 is displayed more clearly by the display light reflected by the second mirror 444b.

The size, projection angle, and light quantity of at least a portion of the virtual screen 720 displayed on the projection surface 101 by the display light by the optical path from the display panel 442 to the windshield 101, Brightness, etc. can be adjusted.

The processor 470 is functionally connected to the indoor camera 461, the outdoor camera 462, and the display panel 442, and generates a virtual image based on the image provided from the indoor camera 461 and/or the outdoor camera 462. Image data for forming 730 may be generated and provided to the display panel 442. For example, the image data may be in a bitmap format.

In one embodiment, the processor 470 detects a specific object 710 existing in front of the vehicle 100 based on the front image provided from the outdoor camera 462 and corresponds to the detected object 710. video data may be provided to the display panel 442.

At this time, a virtual image 730 appears on the virtual screen 720 displayed on the projection surface 101 by the reflected display light, and in a predetermined range (e.g., the position of the user seated in the driver's seat) within the vehicle 100, it appears as if The virtual image 730 may appear to be displayed in front of the vehicle 100 beyond the projection surface 101. In other words, the virtual image 730 may be perceived by the driver as if it is floating a predetermined distance in front of the vehicle 100. For example, the virtual image 730 may be a graphic object that provides information about the outline of the object 710, driving speed, collision warning, etc.

The posture adjustment unit 446 is directly or indirectly connected to one side of the second mirror 444b included in the reflection unit 444, and is configured to adjust at least one of the posture, that is, the position and the rotation angle, of the second mirror 444b. You can. For example, the posture control unit 446 may include at least one motor (eg, step motor). The motor generates a driving force corresponding to a control signal provided from the processor 470, and the second mirror 444b can rotate up and down, left and right, or its front and rear position can be adjusted by the driving force provided from the motor. Meanwhile, the posture adjustment unit 446 may be omitted from the display unit 440 depending on the embodiment.

In one embodiment, the processor 470 may detect the driver's gaze direction from the driver's image provided from the indoor camera 461. In one embodiment, the processor 470 detects the driver's eyes appearing in the driver image using an eye tracking technique, and detects the driver's gaze direction based on the position and movement of the detected eyes. You can.

Based on the detected gaze direction, the processor 470 changes the position or size of at least a portion of the virtual screen 720, or applies the time to at least a portion of the entire area of the virtual screen 720 corresponding to the gaze direction. effect can be determined. For example, the processor 470 may provide the posture adjustment unit 446 with a target rotation angle corresponding to the detected gaze direction. The posture adjusting unit 446 may change the posture of the second mirror 444b by the target rotation angle. As the posture of the second mirror 444b changes, the position or size of the virtual screen 720 may change.

Meanwhile, Figure 7a shows the display unit 440 including one display panel 442, a first mirror 444a, a second mirror 444b, and a posture adjustment unit 446, but the display unit 440 is not limited thereto. .

For example, the head-up display device 400 may include two or more display units 440. In this case, one display unit 440 may display a virtual screen with a different projection distance (or focal length) than that of the other display unit 440.

When the head-up display device 400 includes two or more display units 440 that display virtual screens of different projection distances (or focal lengths), the head-up display device 400 includes a plurality of display units 440 can be controlled individually.

Specifically, the processor 470 displays a first virtual screen at a first projection distance through one display unit 440, and displays a second virtual screen at a second projection distance through another display unit 440. You can.

For example, assume that a first object (eg, a pedestrian) is located on a first street in front of the vehicle, and a second object (eg, another vehicle) is located on a second street in front of the vehicle. In this case, the processor displays a first virtual screen guiding information about the first object through one display unit 440, and a second virtual screen guiding information about the second object through the other display unit 440. A virtual screen can be displayed.

FIG. 7B illustrates various types of information guided by the virtual screen 720 shown in FIG. 7A.

The processor 470 divides the virtual screen 720 into a plurality of sub-screens 721-724 and controls the display unit 440 to display at least one indicator guiding information of a different category for each sub-screen. can do. To aid understanding, Figure 7b shows the virtual screen 720 divided into four sub-screens 721-724.

Specifically, an indicator based on augmented reality technology that guides the driving path of the vehicle 100 or obstacles ahead may be displayed on the first sub-screen 721. For example, as shown, a turn-by-turn indicator 721a and a collision prevention indicator 721b may be displayed on the first sub-screen 721. The turn-by-turn indicator 721a may be displayed in a manner that overlaps the road surface on the driving path of the vehicle 100. In this case, the processor 470 may change the size, shape, color, etc. of the turn-by-turn indicator 721a based on road information and route data. The collision avoidance indicator 721b may be displayed in a way that it overlaps an obstacle around the vehicle 100 and an empty space between the vehicle 100. Of course, if there are no obstacles around the vehicle 100 or there is no risk of collision with the vehicle 100, the collision avoidance indicator 721b may be omitted.

A blind spot indicator 722a, a lane departure warning indicator 722b, etc. may be displayed on the second sub screen 722. The blind spot indicator 722a can guide objects existing in the blind spot that are not visible to the driver of the vehicle 100. The lane departure warning indicator 722b may be displayed when the vehicle 100 crosses the left or right lane without turning on the turn signal.

A driving speed indicator 723a and a speed limit 723b may be displayed on the third sub screen 723. The driving speed indicator 723a guides the current speed of the vehicle 100, and the speed limit 723b guides the speed limit of the road on which the vehicle 100 is currently located. The speed limit may be included in map data stored in the memory 430, etc., or may be received through the communication unit 410.

The fourth sub screen 724 may display route guidance indicators 724a and 724b for the route to the destination.

Meanwhile, the shape of the virtual screen 720 and the types of indicators included therein are not limited to the example shown in FIG. 7B, and it should be understood that various other modifications are possible.

Figure 8 shows an exemplary process S800 performed by the head-up display device 400 according to one embodiment of the present invention.

In step S810, the processor 470 may display a virtual screen on a projection surface provided in the vehicle 100 through the display unit. At this time, the virtual screen may include at least one indicator guiding various driving information.

In step S820, the processor 470 may receive movement information of the vehicle 100. In this case, the motion information may include at least tilt information of the vehicle 100. As an example, the motion information received by the processor 470 may be provided from the sensing unit 160 of the vehicle 100. As another example, the movement information received by the processor 470 may be provided from the posture detection sensor 464.

In step S830, the processor 470 may determine whether the front-to-back inclination of the vehicle 100 included in the received motion information is greater than or equal to the first reference value. The first reference value may be a positive number (eg, 10 degrees). Here, the first reference value compared with the front-to-back inclination of the vehicle 100 may be previously stored in the memory 430 and may be changed by the user of the vehicle 100. For example, at the point when the front wheels of the vehicle 100 pass the bump, it may be determined that the front-to-back inclination of the vehicle 100 is greater than or equal to the first reference value. For another example, when the vehicle 100 is driving uphill, the front-to-back inclination of the vehicle 100 may be determined to be greater than or equal to the first reference value. If the front-to-back inclination of the vehicle 100 is greater than or equal to the first reference value, step S840 may be performed.

In step S840, the processor 470 may change the position of the virtual screen currently being displayed to the bottom based on the front-to-back tilt of the vehicle 100 that is greater than or equal to the first reference value. That is, the processor 470 can move the virtual screen a predetermined distance to the bottom along the z-axis (see FIG. 7A).

For example, when the vehicle 100 passes a bump or enters an uphill road, a situation may occur where the front of the vehicle 100 is higher than the rear of the vehicle. In this case, if the position of the virtual screen is fixed, a phenomenon occurs in which the driver's field of view is located lower than the virtual screen, and as the front-to-back tilt of the vehicle 100 increases in the positive (+) direction, the virtual screen and The difference between the driver's field of vision increases. At this time, the field of view may mean the range that a human can look at without changing the head position and eye position.

The processor 470 increases the amount of change in the position of the virtual screen as the front-to-back inclination of the vehicle 100 increases, thereby reducing the difference between the position of the virtual screen and the driver's field of view even in situations where there is a front-to-back inclination in the (+) direction. can do.

In step S850, the processor 470 may determine whether the front-to-back inclination of the vehicle 100 included in the received motion information is less than or equal to the second reference value. The second reference value may be a negative number (eg, -10 degrees). Here, the second reference value compared to the front-to-back inclination of the vehicle 100 may be previously stored in the memory 430 and may be changed by the user of the vehicle 100. For example, at the point when the rear wheels of the vehicle 100 pass the bump, the front-to-back inclination of the vehicle 100 may be determined to be less than or equal to the second reference value. For another example, when the vehicle 100 is driving down a hill, the front-to-back inclination of the vehicle 100 may be determined to be less than or equal to the second reference value. If the front-to-back inclination of the vehicle 100 is less than or equal to the second reference value, step S860 may be performed.

In step S860, the processor 470 may change the position of the virtual screen currently being displayed to the top based on the front-to-back inclination of the vehicle 100 that is less than or equal to the second reference value. That is, the processor 470 can move the virtual screen a predetermined distance to the top along the z-axis (see FIG. 7).

For example, when the rear wheels of the vehicle 100 pass a bump or enter a downhill road, a situation may occur where the rear portion of the vehicle 100 is higher than the front portion of the vehicle body. In this case, if the position of the virtual screen is fixed, a phenomenon occurs in which the driver's field of view is located above the virtual screen, and as the front-to-back tilt of the vehicle 100 increases in the negative (-) direction, the virtual screen and The difference in the driver's field of vision increases.

The processor 470 increases the amount of change in the position of the virtual screen as the front-to-back inclination of the vehicle 100 increases in the negative (-) direction, so that even in a situation where the front-to-back inclination in the (-) direction exists, the position of the virtual screen The difference between the driver's field of view and the driver's field of view can be reduced.

FIGS. 9A and 9B are diagrams referenced to explain the operation of the head-up display device 400 according to an embodiment of the present invention to change the position of the virtual screen based on the forward-backward tilt of the vehicle 100. .

FIG. 9A illustrates a change in the position of the virtual screen 921 while the front wheels of the vehicle 100 pass the bump 910. Before the vehicle 100 goes over the bump 910, the virtual screen 921 may be displayed at a predetermined reference position (R1) within the projection surface 101.

As shown, when the front wheels of the vehicle 100 pass a portion protruding from the ground, such as a bump 910, the front portion of the vehicle body 100 becomes higher than the rear portion of the vehicle body. That is, the front-to-back inclination of the vehicle 100 may increase beyond the first reference value (see FIG. 8).

When the front-to-back inclination of the vehicle 100 is greater than or equal to the first reference value (see FIG. 8) while the virtual screen 921 is displayed at the reference position (R1), the virtual screen 921 is displayed more than the driver's field of view of the vehicle 100. Because it is located relatively above, it may not be easy for the driver to check the information provided through the virtual screen 921. For example, as shown, the virtual screen 921 displayed at the reference position R1 may be excessively spaced from the ground.

In this case, the processor 470 may set the target position A1 based on the forward-forward tilt included in the motion information. To facilitate understanding, the reference position (R1) is shown as a solid line, and the target position (A1) is shown as a dotted line.

For example, the target position (A1) may be lower than the reference position (R1), and as the front-to-back slope increases in the positive (+) direction, the difference between the target position (A1) and the reference position (R1) may increase. there is. That is, by moving the virtual screen 921 from the reference position R1 to the lower target position A1, the consistency between the driver's field of view and the virtual screen 921 can be improved.

In this case, the processor 470 may adjust the moving speed of the virtual screen 921 based on the difference between the target position (A1) and the reference position (R1). For example, as the difference between the target position A1 and the reference position R1 increases, the moving speed of the virtual screen 921 can be increased. As an example, the processor 470 increases the rotation speed of the motor of the posture control unit 446 as the difference between the target position (A1) and the reference position (R1) increases, so that the virtual screen 921 moves to the reference position (R1). It can be controlled to move quickly from to the target location (A1).

Meanwhile, when the front wheels of the vehicle 100 pass the bump 910 and the front-to-back inclination of the vehicle 100 returns to less than the first reference value, the processor 470 displays the virtual screen 921 at the reference position (R1). ) can be moved again.

Compared to FIG. 9A, FIG. 9B illustrates a change in the position of the virtual screen 921 while the rear wheels of the vehicle 100 pass the bump 910.

As shown, when the rear wheels of the vehicle 100 pass a part protruding from the ground, such as a bump 910, the rear portion of the vehicle body 100 becomes higher than the front portion of the vehicle body. That is, the front-to-back inclination of the vehicle 100 may decrease below the second reference value (see FIG. 8).

When the front-to-back inclination of the vehicle 100 is less than or equal to the second reference value (see FIG. 8) while the virtual screen 921 is displayed at the reference position (R1), the virtual screen 921 is larger than the driver's field of view of the vehicle 100. Because it is located relatively lower, it may not be easy for the driver to check the information provided through the virtual screen 921. For example, as shown, the virtual screen 921 displayed at the reference position R1 may be located too close to the ground and may not be clearly visible to the driver.

In this case, the processor 470 may set the target position A2 based on the forward-forward tilt included in the motion information. To aid understanding, the target location (A2) is shown as a dotted line. For example, the target position (A2) may be higher than the reference position (R1), and as the front-to-back slope increases in the negative (-) direction, the difference between the target position (A2) and the reference position (R1) may increase. there is. That is, by moving the virtual screen 921 from the reference position R1 to the upper target position A2, the consistency between the driver's field of view and the virtual screen 921 can be improved.

In this case, similar to what was described above with respect to FIG. 9A, the processor 470 may adjust the moving speed of the virtual screen 921 based on the difference between the target position A2 and the reference position R1. For example, as the difference between the target position A2 and the reference position R1 increases, the moving speed of the virtual screen 921 can be increased.

FIG. 9C illustrates a change in the position of the virtual screen 921 when the vehicle 100 is driving on an uphill road 941. Before the vehicle 100 enters the uphill road 941, for example, on flat ground, the virtual screen 921 may be displayed at a predetermined reference position R1 within the projection surface 101.

As shown, when the vehicle 100 is going uphill 941, the front portion of the vehicle body 100 becomes higher than the rear portion of the vehicle body. That is, similar to FIG. 9A, the front-to-rear inclination of the vehicle 100 may increase beyond the first reference value (see FIG. 8).

When the front-to-back inclination of the vehicle 100 is greater than or equal to the first reference value (see FIG. 8) while the virtual screen 921 is displayed at the reference position (R1), the virtual screen 921 is displayed more than the driver's field of view of the vehicle 100. Because it is located relatively above, it may not be easy for the driver to check the information provided through the virtual screen 921. For example, as shown, the virtual screen 921 displayed at the reference position R1 may be excessively spaced from the ground. Accordingly, it may be difficult for the driver to check the front obstacle 931, such as a pedestrian.

In this case, the processor 470 may set the target position A1 based on the forward-forward tilt included in the motion information. At this time, the target position A1 is the same as that already described above with reference to FIG. 9A, and detailed description thereof will be omitted.

Meanwhile, when the vehicle 100 passes all of the uphill roads 941 and the front-to-back inclination of the vehicle 100 returns to less than the first reference value, the processor 470 moves the virtual screen 921 to the reference position (R1). It can be moved again.

Compared to FIG. 9C, FIG. 9D illustrates a change in the position of the virtual screen 921 when the vehicle 100 is traveling on a downhill road 942. Before the vehicle 100 enters the downhill road 942, for example, on flat ground, the virtual screen 921 may be displayed at a predetermined reference position (R1) within the projection surface.

As shown, when the vehicle 100 goes down the downhill road 942, the rear portion of the vehicle body 100 becomes higher than the front portion of the vehicle body. That is, similar to FIG. 9B, the front-to-rear inclination of the vehicle 100 may increase below the second reference value (see FIG. 8).

When the front-to-back inclination of the vehicle 100 is less than or equal to the second reference value (see FIG. 8) while the virtual screen 921 is displayed at the reference position (R1), the virtual screen 921 is larger than the driver's field of view of the vehicle 100. Because it is located relatively lower, it may not be easy for the driver to check the information provided through the virtual screen 921. For example, as shown, the virtual screen 921 displayed at the reference position R1 may appear very close to the ground. Accordingly, it may be difficult for the driver to check the front obstacle 932, such as a pedestrian.

In this case, the processor 470 may set the target position A2 based on the forward-forward tilt included in the motion information. At this time, the target position A2 is the same as that already described above with reference to FIG. 9B, and detailed description thereof will be omitted.

Meanwhile, when the vehicle 100 passes all of the downhill road 942 and the front-to-back inclination of the vehicle 100 returns to a value greater than the second reference value, the processor 470 sets the virtual screen 921 to the reference position ( It can be moved back to R1).

Figure 10 shows an exemplary process S1000 performed by the head-up display device 400 according to an embodiment of the present invention.

In step S1010, the processor 470 may display a virtual screen on the projection surface provided in the vehicle 100 through the display unit 440. At this time, the virtual screen may include at least one indicator guiding various driving information.

In step S1020, the processor 470 may receive movement information and road information of the vehicle 100. In this case, the motion information may include at least the steering angle of the vehicle 100. Here, the steering angle of the vehicle 100 may correspond to the rotation direction and rotation angle of the steering wheel provided in the vehicle 100. The motion information received by the processor 470 may be provided from the sensing unit 160 of the vehicle 100. For example, the steering angle of the vehicle 100 may be measured in real time or periodically by the steering angle sensor of the steering driver 152 and provided to the processor 470. Alternatively, the movement information received by the processor 470 may be provided from the posture detection sensor 464.

Additionally, the road information may include at least one of (i) a front image captured by the outdoor camera 462, (ii) route data corresponding to the current location of the vehicle 100, and (iii) map data. At this time, the route data may be data that guides the route from the current location of the vehicle 100 to a preset destination.

For example, the processor 470 detects the road on which the vehicle 100 is traveling in the front image captured by the outdoor camera 462, and detects characteristics of the detected road (e.g., number of lanes, curvature, lane type, lane width). , shape of surrounding facilities), and then match it with map data to calculate the location or driving direction of the vehicle 100.

In step S1030, the processor 470 may determine the driving lane of the vehicle 100 based on the steering angle and road information of the vehicle 100. In one embodiment, when the vehicle 100 is driving on a multi-lane road including a plurality of lanes, the processor 470 determines which lane the vehicle 100 should move from the current driving lane based on the steering angle and path data. You can decide whether to move. For example, when the same steering angle is received, the processor 470 determines whether the driver intended to change route (i.e., turn left, turn right, or U-turn) or change lane, based on the route data of vehicle 100. It can be interpreted. As another example, when there is a left turn lane and a straight lane in front of the vehicle 100, and the route data of the vehicle 100 indicates a left turn, the processor 470 determines the left turn lane as the expected driving lane of the vehicle 100. And, if the route data of the vehicle 100 indicates going straight, the processor 470 may determine the straight driving lane as the vehicle 100's scheduled driving lane.

In step S1040, the processor 470 may change the position of the virtual screen in the left and right directions based on the position of the lane in which the vehicle 100 is scheduled to travel.

In one embodiment, when the scheduled travel lane is located on the left side of the vehicle 100, the processor 470 moves the virtual screen to the left, and when the scheduled travel lane is located on the right side of the vehicle 100, the processor 470 ( 470) can move the virtual screen to the right. In this case, the processor 470 may calculate the distance to move the virtual screen to the right based on the position difference between the vehicle 100 and the lane in which it is scheduled to travel. As an example, the processor 470 may increase the moving distance of the virtual screen in proportion to the difference in position between the vehicle 100 and the lane in which it is scheduled to travel. As another example, the processor 470 may calculate the moving distance of the virtual screen based on the difference between the driving direction of the vehicle 100 and the direction of the planned driving lane.

In one embodiment, the processor 470 may change the position of the virtual screen in the left and right directions based on whether the virtual screen matches the scheduled driving lane. For example, if the virtual screen deviates from the left side of the lane in which the vehicle is scheduled to be driven, the processor 470 may move the position of the virtual screen to the right. As another example, when the virtual screen deviates from the right side of the lane in which the vehicle is scheduled to be driven, the processor 470 may move the position of the virtual screen to the left.

Figure 11 shows an example of an operation in which the head-up display device 400 changes the position of the virtual screen in the left and right directions according to an embodiment of the present invention.

Figure 11 shows a top view of a road with a junction dividing into a first lane 1111 and a second lane 1112.

Referring to FIG. 11 , the processor 470 may detect the first lane 1111 and the second lane 1112 based on the front image provided from the outdoor camera 462. For example, the processor 470 may calculate the width, position, or curvature of each of the detected first lane 1111 and the second lane 1112 based on the front image. Additionally, the route data of the vehicle 100 may indicate driving on the second lane 1112.

When the driver of the vehicle 100 turns the steering wheel to the right so that the vehicle 100 drives in the second lane 1112, the steering wheel manipulation amount is determined by the posture detection sensor 463 or the steering angle sensor of the steering drive unit 152. The steering angle corresponding to can be measured. Accordingly, the processor 470 may set the second lane 1112 as the travel lane based on the steering angle and path data.

Meanwhile, when the position control operation of the virtual screen 1121 is not performed, the processor 470 may display the virtual screen 1121 at the reference position R1. In this case, as shown, the driver may perceive the virtual screen 1121 as being displayed in an area that does not correspond to the second lane 1112. In other words, although the virtual screen 1121 must be displayed aligned with the area within the second lane 1112, the virtual screen 1121 visible to the driver deviates to the left side of the second lane 1112, which causes confusion for the driver. can be aggravated.

The processor 470 determines the target position (A11) of the virtual screen 1121 based on the position of the second lane 1112, which is the expected driving lane of the vehicle 100, and the steering angle (or driving direction) of the vehicle 100. It can be calculated. For example, as shown, when the virtual screen 1121 of the reference position R1 is displayed on the left side of the second lane 1112, the target position A11 may be to the right of the reference position R1. The processor 470 can move the virtual screen 1121 from the reference position R1 to the target position A11, and accordingly, the driver of the vehicle 100 can move the virtual screen 1121 to the area of the second lane 1112. (1121) can be provided.

Meanwhile, in FIG. 11, the description is made based on a situation in which the virtual screen 1121 deviates from the scheduled driving lane 1112 to the left, but in the opposite case, the position of the virtual screen 1121 can be changed to the left and right in a similar manner. This is obvious to those skilled in the art.

Figure 12 shows another example of an operation in which the head-up display device 400 changes the position of the virtual screen in the left and right directions according to an embodiment of the present invention.

Figure 12 shows a top view of a road including first to third lanes 1211-1213. As shown, it is assumed that the first to third lanes 1211-1213 are all straight lanes, and the vehicle 100 is driving within the first lane 1211.

Referring to FIG. 12 , the processor 470 may detect first to third lanes 1211 to 1213 based on the front image provided from the outdoor camera 462. For example, the processor 470 may calculate the width, position, or curvature of each of the detected first to third lanes 1211 to 1213 based on the front image. Additionally, the route data of the vehicle 100 may indicate driving in the second lane 1212.

When the driver of the vehicle 100 turns the steering wheel to the right so that the vehicle 100 moves from the first lane 1211 to the second lane 1212, the steering angle of the posture detection sensor 463 or the steering driver 152 The steering angle corresponding to the amount of manipulation of the steering wheel may be measured by the sensor. Accordingly, the processor 470 may set the second lane 1212 as the travel lane based on the steering angle and path data.

Meanwhile, when the position control operation of the virtual screen 1221 is not performed, the processor 470 may display the virtual screen 1221 at the reference position R1. In this case, as shown, due to the projection distance of the virtual screen 1221, the driver perceives the virtual screen 1221 to be displayed in an area corresponding to the third lane 1213, not the second lane 1212. It can be. In other words, although the virtual screen 1221 should be displayed aligned with the area within the second lane 1212, the virtual screen 1221 visible to the driver deviates to the right side of the second lane 1212, which causes confusion for the driver. can be aggravated.

The processor 470 determines the target position (A12) of the virtual screen 1221 based on the position of the second lane 1212, which is the expected driving lane of the vehicle 100, and the steering angle (or driving direction) of the vehicle 100. It can be calculated. For example, as shown, when the virtual screen 1221 of the reference position R1 is displayed on the right side of the second lane 1212, the target position A12 may be to the left of the reference position R1. The processor 470 may move the virtual screen 1221 to the left from the reference position R1 to the target position A12, and accordingly, the driver of the vehicle 100 may move the virtual screen 1221 to the left in the area of the second lane 1212. A virtual screen 1221 may be provided.

In FIG. 12 , the description is made based on a situation where the virtual screen 1221 deviates from the scheduled driving lane 1212 to the right. However, in the opposite case, it is known to those skilled in the art that the position of the virtual screen 1221 can be changed to the left and right using a similar method. It is self-explanatory.

Meanwhile, comparing FIGS. 11 and 12, even when the steering angle in the same direction is measured, the processor 470 determines the virtual The target position of the screen 1221 can be set in the opposite direction.

Meanwhile, it has been revealed through travel research that as the driving speed of the vehicle 100 increases, the driver's viewing angle narrows. As the driving speed of the vehicle 100 increases, the driver's field of view narrows, so the driver looks further ahead than when the driving speed is relatively slow. Figure 13 explains the operation of controlling a virtual screen through a method that takes into account the above-described human visual characteristics.

FIG. 13 shows an exemplary process S1300 performed by the head-up display device 400 according to an embodiment of the present invention.

In step S1310, the processor 470 may display a virtual screen on the projection surface provided in the vehicle 100 through the display unit. At this time, the virtual screen may include at least one indicator guiding various driving information. For example, a driving speed indicator of the vehicle 100 may be displayed on the virtual screen.

In step S1320, the processor 470 may receive movement information of the vehicle 100. In this case, the motion information may include at least the driving speed of the vehicle 100. Here, the driving speed of the vehicle 100 may be measured by the speed sensor of the vehicle 100.

Depending on the embodiment, the motion information may further include the speed limit of the road on which the vehicle 100 is traveling. At this time, the speed limit information may be included in map data or may be received by the communication unit 410. Alternatively, the processor 470 detects a traffic sign indicating the speed limit based on the image provided from the outdoor camera 462, and obtains speed limit information from the detected traffic sign using TSR (Traffic Sign Recognition) technology. can do.

In step S1330, the processor 470 may determine whether the traveling speed is greater than or equal to the threshold speed. Here, the threshold speed is a value previously stored in the memory 130, and may be set by the user or set as default at the time of shipment. Alternatively, the threshold speed may be input by the driver of the vehicle 100. Alternatively, the threshold speed may be periodically updated by the processor 470 depending on driver status, vehicle 100 status, road conditions, traffic congestion, etc.

For example, the processor 470 may determine whether the driver is drowsy driving based on the image provided from the indoor camera 461, and reduce the critical speed if the driver is drowsy driving.

As another example, the processor 470 may reduce the critical speed when a failure of a tire or engine of the vehicle 100 is detected.

As another example, the processor 470 may adjust the critical speed based on the speed limit of the road on which the vehicle 100 is traveling. As an example, the speed limit and the critical speed may be proportional.

As another example, the processor 470 may calculate the friction coefficient of the road surface based on the slip angle of the vehicle 100 and adjust the critical speed based on the calculated friction coefficient. For example, the friction coefficient and critical speed of a road may be proportional.

As another example, the processor 470 may reduce the critical speed when the vehicle 100 is driving in a section with severe traffic congestion. Conversely, when driving in a section with smooth traffic, the critical speed can be increased.

If it is determined in step S1330 that the driving speed of the vehicle 100 is higher than the threshold speed, the processor 470 may perform step S1340.

In step S1340, the processor 470 may enlarge the size of the driving speed indicator included in the virtual screen. At this time, as the driving speed of the vehicle 100 increases, the rate at which the size of the driving speed indicator increases can increase. For example, if the driving speed is within the first speed or second speed (>first speed), the size of the driving speed indicator is doubled, and if the driving speed is within the second speed or third speed (>second speed), the driving speed indicator is doubled in size. The size of the speed indicator can be enlarged by 3 times. The processor 470 can control the display unit 440 to increase the size, adjust the transparency, color, and brightness of the driving speed indicator, or generate visual effects such as flicking. Accordingly,

Together or separately from this, the processor 470 can change or delete the status of other indicators included in the virtual screen and control the display unit 440 to add and display a new indicator. For example, as the driving speed increases, the processor 470 may increase the blinking speed of the speed limit indicator displayed on the virtual screen. For another example, if the driving speed exceeds 1.2 times the threshold speed, the processor 470 deletes the blind spot indicator from the virtual screen and adds the engine speed (rpm) indicator of the vehicle 100 to the virtual screen. can do. For another example, if the driving speed exceeds 1.3 times the threshold speed, the processor 470 may add a warning indicator to induce deceleration of the vehicle 100 to a blank area of the virtual screen.

Together or separately, the processor 470 may change the projection distance of the virtual screen based on the traveling speed of the vehicle 100. At this time, the projection distance may mean a straight line distance from the projection surface 101 to the virtual image displayed on the virtual screen. For example, the processor 470 may increase the projection distance of the virtual screen as the driving speed of the vehicle 100 increases.

14A to 14C show an operation of the head-up display device 400 to control a virtual screen according to the driving speed of the vehicle 100 according to an embodiment of the present invention.

FIG. 14A illustrates a change in the driver's field of view according to the driving speed of the vehicle 100. Referring to FIG. 14A, the first viewing range 1401 is a viewing range when the traveling speed is a first speed, and the second viewing range 1402 is a viewing range when the traveling speed is a second speed faster than the first speed. range, and the third viewing range 1403 may be a viewing range when the driving speed is a third speed that is faster than the second speed. As shown, as the driving speed increases, the driver's viewing angle narrows and the gaze distance may increase.

FIG. 14B illustrates that the projection distance of the virtual screen is adjusted according to the driving speed of the vehicle 100. Referring to FIG. 14B, the processor 470 controls the display unit 440 to display the first virtual screen 1411 at the first projection distance D1 when the driving speed of the vehicle 100 is the first speed. can do. When the driving speed of the vehicle 100 is a second speed faster than the first speed, the processor 470 controls the display unit 440 to display the second virtual screen 1412 at the second projection distance D2. You can. When the driving speed of the vehicle 100 is a third speed faster than the second speed, the processor 470 controls the display unit 440 to display the third virtual screen 1413 at the third projection distance D3. You can.

At this time, the first to third virtual screens 1411-1413 may be configured the same or different from each other. For example, the first to third virtual screens 1411-1413 may include the same indicators. As another example, an indicator included in one of the first to third virtual screens 1411-1413 and an indicator included in the remaining virtual screens may have different locations, sizes, colors, or types.

That is, the processor 470 may increase the projection distance of the virtual screen as the driving speed of the vehicle 100 increases, and may decrease the projection distance of the virtual screen as the driving speed of the vehicle 100 decreases. As described above, the faster the driving speed of the vehicle 100, the farther the driver's gaze distance becomes, so the processor 470 adjusts the projection distance of the virtual screen based on the driving speed to provide the driver with a virtual screen with improved visibility. can be provided.

FIG. 14C shows changes in the configuration of the first to third virtual screens 1411-1413 shown in FIG. 14B according to changes in driving speed.

First, a plurality of indicators 1411a-1411e may be displayed on the first virtual screen 1411. For example, the first indicator 1411a is an indicator that guides the driving direction of the vehicle 100, the second indicator 1411b is an indicator that guides whether the vehicle 100 deviates from its lane, and the third indicator 1411c is an indicator. It is an indicator that guides the speed limit, the fourth indicator 1411d may be an indicator that guides the driving speed, and the fifth indicator 1411e may be an indicator that guides the direction of the planned route and the remaining distance.

The indicators 1411a - 1411e included in the first virtual screen 1411 may be displayed in the same manner as the second virtual screen 1412 . However, when comparing the first virtual screen 1411, there is a difference in that the driving speed guided by the fourth indicator 1411d is 93 km/h, which is faster than the speed limit of 90 km/h. When the threshold speed (see FIG. 13) is set equal to the limit speed, the processor 470 sets the size of the fourth indicator 1411d of the second virtual screen 1412 to a predetermined size than the size of the first virtual screen 1411. It can be enlarged by the ratio. Accordingly, the driver of the vehicle 100 can quickly recognize that he or she is moving faster than the current speed limit.

In the third virtual screen 1403, the second indicator 1411b and the fifth indicator 1411e included in the second virtual screen 1402 are deleted, and the sixth indicator 1411f and the seventh indicator 1411g are newly added. Additional information may be displayed. The sixth indicator 1411f may be an indicator that guides the rotation speed of the engine of the vehicle 100, and the seventh indicator 1411g may be an indicator that induces deceleration.

As described above, the processor 470 may change the configuration of the virtual screen for each driving speed based on changes in the driving speed of the vehicle 100. Accordingly, appropriate information tailored to the constantly changing driving speed can be provided to the driver through a virtual screen.

FIG. 15 shows an operation of the head-up display device 400 to control a virtual screen according to the driving speed of the vehicle 100 according to an embodiment of the present invention.

FIG. 15 shows a situation in which the vehicle 100 is driving toward a ramp 1501 on level ground. For convenience of explanation, it is assumed that the ramp 1501 is an uphill road as shown.

The processor 470 may obtain information about the ramp 1501 before reaching the ramp 1501. For example, the processor 470 may calculate a change in the slope of the road in front of the vehicle 100 based on the image provided from the outdoor camera 462 and detect the slope 1501 based on the calculated slope. . As another example, the processor 470 may detect the ramp 1501 closest to the vehicle 100 based on map data corresponding to the current location of the vehicle 100. The map data may include slope information of the ramp 1501.

The processor 470 may set the target position of the virtual screen based on the slope of the ramp 1501. For example, the processor 470 may set a position farther from the reference position as the target position as the slope of the ramp 1501 increases. Once the target position is set, the processor 470 may gradually change the position of the virtual screen to the target position as the vehicle 100 approaches the ramp 1501. That is, the processor 470 can move the virtual screen to the target position.

Referring to FIG. 15, when the vehicle 100 is at a first point (P1) that is a predetermined distance or more away from the ramp 1501, the processor 470 may display the virtual screen 1511 at the reference position (R1). there is. Together or separately, the processor 470 may calculate the position and inclination of the ramp 1501 with respect to the vehicle 100. Additionally, the processor 470 may calculate the target location A22. Since the ramp 1501 is an uphill road, the target position A22 may be higher than the reference position R1. At this time, the distance between the reference position (R1) and the target position (A22) may correspond to the slope of the ramp 1501.

Thereafter, while the vehicle 100 is traveling from the first point (P1) to the second point (P2), the processor 470 moves the virtual position (A21) to the intermediate position (A21) between the reference position (R1) and the target position (A22). The screen 1511 can be moved.

Thereafter, while the vehicle 100 drives from the second point (P2) to the third point (P3), the processor 470 moves the virtual screen 1511 from the intermediate position (A21) to the target position (A22). You can. At this time, the third point P3 may be the starting point of the ramp 1501.

The processor 470 may adjust the moving speed of the virtual screen 1511 based on the remaining distance from the vehicle 100 to the starting point P3 of the ramp 1501. For example, as the remaining distance from the vehicle 100 to the starting point P3 of the ramp 1501 becomes shorter, the moving speed of the virtual screen 1511 can be increased.

According to FIG. 15, while the vehicle 100 is driving toward the ramp 1501, the position of the virtual screen 1511 changes gradually according to the inclination of the ramp 1501, so the moment the driver enters the ramp 1501 From now on, highly visible information can be provided through the virtual screen 1511.

FIG. 16 shows an operation of the head-up display device 400 to control the position of a virtual screen based on the headlights of an oncoming vehicle according to an embodiment of the present invention.

Figure 16 shows a top view of the road including first through third lanes 1601-1603. As shown, it is assumed that the first to third lanes 1601-1603 are all straight lanes, and the vehicle 100 is driving within the second lane 1602.

Referring to FIG. 16, the processor 470 may detect the headlights 1611 of the oncoming vehicle 1610. For example, the processor 470 may detect the headlight 1611 based on the front image provided from the outdoor camera 462. As another example, the processor 470 may detect the headlight 1611 based on sensing data provided from the light detection sensor 464. When the virtual screen 1620 is displayed at the reference position (R1), it may be difficult for the driver of the vehicle 100 to properly perceive the information provided through the virtual screen 1620 due to glare caused by the headlights 1611. there is.

Accordingly, the processor 470 may set the target position A31 for the virtual screen 1620 based on at least one of the position and brightness of the detected headlight 1611. For example, as shown, when the headlight 1611 is located on the left side of the vehicle 100, the processor 470 may set the target position A31 to the left of the reference position R1. As another example, the processor 470 may increase the distance difference between the reference position R1 and the target position A31 as the brightness of the headlight 1611 increases.

For example, as shown, the processor 470 may set the target position A31 in a diagonal direction corresponding to the lower right corner based on the reference position R1. The processor 470 may move the virtual screen 1620 toward the target position A31, and as the position of the virtual screen 1620 changes toward the target position A31, the virtual screen 1620 and the headlight The area where (1611) overlaps each other may decrease or disappear. As a result, visibility of the virtual screen 1620 may be improved.

The embodiments of the present invention described above are not only implemented through devices and methods, but may also be implemented through a program that realizes the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded. The implementation can be easily implemented by an expert in the technical field to which the present invention belongs based on the description of the embodiments described above.

In addition, the present invention described above is capable of various substitutions, modifications and changes without departing from the technical spirit of the present invention to those of ordinary skill in the technical field to which the present invention pertains. It is not limited by the drawings, and all or part of each embodiment can be selectively combined so that various modifications can be made.

100: vehicle
400: Head-up display device

Claims (10)

a display unit that displays a virtual screen including at least one indicator guiding driving information of the vehicle on at least a portion of a projection surface provided in the vehicle; and
A processor that changes at least one of the position and size of the virtual screen based on the movement information of the vehicle,
The processor,
Changing the position of the virtual screen in the vertical direction based on the forward and backward tilt included in the motion information of the vehicle,
The speed at which the position of the virtual screen changes in the vertical direction is,
A head-up display device proportional to the tilt. According to paragraph 1,
The display unit,
a display panel that outputs display light corresponding to the driving information; and
A reflection unit including at least one mirror that reflects the display light onto the projection surface,
The virtual screen is an area in which a virtual image of the display light reflected by the reflector is displayed among the entire area of the projection surface. According to paragraph 2,
A head-up display device further comprising a posture adjustment unit that adjusts the rotation angle or position of the reflector according to a control signal provided from the processor. delete According to paragraph 1,
The processor,
A head-up display device that changes the position of the virtual screen in the left and right directions based on the steering angle included in the movement information of the vehicle. According to clause 5,
The processor,
Based on the steering angle and road information, determine the vehicle's expected driving lane,
A head-up display device that changes the position of the virtual screen in the left and right directions based on the position of the lane in which the vehicle is scheduled to travel. According to paragraph 1,
The processor,
A head-up display device that changes the size of at least a portion of the virtual screen based on the driving speed included in the motion information of the vehicle. In clause 7,
The processor,
A head-up display device that enlarges the size of an indicator guiding the driving speed among at least one indicator included in the virtual screen based on an increase in the driving speed included in the motion information of the vehicle. According to paragraph 1,
Further including a light detection sensor,
The processor,
When the headlights of an oncoming vehicle are detected by the light detection sensor, setting a target position of the virtual screen based on the position of the headlights with respect to the vehicle,
A head-up display device that changes the position of the virtual screen to the target position. According to paragraph 1,
The processor,
Setting the target position of the virtual screen based on the slope of the ramp closest to the vehicle,
A head-up display device that gradually changes the position of the virtual screen to the target position as the vehicle approaches the ramp.

Images