JP2015069656A - Three-dimensional (3d) navigation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional navigation system using a three-dimensional head-up display having a dynamic focal plane.SOLUTION: An HUD component performs: projecting a graphic element on a focal plane throughout a vehicle surrounding environment; three-dimensional visualization of the graphic element by moving the focal plane or adjusting a distance between the focal plane and a vehicle, and adjusting a target position of the graphic element, enabling projection of the graphic element as an avatar, and, namely adjusting the distance of the focal plane and the target position, enabling three-dimensional projection of the graphic element along x, y, z axes; and sequentially projecting a moving avatar on different focal planes, animating the avatar. This makes it possible to provide a passenger with a feeling of the avatar approaching or departing from the vehicle.

Description

Cross-reference of related applications

  This application is entitled “Stereoscopic head-up display with dynamic focal plane” filed on March 15, 2013, serial number: 13/832918 (attorney docket number: HRA-36332.01). This is a continuation-in-part (CIP) application of a pending US non-provisional patent application. The entire application is incorporated herein by reference.

  In order to improve the convenience for the driver, the vehicle can be provided with a head-up display (HUD) for displaying information to the driver. The information displayed by the HUD can be projected on the windshield of the vehicle and presented to the driver's view while the driver is driving. By displaying information in the driver's field of view, the driver looks away from the windshield to view the information presented while driving (eg, facing the instrument display on the center dashboard) ) No need.

  The HUD can display vehicle information that is typically displayed on the vehicle's center dashboard, such as information regarding vehicle speed, fuel level, engine temperature, and the like. Further, the HUD can present map information and communication events (for example, navigation instructions, driving instructions, warnings, warnings, etc.) to the driver. A vehicle HUD can display information to the driver in a manner similar to that used in the vehicle dashboard, for example, by displaying gauges and text boxes that appear as graphic elements on the windshield. it can. Furthermore, the vehicle HUD can also display augmented reality graphic elements that extend the physical environment around the vehicle with real-time information.

  However, existing HUD devices used in vehicles may not be able to display augmented reality graphic elements with consistent depth cues. Therefore, augmented reality graphic elements displayed by existing vehicle HUDs may be presented as superficial overlap.

  A brief description of this section is provided to select and introduce in simplified form the concepts described in the detailed description that follows. This brief description is not intended to present the subject matter of the claims broadly, nor is it intended to identify key elements or essential features of the subject matter of the claims. It is not intended to be used to limit the scope of matters.

  According to one embodiment, a vehicle head-up display device for displaying a graphic element in a vehicle driver's field of view includes a first projector and a first actuator. The first projector may be configured to project the first graphic element onto a first focal plane within the driver's field of view. The first focal plane can be oriented substantially perpendicular to the driver's line of sight and at a distance away from the vehicle. The first projector can be installed on the first actuator. The first actuator can be configured to move the first projector linearly. By moving the first projector linearly, the first focal plane of the first graphic element can be moved in the direction of the driver's line of sight.

  According to another embodiment, a vehicle head-up display system includes a vehicle head-up display device and a controller. The vehicle head-up display device displays a graphic element in the field of view of the vehicle driver and has a first projector and a second projector. The first projector may be configured to project the first graphic element onto a first focal plane in the driver's field of view. The first focal plane can be oriented substantially perpendicular to the driver's line of sight. The first projector can be configured to move the first focal plane in the direction of the driver's line of sight. The second projector may be configured to project the second graphic element onto a second focal plane in the driver's field of view. The second focal plane can be stationary and oriented substantially parallel to the ground. The controller communicates with one or more corresponding vehicle control systems, controls a vehicle head-up display device, and based on communication with the one or more corresponding vehicle control systems The first and second graphic elements can be configured to be displayed.

  According to yet another embodiment, a method for presenting an augmented reality graphic element on a vehicle head-up display includes a first graphic element in a first focal plane within the driver's field of view and the driver's field of view. Projecting a second graphic element onto the second focal plane of the image. The first focal plane can be oriented substantially perpendicular to the driver's line of sight, and the second focal plane can be stationary and oriented substantially parallel to the ground. The method can include moving or aligning the first focal plane in the direction of the driver's field of view.

  One or more embodiments of techniques and systems for three-dimensional (3-D) navigation are disclosed herein. For example, a 3D navigation system can project graphical elements or avatars that appear to move into the vehicle occupant's field of view. In one or more embodiments, a heads up display (HUD) component can be configured to project the graphic element or avatar onto one or more focal planes in the environment surrounding the vehicle. . In other words, the HUD component projects a plurality of graphic elements or avatars onto a plurality of adjustable distances, i.e. a plurality of adjustable focal planes, so that one avatar or graphic element moves, flies, animations, etc. This gives the vehicle occupant a sense of

  For example, the HUD component can be configured to “animate” or give motion to an avatar by continuously projecting the avatar onto one or more different focal planes. The projection onto the plurality of focal planes may be realized by using, for example, an actuator that moves the projector of the HUD component. As a result, depth cues such as focus adjustment and convergence corresponding to graphic elements and avatars are generally maintained. If a route is generated from a first location to a second location, the HUD component may generate one or more graphic elements that a vehicle driver or occupant should “track”. it can. The HUD component can project onto a plurality of focal planes or move a plurality of projected graphic elements from one focal plane to another, so that a graphic element or projected image is more realistic. It looks the same as the image in the mirror.

  When a vehicle occupant requests navigation guidance, a graphic element such as an avatar can be presented. The avatar appears to move, glide, fly, etc. in front of the vehicle, similar to what an occupant or driver sees when tracking a fellow vehicle, for example. In addition, the avatar may appear to navigate around obstacle situations, obstacles, pedestrians, object debris, depressions, etc., like an actual vehicle. In one or more embodiments, the avatar appears to “drive”, move, and move in response to real-time traffic. For example, if the route requires a driver or vehicle to cross the track, the avatar will stop on the track when the train crosses. In another example, the avatar makes a lane change so that it does not look like a “collision” or other traffic obstruction to other vehicles.

  The following description and the annexed drawings set forth some illustrative embodiments and implementations. These are descriptions of various methods using one or more embodiments, but only a few of them. Other embodiments, advantages or novel features of the invention will become apparent from the following detailed description when considered in conjunction with the accompanying drawings.

  Embodiments of the present invention can be understood from the following detailed description when taken in conjunction with the accompanying drawings. Elements, configurations, etc. in the drawings are not necessarily drawn to scale. Thus, for example, the same dimensions may be arbitrarily expanded or reduced for clarity of discussion.

1 is a schematic diagram of an example vehicle head-up display system, according to one or more embodiments. FIG.

1 is a schematic diagram of an example of a vehicle comprising a vehicle head-up display system according to one or more embodiments. FIG.

1 is a side view of an example of a vehicle and a four focal plane onto which a graphic element is projected by a vehicle head-up display system, according to one or more embodiments. FIG.

One example of a driver's field of view from a windshield while driving a vehicle and exemplary graphic elements projected by the vehicle head-up display system, according to one or more embodiments. It is explanatory drawing.

FIG. 3 is an illustration of an example of a block diagram of a 3D navigation system, according to one or more embodiments.

FIG. 4 is an illustration of an example flow diagram of a method of 3-D navigation, according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

FIG. 6 is an illustration of an example of a 3-D navigation avatar according to one or more embodiments.

One of the computer-readable media or computer-readable devices storing processor-executable instructions configured to integrate and implement one or more of the conditions described herein, according to one or more embodiments. It is explanatory drawing of an example.

FIG. 6 is an illustration of an example computing environment in which one or more of the conditions described herein are implemented, according to one or more embodiments.

  The embodiments or examples shown in the drawings are disclosed below using specific terminology. However, it should be understood that the disclosed embodiments or examples are not intended to be limiting. Any changes, modifications, and further applications of the principles disclosed herein will generally be conceivable to one skilled in the relevant arts in the disclosed embodiments.

  With respect to one or more of the drawings herein, one or more borders, such as border 116 in FIG. 2, are for purposes of illustration only and have different heights, widths, perimeters, aspect ratios, It is drawn in shape and the like, and is not necessarily drawn to scale. For example, broken lines or dotted lines are used to represent different boundaries, so if broken lines or dotted lines are drawn on top of each other, these lines cannot be identified in the drawing. For this reason, in one or more figures, they are drawn slightly apart from each other and are distinguishable from each other. As another example, if a boundary corresponds to an irregular shape, a boundary such as a box drawn with a dashed line, dotted line, etc., does not necessarily include an entire component in one or more examples. Not exclusively. Conversely, a drawn box does not necessarily include only one corresponding component in one or more examples, and may include part of one or more other components. It is possible.

  Graphic elements that are visually placed by the vehicle HUD device over environmental elements in the driver's direct field of view are often referred to as field-like augmented reality elements or conformal augmented reality graphic elements. The successful presentation of field-similar augmented reality graphic elements to the vehicle driver may depend on the vehicle's ability to accurately reproduce the depth cues. This depth cue includes focus adjustment and convergence. Focus adjustment is a depth cue that causes the eye muscles to actively change refractive power and change the focus to various distances. Convergence is a simultaneous or parallel inward rotation of both eyes in the direction of each other, performed to maintain a single binocular image when viewing one object.

  The embodiments described herein may describe one vehicle driver, but may project, present, and render graphic elements within the field of view of one or more other occupants, such as passengers. it can. As such, these examples are not intended to be limiting and are disclosed merely to illustrate one or more embodiments of exemplary applications.

  When the HUD device displays a graphic element on the vehicle windshield, the focus adjustment causes the human eye to move between the environmental element and the information displayed by the HUD device. Convergence can cause both eyes to concentrate on a point in the environment beyond the windshield, which can cause the HUD graphic element displayed on the windshield to appear as a double image. Thus, to render a field-similar augmented reality graphic element with correctly reproduced depth cues, the graphic element is on the same space as the real environment (eg, the corresponding focal plane) rather than on the vehicle windshield. Need to be rendered.

  A vehicle head-up display device is provided for displaying graphic elements in the driver's field of view while the vehicle driver is viewing the environment through the windshield. The head-up display device includes one or more projectors that project graphic elements onto a front focal plane in the driver's field of view while the driver is viewing the environment through the windshield; One or more projectors that project graphic elements onto a focal plane parallel to the ground in the driver's field of view while the person is viewing the environment through the windshield. The projector that projects the graphic element on the front focal plane can be mounted on an actuator that linearly moves the projector so as to move the front focal plane in the direction of the driver's line of sight. The projector that projects onto the focal plane parallel to the ground surface can be fixedly arranged so that the focal plane parallel to the ground surface is stationary.

  Referring to FIG. 1, render a stereoscopic field-similar augmented reality graphic element (eg, a three-dimensional or “3-D” graphic element rendered in the same space as the real environment) with correctly reproduced depth cues. A three-dimensional head-up display system 100 ("HUD system 100" or "HUD component 100") for a vehicle is described. The HUD system 100 includes a vehicle head-up display device 102 (“HUD device 102”) and a controller 104 or (or “control component 104”). Referring to FIG. 2, the HUD system 100 can be provided on a vehicle 106 and includes a driver's seat 108, a dashboard housing 110, and a windshield 112.

  The configuration of the vehicle 106 may be conventional with respect to the relative positions of the driver's seat 108, the dashboard housing 110, and the windshield 112, for example. In order to accommodate the HUD system 100 described herein, a dashboard housing 110 defines an accommodation space in which the HUD system 100 is accommodated. Further, the dashboard housing 110 has a HUD exit hole 114 defined by the upper surface of the dashboard housing 110. The HUD system 100 housed in the dashboard housing 110 projects a graphic element such as a visual field similarity augmented reality graphic element onto the windshield 112 through the HUD exit hole 114. The windshield 112 can be used as a display screen for the HUD system 100. As described in more detail below, augmented reality graphic elements can be rendered to the driver as if they were in the same space as the real environment.

  The driver of the vehicle 106 drives the vehicle 106 while sitting on the driver's seat 108. Therefore, the driver is subjected to positional restrictions on the seating position of the driver's seat 108 in the vehicle 106. In view of this positional constraint, the HUD system 100 can be designed with the assumption that the driver's field of view is out of the eye box 116 in the vehicle. The eye box 116 can be considered to include the area inside the vehicle 106 where the driver's eyes are placed while the driver is seated in the driver's seat 108.

  The eye box 116 can be sized to encompass any possible head position of the driver, regardless of the position and attitude of the driver's seat 108. Alternatively, the HUD system 100 can be configured to detect the position and attitude of the driver's seat 108 and adjust the position and size of the eye box 116 based thereon. In one or more embodiments, the HUD system 100 can be designed assuming that the eye box 116 has a fixed size and is in a fixed position. For example, the eye box can have a size of 20 cm x 10 cm x 10 cm. In any case, the HUD system 100 allows the driver's eyes to be in the eye box 116 when the driver is looking forward / looking through the windshield 112 of the vehicle 106. Can be configured to display field-similar augmented reality graphic elements. Although the eye box 116 of FIG. 2 is illustrated for a driver of the vehicle 106, the eye box 116 may be configured to include one or more other occupants of the vehicle. In one or more embodiments, one or more additional eye boxes or HUD devices may be provided, for example, for passengers or other passengers.

  The HUD device 102 displays one or more graphic elements in the driver's view of the vehicle 106 while the driver is viewing the environment through the windshield 112 of the vehicle 106. Any graphic that the driver is looking through the windshield 112 while the driver's eyes are in the eye box 116 and the driver is looking forward / looking through the windshield 112 Elements and environmental elements can be considered within the driver's view. As used herein, the driver's view of the vehicle 106 is the front, except for the dashboard display that is in the vehicle 106, while the driver is viewing the environment through the windshield 112 of the vehicle 106. It is intended to include the area visible through the glass 112. In other words, the HUD device 102 presents graphic elements so that the driver can see the graphic elements without looking away from the road.

  Returning to FIG. 1, the HUD device 102 of the HUD system 100 includes a first projector 118, a second projector 120, a third projector 122, and a fourth projector 124. The first projector 118 and the third projector 122 share the first beam splitter 126 and the first objective lens 128, and the second projector 120 and the fourth projector 124 are the second beam splitter. 130 and the second objective lens 132 are shared. As a result, the outputs of the first projector 118 and the third projector 122 are received by the first beam splitter 126 and directed to the first objective lens 128 and pass through the first objective lens 128. Integrated into a single output. Similarly, the outputs of the second projector 120 and the fourth projector 124 are received by the second beam splitter 130, directed to the second objective lens 132, and passed through the second objective lens 132. Integrated into a single output through.

  The HUD device 102 is further configured to receive output from the first and second objective lenses 128, 132 and is arranged downstream of the first and second objective lenses 128, 132. It has a beam splitter 134. Outputs from the first and second objective lenses 128, 132 are combined into a single output by a third beam splitter 134, which outputs the first, second, third, and fourth projectors 118. , 120, 122, 124 can be a combination of all outputs, directed toward and through the third objective lens 136 and eyepiece 138 and then out of the HUD exit aperture 114 Directed to the glass 112. The windshield 112 can be used as a display screen for the HUD system 100.

  Each of the first projector 118, the second projector 120, the third projector 122, and the fourth projector 124 includes projector units 140, 142, 144, and 146 and diffusion screens 148, 150, 152, and 154. . The diffusion screens 148, 150, 152, and 154 are securely fixed at installation distances from the projector units 140, 142, 144, and 146, and are emitted from the projector units 140, 142, 144, and 146. Are arranged with respect to the projector units 140, 142, 144, 146 so as to pass through the diffusion screens 148, 150, 152, 154. The projector units 140, 142, 144, 146 can be light emitting units that project image or graphic elements that pass through corresponding diffusion screens 148, 150, 152, 154. Diffusion screens 148, 150, 152, 154 act as luminescent image sources or (or objects) to other parts of the optical system of HUD device 102 and exit diffusion screens 148, 150, 152, 154 Most of the light is optically behind the diffusing screens 148, 150, 152, 154 (eg, first beam splitter 126, first objective lens 128, second beam splitter 130, second objective). Lens 132, third beam splitter 134, third objective lens 136 and eyepiece 138). At the same time, the diffusing screens 148, 150, 152, and 154 will ultimately allow the brightness of the image and graphic elements to remain constant even when the driver's head is moving within the eye box 116. Diffuse the light so that it fills the eye box 116. In this way, the use of the diffusing screens 148, 150, 152, 154 substantially prevents different portions of the image element and / or graphic element (s) from being seen from different points in the eye box 116. Prevents different visual movements from occurring with slight head movements.

  Projector units 140, 142, 144, and 146 may take the form of any light emitting unit suitable for the applications described herein. Projector units 140, 142, 144, 146 may take the form of any light emitting unit capable of projecting image elements and graphic elements in accordance with the applications described herein. Similarly, the diffusing screens 148, 150, 152, 154 can take the form of any light diffusing screen suitable for the applications described herein.

  The first projector 118 can be attached to the first actuator 156 of the HUD device 102. The first actuator 156 is a linear actuator capable of moving the first projector 118 in a linear direction toward or away from the first beam splitter 126. It can be. Further, the third projector 122 can be attached to the second actuator 158 of the HUD device 102. The second actuator is a linear actuator that can move the third projector 122 in a linear direction toward the first beam splitter 126 or away from the first beam splitter 126. obtain. The first and second actuators 156, 158 can take the form of any linear actuator suitable for the applications described herein. The linear movement capabilities of the first projector 118 and the third projector 122 allow the first projector 118 and the third projector 122 to project graphic elements on a dynamic focal plane or a movable focal plane. Become. In contrast to the first and third projectors 118, 122, the second and fourth projectors 120, 124 are fixedly disposed within the HUD device 102, and are thus capable of projecting graphic elements onto the stationary focal plane. it can.

  Using the first, second, third and fourth projectors 118, 120, 122, 124, the HUD device 102 can display graphic elements (field-similar augmented reality graphic elements or other graphic elements) as windshields. Rendered to four different focal planes in the environment viewed by the driver via 112. In this regard, the first projector 118 can be configured to project the first graphic element 160 onto the first focal plane 162, and the second projector 120 can cause the second graphic element 164 to be projected to the second focal plane 162. The third projector 122 can be configured to project onto the focal plane 166, the third projector 122 can be configured to project the third graphic element 168 onto the third focal plane 170, and the fourth projector 124 can The fourth graphic element 172 can be configured to project onto the fourth focal plane 174 (described below with reference to FIGS. 3 and 4). Driving all of the first, second, third and fourth graphic elements 160, 164, 168, 172 and the corresponding first, second, third and fourth focal planes 162, 166, 170, 174 When a person drives the vehicle 106 looking forward through the windshield 112 and the driver's eyes are in the eye box 116, it can be rendered into an environment within the driver's field of view.

  3 and 4, the first, second, third and fourth focal planes 162, 166 of the first, second, third and fourth graphic elements 160, 164, 168, 172, 170 and 174 will be described with reference to the ground surface 176 and the driver's line of sight 178. Here, the ground surface 176 is the surface of the road ahead of the vehicle 106. For simplicity of explanation, it is assumed that the ground surface 176 is a substantially flat surface. The driver's line of sight 178 is a line extending forwardly from the eye box 116 and substantially parallel to the ground surface 176. As used herein, the direction of the line of sight 178 is a direction that extends along the line of sight 178 toward the driver and the vehicle 106 or away from the driver and the vehicle 106.

  The first focal plane 162 is a front focal plane that can be oriented substantially perpendicular to the driver's line of sight 178. The third focal plane 170 is also a front focal plane that can be oriented substantially perpendicular to the driver's line of sight 178. The first and third focal planes 162 and 170 are dynamic focal planes that can move both in the forward direction (a direction away from the vehicle 106) and backward (in the direction toward the vehicle 106) in the direction of the line of sight 178. Can do. The second focal plane 166 is a focal plane parallel to the ground that can be oriented substantially parallel to the ground surface 176, and is disposed on the ground surface 176 so that the second focal plane 166 becomes the ground focal plane. can do. The fourth focal plane 174 is also a focal plane parallel to the ground surface that can be oriented substantially parallel to the ground surface 176 and can be disposed above the ground surface 176. The fourth focal plane 174 can be placed above the ground surface 176 and the driver's line of sight 178 to provide a sky or ceiling focal plane. As a result, the second and fourth focal planes 166 and 174 can be stationary focal planes.

  Referring to FIG. 4, the first, second, third and fourth graphic elements 160, 164, 168, 172 can be used to present different information to the driver. The fine types of information displayed by the first, second, third and fourth graphic elements 160, 164, 168, 172 can be changed. For example, the first graphic element 160 and the third graphic element 168 can present a warning to the driver instructing to bypass a dangerous situation or an obstacle. In addition, navigation commands and driving commands corresponding to road traffic regulations (eg, stop signs, YIELD signs, etc.) can be displayed. The second graphic element 164 and the fourth graphic element 172 can present navigation instructions to the driver as a graphical overlay that is presented on the ground surface 176 and also provides the driver with a vehicle perimeter sign. Can be presented. The first, second, third and fourth graphic elements 160, 164, 168, 172 may present different information or graphic elements to the driver than those described herein, and A partial set of the first, second, third and fourth graphic elements 160, 164, 168, 172 may be presented.

  Returning to FIG. 1, the controller 104 may include one or more computers (eg, for arithmetic calculations) that can communicate with one or more vehicle control systems 180 to control the HUD device 102. A processor or any other device may be included. One or more vehicle control systems 180 (herein “vehicle control system 180” or “vehicle control component 180”) take the form of any vehicle control system 180 (s) and It can be used to facilitate control either passively or passively. The vehicle control system 180 may include or communicate with one or more sensors (not shown) that sense driving and environmental conditions related to the operation of the vehicle 106.

  Looking briefly at the operation of the HUD system 100, the control unit 104 communicates with the vehicle control system 180, and based on the communication with the vehicle control system 180, the types and positions of graphic elements presented to the driver of the vehicle 106. To decide. The control unit 104 is presented as first, second, third and fourth graphic elements 160, 164, 168, 172 by the first, second, third and fourth projectors 118, 120, 122, 124. The type of graphic element to be determined and the first, second, third and fourth projectors 118, 120, 122, 124 are controlled to provide the first, second, third and fourth graphics. Elements 160, 164, 168, 172 are projected as determined graphic elements. The control unit 104 determines the target position of the first graphic element and the target position of the third graphic element, and the target that the first and third graphic elements 160 and 168 are to be rendered to the driver within the environment. It can be determined as a position. Next, the control unit 104 controls the first and second actuators 156 and 158 to linearly move the first and third projectors 118 and 122, and the first and third focal planes 162, 170 can be moved to the target positions of the first and third graphic elements, respectively.

  In this way, the first projector 118 projects the first graphic element 160 onto the first focal plane 162. The first focal plane 162 can be oriented substantially perpendicular to the driver's line of sight and directs the driver's line of sight 178 through linear motion of the first projector 118 by the first actuator 156. It is possible to move toward or away from the vehicle 106. The second projector 120 projects the second graphic element 164 onto the second focal plane 166. The second focal plane 166 can be stationary, oriented parallel to the ground surface 176, and disposed on the ground surface 176. The third projector 122 projects the third graphic element 168 on the third focal plane 170. Third focal plane 170 is oriented substantially perpendicular to the driver's line of sight, and through the linear motion of third projector 122 by second actuator 158, the direction of driver's line of sight 178 is directed toward vehicle 106. Alternatively, it can be moved or adjusted away from the vehicle 106. The fourth projector 124 can project the fourth graphic element 172 onto the fourth focal plane 174. The fourth focal plane 174 is stationary, oriented parallel to the ground surface 176, and can be positioned above the driver's line of sight 178. The control unit 104 controls the first and second actuators 156 and 158 to move the first and third projectors 118 and 122 and move the first and third focal planes 162 and 170.

  First and third projectors 118, 122 have first and third graphic elements 160, 168 on first and third focal planes 162, 170 oriented substantially perpendicular to the driver's line of sight 178. By projecting, the focus of an object at a different distance from the vehicle 106 can be adjusted. This facilitates the provision of accurate depth cues to the driver for the first and third graphic elements 160, 168, and in particular the HUD system 100 can be a vehicle application, thus serving as a mobile platform. The vehicle 106 can be provided.

  The second and fourth projectors 120, 124 project the second and fourth graphic elements 164, 172 onto the second and fourth static focal planes 166, 174 at the same time as the second and fourth focal points. The surfaces 166, 174 can be continuous without breaks. In order to make the second and fourth focal planes 166, 174 parallel to the ground 176, the diffusing screens 150, 154 of the second and fourth projectors 120, 124 can be tilted. The optical system of the HUD device 102 has very little distortion and is almost telecentric with respect to an image in a focal plane parallel to the ground, so that the light beam is nearly parallel to the optical axis. This allows the projected second and fourth graphic elements 164, 172 to be projected or rendered without distortion or without changing magnification while the second and fourth focal planes 166, 174 are tilted. It becomes possible. Accordingly, the resulting second and fourth graphic elements 164, 172 are displayed on a continuous focal plane (second and fourth focal planes 166, 174) parallel to the ground surface 176. In this regard, the second and fourth graphic elements 164, 172 render with actual three-dimensional (3-D) solid shapes instead of line segments and add monocular cues to enhance depth perception. You can also.

  The continuous and stationary second and fourth focal planes 166, 174 facilitate the driver's perception of depth relative to the second and fourth graphic elements 164, 172. The continuous and stationary second and fourth focal planes 166, 174 provide the correct generation of actual images or graphic elements across the entire anteroposterior direction in 3-D space (eg, in the direction of the driver's line of sight 178). This makes it possible to generate appropriate motion parallax cues. Thus, even if the driver's head moves from side to side or up and down, the driver does not move the second and fourth graphic elements 164, 172 around, but rather has a fixed position in the environment. Looks like it has been. Thus, the HUD system 100 does not require a head tracking function to compensate for the driver's head movement.

  With respect to the exemplary information that can be presented to the driver, the vehicle control system 180 can perform functions such as detection of dangerous situations and obstacles, navigation, navigation commands, vehicle periphery (for example, blind spot) monitoring, and the like. And a sensor. The vehicle control system 180 can include processes and sensors that can perform other vehicle control functions (eg, highway merge assistance, etc.). The other vehicle control functions can alternatively or additionally be coupled to information presented to the driver using the HUD system 100. Regardless of the function performed by the vehicle control system 180, the detailed operational configuration of the vehicle control system 180 to perform the function, including the associated sensors and processes, is not associated with the operation of the HUD system 100. Good.

  The controller 104 communicates with the vehicle control system 180 and receives input from the vehicle control system 180 corresponding to the above or (or other) functions related to the operation of the vehicle 106. Next, the control unit 104 controls the HUD device 102 based on the input received from the vehicle control system 180. Here, one or both of the control unit 104 and the vehicle control system 180 may display the types of graphic elements displayed as the first, second, third, and fourth graphic elements 160, 164, 168, 172, The position of the second, third and fourth graphic elements 160, 164, 168, 172 and which of the first, second, third and fourth graphic elements 160, 164, 168, 172 to display. decide. The above determination can also be based on one or more vehicle functions used by the driver, such as whether the driver is using a navigation function.

  Regardless of whether the determination is made using the controller 104 or the vehicle control system 180, the controller 104 controls the HUD device 102 to display the appropriate graphic element at the appropriate location. Such control includes controlling the first, second, third, and fourth projectors 118, 120, 122, 124 to provide first, second, third, and fourth graphic elements 160, 164, 168 and 172 may be appropriately projected. For this control, the first and second actuators 156 and 158 are controlled to move the first and third projectors 118 and 122 linearly, and the first and third focal planes 162 and 170 are appropriately adjusted. A process of moving to a different position (for example, a target position) may be included. For example, one or more actuators, such as 156, 158, can be configured to move one or more focal planes, such as 162,170. With respect to the third focal plane 170, for example, the distance between the third focal plane 170 and the windshield (eg, 302) of the vehicle 106 can be adjusted by adjusting the distance 170 '. Similarly, the distance 162 'can be adjusted to change the target position of the focal plane 162.

  With reference to the exemplary information associated with the first, second, third, and fourth graphic elements 160, 164, 168, 172, the operation of the HUD system 100 includes a vehicle control system 180. Description will be made with reference to 106. The vehicle control system 180 enables the following functions, such as detection and warning functions for dangerous situations and obstacles, navigation functions, navigation command functions, and vehicle periphery (for example, blind spot) monitoring functions. Again, the vehicle 106 may have some or additional functions of the above functions, and the HUD system 100 may be used in connection with some or additional functions of the above functions. The description of the HUD system 100 regarding the above functions is merely an example, and is used to facilitate the description of the HUD system 100. One or both of the controller 104 and the vehicle control system 180 can make decisions related to the operation of the HUD system 100, but in the following description, the controller 104 received from the vehicle control system 180. The description will be made assuming that the determination is made based on the input.

  Information regarding obstacle detection and warning functions can be presented to the driver as a visual field-similar augmented reality graphic element projected by the first projector 118 of the HUD device 102. In this regard, the vehicle control system 180 can detect various obstacles on the road on which the vehicle 106 is traveling. For example, obstacles include pedestrians crossing roads, other vehicles, animals, debris on roadways, depressions, and the like. These obstacles are detected by processing information from the environment sensed by a sensor (not shown) provided on the vehicle 106. Obstacle detection can be performed by any method.

  When the obstacle is detected, the vehicle control system 180 transmits the obstacle information to the control unit 104. When receiving the obstacle information from the vehicle control system 180, the control unit 104 determines the type of the graphic element to be presented as the first graphic element 160 and the target position of the first graphic element based on the received obstacle information. decide. While various types of graphic elements can be used, such as flash icons, other displays, etc., the examples herein will be described with reference to a “YIELD” indicator that is presented when an obstacle is detected.

  Referring to FIG. 4, the obstacle detected by the vehicle control system 180 may be a pedestrian 182 crossing a road on which the vehicle 106 is traveling. In the driver's exemplary view of FIG. 4, the vehicle 106 is traveling on the road that the pedestrian 182 is crossing. Therefore, the vehicle control system 180 can transmit obstacle information regarding the pedestrian 182 to the control unit 104. Based on the obstacle information, the control unit 104 can determine the type of graphic element to be displayed as the first graphic element 160. In this case, for example, the graphic element may be a “YIELD” indicator, but other graphics may be used. The controller 104 perceives the target position of the first graphic element as being at the same depth (eg, focal plane) as the pedestrian 182 by the driver being projected and rendered by the first graphic element 160. Can be determined. Further, the control unit 104 may adjust the target position of the first graphic element so that the first graphic element 160 “tracks” or “tracks” the pedestrian 182 when the pedestrian 182 walks, for example. It can be configured.

  The controller 104 then controls the first projector 118 to project a “YIELD” sign as the first graphic element 160 and controls the first actuator 156 to linearly. The first projector 118 is moved and the first graphic element 160 is projected and rendered by the driver (eg, the driver's eyes are in the eye box 116 and the driver is over the windshield 112). It can be perceived to be at the same depth as the pedestrian 182 (while looking forward). The first actuator 156 is controlled to project the first graphic element 160 onto the first focal plane 162. The first focal plane 162 can be located at the target location of the first graphic element and can be oriented substantially perpendicular to the line of sight 178.

  When the vehicle 106 and the pedestrian 182 pass on the road, the relative distance between them changes. This change in distance can be transmitted to the control unit 104 by the vehicle control system 180, the target position of the first graphic element can be changed accordingly, and the first actuator 156 is controlled by the control unit 104. To move the first focal plane 162 to remain at the target position (eg, changed / changed). In this way, the first graphic element 160 is projected onto the first focal plane 162 that is movable along the direction of the driver's line of sight 178, and the depth cue corresponding to the first graphic element 160 is correctly reproduced. Thus, the driver can accurately determine the position of the first graphic element 160 (eg, detected obstacle).

  Further, the information regarding the navigation function can be presented to the driver as a visual field similarity augmented reality graphic element projected by the second projector 120 of the HUD device 102. In this regard, when the vehicle control system 180 receives a navigation request (eg, input of a desired location) from the driver, the vehicle control system 180 generates a navigation path that the driver must take to reach the desired location. The navigation route includes a series of driving instructions to be followed by the driver, and the instructions include instructions for reaching a road on the route to a desired location. The navigation function can be performed in any way. When the navigation function is activated, the vehicle control system 180 transmits a driving instruction regarding the navigation function to the control unit 104.

  The control unit 104 receives the driving instruction from the vehicle control system 180 and determines the type of graphic element to be presented as the second graphic element 164. For the types of graphic elements associated with the navigation function, continue on the current road (eg straight line or arrow), turn left or right and enter the next crossing road (eg left / right arrow or appropriate A graphic element is included that directs the driver to turn in a direction), enter the highway, merge or exit the highway (eg, a line or arrow indicating an appropriate passage), and the like. The control unit 104 selects an appropriate graphic element to be presented as the second graphic element 164 based on the driving instruction transmitted from the vehicle control system 180.

  Referring to the driver view example of FIG. 4, the driving instructions for the driving route determined by the navigation function of the vehicle control system 180 include an instruction to turn left and enter the next road. Therefore, the control unit 104 controls the second projector 120 to generate and project a left-turn graphic element on the second focal plane 166 as the second graphic element 164. As shown in FIG. 4, the second focal plane 166 is oriented parallel to the ground surface 176 and can be placed on the ground surface 176. As described above, the second projector 120 can be fixedly disposed on the HUD device 102 so that the second focal plane 166 is stationary. As described above, the second focal plane 166 can be continuous and the second graphic element 164 can be rendered as a 3-D image with appropriate depth cues for the driver.

  Similarly, information regarding the navigation command function is presented to the driver as a field-similar augmented reality graphic element projected by the third projector 122 of the HUD device 102. Here, the vehicle control system 180 uses a sensor or information stored in a database and associated with the map to monitor the road on which the vehicle 106 is traveling, and to perform next navigation related to traveling on the road. Instructions can be determined. For example, the vehicle control system 180 detects the next required stop, yield (YIELD) or other situation (collectively referred to herein as “road situation”) on the road on which the vehicle 106 is traveling. be able to. The vehicle control system 180 determines a navigation command (for example, a stop command associated with the stopped road condition) related to the detected road condition. The navigation instruction function can be executed in any way, but the details of the execution method are not necessarily related to the operation of the HUD system 100. In addition, road conditions include, among other things, traffic volume on road sections, obstacle events, obstacles, weather conditions, surface conditions on road sections, or speed limits related to roads or parts of road sections, Etc. are included. That is, road conditions can generally include, for example, reasons for driving, accelerating, decelerating, detouring, stopping, alerting, and the like.

  The vehicle control system 180 transmits a road condition or a navigation command corresponding to the road condition and information on the position of the road condition to the control unit 104. Accordingly, the control unit 104 can control the third projector 122 to project the third graphic element 168 and transmit information regarding road conditions or corresponding navigation commands to the driver. The control unit 104 receives the road condition or navigation command information and the position information from the vehicle control system 180, and determines the type of graphic element to be displayed as the third graphic element 168 and the target position of the third graphic element. Can be determined.

  Various types of graphic elements can be used in combination with the navigation instruction function. For example: STOP (stop) sign, YIELD sign, ONE WAY (one-way) sign, NO TURN ON RED (red turn prohibited) sign, etc. The type of graphic element can be selected to convey navigation instructions associated with road conditions. Whatever type of graphic element is determined by the control unit 104 as the third graphic element 168, the graphic element can be projected to be displayed at the position of the driving situation. . At this point, the target position of the third graphic element is determined based on the detected position of the road condition relative to the vehicle 106 as the position where the third graphic element 168 is to be rendered in the driver's field of view. There is a need.

  The control unit 104 can be configured to control the third projector 122 to project an appropriate graphic element as the third graphic element 168. The control unit controls the second actuator 158 to linearly move the third projector, and the third graphic element 16 is moved by the driver (for example, the driver's eyes are in the eye box 116). And while the driver is looking forward through the windshield 112, can be projected and rendered to perceive that it is at the same depth as the road conditions (eg, having the same focal plane). it can. The second actuator 158 can be controlled to project the third graphic element 168 onto the third focal plane 170. The third focal plane 170 can be positioned at the target location of the third graphic element and oriented substantially perpendicular to the line of sight 178. The control unit 104 controls the second actuator 158 to move the third projector 122 continuously and linearly, and the third focal plane 170 is detected as the vehicle 106 (for example, the first road condition). 3 (target position of the three graphic elements) can be moved in response to a change in distance (detected by the vehicle control system 180 and transmitted to the control unit 104). The change occurs, for example, as a result of the vehicle 106 traveling toward the detected road condition.

  In the example of the field of view from the driver's viewpoint in FIG. 4, the vehicle 106 is approaching the crossroad where the vehicle 106 should stop. Accordingly, the vehicle control system 180 detects a stop road condition at the intersection entrance position, and determines that the navigation instruction corresponding to the stop road condition should be a stop instruction. The stop road condition or command and the position of the stop road condition are transmitted to the control unit 104. The control unit 104 determines that the STOP indication should be displayed as the third graphic element 168. The control unit 104 determines that the third graphic element 168 (eg, a STOP indication) should be displayed at the entrance position of the crossroads. Thus, the position of the entrance of the intersection can be determined as the target position of the third graphic element.

  The control unit 104 can control the third projector 122 to project the “STOP” indication as the third graphic element 168, and can control the second actuator 158 to control the third projector 122. The third graphic element 168 is moved by the driver (eg, while the driver's eyes are in the eye box 116 and the driver is looking forward through the windshield 112) It can be projected and rendered so that it is perceived to be at the same depth. The second actuator 158 can be controlled such that the third graphic element 168 is projected onto the third focal plane 170. The third focal plane 170 can be at a target position of the third graphic element and can be oriented substantially perpendicular to the line of sight 178. When the vehicle 106 is traveling on the road, the relative distance between the vehicle 106 and the entrance of the crossroad changes. This change in distance can be communicated to the control unit 104 by the vehicle control system 180, thus changing the target position of the third graphic element, and the control unit 104 controls the second actuator 158. Thus, the third focal plane 170 can be moved so that the third focal plane 170 remains at the target position of the third graphic element (eg, changed / being changed). In this way, the third graphic element 168 is projected onto the third focal plane 170 that is movable along the direction of the driver's line of sight 178 so that the depth cues corresponding to the third graphic element 168 are accurate. And the driver can accurately determine the position of the third graphic element 168 (eg, detected road condition).

  Information related to the monitoring function around the vehicle (eg, blind spot) can be presented to the driver by the fourth projector 124 of the HUD device 102. In this regard, the vehicle control system 180 can detect the presence of other vehicles in the area immediately around or around the vehicle 106. The detection of other vehicles immediately around the vehicle 106 can be performed by processing information related to the surroundings detected by a sensor provided in the vehicle 106. The determination around the vehicle can be performed by any method.

  Information about the surroundings of the vehicle is determined by the vehicle control system 180 and transmitted to the control unit 104. The controller 104 receives vehicle periphery information from the vehicle control system 180 and, if so, determines how to change the fourth graphic element 172 projected on the fourth focal plane 174. In this regard, as the fourth graphic element 172, the graphic element used to support the vehicle periphery (eg, blind spot) monitoring function may be the vehicle periphery indicator shown in FIG.

  The vehicle perimeter sign includes a central marker representing the vehicle 106 and eight surrounding markers representing locations directly surrounding the vehicle 106. The vehicle control system 180 communicates information about the position of the vehicle in the immediate vicinity of the vehicle 106, and the control unit 104 controls the fourth projector 124 to change the fourth graphic element 172, Alternatively, a plurality of corresponding eight surrounding markers are highlighted. Eight surrounding marker highlights indicate to the driver the location of other vehicles in the area directly surrounding the vehicle 106.

  In FIG. 4, the fourth graphic element 172 can be projected onto the fourth focal plane 174. The fourth focal plane 174 can be oriented parallel to the ground surface 176 and can be positioned above the ground surface 176 and line of sight 178. As described above, the fourth projector 124 can be fixedly disposed within the HUD device 102 so that the fourth focal plane 174 is stationary. As described above, the fourth focal plane 174 can be continuous without interruption, and the fourth graphic element 172 can be rendered to the driver with appropriate depth cues as a 3-D image.

  The fourth graphic element 172 may be provided in a different form than the vehicle perimeter sign of FIG. In any event, the fourth graphic element 172 can be projected onto the fourth focal plane 174. The fourth focal plane 174 can be oriented parallel to the ground surface 176 and can be positioned above the ground surface 176 and the driver's line of sight 178. Accordingly, the fourth graphic element 172 can be presented on an empty focal plane. This may be appropriate because the information conveyed by the fourth graphic element 172 need not interact with the environment.

  The HUD system 100 described above has graphic elements, some of which are visual-field-augmented augmented reality graphic elements, continuously changing focal length and front-to-back along the direction of the driver's line of sight 178. Can be projected onto a focal plane parallel to the earth's surface with a constantly changing focal point. Accordingly, depth perception cues can be improved, facilitating focus adjustment, and increasing the driver's attention to the environment at the same time or in parallel or (or nearly simultaneously). This allows the driver to observe the environment as well as the information presented via graphic elements. In this regard, through experimentation, the inventor has shown that spatial perception can be greatly influenced by focus cues and that the graphic elements, along with the focal plane adjustment function of the HUD system 100 described herein, are continuous, and It was determined that the ability to display on a stationary, parallel focal plane would improve spatial cognition. For this purpose, a significant improvement in spatial cognition is observed when adjusting the focus cues described herein, rather than when adjusting the size of the graphic elements.

  In the configuration of the HUD device 102, including the beam splitters 126, 130, 134 and the lenses 128, 132, 136, 138, the size of the HUD device 102 can be relatively compact. Furthermore, the lenses 128, 132, 136, and 138 can extend the depth range from a few meters in front of the vehicle 106 to infinity in the physical space assigned to the optics of the HUD device 102. Furthermore, the beam splitters 126, 130, 134 can be used as optical combiners, with different sets of projection rays from the first, second, third and fourth projectors 118, 120, 122, 124. All are merged through lenses 128, 132, 136, 138 and individual images from the first, second, third and fourth projectors 118, 120, 122, 124 are brought into the driver's field of view. It can be combined into one integrated image or (or graphic element, for example) to be projected.

  In one or more embodiments, any of the above-disclosed and other features and functions, or alternatives or variations thereof, may be combined as desired into many other different systems or applications. Can do. Also, various alternatives, modifications, variations or improvements presently unforeseeable or unforeseeable in the system or application may be made later by those skilled in the art which are intended to be covered by the following claims.

  For example, fewer or more projectors may be used in the HUD system 100 to project fewer or more graphic elements. Further, although the HUD system 100 is described as having two projectors that project graphic elements onto a front focal plane and two projectors that project graphic elements onto a focal plane parallel to the ground, the front focal plane is described. The ratio of the focal plane parallel to the ground can be varied. The vehicle functions described above corresponding to the HUD system 100 are exemplary and can be changed or modified.

  Furthermore, the mechanism for moving the front focal plane can be changed from that described above. For example, rather than moving the entire projector (e.g., the first and third projectors 118, 122 using the first and second actuators 156, 158), simply a diffusing screen (e.g., the first and third projectors). Diffusing screens 148, 152) of the projectors 118, 122 may be moved relative to each projector unit (eg, projector units 140, 144).

  Further, although the HUD system 100 is described with reference to a vehicle 106, the vehicle 106 was an outdoor four-wheel vehicle, the HUD system 100 can be used with different types of vehicles. For example, the HUD system can be used on marine vehicles (eg, boats), aerial vehicles (eg, airplanes or jets), or vehicles intended for indoor use (eg, transportation carts, material handling vehicles such as forklifts, etc.). It may be provided.

  FIG. 5 is an illustration of an example configuration of a 3D navigation system 500 in accordance with one or more embodiments. The system 500 includes a HUD component 100, a vehicle control component 180, a control component 104, a navigation component 540, a depth map component 550, a depth buffer component 560, and one or more sensor components 570. One or more control area networks (CAN) 580 may be included. The HUD component 100 can be a vehicle stereoscopic HUD system, such as the HUD system 100 of FIG. 1, and can include the components described above. In one or more embodiments, the HUD component 100 may be a 3-D HUD, a variable distance HUD, an augmented reality HUD (AR-HUD), etc., among others.

  The navigation component 540 can be configured to receive or identify an origin location (eg, point A) and one or more destination locations (eg, point B). The navigation component 540 can be configured to calculate or determine one or more routes from point A to point B, for example. In general, the navigation component 540 corresponds to one vehicle. For example, the navigation component 540 may be mounted on a vehicle, integrated with one or more systems or components of the vehicle, housed within the vehicle, coupled to one or more components of the vehicle, or Communication connection or placement in a vehicle is possible. In either case, the navigation component 540 can identify or receive the origin location and the destination location. In one or more embodiments, the navigation component 540 can include a telematics component (not shown) that can be configured to determine the current position or current coordinates of the vehicle.

  Further, the navigation component 540 can be configured to generate one or more routes from the origin location to one or more destination locations. In one or more embodiments, the navigation component 540 can be configured to generate one or more groups of routes from the vehicle's current position or current coordinates to one or more destination positions. . One path in the one or more path groups may include one or more parts or one or more parts of the path. For example, one or more portions of the one route may include one or more navigation instructions or navigation operations corresponding to one or more road sections or one or more intersections of road sections. it can. In other words, one or more portions of the route may include one or more turns, navigation operations, road segments, intersections, landmarks, or other elements along the route. The navigation component 540 identifies one or more of these turns, navigation operations, landmarks, etc., and accordingly sends one or more navigation instructions or one or more navigation instructions to the vehicle. It can be configured to be issued to a driver or the like.

  The navigation component 540 can issue the one or more navigation instructions and navigation commands via audio prompts, visual prompts, tactile prompts, and the like. For example, the navigation component 540 transmits one or more prompts via one or more control area networks (CAN) 580 to one or more peripheral components (not shown). Can be interfaced. The navigation component 540 needs to take a driving action to the driver by reproducing an audible command such as “turn left on the main street”, flashing light on the left side of the display, or vibrating the steering wheel. You can show that there is. The navigation component 540 can facilitate the transmission or delivery of one or more driving instructions in coordination with one or more other components.

  For example, the HUD component 100 is configured to project one or more navigation instructions or one or more navigation operations as one or more graphic elements or avatars onto a vehicle occupant or driver's field of view. be able to. These navigation instructions may be received from the navigation component 540 (eg, directly or indirectly). The HUD component 100 projects an avatar onto successive focal planes so that the avatar appears to move to an occupant, such as a driver, with a field of view from the eye box 116 of FIG. be able to. In this way, the HUD component 100 allows the driver to recognize a stereoscopic image within the driver's field of view. Here, the stereoscopic image can serve as a “virtual” guidance vehicle that the vehicle driver should follow. In other words, it appears to the driver of the vehicle that he or she is simply following, for example, a guidance vehicle to the target position. In addition, as will be described later in this specification, one or more other navigation instructions or navigation commands may be projected as a three-dimensional placeholder, marker or flagpole. is there.

  The HUD component 100 can be configured to project one or more graphic elements. The graphic elements can be visual field-like augmented reality graphic elements, conformal augmented reality graphic elements, avatars, icons, and the like. These graphic elements can be projected in three dimensions by the HUD component 100. As a result, one or more visual cues or one or more depth cues corresponding to the graphic element can be substantially maintained. Maintenance of these one or more visual cues or depth cues can be achieved by projecting or rendering the graphic element onto a dynamic or movable focal plane. That is, the HUD component 100 can be configured to project or render one or more graphic elements onto a movable or adjustable focal plane. The dynamic focal plane or the movable focal plane can be moved or adjusted along a path or track, such as the line of sight of a vehicle occupant, as described with reference to FIGS. In other words, the dynamic focal plane can be movable in a direction toward or away from the vehicle or windshield.

  In one or more embodiments, the focal plane can be dynamic, for example, as a result of movement of the projector or screen of the HUD component 100 through the use of an actuator. That is, one or more projectors of the HUD component 100 can be configured to move linearly, whereby each projector can dynamically, moveably or adjust one or more graphic elements. It is possible to project onto a possible focal plane. The focal plane moves in accordance with the movement of the projector. In other embodiments, one or more other means or alternative means may be used for adjustment.

  Stated another way, when a graphic element is projected onto a dynamic, movable or adjustable focal plane, the graphic element may be distant from the vehicle (eg, distance 162 ′ of FIG. 3). Alternatively, the distance 170 ′) can be projected onto the adjusting focal plane. The projector of the HUD component 100 can project or render graphic elements onto a movable focal plane so that the projected graphic elements can be focused at various distances from the vehicle. As described above, one or more focal planes can be oriented substantially perpendicular or substantially parallel to the line of sight of the vehicle occupant. That is, the focal plane can be perpendicular to the ground or parallel to the ground. Also, one or more of the focal planes may be movable or stationary with respect to the occupant's line of sight or the ground. This allows the depth cues associated with the graphic elements to be accurately presented to a vehicle occupant, such as a driver, when the vehicle moves or travels (and thus acts as a mobile platform, for example). it can.

  The HUD component 100 of FIG. 5 can be configured to project or render a stereoscopic field-like augmented reality graphic element. This means that these graphic elements can be projected to be displayed at various distances. In other words, the HUD component 100 can project graphic elements onto multiple focal planes or in a focusable manner. In yet another way, the focal plane of the graphic element projected by the HUD component 100 can be adjusted to a distance extending beyond the windshield, such as next to a pedestrian on the sidewalk, This allows the occupant to focus on the operating and operating environment and eliminates the need to switch eye focus between the vehicle windshield or instrument panel and the driving environment. Thus, safety can be increased by the 3D navigation system 500.

  Thus, the graphic elements can be projected or visually positioned (eg, by the HUD component 100) in an environment within the occupant's direct field of view. This means that the graphic element can be rendered in the same space as the real environment, not rendering on the windshield. Then, the depth cue corresponding to the graphic element can be accurately and correctly reproduced. As a result, graphic elements can be projected onto the same focal plane as real-world objects (eg, roads), allowing vehicle occupants to view graphic elements without looking away from the road, for example. It can be so.

  The reason that these multiple focal planes or adjustable focal planes can be realized is that when the projector of the HUD component 100 moves, the light rays are reconstructed or deformed so that the projected graphic elements or virtual objects are better than the windshield. This is because it can have a focal plane that appears far away or is not on the windshield. That is, the projected graphic element or virtual object can have the same focus characteristics as a real object (eg, pedestrian, vehicle, sign, etc.) that is far away (eg, 10 meters away). The rays are reflected off the windshield and the outgoing rays diverge, thereby creating a “reflected” image, ie a real image, which is projected as a graphic element.

  When the occupant moves his head, the rays are reflected off the windshield and are not emitted or appear from the windshield (eg, with a special coating). There is no need to re-render the graphic elements. For example, the continuous stationary focal plane of FIG. 3 makes it possible to generate an optically “correct” or real image along the front-rear direction in the three-dimensional space (eg, the direction of the sight line of the occupant). This makes it possible to generate appropriate motion parallax cues. Therefore, when the occupant's head moves, the graphic elements corresponding to these focal planes appear to be fixed in position in the environment and do not appear to swing. As mentioned above, this means that the HUD component 100 does not require a head tracking function to compensate for occupant head movement.

  The HUD component 100 can be raster based rather than vector based. This means that the graphic element projected by the HUD component 100 is a bit map, has a dot matrix structure, or is a pixel in a rectangular grid. Further, the HUD component 100 can be configured to project one or more portions of one or more graphic elements having different shading, transparency, color, brightness, etc.

  As such, the HUD component 100 can be configured to render or project graphic elements or avatars with varying degrees of freedom. That is, the focus adjustment can be maintained so that the occupant's eyes actively change their refractive power and concentrate on the graphic elements projected on the focal plane. Similarly, when a graphic element is projected to move “closer” (eg, by projecting onto a contiguous focal plane), congestion is maintained and the occupant is able to simultaneously Inward rotation can be obtained.

  In one or more embodiments, the HUD component 100 may project graphic elements as avatars or moving avatars that a vehicle driver or occupant should follow as navigation instructions, navigation operations, or navigation instructions. it can. For example, the HUD component 100 can be configured to project or render one or more graphic elements as moving avatars, display location securing symbols, identifiers, flag poles, markers, and the like. These graphic elements can be projected onto one or more focal planes around the vehicle's surrounding environment and projected into the vehicle's occupant's field of view. The avatar or graphic element projected by the HUD component 100 guides the vehicle driver over one or more parts of the path and proceeds to sew around the obstacle, navigate, move, Or by projecting so that it may drive | work, the collision with an obstruction, an obstacle condition, or a road condition can be avoided. The sensor component 570 is configured to sense one or more obstacles or road conditions, and the control component 104 instructs the HUD component 100 to project graphic elements, eg, a traffic regulation barrel. For example, by changing the lane to avoid the situation, the vehicle can travel or detour around the road condition.

  In one or more embodiments, the sensor component 570 can be configured to sense, identify, or detect one or more road conditions around or surrounding the vehicle. The sensor component 570 may include road sections, sidewalks, objects, pedestrians, other vehicles, obstacle situations, obstacles, debris, depressions, road surface conditions (eg, ice, rain, sand, gravel, etc.), traffic conditions or traffic signs. (Eg, red light, speed limit sign, stop sign, railroad crossing, train, etc.) can be detected or identified. These road conditions are transmitted to the control component 104 or the vehicle control component 180. For example, one or more CANs 580 can be used to facilitate communication between the sensor component 570 and the control component 104 or vehicle control component 180. In one or more embodiments, sensor component 570 can include one or more imaging devices, microphones, blind spot monitors, parking sensors, proximity sensors, objective sensors, infrared sensors, motion sensors, and the like.

  The vehicle control component 180 can be configured to receive data relating to one or more road conditions or data related to the environment surrounding the vehicle (eg, operating environment, driving environment, surrounding environment, etc.). In one or more embodiments, the vehicle control component 180 can receive one or more road conditions from the sensor component 570. In addition, the vehicle control component 180 can receive one or more road conditions from one or more other sources, such as a server (not shown) or a database (not shown). The vehicle control component 180 can be communicatively connected to a server, third party, database or others via a telematics channel activated via a telematics component (not shown). In this manner, the vehicle control component 180 can collect information regarding one or more portions of the route from the origin location to the destination location.

  For example, the vehicle control component 180 can receive road status information including traffic information for road segments (eg, whether traffic is congested when there is an accident on the road, etc.). In addition, the vehicle control component 180 can receive speed limit information corresponding to one or more road segments of the route. This information is used to determine how to project one or more graphic elements to a vehicle driver or occupant. That is, a road segment corresponds to a speed limit of 65 mph (104.605 km), but the current speed of the vehicle (detected by sensor component 570, for example) is 25 mph (40.233 km). In some cases, the vehicle control component 180 instructs the HUD component 100 to project an avatar and displays it to accelerate when it enters the road segment.

  As another example, if the sensor component 570 detects a traffic regulation barrel in the current lane in which the vehicle is traveling, the vehicle control component 180 receives this information and a lane change navigation instruction is received by the HUD component 100. A decision can be made that it needs to be projected. The indication can be sent to the HUD component 100 through one or more CANs 580. The HUD component 100 can project, render, or animate lane change or relocation avatars or graphic elements in response to detected traffic control barrels. In other words, the HUD component 100 can project an avatar or icon that is displayed as it proceeds to sew or pass around the traffic control barrel. The traffic regulating barrel is located in front of the vehicle in an operating environment surrounding the vehicle. Also, the vehicle control component 180 can be configured to cause the HUD component 100 to project a turn signal light on the avatar of the vehicle for display when the actual vehicle changes lanes. In addition, the vehicle control component 180 can adjust the perception speed of the avatar as the avatar approaches the traffic control barrel. This can be achieved by continuously projecting an avatar or graphic element onto a closer focal plane or by adjusting the dynamic focal plane of the graphic element and the vehicle or vehicle windshield. It can be realized such that the distance between them decreases. (Conversely, if it is desired to project the avatar to accelerate, the dynamic focal plane is adjusted so that the distance between the dynamic focal plane and the vehicle or the vehicle windshield increases. be able to).

  In other words, the vehicle control component 180 can be configured to receive one or more road conditions. One of the one or more road conditions includes traffic information of one or more road sections or speed limit information corresponding to one or more road sections. In addition, the vehicle control component 180 activates the HUD component 100 based on one or more graphic elements based on one or more road conditions, such as a road segment speed limit and the vehicle's current speed. Can be configured to project. In this manner, the vehicle control system 180 determines one or more appropriate actions (eg, stop, acceleration, lane change, deceleration, etc.) or navigation commands to be projected by the HUD component 100. be able to.

  In one or more embodiments, the system 500 can include a view management component (not shown) that manages one or more aspects of one or more graphic elements projected by the HUD component 100. . In one or more embodiments, the control component 104 can be configured to manage functions associated with one or more of the side or vehicle control components 180. For example, the control component 104 can be configured to receive one or more road conditions.

  The control component 104 can be configured to determine the types of graphic elements that are displayed, projected, animated or rendered by the HUD component 100. For example, when the vehicle is traveling along one or more portions of a route that includes a relatively straight road segment, the control component 104 projects a graphical element that is an avatar. The avatar is displayed or projected as a vehicle or a guidance vehicle. In situations where a vehicle is traveling along one or more portions of a route that includes one or more turns or other navigation operations, the control component 104 instructs the HUD component 100 to provide one or more A graphic element that is a marker can be projected at a location corresponding to a plurality of corners. For example, if the route includes a right turn from the first street to the second street, the control component 104 instructs the HUD component 100 to place the marker or identifier at the intersection of the first street and the second street. Can be projected toward or around the intersection. As such, the control component 104 can be configured to determine one or more types of graphic elements to be displayed (eg, markers, identifiers, flagpoles, guidance avatars, etc.).

  Further, the control component 104 can be configured to determine one or more positions at which to project the graphic element. In other words, the control component 104 can determine when and where to project a graphic element or how to display the graphic element. The position of the graphic element includes the focal plane, the vehicle or the distance from the vehicle windshield to the focal plane, eg, x, y, z-axis x, y, z coordinates, etc. The location can be referred to as a target location for one or more graphic elements. In one or more embodiments, the control component 104 determines the distance between one or more focal planes of the one or more graphic elements and the vehicle (or vehicle windshield, for example), Can be configured to adjust based on one or more road conditions corresponding to one or more portions of the route, the current position of the vehicle, the current speed of the vehicle, etc.

  That is, a road segment (eg, the portion of the route on which the vehicle is currently located or located) corresponds to a speed limit of 65 miles per hour (eg, 104.605 km) (eg, road conditions) , Detected by sensor component 570), if the current speed is 25 mph (40.233 km) (eg, the current speed of the vehicle), control component 104 instructs HUD component 100 to provide approximately 65 mph It can be configured to project an avatar or graphic element that appears to be traveling at (104.605 km). In one or more embodiments, the avatar can be projected to represent a gradual acceleration from 25 miles per hour (40.233 km) to 65 miles per hour (104.605 km). This means that the distance between the focal plane of the avatar and the vehicle can be adjusted as appropriate. For example, in a scene where the vehicle accelerates at approximately the same pace, the distance between the focal plane and the vehicle may remain approximately the same. If the vehicle accelerates at a slower pace than the avatar, it can be adjusted by the control component 104 to increase the distance between the focal plane and the vehicle. In any case, the adjustment can be made based on the current position of the vehicle or the current speed of the vehicle and the road conditions of the associated route.

  Further, the control component 104 can be configured to adjust or determine the size of a graphic element in conjunction with the HUD component 100 according to or based on the distance of the graphic element's focal plane and the vehicle. This means that the control component 104 can adjust the height, size, width, depth, etc. of the graphic element, guidance icon or avatar based on the target perceptual image. For example, to allow the avatar to appear to accelerate, the control component 104 projects the avatar onto a focal plane that continuously moves away while adjusting the size of the avatar to shrink or shrink, or The dynamic focal plane can be adjusted gradually away from the vehicle.

  In one or more embodiments, the size of the graphic element can be used as an indication of the importance of a navigation instruction or message. In other words, the more important the message or navigation instruction, the larger the avatar, icon or graphic element that is projected.

  The control component 104 can be configured to determine that the HUD component 100 projects one or more actions on one or more graphic elements. For example, the control component 104 issues an instruction to the HUD component 100 to project an avatar, and the avatar accelerates, decelerates, stops, changes lane, starts a direction light before changing lane, flashes, blinks, and avatar direction It is possible to change the angle and the avatar color. Further, the control component 104 can adjust the target position of the one or more graphic elements based on road conditions, the current position of the vehicle, the current speed of the vehicle, or other attributes, characteristics, or measurement results. it can. In one or more embodiments, the control component 104 can interface or communicate with the navigation component 540 via one or more CANs 580.

  The control component 104 can be configured to mitigate fault conditions, confusion, or other situations that can be an obstacle to the driver or occupant of the vehicle. In one or more embodiments, the control component 104 can be configured to receive a horizontal position from the sensor component 570 or the like and to project the graphic element onto a horizontal or empty plane. The control component determines the color, transparency or shadow of one or more graphic elements based on the time of day, the traffic congestion associated with the route, the driver's familiarity with the route, etc. Or it can be configured to adjust.

  The depth map component 550 can be configured to build or receive a depth map of an environment around or surrounding a vehicle, such as an operating environment. The HUD component 100 can project one or more graphic elements using the depth map accordingly. This is because if the avatar turns around a corner and hides in the building (eg, the building is between the vehicle occupant ’s line of sight and the perceived or target position of a graphic element or avatar), the HUD component 100 It means that the projection of one or more parts of an avatar or graphic element that is on the line of what is to be visible can be enabled or disabled.

  The depth map component 550 can be configured to receive a depth map from a server or a third party server. For example, the depth map component 550 can download a depth map from a server via a telematics channel activated via a telematics component (not shown). In other embodiments, the sensor component 570 can be configured to detect depth information that the depth map component 550 can use to construct a depth map. That is, the depth map component 550 can interface or communicate with one or more sensors to build a depth map or receive a pre-built depth map from a database. In any case, the depth map component 550 can construct or receive a depth map based on the depth information. The depth map represents the distance of one or more surfaces, objects, obstacle situations, spatial shapes, etc. within the environment or area around the vehicle.

  The depth map can be passed to or transmitted to the control component 104. The control component 104 can instruct the HUD component 100 to render one or more graphic elements accordingly. For example, the HUD component 100 projects or renders graphic elements based on the height of the eye box corresponding to the vehicle occupant, the position of the vehicle, and the depth map of the region. The depth map can be proactively sensed or received from a database. The HUD component 100 can project one or more graphic elements based on the depth map to display a perspective view for one or more occupants of the vehicle.

  The depth buffer component 560 can be configured to facilitate perspective control of one or more occupants of a vehicle using the depth map generated or received by the depth map component 550. That is, the depth buffer component can be configured to facilitate rendering of graphic elements such that the graphic elements are visually “accurate” to the occupant. For example, if a graphic element needs to be projected behind a real-world object, the depth buffer component 560 “hides” the graphic element from the occupant without projecting or rendering the graphic element. "be able to. In other words, the depth buffer component 560 can control which portions (eg, pixels) of a graphic element are drawn, projected or rendered, and which portions are not. For this purpose, the depth buffer component 560 can be configured to enable or disable rendering of one or more portions of one or more of the graphic elements based on the depth map. it can.

  Additionally, the depth buffer component 560 can be configured to blur real world objects, thereby hiding what a vehicle occupant can see. For example, the depth buffer component 560 instructs the HUD component 100 to project a white graphic element that is detected by a real-world object such as a signboard (eg, sensor component 570). Can be covered. As a result, the sign cannot be seen by the occupant or may appear unclear. In this manner, the depth buffer component can be configured to mitigate vehicle driver and occupant confusion by providing graphic elements that enable impaired reality.

  Examples of navigation commands that can be projected by the HUD component 100 include following a guided vehicle, acceleration (eg, changing the dynamic focal plane to increase the distance from the focal plane to the vehicle, thereby Adjusting the perspective that the driver or occupant may have on the graphic element), decelerating (eg adjusting the distance between the focal plane and the vehicle to decrease), lane changing (eg the target of the graphic element) Adjust position), circumvent obstacles, turn, arrive, mark location, etc. As an example, when the pedestrian steps into a road section, road, pedestrian crossing, etc., the control component 104 can instruct the HUD component 100 to project the avatar to decelerate. As another example, the control unit 104 may instruct the HUD component 100 to project the avatar to decelerate based on the turning angle, the speed limit corresponding to the road section, and road conditions such as ice. That is, when ice is on the road surface, the control unit 104 can instruct the HUD component 100 to project an avatar that travels slower than when ice is not present on the road surface.

  In one or more embodiments, the control component 100 can mark or identify the next turn or intersection with a marker, flag post, flagpole, identifier, or the like. For example, the HUD component 100 can render or project a display location securing symbol or marker along a vehicle occupant's view. The depth map component 550 may be configured to provide a depth map in which multiple real objects, such as buildings, trees, etc., act as a line of sight obstruction for one or more portions of the display location securing symbol. it can. As an example, the display position securing symbol has a height to be perceived of 100 feet (30.48 m), and a building with a height of 50 feet (15.24 m) is in front of the display position securing symbol. , The depth buffer component 560 can be corrected to obstruct the line of sight by disabling rendering or projection of the bottom of the graphic element of the display position securing symbol, and thus the driver or occupant perspective. The display position securing symbol can be rendered according to the above.

  In one or more embodiments, one or more graphic elements are projected onto a vehicle occupant's field of view based on the path (eg, guided vehicle following mode). In one or more embodiments, the graphic element can be projected as an avatar or other guidance icon. The avatar appears to be flying and displayed towards the real world environment around the vehicle. An avatar can move, navigate or “fly” in 3-D space or in three dimensions. For this reason, an avatar and a graphic element seem to move within 3-D, and can give a more intuitive feeling and a sense of security to the crew member or driver who follows an avatar by it. As an example, an avatar, graphic element or guidance icon can be projected such that the height or size appears to change based on the perceived distance from the vehicle occupant. The avatar can be animated by sequentially projecting the moving avatar onto one or more different focal planes. Further, the avatar may appear to bypass around obstacle situations, obstacles, pedestrians, debris, dents, etc. as an actual vehicle does. In one or more embodiments, an avatar may appear to “drive”, move, or move according to real-time traffic conditions. The avatar can change lanes in such a way that it does not appear to "clash" with other vehicles or obstruct traffic. As another example, if a driver or vehicle needs to cross a track on a route, the avatar can stop on the track when the train crosses. In other embodiments, the HUD component 100 may be configured to stop with a stop sign or red light, or to project an avatar or graphic element that complies with traffic laws. When arriving at the destination, the HUD component 100 can be configured to render or project a resting avatar, for example.

  In this manner, the 3D navigation system 500 can generate intuitive messages, instructions, or instructions for a vehicle occupant such as a driver. The instructions can be based on one or more attributes related to perspective, where the perspective projects or renders a stereoscopic 3-D graphic element along one or more adjustable focal planes. It depends on. For example, the 3-D effect can be determined based on distance, perspective, perceived distance, road conditions, and the like.

  FIG. 6 is an illustration of an example flow 600 of a 3-D navigation method, according to one or more embodiments. At 602, a route is generated from the origin position to the destination position. In one or more embodiments, the origin location or destination location may be received via a telematics channel, for example, from a Global Positioning System (GPS) unit. At 604, the one or more graphic elements are projected onto one or more focal planes within the field of view of the vehicle occupant. Here, the graphic element can be displayed as an avatar, an image, an icon, an identifier, a marker, or the like. Furthermore, these graphic elements can be based on one or more parts of the path. This means that these graphic elements can be projected at various distances depending on the part of the path in which the vehicle may be located (eg the current position of the vehicle).

  At 606, the distance between the focal plane and the vehicle is adjusted based on road conditions corresponding to one or more portions of the route. Furthermore, the distance can also be adjusted based on the current speed of the vehicle. For example, if a vehicle travels along a part of a route with a speed limit of 65 mph (104.605 km) and the current speed of the vehicle is 25 mph (40.233 km) Increase the distance between the rendered graphic element and the focal plane of the avatar (eg, to instruct the driver or occupant to accelerate). In other words, the graphic element can be projected to appear to run at about 65 miles per hour (104.605 km), thereby urging the occupant or driver to accelerate and "catch up" the avatar. (E.g., mimicking following a guidance vehicle or simulating it).

  FIG. 7A is an illustration of an example avatar 700 for 3-D navigation, according to one or more embodiments. The avatar 700 of FIG. 7A appears in front of the vehicle and can fly, glide, move, maneuver, etc. around elements, obstacles, traffic, road conditions, etc. FIG. 7B is an illustration of an example 710 of avatar (s) for 3-D navigation, according to one or more embodiments. The avatar (s) 710 in FIG. 7B is visible from a high field of view, for example, as seen in a bird's eye view from slightly behind the avatar 710. It can be seen that one or more avatars 710 are projected onto one or more different focal planes or target positions, thereby giving the driver or occupant the feeling of tracking an actual vehicle.

  FIG. 8A is an illustration of an example avatar 800 for 3-D navigation, according to one or more embodiments. The avatar 800 in FIG. 8A rotates counterclockwise to indicate a left turn. FIG. 8B is an illustration of an example avatar 810 for 3-D navigation, according to one or more embodiments. In one or more embodiments, the avatar 810 of FIG. 8B can indicate a left turn by blinking, flashing, changing colors, and the like. For example, the left wing of a paper airplane avatar 810 may indicate that it will shine or change brightness and then turn left. In one or more embodiments, an avatar can be projected onto a focal plane close to the vehicle so that the avatar appears to “decelerate” before it turns.

  FIG. 9A is an illustration of an example avatar 900 for 3-D navigation, according to one or more embodiments. FIG. 9B is an illustration of an example avatar 910 for 3-D navigation, according to one or more embodiments. The avatar 900 of FIG. 9A can be projected as a navigation command for a vehicle driver to decelerate, for example. In FIG. 9B, the avatar 910 is projected above the horizon or onto an empty surface so that the avatar 910 does not interfere with the driver or occupant looking at one or more parts of the surrounding environment of the vehicle.

  In yet another embodiment, a computer-readable medium that includes processor-executable instructions configured to implement one or more embodiments of the technology disclosed herein is included. FIG. 10 shows an embodiment of a computer readable medium or a computer readable device devised by the above method. In FIG. 10, implementation 1000 includes a computer readable medium 1008, such as a CD-R, DVD-R, flash drive, hard disk drive platter, on which computer readable data 1006 is encoded and recorded. included. Computer readable data 1006, such as binary data including a plurality of zeros or ones, as shown at 1006, includes a set of computer instructions configured to operate in accordance with one or more of the principles described herein. Includes in order. In one such embodiment 1000, processor-executable computer instructions 1004 are configured to perform a method 1002, such as the method 600 of FIG. In another embodiment, the processor executable instructions 1004 are configured to implement a system such as the system 500 of FIG. Many of the above computer readable media are devised by those skilled in the art to operate in accordance with the techniques introduced herein.

  As used herein, the terms “component”, “module”, “system”, “interface”, etc., generally refer to computer-related entities, hardware, a combination of hardware and software, software or running Is intended to point to any of the software. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, an execution thread, a program, or a computer. Depending on the description, both the application running on the control unit and the control unit may be components. One or more components present in a process or execution thread and one component can be localized on one computer or distributed between two or more computers.

  Further, the claimed subject matter uses methods of standard programming or engineering to produce software, firmware, hardware, or any combination thereof that controls a computer that implements the disclosed subject matter, Realized as a device or manufactured article. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer readable device, computer readable carrier, or computer readable medium. Of course, many modifications may be made to the structure without departing from the scope or spirit of the claimed subject matter.

FIG. 11 and the following discussion describe a computing environment suitable for implementing one or more of the condition embodiments described herein.
The operating environment of FIG. 11 is merely an example of a suitable operating environment and does not suggest any limitation with respect to the usage range or functional range of the operating environment. Exemplary computing devices include personal computers, server computers, handheld or laptop devices, mobile devices such as mobile phones, personal digital assistants (PDAs), media players, multiprocessor systems, consumption Consumer electronics, mini computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

  In general, embodiments are described in the general flow of “computer readable instructions” executed by one or more computing devices. The computer readable instructions are distributed by computer readable media as described below. A computer readable instruction is a program, such as a function, object, application programming interface (API), data structure, etc., that performs one or more tasks or implements one or more abstract data types. Implemented as a module. In general, the functionality of computer-readable instructions is combined or distributed as desired in various environments.

  FIG. 11 is an illustration of a system 1100 that includes a computing device 1112 configured to implement one or more embodiments provided herein. In one configuration, computing device 1112 includes one or more processing units 1116 and memory 1118. The memory 1118 can be non-volatile such as RAM, volatile such as ROM, flash memory, or the two, depending on the detailed configuration and type of computing device. Can be combined. This configuration is indicated by the dashed line 1114 in FIG.

  In other embodiments, device 1112 has additional features or functions. For example, the device 1112 can include a removable storage device or a non-removable storage device. This includes, but is not limited to, additional storage devices such as magnetic storage devices and optical storage devices. Such an additional storage device is indicated by a storage unit 1120 in FIG. In one or more embodiments, computer readable instructions that implement one or more embodiments provided herein are stored in storage 1120. The storage unit 1120 can store other computer-readable instructions that implement an operating system, application programs, and the like. Computer readable instructions are loaded into memory 1118 for execution by processing unit 1116, for example.

  The term “computer-readable medium” as used herein includes computer storage media. Computer storage media includes volatile and non-volatile removable and non-removable media implemented with any method or technique for storing information such as computer-readable instructions or other data. The memory 1118 and the storage unit 1120 are examples of computer storage media. Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, magnetic cassette, magnetic tape, magnetic disk storage device or This includes, but is not limited to, other magnetic storage devices or any other medium that can be used to store desired information and that can be accessed by device 1112. Any computer storage medium as described above is part of device 1112.

  The term “computer-readable medium” includes communication media. Communication media typically embodies computer readable instructions or other data in the form of a “modulated data signal” such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” includes a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

  Device 1112 includes input device (s) 1124 such as a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device or any other input device. Output device (s) 1122 may be included in device 1112, such as one or more displays, speakers, printers or any other output device. Input device (s) 1124 and output device (s) 1122 are connected to device 1112 via a wired connection, a wireless connection, or any combination thereof. In one or more embodiments, input device (s) or output device (s) from another computing device may be input device (s) 1124 or output device (s) 1122 for computing device 1112. Used as. Device 1112 can include communication connection (s) 1126 to facilitate communication with one or more other devices.

  According to one or more embodiments, a three-dimensional (3-D) navigation system is provided that includes a navigation component configured to receive an origin location and a destination location. The navigation component is associated with one vehicle and can be configured to generate a route from a starting position to a destination position. One or more portions of the route may include one or more navigation instructions associated with one or more road sections or one or more intersections of the road sections. The system may include a head up display (HUD) component configured to project one or more graphic elements onto one or more focal planes around the vehicle's surrounding environment. The HUD component may be configured to project one or more graphic elements onto the vehicle occupant's field of view based on the path. The system determines a distance between one or more focal planes of one or more graphic elements and the vehicle, one or more road conditions corresponding to one or more portions of the route, and the A control component can be included that is configured to adjust based on the current position of the vehicle.

  In one or more embodiments, the control component can be configured to adjust the target position of one or more graphic elements based on one or more road conditions or the current position of the vehicle. . The system can include a vehicle control component configured to receive one or more road conditions. Further, the system can include a sensor component configured to detect one or more road conditions. One road condition of the one or more road conditions may include traffic information of one or more road sections or speed limit information corresponding to the one or more road sections. it can. Furthermore, the road conditions can include, for example, obstacle conditions, obstacles, pedestrians, debris or depressions.

  The system can include a depth map component configured to build a depth map of the surrounding environment of the vehicle. The HUD component can be configured to project one or more graphic elements based on a depth map of the environment. The depth map component can be configured to build a depth map based on the depth information. In one or more embodiments, the system can include a sensor component configured to detect depth information from the vehicle's surrounding environment. The depth map component can be configured to receive a depth map based on a telematics channel. The system can include a depth buffer component configured to enable or disable rendering of one or more portions of one or more graphic elements based on the depth map.

  The HUD component can be configured to project one or more graphic elements, such as flagpoles, markers, identifiers, as mobile avatars or display location securing symbols.

  According to one or more embodiments, a method for three-dimensional (3-D) navigation is provided that includes generating a route from a starting location to a destination location for a vehicle. One or more portions of the route may include one or more navigation instructions associated with one or more road sections or one or more intersections of the road sections. The method can include projecting one or more graphic elements onto one or more focal planes around the environment surrounding the vehicle. One or more graphic elements can be projected onto the occupant's field of view of the vehicle based on the path. The method includes determining a distance between one or more focal planes of one or more graphic elements and the vehicle, one or more road conditions and vehicles associated with one or more portions of the route. Adjusting based on the current position of the current position. One or more parts of the method may be performed via the processing unit.

  The method may include adjusting the target position of the one or more graphic elements based on the one or more road conditions and the current position of the vehicle. The method can include receiving or detecting one or more road conditions. One road condition of the one or more road conditions includes traffic information of one or more of the road sections, speed limit information corresponding to the one or more of the road sections, fault conditions, and faults. It can include objects, pedestrians, debris or depressions.

  The method includes, among other things, constructing a depth map of the vehicle's ambient environment, projecting one or more graphic elements based on the environment's depth map, detecting depth information from the vehicle's ambient environment, Constructing a depth map based on the detected depth information can include enabling or disabling rendering of one or more portions of one or more graphic elements based on the depth map.

  According to one or more embodiments, a computer-readable storage medium includes computer-executable instructions that perform a plurality of operations when executed through a processing unit of a computer. The step included in the operation is a step of generating a route from the starting position to the target position for the vehicle, wherein one or more parts of the route are one or more road sections, or road sections Projecting one or more graphical elements onto one or more focal planes around the vehicle's surrounding environment, including one or more navigation instructions associated with the one or more intersections Projecting one or more of the graphic elements onto a field of view of an occupant of the vehicle based on the route, or one or more roads associated with one or more portions of the route Adjusting the distance between one or more focal planes of the one or more graphic elements and the vehicle based on the situation and the current position of the vehicle is included.

  In one or more embodiments, projecting the one or more graphic elements utilizes raster-based graphics. Further, in one or more embodiments, the moving avatar is projected by projecting one or more graphic elements as moving avatars or sequentially projecting the moving avatar onto one or more different focal planes. Providing one or more navigation instructions via the animating step.

While the claimed subject matter has described structural features or methodological actions in a particular language, it should be understood that the subject matter is not necessarily limited to the particular features or actions described above. . Rather, the specific features and acts described above are disclosed as example embodiments.

Various operations of the embodiments are described herein. The order in which one or more or all of the operations are described should not necessarily be construed as implying that these operations are order dependent. Alternative sequences will be understood based on the description herein. Moreover, not all operations can be illustrated in the embodiments provided herein.

  As used in this application, “or” means generic “or” and is not intended to mean an exclusive “or”. Also, the articles “a” and “an” as used in this application generally refer to “one or more” unless the context clearly dictates or unless otherwise clear from the context. Should be interpreted as "multiple". Furthermore, at least one of A and B and / or similar expressions generally means A or B, or both A and B. Further, “includes”, “having”, “has”, “with”, or variations thereof are described in the foregoing detailed description or claims. As used herein, the term is meant to encompass as well as the term “comprising”.

  Further, unless otherwise specified, “first”, “second”, etc. are not intended to mean temporary, spatial, ordering, etc. Rather, such terms are merely used as identifiers, names, etc., for features, elements, items, etc. For example, the first channel and the second channel generally correspond to channel A and channel B or two different channels or two equal channels or the same channel.

  Although this disclosure has illustrated and described with respect to one or more implementations, equivalent changes and modifications may be made based on reading and understanding of this specification and the accompanying drawings. The present invention includes all such modifications and changes and is limited only by the scope of the following claims.

DESCRIPTION OF SYMBOLS 100 Head-up display system, HUD system, HUD component 102 Head-up display device, HUD device 104 Control part, control component 106 Vehicle 108 Driver's seat 110 Dashboard housing 112 Front glass 114 HUD exit hole 116 Eye box 118, 120, 122, 124 First, second, third, fourth projector 126, 130, 134 First, second, third beam splitter 128, 132, 136 First, second , Third objective lens 138 eyepiece 140, 142, 144, 146 projector unit 148, 150, 152, 154 diffusion screen 156, 158 first, second actuator 160, 164, 168, 172 first, 2, third and fourth graphic elements 162, 166, 170, 174 first, second, third and fourth focal planes 162 ', 170' distance 176 ground surface 178 driver's line of sight 180 vehicle control system , Vehicle control component 182 pedestrian 200 vehicle with HUD system 300 side view of example vehicle and 4 focal planes 302 windshield 400 driver view and example of graphic elements 1000 computer readable media / device embodiments 1002 Method 1004 Processor-executable instructions 1006 Computer-readable data 1008 Computer-readable medium 1100 System including computing device 1112 Computing device 1114 Processing unit and memory 1116 Processing unit 111 8 Memory 1120 Storage unit 1122 Output device 1124 Input device 1126 Communication connection unit 1128 Network 1130 Computing device

Claims (20)

It is configured to receive a starting position and a destination position and generate a route from the starting position to the destination position, and one or more parts of the route are one or more road sections or the road A navigation component that includes one or more navigation instructions corresponding to one or more intersections of a section and is associated with a vehicle;
A head-up display component that projects one or more graphic elements onto one or more focal planes surrounding the environment surrounding the vehicle;
One or more road conditions corresponding to one or more portions of the route, the distance between one or more of the focal planes of the one or more graphic elements and the vehicle, and the vehicle A control component that adjusts based on the current position of the
A three-dimensional navigation system comprising:   The system of claim 1, wherein the control component adjusts target positions of one or more of the graphic elements based on one or more of the road conditions and the current position of the vehicle. .   A vehicle control component that receives one or more road conditions, wherein one of the one or more road conditions includes one or more traffic information of the road section or one or more of the one or more of the road conditions. The system according to claim 1, comprising speed limit information corresponding to a road section.   Detecting one or more of the road conditions, wherein one of the one or more road conditions includes a sensor component that includes an obstacle condition, an obstacle, a pedestrian, a fragment, or a depression; The system according to claim 1.   A depth map component that constructs a depth map of the environment that surrounds the vehicle and the HUD component that projects one or more of the graphic elements based on the depth map of the environment. Item 4. The system according to Item 1.   6. The system of claim 5, including a sensor component that detects depth information from the environment surrounding the vehicle, the depth map component constructing the depth map based on the depth information. .   The system of claim 5, wherein the depth map component receives the depth map via a telematics channel.   The depth buffer component for enabling or disabling rendering of one or more portions of one or more of the graphic elements based on the depth map. system.   The system of claim 1, wherein the HUD component projects one or more graphic elements as moving avatars.   The system of claim 1, wherein the HUD component projects one or more of the graphic elements as a display position securing symbol. Generating a route from a starting position to a destination position for a vehicle, wherein one or more portions of the route include one or more road sections or one or more of the road sections Including one or more navigation instructions corresponding to the intersection;
One or more graphic elements are projected onto one or more focal planes surrounding the environment surrounding the vehicle, and the one or more graphic elements are projected based on the path of the occupant of the vehicle. Projecting into the field of view;
The distance between one or more focal planes of the one or more graphic elements and the vehicle, the one or more road conditions corresponding to one or more parts of the route, and the vehicle's Adjusting based on the current position,
3. The three-dimensional navigation method, wherein the generating step or the adjusting step is performed through a processing unit.   The method of claim 11, comprising adjusting a target position of one or more of the graphic elements based on one or more of the road conditions and the current position of the vehicle.   Receiving or detecting one or more of the road conditions, wherein one of the one or more road conditions is traffic information of one or more of the road sections or one or more The method according to claim 11, comprising speed limit information, obstacle conditions, obstacles, pedestrians, debris or depressions corresponding to a plurality of the road sections.   12. The method of claim 11, comprising building a depth map of the environment surrounding the vehicle and projecting one or more of the graphic elements based on the depth map of the environment. The method described. Detecting depth information from the environment surrounding the vehicle;
15. The method of claim 14, comprising building the depth map based on the detected depth information.   The system of claim 14, comprising enabling or disabling rendering of one or more portions of one or more of the graphic elements based on the depth map. Generating a route from a starting position to a destination position for a vehicle, wherein one or more portions of the route are one or more road sections or one or more intersections of the road sections Including one or more navigation instructions corresponding to
One or more graphic elements are projected onto one or more focal planes surrounding the environment surrounding the vehicle, and the one or more graphic elements are projected based on the path of the occupant of the vehicle. Projecting into the field of view;
One or more road conditions corresponding to one or more portions of the route, the distance between one or more of the focal planes of the one or more graphic elements and the vehicle, and the vehicle Adjusting based on the current position of
A computer-readable storage medium comprising computer-executable instructions for execution when executed through a processing unit of a computer.   The computer-readable storage medium of claim 17, wherein projecting the one or more graphic elements utilizes raster-based graphics.   18. The computer-readable storage medium of claim 17, comprising presenting one or more navigation instructions by projecting one or more of the graphic elements as a moving avatar.   The computer-readable storage medium of claim 19, comprising animating the moving avatar by sequentially projecting the moving avatar onto one or more different focal planes.

Images