KR20200017141A - Appartus and method for displaying image of vehicle - Google Patents

Abstract

An apparatus for displaying an image of a vehicle of the present invention includes: a route search unit for searching for a route to a destination; a camera for photographing a surrounding environment; and an image producing unit which obtains similar path information from a server based on the searched route and produces a virtual reality image corresponding to the photographed surrounding environment based on the similar route information. Therefore, the present invention provides a driver′s preferred environment compared to an environment in which the vehicle is being currently driven, so that a driver can feel as if the driver drives the vehicle in a new environment.

Description

Vehicle image display device and method {APPARTUS AND METHOD FOR DISPLAYING IMAGE OF VEHICLE}

The present invention relates to an image display apparatus of a vehicle, and more particularly, to an image display apparatus and method for displaying a virtual reality image corresponding to the driving environment of the vehicle.

In general, a vehicle terminal mounted on a vehicle to provide vehicle information to a driver is provided at the center between the driver's seat and the auxiliary seat. When the driver normally runs while looking ahead, when the information is provided to the vehicle terminal, the driver moves the gaze to the vehicle terminal to check the provided information.

If the driver's gaze of a vehicle traveling at 100 km / h is only 2 seconds away from the vehicle, the vehicle will travel 55 meters defenselessly. Thus, the vehicle can be easily exposed to an accident situation. As the number of devices in the car, such as navigation and smartphones, increases, this risk can be even greater.

Accordingly, a head-up display device that displays information of a vehicle in front of the driver's field of view has been proposed. The head-up display device allows the driver to view the information provided from the driver's front position while watching the front without changing the line of sight or the head to confirm the provided information, thereby reducing the risk of distraction.

 Recently, Augmented Reality (AR) technology has been introduced to provide the driver with safe driving information by aligning the position of the driver's forward real-world information with the information displayed on the head-up display device based on the driver's field of view.

Meanwhile, a technology for displaying a virtual reality image corresponding to an environment in which a vehicle is currently driven according to a driver's selection requires development.

An object of the present invention is to provide an image display apparatus and method of a vehicle for displaying a virtual reality image corresponding to an environment in which a driver is currently driving on a windshield of the vehicle.

An image display apparatus of a vehicle of the present invention obtains similar path information from a server based on a path search unit for searching a path to a destination, a camera photographing a surrounding environment, and the searched path, and based on the similar path information. It includes an image generating unit for generating a virtual reality image corresponding to the photographed surrounding environment.

The surrounding environment includes at least one of a lane of a road on which the vehicle is running, another vehicle, a pedestrian, a building around the road, and a natural environment around the road.

The similar path information includes similar path and similar path images obtained from the server based on the similarity between the predetermined path searched from the server and the searched path using the latitude and longitude of the origin and destination of the searched path. do.

The similarity may include at least one of a distance between the predetermined path and the searched path, a curvature of the predetermined path and the searched path, an altitude of the predetermined path and the searched path, and a bow of the predetermined path and the searched path. Calculated on the basis of

The similar path image includes an image photographed by a vehicle traveling along the similar path.

The virtual reality image is generated by a combination of similar path images corresponding to the surrounding environment photographed by the camera.

The similar path images corresponding to the surrounding environment photographed by the camera are obtained by matching the surrounding environment objects from the surrounding environment and the similar path image photographed by the camera, and obtained from the server according to the number of matched objects. .

The surrounding object includes at least one of a lane of a road, another vehicle, a pedestrian, a building around a road, and a natural environment around a road.

The display apparatus may further include a display unit configured to display the generated virtual reality image.

The display unit includes a windshield of the vehicle.

The image display method of the vehicle of the present invention includes the steps of: searching for a route, photographing a surrounding environment, acquiring similar route information based on the found route, and based on the similar route information. Generating an image of the virtual reality corresponding to the environment.

The surrounding environment includes at least one of lanes, lane colors, other vehicles, pedestrians, buildings around roads, and natural environments around roads.

The similar path information includes similar path and similar path images obtained from the server based on the similarity between the predetermined path searched from the server and the searched path using the latitude and longitude of the origin and destination of the searched path. do.

The similarity may include at least one of a distance between the predetermined path and the searched path, a curvature of the predetermined path and the searched path, an altitude of the predetermined path and the searched path, and a bow of the predetermined path and the searched path. Calculated on the basis of

The similar path image includes an image photographed by a vehicle traveling along the similar path.

The generating of the image of the virtual reality combines similar path images corresponding to the surrounding image photographed by the camera.

The similar path images corresponding to the surrounding image photographed by the camera are obtained by matching the surrounding environment object from the surrounding image photographed by the camera and the similar path image, and obtained from the server according to the number of matching objects. .

The surrounding object includes at least one of a lane of a road, a lane color, another vehicle, a pedestrian, a building around a road, and a natural environment around a road.

The method may further include displaying the generated virtual reality image on a display unit after generating the image of the virtual reality.

The display unit includes a windshield of the vehicle.

The image display apparatus and method of the vehicle of the present invention may display a virtual reality image corresponding to the environment in which the vehicle is currently traveling to the driver so that the driver may feel that the vehicle is running in the new environment.

1 is a block diagram showing a video display device of a vehicle of the present invention.
2 to 4 are schematic diagrams illustrating a process of searching for a similar path according to an embodiment of the present invention.
5 is a diagram illustrating a surrounding environment photographed by a camera according to an exemplary embodiment of the present invention.
6 is a diagram illustrating a similar path image corresponding to a surrounding environment photographed by a camera according to an exemplary embodiment of the present invention.
7 is a flowchart illustrating a method of displaying an image of a vehicle according to an exemplary embodiment of the present invention.
8 is a flowchart illustrating a method of displaying an image of a vehicle according to another exemplary embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are used to refer to the same components as much as possible, even if displayed on different drawings. In addition, in describing the embodiments of the present invention, if it is determined that the detailed description of the related well-known configuration or function interferes with the understanding of the embodiments of the present invention, the detailed description thereof will be omitted.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. In addition, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

According to embodiments of the present invention, when the autonomous vehicle autonomously travels a route searched to a destination, a virtual reality image corresponding to a road environment currently running in a wind shield of the vehicle or an environment in which the driver is currently driving, according to the driver's selection The present invention relates to an image display apparatus and method for displaying a vehicle, but is not limited thereto.

1 is a configuration diagram of a video display device of a vehicle of the present invention.

As shown in FIG. 1, the image display apparatus 100 of the present invention may include a camera 10, a path search unit 12, a display unit 16, and an image generator 18, and display an image. The device 100 may communicate with the server 200.

The camera 10 may photograph a vehicle surrounding environment. To this end, the camera 10 may include a front ultra wide angle camera photographing the front of the vehicle, a rear ultra wide angle camera photographing the image of the rear of the vehicle, a left ultra wide angle camera and a right ultra wide angle camera which captures the images of the left and right sides of the vehicle. If the camera 10 can obtain images of the front, rear, left and right sides of the vehicle, there is no limitation on the installation position or the number of the camera.

More specifically, the camera 10 may photograph the surrounding environment of the path searched by the path search unit 12. The surrounding environment of the searched route may include at least one of lanes, lane colors, and vehicles located near the road, pedestrians, buildings around the road, and natural environments around the road.

The route search unit 12 may include a storage means, and may store map information including the number of lanes of a lane, a curvature, a lane state, an altitude, and the like in the storage means. The route search unit 12 may store a precise map including detailed information, such as a marking state of a lane of a driving lane. The route search unit 12 may search the route to the destination based on the stored map, but is not limited thereto. The route search unit 12 may receive the map through the communication with the server 200 and search the route to the destination. The route may be searched based on the shortest distance, time required, and the like.

The path search unit 12 may include a position measuring unit, but may be separately provided. The position measuring unit measures the current position of the vehicle. The position measuring unit may be implemented by a global positioning system (GPS) module. The GPS module calculates the current position of the vehicle using signals transmitted from three or more GPS satellites. The GPS module calculates the distance between the satellite and the GPS module by using the time difference between the time when the satellite transmits a signal and the time when the GPS module receives the signal. The GPS module calculates the current position of the vehicle using the calculated distance between the satellite and the GPS module and the position information of the satellite included in the transmitted signal. In this case, the GPS module may calculate the current position of the vehicle by using triangulation.

The communication unit 14 may communicate with the server 200. The communication unit 14 may receive a map and a road image from the server 200 through a wireless network, and may receive similar path information. In addition, the communication unit 14 may transmit an image captured by the camera 10 and a path searched by the path search unit 12 to the server 200.

The display unit 16 may include a head-up display, and more specifically, may include a wind shield provided on the front of the vehicle. The display unit 16 may output the virtual reality image generated by the image generator 18. If the display unit 16 does not output an image of the virtual reality, the display unit 16 may display a road environment currently being driven.

The image generator 18 may obtain virtual path information from the server 200 based on the searched path, and generate a virtual reality image corresponding to an environment in which the searched path travels. Here, the server 200 may search for similar path information based on the found path. The similar path information may include a similar path searched by the server 200 and a similar path image photographed by a vehicle traveling along the similar path.

The server 200 searches for a predetermined route based on latitude and longitude of the origin and destination of the searched route, calculates similarity between the searched route and the searched predetermined route, and searches for the similar route based on the calculated similarity. can do. More details will be described with reference to FIGS. 2 to 4. 2 to 4 schematically illustrate a process of searching for a similar path according to an embodiment of the present invention.

More specifically, as shown in FIG. 2, the server 200 may check the longitude and latitude of the starting point and the longitude and latitude of the destination based on the found route. As shown in FIG. 3, a route searched from a map may be extracted and analyzed based on a difference between a longitude and a latitude, a distance, a curvature, an altitude, and the number of lanes of a starting point and a destination. According to an embodiment, the difference between the latitude between the starting point and the destination of the searched route may be a difference of 0.037 longitude and 0.01.

The server 200 may search for a predetermined path having a distance of the searched path based on the analyzed result. Preferably, the driver may search for a predetermined route in the city preferred by the driver.

The server 200 may calculate a similarity degree between the found path and the predetermined path. The server 200 may consider certain items to calculate the similarity. According to an embodiment, the predetermined item may include a distance difference, a curvature difference, an altitude difference, and a lane difference. Here, the distance difference may mean the difference between the distance of the search path and the distance of the predetermined path, and the curvature difference may mean the sum of the absolute value of the difference of the curvature difference of the search path and the difference of the curvature of the predetermined path, and the altitude difference is It can mean the sum of the absolute value of the altitude difference of the search path and the absolute value of the altitude difference of the predetermined path, and the lane difference means the difference of the number of lanes at the predetermined point in the similar path corresponding to the predetermined point of the search path. can do.

According to an embodiment, the server 200 may calculate the similarity by multiplying a coefficient by a value obtained by scoring the values obtained in the aforementioned items as shown in Tables 1 to 4. For more detailed calculation of similarity, refer to the following equation.

Similarity (%) = a * item 1 score + b * item 2 score + c * item 3 score + d * item 4 score

(Where a, b, c, and d are coefficients according to the priorities of items 1, 2, 3, and 4)

Distance 0 km Within 1km Within 2km Within 3km More than 3km score 100 80 70 60 except

Curvature difference 0 degrees Within 3 degrees Within 5 degrees Within 7 degrees 7 degrees or more score 100 80 70 60 except

Altitude difference 0m Within 5m Within 10m More than 10m score 100 80 70 except

Lane difference Lane 0 1 lane 2 lane or more score 100 70 except

According to an embodiment, the sum of the coefficients a, b, c, and d used in calculating the similarity is preferably 1, and the server 200 has the highest priority in the items considered to calculate the similarity. The similarity can be calculated by setting the value of the coefficient of the item.

For example, when the server 200 sets the priority to condition 1 as follows, the similarity may be calculated as follows. Here, condition 1 includes a priority order of distance difference> curvature difference> elevation difference> lane difference, and includes a case where the distance difference is within 1 km, the curvature difference is within 5 degrees, the altitude difference is 0 m, and the lane difference is 1 lane difference. Can be. Although the value of each coefficient may vary, in an embodiment, a is 0.5, b is 0.35, c is 0.1, and d is 0.05.

Similarity for Condition 1 = 0.5 * 80 (Score for Distance Difference) + 0.35 * 70 (Score for Curvature Difference) + 0.1 * 100 (Score for Elevation Difference) + 0.05 * 100 (Score for Lane Difference)

The server 200 may calculate the similarity with respect to condition 1 as 79.5%.

For another example, when the server 200 sets the priority to condition 2 as follows, the similarity may be calculated as follows. Here, condition 2 may include a case where the priority is set in the order of curvature difference> lane difference> distance difference> altitude difference, and the distance difference is within 2 km, the curvature difference is within 0 degrees, the altitude difference is within 5 m, and the lane difference is 0 lane. Can be. Although the value of each coefficient may vary, in an embodiment, a is 0.5, b is 0.35, c is 0.1, and d is 0.05.

Similarity for Condition 2 = 0.5 * 100 (Score for Curvature Difference) + 0.35 * 100 (Score for Lane Difference) + 0.1 * 70 (Score for Distance Difference) + 0.05 * 80 (Score for Elevation Difference)

The server 200 may calculate the similarity with respect to condition 2 as 94%.

According to an embodiment, when the calculated similarity is 90% or more, the server 200 may search for the predetermined path as the similar path as shown in FIG. 4.

The server 200 searches for a similar path image photographed by a vehicle traveling along the similar path.

The server 200 is an environment of any one of a lane of a road, another vehicle, a pedestrian, a building around a road, and a natural environment around the road from an image and a similar path image of the camera 10 which photographs the surrounding environment currently driving. You can extract the object.

The server 200 matches the environment object extracted from the environment captured by the camera 10 with the environment object extracted from the similar path image, and matches the environment object captured by the camera 10 according to the number of objects that are matched. A corresponding similar path image may be searched for. Preferably, when there are two or more matching objects, the similar path image corresponding to the surrounding environment captured by the camera 10 may be searched.

Meanwhile, the image generator 18 may obtain similar path images corresponding to the surrounding environment captured by the camera 10 from the server 200 and combine them to generate a virtual reality image.

See FIG. 5 and FIG. 6 for more details. 5 is a diagram illustrating a surrounding environment photographed by a camera, and FIG. 6 is a diagram showing a similar path image corresponding to the surrounding environment photographed by a camera.

The image generator 18 may acquire similar path images from the server 200 corresponding to the surrounding environment (see FIG. 5) captured by the camera 10.

According to an embodiment, the server 200 may extract a bus, a truck, and a building located on the right side of the vehicle from the surrounding environment and the similar path image captured by the camera 10 as the surrounding object. The server 200 may determine the surrounding objects extracted from the similar path image matched with the surrounding objects extracted in FIG. 5 as buses, trucks, and buildings, and thus correspond to the surroundings captured by the camera 10. The similar path image (see FIG. 6) may be searched.

The image generator 18 may generate a virtual reality image by combining similar path images corresponding to the surrounding environment captured by the camera 10.

The controller 20 may control to output the virtual reality image on the front of the display unit 16 when the image generator 18 generates the virtual reality image. The controller 20 may control to output the virtual reality image based on the driver input. The controller 20 may determine that the virtual reality image is output when there is a driver input. On the other hand, if there is no driver input, it may be determined that the virtual reality image is not output.

In addition, the controller 20 may not output the virtual reality image when the vehicle deviates from the searched route, and may display a road currently being driven.

The controller 20 may determine whether to output the virtual reality image even when the route to the destination is not found. If the controller 20 determines that the virtual reality image is output even when the route to the destination is not found, the controller 20 guides the driver to set the destination. Thereafter, a destination is set, and the controller 20 searches for a path according to the destination, and when there is a found path, obtains similar path information based on the found path, and the camera 10 photographs based on the similar path information. The virtual reality image corresponding to a surrounding environment may be output through the display unit 16.

7 is a flowchart illustrating a method of displaying an image of a vehicle according to an exemplary embodiment of the present invention. As shown in FIG. 7, the path search and the detected path are taken (S110). In S110, the route search unit 12 searches for a route to the destination when the destination is input. In operation S110, the camera 10 photographs the surrounding environment within the searched path. For example, the camera 10 may photograph a lane, a lane color, and a vehicle, a pedestrian, a building around a road, and a natural environment around a road on which a vehicle is traveling.

The path search unit 12 obtains similar path information from the server 200 (S120). In SS 120, the path search unit 12 may obtain similar path information searched by the server 200. The server 200 may search for similar path information based on the found path. The similar path information may include a similar path and a similar path image. The server 200 may search for a predetermined road based on the latitude and longitude of the origin and destination of the searched route, calculate the similarity between the searched route and the predetermined route, and search for the similar route based on the similarity. In addition, the server 200 may search for an image taken by the vehicle traveling along the similar path as the similar path image.

The controller 20 determines whether to output the virtual reality image through the display 16 (S130). In operation S130, the controller 20 may determine whether to output the virtual reality image based on a driver input. The controller 20 may determine that the virtual reality image is output when there is a driver input. On the other hand, if there is no driver input, it may be determined that the virtual reality image is not output.

If it is determined in step S130 that the controller 20 outputs the virtual reality image (Yes), the image generator 18 generates the virtual reality image based on the similar path information (S140). In operation S140, the image generator 18 obtains similar-path images from the server 200 corresponding to the surrounding environment captured by the camera 10, and generates a virtual reality image by combining them. The similar path images corresponding to the surrounding environment photographed by the camera 10 are extracted by matching the surrounding environment objects from the surrounding environment and similar path images captured by the camera 10, and the server 200 according to the number of matched objects. Can be retrieved from

If it is determined in step S130 that the virtual reality image is not output (No), the controller 20 controls to display a road currently being driven through the display unit 16 (S170).

The controller 20 controls to output the virtual reality image to the front of the display unit 16 (S150).

The controller 20 checks whether the vehicle has left the searched path (S160). If it is confirmed that the route is deviated, the control unit is configured to display the road currently being driven without outputting the virtual reality image (S170).

8 is a flowchart illustrating a method of displaying an image of a vehicle according to another exemplary embodiment of the present invention.

As shown in FIG. 8, it is confirmed that there is no searched path (S200). In S200, the route search unit 12 may check that the destination is not input if there is no route found.

The controller 20 determines whether to output the virtual reality image through the display 16 (S210). In S210, the controller 20 may determine whether to output the virtual reality image based on a driver input. The controller 20 may determine that the virtual reality image is output when there is a driver input. On the other hand, if there is no driver input, it may be determined that the driver does not output the virtual reality image. In this case, S270 is performed.

If it is determined in step S210 that the control unit 20 outputs the virtual reality image through the display unit (Yes), the route search unit 12 guides the destination input and searches the route to the input destination input, the camera 10 photographs the surrounding environment of the searched path (S220). In S220, the camera 10 photographs at least one of a lane of a road on which the vehicle is traveling, another vehicle, a pedestrian, a building around a road, and a natural environment around the road in the searched path.

The path search unit 12 obtains similar path information from the server 200 (S230). Since S230 is the same as S120, the description of S120 is referred to.

The image generator 18 generates a virtual reality image based on the similar path information (S240). In operation S240, the image generator 18 obtains similar path images corresponding to the surrounding environment captured by the camera 10 from the server 200, and generates a virtual reality image by combining them. Similar path images corresponding to the surrounding environment photographed by the camera 10 are extracted by matching the surrounding environment objects from the surrounding environment and similar path images photographed by the camera 10, and the server 200 according to the number of matched objects. Can be retrieved from

The controller 20 controls to output the virtual reality image to the front of the display unit 16 (S250).

The controller 20 checks whether the vehicle has left the searched path (S260). If it is confirmed that the route is deviated, the virtual road image is displayed without displaying the road currently being driven (S270).

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Camera 10
Route search unit 12
Communications Department 14
Display unit 16
Image generator 18
Control unit 20
Video display device 100
Server 200

Claims (20)

A route search unit for searching a route to a destination;
A camera for photographing the surrounding environment; And
And an image generator for acquiring similar path information from a server based on the found path, and generating a virtual reality image corresponding to the photographed surrounding environment based on the similar path information. The method according to claim 1,
The surrounding environment
An image display apparatus of a vehicle including at least one of a lane of a road on which the vehicle is running, another vehicle, a pedestrian, a building around the road, and a natural environment around the road. The method according to claim 1,
The similar path information is
And a similar path and similar path image obtained from the server based on the similarity between the predetermined path searched from the server and the searched path using the latitude and longitude of the origin and destination of the searched path. The method according to claim 3,
The similarity is
A vehicle calculated based on at least one of a distance between the predetermined path and the searched path, a curvature of the predetermined path and the searched path, an altitude of the predetermined path and the searched path, and a player of the predetermined path and the searched path Video display device. The method according to claim 3,
The similar path image is
And an image captured by the vehicle traveling along the similar path. The method according to claim 3,
The virtual reality image is
And an image of a similar path image corresponding to the surrounding environment captured by the camera. The method according to claim 6,
Similar path images corresponding to the surrounding environment photographed by the camera
And extracting and matching the surrounding object from the surrounding environment and the similar path image photographed by the camera, and obtaining from the server according to the number of matched objects. The method according to claim 7,
The environment object
An image display apparatus of a vehicle including at least one of a lane of a road, another vehicle, a pedestrian, a building around the road, and a natural environment around the road. The method according to claim 1,
And a display unit for displaying the generated virtual reality image. The method according to claim 9,
The display unit
An image display apparatus of a vehicle including a wind shield of a vehicle. Searching for a route;
Photographing the environment;
Acquiring similar path information based on the found path; And
And generating an image of a virtual reality corresponding to the captured surrounding environment based on the similar path information. The method according to claim 11,
The surrounding environment
An image display method of a vehicle including at least one of lanes, lane colors, other vehicles, pedestrians, buildings around a road, and a natural environment around a road on which a vehicle is traveling. The method according to claim 11,
The similar path information is
And a similar path and a similar path image obtained from the server based on the similarity between the predetermined path searched from the server and the searched path using the latitude and longitude of the origin and destination of the searched path. The method according to claim 13,
The similarity is
A vehicle calculated based on at least one of a distance between the predetermined path and the searched path, a curvature of the predetermined path and the searched path, an altitude of the predetermined path and the searched path, and a player of the predetermined path and the searched path Video display method. The method according to claim 13,
The similar path image is
The image display method of a vehicle comprising an image taken by the vehicle traveling along the similar path. The method according to claim 13,
Generating the image of the virtual reality is
And displaying similar path images corresponding to the surrounding images captured by the camera. The method according to claim 16,
Similar path images corresponding to the surrounding image captured by the camera
And extracting and matching the surrounding object from the surrounding image photographed by the camera and the similar path image, and obtaining the image from the server according to the number of matching objects. The method according to claim 17,
The environment object
An image display apparatus of a vehicle including at least one of a lane of a road, a lane color, another vehicle, a pedestrian, a building around the road, and a natural environment around the road. The method according to claim 11,
After generating the image of the virtual reality
And displaying the generated virtual reality image on a display unit. The method according to claim 19,
The display unit
An image display method of a vehicle including a wind shield of the vehicle.

Images