KR102348127B1 - Electronic apparatus and control method thereof - Google Patents

Abstract

A method of controlling an electronic device is disclosed. This control method includes the steps of receiving a user input requesting route guidance, generating a route guide line based on destination information corresponding to the request, and correcting the route guide line generated by reflecting the radius of the driving trajectory of the actual driving vehicle. performing variable three-dimensionalization by varying the height of the corrected route guide line according to the distance from the host vehicle, mapping the texture to the three-dimensional data generated according to the variable three-dimensionalization to generate a route guide indicator, and route guide indicator and outputting it to the screen through augmented reality.

Description

ELECTRONIC APPARATUS AND CONTROL METHOD THEREOF

The present invention relates to an electronic device and a control method thereof, and more particularly, to an electronic device capable of recognizing a location of a lane in which a vehicle is located and performing driving-related guidance based thereon, and a control method thereof.

The most important thing when driving a vehicle is safe driving and prevention of traffic accidents. have.

In addition, recently, devices such as a black box are located in a vehicle to store driving images of the vehicle and data transmitted from various sensors, thereby identifying the cause of the vehicle accident when the vehicle is in an accident. Portable terminals such as smartphones and tablets can also be equipped with a black box or a navigation application, and thus are being used as such vehicle devices.

However, the current utilization of driving images in such vehicle devices is low. More specifically, even if a driving image of a vehicle is acquired through a vision sensor such as a camera mounted on a current vehicle, the electronic device of the vehicle simply displays and transmits it, or generates only simple surrounding notification information such as lane departure.

In addition, although a head-up display (HUD) or an augmented reality interface has been proposed as an electronic device for a vehicle that is newly emerging, the utilization of the driving image of the vehicle is limited to a simple display or simple notification information generation here.

The present invention has been devised to solve the above-described problems, and an object of the present invention is to generate location information on the lane in which the vehicle is located by using driving-related image data of the vehicle, and based on this, an electronic device capable of providing driving-related guidance To provide an apparatus and a method for controlling the same.

Another object of the present invention is to provide an electronic device capable of effectively performing driving-related guidance based on augmented reality and a control method thereof.

A method of controlling an electronic device according to an embodiment of the present invention for achieving the above object includes: identifying a lane area from driving-related image data of a vehicle; generating lane information corresponding to the lane; generating lane location information on the lane on which the vehicle is located using at least one of the generated lane information and lane information on the road on which the vehicle is located; and using the generated lane location information and performing driving-related guidance of the vehicle using the .

In addition, the lane information may include lane type information and lane color information corresponding to each lane located on both sides of the lane in which the vehicle is located.

In addition, the generating of the lane location information includes: obtaining lane information of a road on which the vehicle is located from map data; The method may include determining whether the vehicle is located, and when the vehicle is located on the first or last lane, generating information about the lane on which the vehicle is located by reflecting information on the lane of the road.

In the generating of the lane location information, when the lane in which the vehicle is located is changed to a lane located between the first or last lanes according to the lane change of the vehicle, the generated lane location information is converted into the changed lane. The method may further include updating the location information.

In addition, when the lane on which the vehicle is located is changed from the lane located between the lanes to the first or last lane according to the lane change of the vehicle, the lane location information on the lane on which the vehicle is located is reflected by reflecting the lane information of the road on which the vehicle is located It may further include the step of regenerating.

In addition, the performing driving-related guidance of the vehicle may include outputting a lane change guidance using a navigation route of the vehicle and the lane location information.

In addition, the step of performing the driving-related guidance of the vehicle may include outputting a guidance on a lane in which the vehicle is located by using the lane location information.

The method may further include the step of selecting and outputting an appropriate lane departure guide according to the types of both lanes of the lane in which the vehicle identified based on the lane information is located.

Also, the outputting may include generating an indicator for performing the driving-related guidance and outputting the generated indicator through augmented reality.

On the other hand, an electronic device according to an embodiment of the present invention for achieving the above object identifies a lane area from driving-related image data of a vehicle, and from the image data of the identified lane area part to a lane in which the vehicle is located A lane information generation unit generating corresponding lane information, a lane location information generation unit generating location information on a lane in which the vehicle is located by using at least one of the generated lane information and lane information of a road on which the vehicle is located, and the generation unit and a controller configured to provide driving-related guidance of the vehicle by using the obtained lane location information.

In addition, the lane information may include lane type information and lane color information corresponding to each lane located on both sides of the lane in which the vehicle is located.

In addition, the lane location information generating unit obtains lane information of the road on which the vehicle is located from map data, and determines whether the vehicle is located on the first lane or the last lane of the road using the generated lane information. and if the vehicle is located on the first or last lane, lane location information on which the vehicle is located may be generated by reflecting lane information of the road

The lane location information generating unit may be configured to convert the generated lane location information into the changed lane location information when the lane on which the vehicle is located is changed to a lane located between the first or last lanes according to the lane change of the vehicle. Can be updated.

In addition, the lane location information generating unit may be configured to reflect the lane information of the road on which the vehicle is located when the lane on which the vehicle is located is changed from a lane located between the lanes to the first or last lane according to the lane change of the vehicle. It is possible to regenerate the location information of the vehicle in which the vehicle is located.

In addition, the controller may control the output unit to output a lane change guide using the navigation route of the vehicle and the lane location information.

Also, the controller may control the output unit to output a guide to a lane in which the vehicle is located by using the lane location information.

In addition, the controller may control the output unit to select and output an appropriate lane departure guide according to types of lanes on both sides of the lane in which the vehicle identified based on the lane information is located.

Also, the controller may generate an indicator for performing the driving-related guidance and control the output unit to output the generated indicator through augmented reality.

On the other hand, to achieve the above object, the control method of the electronic device according to an embodiment of the present invention for achieving the above object, the step of receiving a user input requesting a route guide generating a route guide line based on destination information corresponding to the request Step, performing correction of the generated path guide line by reflecting the radius of the driving trajectory of the actual driving vehicle, performing variable three-dimensionalization by varying the height of the corrected path guide line according to the distance of the own vehicle, and applying a texture to the three-dimensional Creating a route guidance indicator by mapping, and outputting the route guidance indicator to a screen through augmented reality.

And, performing the correction of the path guide line includes the steps of adding vertices for maintaining a straight line of the path guide line in front of the own vehicle, adding vertices for curving the curve section to the curve section of the path guide line, and the added The method may include generating a path guide line reflecting the radius of the driving trajectory of the actual driving vehicle by using the vertices.

And, performing the variable stereoscopicization includes generating virtual path guide lines on both sides of the corrected path guide line, and a height value such that the height values of vertices included in the corrected path guide line increase in proportion to the distance of the own vehicle. and performing three-dimensionalization of vertices of the path guide line for which the height value is calculated and vertices included in each of the virtual path guide lines through polygons.

In addition, the generating of the route guide indicator may include mapping a texture having a displacement according to a vehicle speed to the three-dimensional object to generate the route guide indicator.

And, performing calibration for estimating a camera parameter corresponding to the camera from the captured image taken by the camera and generating a virtual 3D space from the image captured by the camera based on the camera parameter, and , the outputting may be output by combining the generated path guide indicator in the generated virtual 3D space.

Meanwhile, in an electronic device according to an embodiment of the present invention for achieving the above object, an input unit for receiving a user input requesting a route guide, and a route guide line for generating a route guide line based on destination information corresponding to the request A generator, a route guide indicator generator for generating a route guide indicator for route guidance in augmented reality using the generated route guide line, and a display unit for outputting the generated route guide indicator to a screen through augmented reality, The path guide indicator generation unit includes a path guide line processing unit that corrects a path guide line generated by reflecting the radius of the driving trajectory of the actual driving vehicle, and performs variable three-dimensionalization by varying the height of the corrected path guide line according to the distance of the vehicle It may include a path guide line three-dimensionalization unit and a texture mapping unit generating a path guide indicator by mapping a texture to the solid.

And, the route guide line processing unit adds vertices for maintaining a straight line of the route guide line in front of the own vehicle, adds vertices for curving the curve section to the curve section of the route guide line, and uses the added vertices to actually drive the vehicle It is possible to generate a path guide line reflecting the radius of the driving trajectory of . The path guide line stereoscopic unit generates virtual path guide lines on both sides of the corrected path guide line, and height values of vertices included in the corrected path guide line are A height value may be calculated to increase in proportion to the distance of the host vehicle, and vertices of the path guide line for which the height value is calculated and vertices included in each of the virtual path guide lines may be three-dimensionalized through polygons.

Also, the texture mapping unit may generate a route guidance indicator by mapping a texture having a displacement according to a vehicle speed to the three-dimensional object.

In addition, the calibration unit for performing calibration (Calibration) for estimating a camera parameter corresponding to the camera from the captured image taken by the camera; and a 3D space generating unit generating a virtual 3D space from an image captured by the camera based on the camera parameters, wherein the display unit may output the generated path guide indicator by combining the generated virtual 3D space.

Meanwhile, in the recording medium according to an embodiment of the present invention for achieving the above object, a program code for executing the above-described control method in a computer may be recorded.

According to various embodiments of the present disclosure described above, lane information corresponding to a lane in which a vehicle is located may be generated from image data of a lane area portion, and necessary processing may be performed. Accordingly, various information processing including outputting a lane interface using lane information and generating an augmented reality interface may be performed.

Also, according to various embodiments of the present disclosure, in a situation in which there are a plurality of center lanes of a road (eg, when the number of lanes is 4 or more), a lane in which the vehicle is located among a plurality of center lanes may be accurately known.

Also, according to various embodiments of the present disclosure, it is possible to determine a lane in which the vehicle is located and guide the driver to the driver assistance role.

In addition, according to various embodiments of the present disclosure, user convenience can be promoted by performing accurate lane change notification using route information of navigation and the lane in which the vehicle is located.

In addition, according to various embodiments of the present disclosure, an appropriate lane change notification can be selectively performed according to the types of lanes on both sides of the driving lane using lane information, thereby improving the performance of the lane change notification and providing rich information. can

In addition, according to various embodiments of the present disclosure, it is possible to realistically and effectively display a three-dimensional route guide indicator on a two-dimensional camera image by building a route guide indicator suitable for augmented reality through three-dimensional processing of route guide lines in real time. have. That is, instead of displaying a simple route guide line as in the conventional augmented reality navigation, it may be displayed as if the route guide line is located on an actual road.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention.
2 is a diagram for explaining a system network connected to an electronic device according to an embodiment of the present invention.
3 is a flowchart illustrating a method of generating lane information of an electronic device according to an embodiment of the present invention.
4 is a flowchart specifically illustrating a method for generating lane information of an electronic device according to an embodiment of the present invention.
5 is a diagram illustrating a gray image conversion and lane area detection process according to an embodiment of the present invention.
6 is a diagram illustrating a lane type ROI in a gray image according to an embodiment of the present invention.
7 is a diagram illustrating binarization and one-dimensional mapping of a lane type region of interest in a gray image according to an embodiment of the present invention.
8 is a flowchart specifically illustrating a method for generating lane location information according to an embodiment of the present invention.
9 is a diagram illustrating a lane determination table according to an embodiment of the present invention.
10 is a diagram illustrating a lane determination table according to another embodiment of the present invention.
11 is a flowchart illustrating a control method according to an embodiment of the present invention.
12 is a block diagram specifically illustrating an augmented reality providing unit according to an embodiment of the present invention.
13 is a diagram comparing before and after processing of a route guide line.
14 is a diagram illustrating a process of three-dimensionalizing a route guide line according to an embodiment of the present invention.
15 is a flowchart illustrating an augmented reality route guidance method according to an embodiment of the present invention.
16 is a diagram illustrating a route guidance screen according to an embodiment of the present invention.
17 is a diagram illustrating an implementation form when a camera and an electronic device are separated according to an embodiment of the present invention.
18 is a diagram illustrating an implementation form when a camera and an electronic device are integrated according to an embodiment of the present invention.
19 is a diagram illustrating an implementation form using a Head-Up Display (HUD) and an electronic device according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art will be able to devise various devices that, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. In addition, all conditional terms and examples listed herein are, in principle, expressly intended only for the purpose of understanding the concept of the present invention, and it should be understood that they are not limited to the specifically enumerated embodiments and states as such. do.

Moreover, it is to be understood that all detailed description reciting specific embodiments, as well as principles, aspects, and embodiments of the invention, are intended to include structural and functional equivalents of such matters. It should also be understood that such equivalents include not only currently known equivalents, but also equivalents developed in the future, i.e., all devices invented to perform the same function, regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of illustrative circuitry embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc. may be tangibly embodied on computer-readable media and be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly shown. should be

The functions of the various elements shown in the drawings including a processor or functional blocks represented by similar concepts may be provided by the use of dedicated hardware as well as hardware having the ability to execute software in association with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, the clear use of terms presented as processor, control, or similar concepts should not be construed as exclusively referring to hardware having the ability to execute software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM (RAM) and non-volatile memory. Other common hardware may also be included.

In the claims of the present specification, a component expressed as a means for performing the function described in the detailed description includes, for example, a combination of circuit elements that perform the function or software in any form including firmware/microcode, etc. It is intended to include all methods of performing the functions of the device, coupled with suitable circuitry for executing the software to perform the functions. Since the present invention defined by these claims is combined with the functions provided by the various enumerated means and in a manner required by the claims, any means capable of providing the functions are equivalent to those contemplated from the present specification. should be understood as

The above objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in the description of the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention. Referring to FIG. 1 , the electronic device 100 includes a storage unit 110 , an input unit 120 , an output unit 130 , a lane information generating unit 140 , a lane location information generating unit 150 , and an augmented reality providing unit. 160 , the control unit 170 , the communication unit 180 , and all or a part of the sensing unit 190 .

Here, the electronic device 100 includes a smart phone, a tablet computer, a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), smart glasses, project glasses, and a navigation system capable of providing driving-related guidance to a vehicle driver. navigation), may be implemented in various devices such as a black-box.

Here, the driving state of the vehicle may include various states in which the vehicle is being driven by the driver, such as a stopped state of the vehicle, a running state of the vehicle, and a parking state of the vehicle.

The driving-related guidance may include various guidance for assisting the driver in driving, such as route guidance, lane departure guidance, forward vehicle departure guidance, traffic light change guidance, forward collision avoidance guidance, lane change guidance, lane guidance, and the like.

Here, the route guidance includes augmented reality route guidance that performs route guidance by combining various information such as the location and direction of the user with an image taken in front of the vehicle being driven, and 2D (2-Dimensional) or 3D (3-D) 2D (2-Dimensional) or 3D (3-Dimensional) route guidance for performing route guidance by combining various types of information such as a user's location and direction with map data of Dimensional) may be included. Here, the route guidance may be interpreted as a concept including route guidance when the user moves while walking or running as well as when the user rides and drives in a vehicle.

In addition, the lane departure guidance may be guiding whether the driving vehicle has departed from a lane.

In addition, the front vehicle departure guidance may indicate whether a vehicle located in front of a stopped vehicle departs.

Also, the traffic light change guidance may be to guide whether a traffic light positioned in front of a stopped vehicle is changed. As an example, when a red light indicating a stop signal is turned on to a blue light indicating a departure signal, this may be guiding.

In addition, the front vehicle collision avoidance guidance may be to guide the vehicle in front of a stopped or driving vehicle to prevent a collision with the vehicle in front when the distance to the vehicle located in front is within a predetermined distance.

In addition, the lane change guidance may indicate a change from a lane in which the vehicle is located to another lane in order to guide a route to a destination.

Also, the lane guidance may indicate a lane in which the vehicle is currently located.

A driving-related image, such as a front image of a vehicle that enables the provision of various guidance, may be captured by a camera mounted on the vehicle. Here, the camera may be a camera integrally formed with the electronic device 100 mounted on the vehicle to photograph the front of the vehicle. In this case, the camera may be integrally formed with a smart phone, a navigation system, or a black box, and the electronic device 100 may receive an image captured by the integrally formed camera.

As another example, the camera may be mounted on a vehicle separately from the electronic device 100 to photograph the front of the vehicle. In this case, the camera may be a separate black box mounted toward the front of the vehicle, and the electronic device 100 receives a photographed image through wired/wireless communication with the separately mounted black box, or receives a photographed image of the black box. When the storage medium to be stored is inserted into the electronic device 100 , the electronic device 100 may receive a captured image.

Hereinafter, the electronic device 100 according to an embodiment of the present invention will be described in more detail based on the above description.

The storage unit 110 functions to store various data and applications required for the operation of the electronic device 100 . In particular, the storage 110 may store data necessary for the operation of the electronic device 100 , for example, an OS, a route search application, map data, and the like. Also, the storage 110 may store data generated by the operation of the electronic device 100 , for example, searched route data, a received image, and the like. Also, the storage unit 110 may store a lane determination table and the like.

Here, the storage unit 110 includes a random access memory (RAM), a flash memory, a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), a register, a hard disk, a removable disk, and a memory. It may be implemented as a storage device in a built-in type such as a card or a Universal Subscriber Identity Module (USIM), as well as a storage device in a removable type such as a USB memory.

The input unit 120 functions to convert a physical input from the outside of the electronic device 100 into a specific electrical signal. Here, the input unit 120 may include all or part of the user input unit 121 and the microphone unit 123 .

The user input unit 121 may receive a user input such as a touch or a push operation. Here, the user input unit 120 may be implemented using at least one of various button shapes, a touch sensor for receiving a touch input, and a proximity sensor for receiving an approaching motion.

The microphone unit 123 may receive a user's voice and a sound generated inside or outside the vehicle.

The output unit 130 is a device that outputs data of the electronic device 100 . Here, the output unit 130 may include all or part of the display unit 131 and the audio output unit 133 .

The display unit 131 is a device that outputs data that can be visually recognized by the electronic device 100 . The display unit 131 may be implemented as a display unit provided on the front of the housing of the electronic device 100 . Here, the display unit 131 may be integrally formed with the electronic device 100 to output visual recognition data, and may be installed separately from the electronic device 100 such as a Head Up Display (HUD) to output visual recognition data. may be

The audio output unit 133 is a device for outputting data that can be recognized aurally by the electronic device 100 . The audio output unit 133 may be implemented as a speaker expressing sound of data to be notified to the user of the electronic device 100 .

The communication unit 180 may be provided for the electronic device 100 to communicate with other devices. The communication unit 180 includes all or part of the location data unit 181 , the wireless Internet unit 183 , the broadcast transceiver unit 185 , the mobile communication unit 186 , the short-distance communication unit 187 , and the wired communication unit 189 . may include

The location data unit 181 is a device for acquiring location data through a Global Navigation Satellite System (GNSS). GNSS refers to a navigation system capable of calculating the position of a receiving terminal using a radio signal received from an artificial satellite. Specific examples of GNSS include GPS (Global Positioning System), Galileo, GLONASS (Global Orbiting Navigational Satellite System), COMPASS, IRNSS (Indian Regional Navigational Satellite System), QZSS (Quasi-Zenith Satellite System), etc. can The location data unit 181 of the electronic device 100 according to an embodiment of the present invention may obtain location data by receiving a GNSS signal serviced in a region where the electronic device 100 is used.

The wireless Internet unit 183 is a device for acquiring or transmitting data by accessing the wireless Internet. The wireless Internet accessible through the wireless Internet unit 183 may be a wireless LAN (WLAN), a wireless broadband (Wibro), a world interoperability for microwave access (Wimax), a high speed downlink packet access (HSDPA), or the like.

The broadcast transceiver 185 is a device for transmitting and receiving broadcast signals through various broadcast systems. Broadcast systems that can be transmitted and received through the broadcast transceiver 185 include Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), Media Forward Link Only (MediaFLO), Digital Video Broadcast Handheld (DVBH), and ISDBT (Digital Multimedia Broadcasting Terrestrial). Integrated Services Digital Broadcast Terrestrial) or the like. The broadcast signal transmitted and received through the broadcast transceiver 185 may include traffic data, life data, and the like.

The mobile communication unit 186 may access and communicate with a mobile communication network according to various mobile communication standards such as 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), and Long Term Evolution (LTE).

The short-distance communication unit 187 is a device for short-distance communication. The short-range communication unit 187, as described above, Bluetooth (Bluetooth), RFID (Radio Frequency Identification), infrared communication (IrDA, Infrared Data Association), UWB (Ultra WideBand), ZigBee, NFC (Near Field Communication), Wi -Communication is possible through Fi (Wireless-Fidelity), etc.

The wired communication unit 189 is an interface device that can connect the electronic device 100 to another device by wire. The wired communication unit 119 may be a USB module capable of communicating through a USB port.

The communication unit 180 includes at least one of a location data unit 181 , a wireless Internet unit 183 , a broadcast transceiver unit 185 , a mobile communication unit 186 , a short-distance communication unit 187 , and a wired communication unit 189 . can be used to communicate with other devices.

For example, when the electronic device 100 does not include a camera function, an image captured by a vehicle camera such as a black box may be received using at least one of the short-range communication unit 187 and the wired communication unit 189 .

As another example, when communicating with a plurality of devices, one may communicate with the short-range communication unit 187 and the other may communicate through the wired communication unit 119 .

The sensing unit 190 is a device capable of detecting the current state of the electronic device 100 . The sensing unit 190 may include all or a part of the motion sensing unit 191 and the light sensing unit 193 .

The motion sensing unit 191 may detect a motion of the electronic device 100 in a three-dimensional space. The motion sensing unit 191 may include a 3-axis geomagnetic sensor and a 3-axis acceleration sensor. A more accurate trajectory of the vehicle to which the electronic device 100 is attached may be calculated by combining the motion data acquired through the motion sensing unit 191 with the position data acquired through the position data unit 191 .

The light sensing unit 193 is a device for measuring peripheral illuminance of the electronic device 100 . By using the illuminance data obtained through the light sensing unit 193, the brightness of the display unit 195 may be changed to correspond to the ambient brightness.

The power supply unit 195 is a device that supplies power necessary for the operation of the electronic device 100 or the operation of another device connected to the electronic device 100 . The power supply unit 195 may be a device that receives power from an external power source such as a battery built in the electronic device 100 or a vehicle. In addition, the power supply unit 195 may be implemented as a wired communication module 119 or a wirelessly supplied device depending on the form of receiving power.

Meanwhile, the controller 170 controls the overall operation of the electronic device 100 . Specifically, the control unit 170 includes a storage unit 110 , an input unit 120 , an output unit 130 , a lane information generating unit 140 , a lane location information generating unit 150 , an augmented reality providing unit 160 , and a communication unit. 180 , all or part of the sensing unit 190 may be controlled.

In particular, the controller 170 controls the lane information generation unit 140 and the lane location information generation unit 150 to identify a lane area from driving-related image data, and to detect the vehicle from the image data of the identified lane area portion. Lane information corresponding to the located lane may be generated, and information on the lane on which the vehicle is located may be generated using at least one of the generated lane information and lane information of the road on which the vehicle is located.

Here, the lane may mean each of both sides of the line forming a lane in which the vehicle is located. In addition, a lane may mean a road formed by lanes such as a primary lane, a secondary lane, .. N lane, and the like, and on which a vehicle travels.

The lane information generating unit 140 identifies a lane area from image data captured in a driving state such as driving or stopping the vehicle, and corresponds to each lane located on both sides of the lane in which the vehicle is located from the image data of the lane area portion lane information can be generated. The lane information may include lane type information and lane color information corresponding to each lane located on both sides of the lane in which the vehicle is located.

Here, in order to generate lane information corresponding to the lane in which the vehicle is located, the lane information generator 140 may perform binarization on the image data of the lane area portion, and obtain lane type information from the binarized partial image data. have. Specifically, the lane information generating unit 140 analyzes the binarized partial image data using at least one of time continuity information of the lane and lane speed information to identify whether both types of lanes of the lane in which the vehicle is located are solid or dotted lines. can

In addition, the lane information generating unit 140 may extract color information corresponding to each lane whose type is identified from the image data and generate the lane information as lane information.

The lane location information generating unit 150 uses at least one of the lane information generated by the lane information generating unit 140 and the lane information of the road on which the vehicle is located under the control of the controller 170 to provide information on the lane location on which the vehicle is located. can create

Specifically, the lane location information generating unit 150 obtains lane information of the road on which the vehicle is located from the map data, and determines whether the vehicle is located on the first or last lane of the road using the generated lane information. And, when the vehicle is located on the first or last lane, information on the lane on which the vehicle is located may be generated by reflecting the information on the lane of the road. Thereafter, when the lane on which the vehicle is located is changed to a lane located between the first or last lanes according to the change of lane of the vehicle, the generated lane location information may be updated with the changed lane location information. Thereafter, when the lane on which the vehicle is located is changed from the lane located between the lanes to the first or last lane according to the change of lane of the vehicle, information on the lane on which the vehicle is located may be regenerated by reflecting the lane information of the road on which the vehicle is located. .

Here, the lane information of the road on which the vehicle is located may include information on the number of lanes on the road on which the vehicle is located, information on the type of road (eg, a highway, an urban high-speed road, a local road, a general road), and the like, and the electronic device It may be obtained from map data stored in the internal storage 110 , obtained from an external map database (DB) separate from the electronic device 100 , or obtained from another electronic device 100 . As an example, when the electronic device 100 is implemented as a black box, the black box may obtain information on the lane of a road from an external navigation device that is connected through communication.

Meanwhile, since lane information used for each country is different, the lane location information generating unit 150 may generate lane location information by using a lane determination table according to traffic laws and the like for each country. Accordingly, the lane position information generating unit 150 may generate the lane position information based on the lane determination table mapped to the set country information.

Meanwhile, the controller 170 may perform driving-related guidance of the vehicle using the lane information generated by the lane information generator 140 and the lane location information generator 150 .

For example, the controller 170 may select an appropriate lane departure guide according to the type of both lanes of a lane in which a vehicle identified based on lane information is located, and output the appropriate lane departure guidance to the user. Specifically, when the electronic device 100 provides a lane departure guidance function, the controller 170 may provide different guidance according to the type and color of the departed lane. For example, the control unit 170 selects a different guide image or a guide voice depending on the case where the vehicle violates the center line, the white solid line, the white dotted line, or the blue line, respectively. can be printed out.

As another example, the control unit 170 may output the lane guidance generated by using the lane position information generated by the lane position information generating unit 150 through the output unit 140 . Specifically, the controller 170 may output the number of lanes in which the vehicle is currently located, for example, whether the first lane, the second lane, or the N lane is recognized as an image or voice.

As another example, the control unit 170 may output the lane change guide generated by using the lane position information generated by the lane position information generation unit 150 through the output unit 140 . Specifically, when the electronic device 100 provides a vehicle navigation function, the controller 170 may output a lane change guidance as an image or a voice guidance according to the route to the destination and the determined lane location. That is, when the distance to the left turn or right turn guide point is less than a predetermined distance, the controller 170 may determine whether the current lane can turn left or right and output a lane change guidance to the user.

Meanwhile, the controller 170 may control the augmented reality providing unit 160 so that the electronic device 100 performs driving-related guidance based on the augmented reality. Here, the augmented reality refers to adding additional information (eg, a graphic element indicating a point of interest (POI), a graphic element indicating a path to a destination, etc.) on a screen containing the real world that the user is actually viewing. It may be a method of providing a visually overlapping method. In this case, the control unit 170 may generate an indicator for performing driving-related guidance in conjunction with the augmented reality providing unit 160 , and output the generated indicator through the output unit 130 . For example, the augmented reality may be provided through a HUD using a vehicle windshield or an image overlay using a separate image output device, etc. can create Through this, augmented reality navigation or a vehicle infotainment system may be implemented.

In particular, according to an embodiment of the present invention, a route guide indicator suitable for augmented reality is built in real time through three-dimensional processing of a route guide line, and a three-dimensional route guide indicator can be displayed realistically and effectively on a two-dimensional camera image. have. This will be described later with reference to the drawings.

2 is a diagram for explaining a system network connected to an electronic device according to an embodiment of the present invention. Referring to FIG. 2 , an electronic device 100 according to an embodiment of the present invention may be implemented as various devices provided in a vehicle, such as a navigation device, a black box, a smart phone, or other device for providing an augmented reality interface for a vehicle. It can connect to a communication network and other electronic devices 61 to 64 .

In addition, the electronic device 100 may calculate the current location and the current time zone by interworking with the GPS module according to the radio signal received from the artificial satellite 20 .

Each of the satellites 20 may transmit an L-band frequency having a different frequency band. The electronic device 100 may calculate the current location based on the time it takes for the L-band frequency transmitted from each satellite 20 to reach the electronic device 100 .

Meanwhile, the electronic device 100 may wirelessly access the network 30 through the control station 40 , ACR, the base station 50 , RAS, etc. through the communication unit 180 . When the electronic device 100 is connected to the network 30 , it may also indirectly connect with other electronic devices 61 and 62 connected to the network 30 to exchange data.

Meanwhile, the electronic device 100 may indirectly access the network 30 through another device 63 having a communication function. For example, when the electronic device 100 does not include a module that can connect to the network 30 , it can communicate with another device 63 having a communication function through a short-range communication module or the like.

3 is a flowchart illustrating a method of generating lane information of an electronic device according to an embodiment of the present invention. Referring to FIG. 3 , first, the electronic device 100 may identify a lane area from driving-related image data of a vehicle ( S101 ). Specifically, the lane information generating unit 140 may convert a driving-related image into a gray image, and perform a lane detection algorithm to determine an area in which lanes located on both sides of the vehicle can be identified as the lane area. Here, the driving-related image of the vehicle may include an image of stopping, driving, etc. of the vehicle. In addition, the driving-related image of the vehicle may be an image captured by a camera module included in the electronic device 100 or an image received by the electronic device 100 from an image captured by another device. In addition, the driving-related image of the vehicle may be a Red Green Blue (RGB) (color) image.

Then, the electronic device 100 may generate lane information corresponding to the lane in which the vehicle is located from the image data of the identified lane region portion ( S102 ). Specifically, the lane information generator 140 may generate lane information by analyzing pattern information and color information of the lane for the detected lane area. The lane information may include at least one of line type information and line color information corresponding to each lane located on both sides of the lane in which the vehicle is located.

Hereinafter, a method of generating lane information will be described in more detail with reference to FIGS. 4 to 7 .

4 is a flowchart specifically illustrating a method for generating lane information of an electronic device according to an embodiment of the present invention. Referring to FIG. 4 , first, the electronic device 100 may convert color image data into a gray image ( S201 ) and detect a lane area from the converted gray image ( S202 ).

Specifically, the lane information generating unit 140 may extract an area for detecting a lane from the captured driving-related image. Also, since it is difficult to detect a lane when a part of the road is affected by the shadow, the lane information generator 140 may perform light source correction on the original image in advance in order to minimize the shadow effect.

Also, the lane information generating unit 140 may detect an area in which a lane may exist as a lane area according to a predetermined camera position or camera installation angle. For example, the lane information generator 140 may determine a lane area based on a starting position of a lane. Also, the lane information generating unit 140 may estimate the starting position of the lane region and the length of the lane region as the lane width (the maximum width between the left lane region and the right lane region) in the driving-related image and the viewing angle of the camera.

Also, the lane information generator 140 may convert a gray image corresponding to the lane detection region into an edge image, and detect the lane region based on a straight line position extracted from the converted edge image. More specifically, the driving-related image may be converted into an edge image through various known algorithms, and the edge image may include edges representing a plurality of straight lines. In this case, the lane information generating unit 140 may recognize the position of the detected straight line as a lane. Also, the lane information generator 140 may determine a lane area based on a position of a straight line having a constant lane width with respect to the driving direction of the vehicle among a plurality of straight line candidates.

5 shows the gray image conversion and lane area detection process. Referring to FIG. 5 , an initially input driving-related image may be converted into a gray image 200 , and linear lane areas 201 and 202 may be detected through a lane detection algorithm such as edge detection. The lane area may be divided into a left lane area 201 and a right lane area 202 based on the location of the vehicle.

Fig. 4 will be described again.

Thereafter, when a lane area is detected, the electronic device 100 may set a lane type ROI based on the lane area ( S203 ). Specifically, when a lane area is detected, the lane information generator 140 may set a lane type region of interest (ROI) based on the detected lane area. The lane type ROI may mean a portion of a driving-related image including a lane for determining the line type and color of the lane and a predetermined area around the lane.

More specifically, FIG. 6 shows a lane type ROI in a gray image.

As shown in FIG. 6 , the lane-type ROIs 210 and 220 may include a previously detected lane-line region and a portion thereof. Also, the lane type ROI may be divided into a left lane type ROI 210 and a right lane type ROI 220 based on the driving direction of the vehicle.

For example, the previously detected lane area may be generally formed in a straight line, and when expressed as an equation such as y = a * x + b , the lane type ROI is y = a * x + b + m and y = a * x + b - m. Unlike the existing simple lane detection method, this is a method for generating specific and various driving lane information. The lane information generating unit 140 may extend the detected straight lane area and set the surrounding area as the area of interest. .

Fig. 4 will be described again.

Thereafter, the electronic device 100 binarizes the lane-type region of interest (S204), maps the binarized partial gray image to the one-dimensional region (S205), and uses at least one of visual continuity and speed to The type can be identified (S206).

The lane information generator 140 may extract a partial gray image of a lane type ROI from the converted gray image, and may perform binarization on the partial gray image. The reference value for binarization may be determined as an average luminance value of a partial gray image of the ROI. Accordingly, the driving lane information generating module 180 may clearly distinguish only a portion determined to be a lane from the partial gray image.

In addition, the lane information generator 140 may map each line (left and right) identified in the binarized partial gray image to a one-dimensional area. Then, the line type can be identified by analyzing the pattern of each line mapped to the one-dimensional area.

More specifically, FIG. 7 illustrates binarization and one-dimensional mapping of a lane type region of interest in a gray image.

As shown in FIG. 7 , when binarization is performed on the lane type ROI, a binarized image 300 may be obtained. In the binarized image 300 , a white portion may be identified as a lane, and other portions may be identified as black.

In addition, each line identified in the binarized image 300 may be mapped to a one-dimensional area. The lane information generator 140 may easily determine the line type by using the image 310 mapped to the one-dimensional area.

For example, the lane information generator 140 may determine whether a dotted line or a solid line is based on the start point and length characteristics of each line mapped in one dimension. In addition, the lane information generating unit 140 may determine whether a dotted line or a solid line by using the time continuity and speed of each line mapped in one dimension. In addition, the lane information generator 140 may first determine a dotted line or a solid line according to the starting point and length characteristics, and then secondarily determine the dotted line or a solid line using time continuity and speed.

More specifically, the lane information generating unit 140 may first determine whether a dotted line or a solid line by comparing and determining line lengths according to the starting point positions of each line. In this case, the lane information generating unit 140 may determine whether a dotted line or a solid line exists with only one image frame.

Also, the lane information generating unit 140 may more clearly determine whether each line is a dotted line or a solid line according to whether each line is continuously formed over time. For example, the driving lane information generating module 180 may preset a degree of continuity according to a line moving speed in the image, and when the continuity of each line is lower than a preset continuity value, it may be determined as a dotted line.

Therefore, according to an embodiment of the present invention, it is possible to determine whether a dotted line or a solid line is in advance through one frame and to verify it through successive frames to determine the final line type.

Referring again to FIG. 4 , the electronic device 100 may detect the color of the line part whose type is identified from the color original image data ( S207 ).

The lane information generator 140 may analyze the color image, detect a color of a portion corresponding to the previously identified line, and classify it. For example, the lane information generator 140 may classify the detected color as white, yellow, or blue.

Thereafter, the electronic device 100 may generate lane information corresponding to the lane in which the vehicle is located based on the identified line type and the classified color ( S208 ).

8 is a flowchart specifically illustrating a method for generating lane location information according to an embodiment of the present invention. Referring to FIG. 8 , the electronic device 100 may obtain information on the lane of a road on which the vehicle is located from the map data ( S301 ). Here, the lane information of the road may be information on the lane of the road on which the vehicle being driven is currently located, and may include information on the number of lanes on the road on which the vehicle is located. In addition, the lane information of the road is obtained from map data stored in the storage unit 110 in the electronic device 100 , or obtained from an external map database (DB) separate from the electronic device 100 , or another electronic device 100 . can be obtained from

Then, the electronic device 100 may determine whether the vehicle is located in the first lane (first lane) or the last lane (end lane) of the road by using the generated lane information ( S302 ). Specifically, the lane location information generating unit 150 may determine whether the vehicle is located in the first lane or the last lane of the road by applying lane information corresponding to the lane in which the vehicle is located to the lane determination table as shown in FIG. 9 . .

That is, the lane determination table may include a primary lane and an end lane determined according to a country, a type and color of a left line, and a type and color of a right line. Here, the lane determination table as shown in FIG. 10 may be an example, and may be set to a different value for each country or situation according to the setting.

Meanwhile, when the vehicle is located on the first or last lane, the electronic device 100 may generate information on the lane on which the vehicle is located by reflecting lane information of the road ( S303 ). For example, if it is determined that the vehicle is located on the last lane, the lane location information generating unit 150 may generate the lane location information as N lanes. In addition, when the number of lanes corresponding to the road lane information is 5, N lanes may be generated as 5 lanes by reflecting this.

Then, when the lane on which the vehicle is located is changed to a lane located between the first or last lanes according to the change of lane of the vehicle, the electronic device 100 may update the generated lane location information with the changed lane location information (S304). ). In this case, the lane location information generating unit 150 may determine whether to depart from a lane by using the lane information, and determine whether to change the lane based on this. For example, if it is determined that the vehicle is located on the 5th lane and changes to the first lane from the left lane, the lane location information generator 150 may update the lane location information from the 5th lane to the 4th lane by reflecting this.

Then, when the lane on which the vehicle is located is changed from a lane on which the vehicle is located to the first or last lane according to the change of lane of the vehicle, the electronic device 100 may re-acquire information on the lane of the road on which the vehicle is located ( S305 ). Then, by reflecting the re-acquired lane information, the lane location information on which the vehicle is located may be regenerated (S306). For example, if it is determined that the vehicle is located in lane 4 and is changed to the first lane to the right lane, the vehicle has moved to lane 5, which is the last previously set lane, so that information on the lane of the road on which the vehicle is currently located may be acquired. And, when the obtained lane information is 4 lanes, the lane location information on which the vehicle is located may be regenerated as 4 lanes.

Meanwhile, the method for generating lane location information according to an embodiment of the present invention is not limited to FIG. 9 . Accordingly, according to another embodiment, the above-described order may be partially changed. As an example, the step of obtaining information on the lane of the road on which the vehicle is located may be performed in step S304 . In this case, when the vehicle is located on the first or last lane, the electronic device 100 may generate location information on the lane on which the vehicle is located ( S303 ). For example, if it is determined that the vehicle is located on the last lane, the lane location information generating unit 150 may generate the lane location information as N lanes.

And, when the lane in which the vehicle is located is changed to a lane located between the first or last lanes according to the lane change of the vehicle, the electronic device 100 uses the generated lane location information and the obtained lane information of the road to locate the vehicle. Information can be updated (S304). For example, when it is determined that the vehicle is located in the N lane corresponding to the last lane and changed to the first lane to the left lane, the lane location information generating unit 150 reflects N = 5, which is information on the number of lanes in the N-1 lane, Lane location information can be updated to 4 lanes.

In addition, according to another embodiment of the present invention, the lane location information generating unit 150 applies lane information corresponding to the lane in which the vehicle is located to the lane determination table as shown in FIG. It is possible to determine whether the vehicle is located on the last lane. However, in this case, in a situation in which there are a plurality of central lanes of a road (eg, when the number of lanes is four or more), it is not possible to accurately determine the lane in which the vehicle is located among the plurality of central lanes, an embodiment of the present invention Preferably, the same method as in FIG. 9 can be used.

11 is a flowchart illustrating a control method according to an embodiment of the present invention. Referring to FIG. 11 , first, the electronic device 100 may identify a lane area from driving-related image data of a vehicle ( S401 ).

Then, lane information corresponding to the lane in which the vehicle is located may be generated from the image data of the identified lane area portion (S402).

Then, information on the lane location on which the vehicle is located may be generated by using at least one of the generated lane information and lane information on the road on which the vehicle is located ( S403 ).

In addition, driving-related guidance of the vehicle may be performed using the obtained lane information (S404).

Here, the step of performing the driving-related guidance of the vehicle ( S404 ) may include outputting a lane change guidance using the vehicle navigation route and lane location information.

In addition, the step of performing the driving-related guidance of the vehicle ( S404 ) may include outputting the guidance on the lane in which the vehicle is located by using the lane location information.

Meanwhile, the control method according to an embodiment of the present invention may further include the step of selecting and outputting an appropriate lane departure guide according to types of both lanes of a vehicle lane identified based on lane information.

Here, the output may be performed by generating an indicator for performing driving-related guidance and outputting the generated indicator through augmented reality.

On the other hand, in the conventional augmented reality navigation, various methods are used for route guidance, but it is difficult to synthesize the route guide line and the actual road environment, so there is a limit to the expression of the route guide line.

However, according to an embodiment of the present invention, a route guide indicator suitable for augmented reality is built in real time through three-dimensional processing of a route guide line, and a three-dimensional route guide indicator can be displayed realistically and effectively on a two-dimensional camera image. have. Hereinafter, the augmented reality providing unit 160 according to the present embodiment for achieving the above object will be described in detail.

12 is a block diagram specifically illustrating the augmented reality providing unit 160 according to an embodiment of the present invention. Referring to FIG. 12 , the augmented reality providing unit 160 may include all or a part of a calibrator 161 , a 3D space generation unit 162 , an indicator generation unit 163 , and a mapping unit 164 .

The calibrator 161 may perform calibration for estimating a camera parameter corresponding to a camera from a captured image captured by the camera. Here, the camera parameter may be a parameter constituting the camera matrix, which is information indicating the relationship between the actual space and the photograph.

The 3D space generator 162 may generate a virtual 3D space based on a captured image captured by the camera. Specifically, the 3D space generator 162 obtains depth information from an image captured by the camera based on the camera parameter estimated by the calibrator 161, and based on the obtained depth information and the captured image You can create a virtual 3D space.

The indicator generating unit 163 may generate an indicator for guidance in augmented reality, for example, a route guidance indicator, a lane change guidance indicator, a lane departure guidance indicator, and the like.

In particular, when a user input requesting route guidance to a destination is received through the input unit 120 , the indicator generator 163 may generate a route guidance indicator for route guidance in augmented reality. Here, the route guide indicator generating unit includes a route guide line processing unit 163-1 and a route guide line three-dimensionalization unit 163-2. It may include a dynamic texture mapping unit 163-3.

When a route guide line to a destination is generated according to a user's route guide request, the route guide line processing unit 163-1 may process the route guide line by reflecting the radius of the driving trajectory of the actual driving vehicle.

Specifically, the electronic device 100 uses the map data acquired from the storage 110 and the map data acquired from the external map database DB separate from the electronic device 100 to the destination according to the user's route guidance request. You can create a route guide to . Here, the generated path guide line may be composed of nodes and links, and may have a form as shown in FIG. 13(a) . That is, referring to FIG. 13A , the generated path guide line 1301 may have a straight line shape similar to the vehicle driving trajectory in the curved section 1302 . Accordingly, when augmented reality is provided by synthesizing the camera image based on the route guide line as shown in FIG. 13( a ), a result different from the driving trajectory of the actual vehicle is displayed.

Accordingly, the route guide line processing unit 163 - 1 according to an embodiment of the present invention may process the route guide line by reflecting the radius of the driving trajectory of the actual driving vehicle. Specifically, the path guide line processing unit 163 - 1 may remove unnecessary vertices such as vertices and overlapping points for an area not currently displayed on the screen among the generated path guide lines. In addition, the path guide line processing unit 163 - 1 may add vertices for maintaining a straight line of the path guide line in front of the host vehicle. In addition, the path guide line processing unit 163 - 1 may add vertices for curving the curve section to the curve section of the path guide line. In addition, the path guide line processing unit 163 - 1 may generate a path guide line reflecting the radius of the driving trajectory of the actual driving vehicle by using the added vertices.

According to an embodiment of the present invention, a path guide line as shown in FIG. 13B may be generated. That is, referring to FIG. 13B , the processed route guide line 1303 may have a smooth shape similar to the vehicle driving trajectory in the curve section 1304 .

The path guide line three-dimensionalization unit 163-2 may perform variable stereoscopicization by varying the height of the path guide line generated by the path guide line processing unit 163-1 according to a distance. This will be described in detail with reference to FIG. 14 .

Referring to FIG. 14 , the path guide line three-dimensionalization unit 163-2 may generate virtual path guide lines 1402 and 1403 on both sides of the path guide line 1401 generated by the path guide line processing unit 163-1. Specifically, the path guide line stereoscopic unit 163-2 calculates a unit vector of the vertices of the path guide line 1401 generated by the path guide line processing unit 163-1, and is converted to the unit vector by the dot product operation of the calculated unit vector. By calculating a vertical normal vector, virtual path guide lines 1402 and 1403 may be generated.

In addition, the path guide line stereoscopic unit 163 - 2 may calculate height values of vertices included in the path guide line 1401 . In this case, the path guide line stereoscopic unit 163 - 2 may calculate a height value such that the height values of the vertices included in the path guide line 1401 increase in proportion to the distance.

In addition, the path guide line stereoscopic unit 163-2 generates a surface through polygons using the vertices of the path guide line 1401 for which the height value is calculated and the vertices included in each of the virtual path guide lines 1402 and 1403 to be three-dimensionalized. can be done

Accordingly, among the route guidance indicators displayed on the screen, the route guidance indicator of a point located far from the own vehicle may be displayed on the screen so that the driver can identify it.

The dynamic texture mapping unit 163 - 3 may map a texture having a displacement according to a vehicle speed on the stereoscopic data generated by the path guide line stereoscopicizing unit 163 - 2 .

Here, the texture may be a texture having a displacement according to the vehicle speed. That is, when the vehicle moves on a path, the dynamic texture mapping unit 163 - 3 may generate a texture having a displacement according to the vehicle speed by changing the mapping position of the texture in the three-dimensional data. In this case, the route guidance indicator displayed on the screen can maximize the effect of being attached to the road surface.

According to this operation, the indicator generator 163 may generate a route guidance indicator for route guidance in augmented reality.

On the other hand, in order to display a more effective screen when there is a turn point and the display technique after passing the turn point, the route guide line stereoscopic unit 163-2 may verticalize the stereoscopic generated according to the above-described operation.

That is, in the case of a left turn, the route guide line stereoscopic unit 163-2 may determine stereoscopic data by erecting a virtual route guide line 1403 corresponding to the right side of the route guide line 1401 in the height direction, and vice versa, in the case of a right turn, The three-dimensional data may be determined by erecting a virtual path guide line 1402 corresponding to the left side of the path guide line 1401 in the height direction.

Meanwhile, the mapping unit 164 may combine the indicator generated by the indicator generating unit 163 with the virtual 3D space generated by the 3D space generating unit 162 .

15 is a flowchart illustrating an augmented reality route guidance method according to an embodiment of the present invention. Referring to FIG. 15 , first, the electronic device 100 may receive a user input requesting route guidance ( S501 ).

Then, the electronic device 100 may generate a route guide line based on destination information according to the route guide (S502).

Then, the electronic device 100 may correct the path guide line generated by reflecting the radius of the driving trajectory of the actual driving vehicle ( S503 ). Specifically, step S503 is a step of removing unnecessary vertices such as vertices and overlapping points for an area not currently displayed on the screen among the generated route guide lines, and adding vertices for maintaining a straight line of the route guide line in front of the own vehicle and adding vertices for curving the curve section to the curve section of the route guide line, and generating a route guide line reflecting the radius of the driving trajectory of the actual driving vehicle using the added vertices.

Then, the electronic device 100 may perform variable three-dimensionalization by varying the height of the corrected route guide line according to the distance from the host vehicle (S504). Specifically, step S504 includes the steps of generating virtual path guides on both sides of the processed path guide, calculating the height value so that the height values of vertices included in the processed path guide line increase in proportion to the distance, and the height value is The method may include the step of three-dimensionalizing the vertices of the calculated path guide line and the vertices included in each of the virtual path guide line by generating a surface through a polygon.

Then, the electronic device 100 may generate a route guidance indicator by mapping a texture having a displacement according to the vehicle speed to the stereoscopic data (S505).

Then, the electronic device 100 may output the route guidance indicator to the screen through augmented reality (S506). Here, the output screen will be described in detail with reference to FIG. 16 .

16 is a diagram illustrating a route guidance screen according to an embodiment of the present invention. Referring to FIG. 16 , the electronic device 100 according to an embodiment of the present invention may display a route guidance screen in augmented reality (left screen) and a route guidance screen on a map (right screen) together.

In this case, the augmented reality providing unit 160 may generate an indicator overlaid on the augmented reality for guidance on the augmented reality.

For example, as shown in FIG. 16 , a route guidance indicator 1601 , a lane change guidance indicator 1602 , and a lane departure guidance indicator 1603 may be generated. In addition, the augmented reality providing unit 160 may output the generated indicator on the augmented reality.

17 is a diagram illustrating an implementation form when a camera and an electronic device are separated according to an embodiment of the present invention. Referring to FIG. 17 , the vehicle black box 200 provided separately from the vehicle navigation system 100 may configure a system according to an embodiment of the present invention using a wired/wireless communication method.

The vehicle navigation device 100 may include a display unit 145 provided on the front surface of the navigation housing 191 , a navigation operation key 193 , and a navigation microphone 195 .

The vehicle black box 200 may acquire vehicle data while driving and stopping. That is, it is possible to capture an image while the vehicle is driving, as well as capture an image even when the vehicle is stopped. The quality of the image acquired through the vehicle black box 200 may be constant or may be changed. For example, by increasing the image quality before and after the occurrence of an accident, and lowering the image quality in a normal case, it is possible to store essential images while minimizing the required storage space.

The vehicle black box 200 may include a black box camera 222 , a black box microphone 224 , and an attachment unit 281 .

Meanwhile, in FIG. 17 , the vehicle navigation system 100 and the vehicle black box 200 provided separately are illustrated as being connected through a wired/wireless communication method, but the vehicle navigation system 100 and the vehicle black box 200 are connected using a wired/wireless communication method. It may not be connected through . In this case, when a storage medium for storing the captured image of the black box 200 is inserted into the electronic device 100 , the electronic device 100 may receive the captured image. On the other hand, it is also possible that the vehicle black box 200 has the function of the vehicle navigation device 100 or the vehicle navigation device 100 is integrated with a camera. This will be described in detail with reference to FIG. 18 .

18 is a diagram illustrating an implementation form when a camera and an electronic device are integrated according to an embodiment of the present invention. Referring to FIG. 18 , when the electronic device includes a camera function, the user may mount the electronic device so that the camera part of the electronic device photographs the front of the vehicle and the display part of the electronic device can be recognized by the user. Accordingly, it is possible to implement a system according to an embodiment of the present invention.

19 is a diagram illustrating an implementation form using a head-up display (HUD) and an electronic device according to an embodiment of the present invention. Referring to FIG. 19 , the electronic device may display the augmented reality guidance screen on the heads-up display through wired/wireless communication with the heads-up display.

Meanwhile, the above-described method for controlling an electronic device according to various embodiments of the present disclosure may be implemented as a program code and provided to each server or device while stored in various non-transitory computer readable media. have.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, and the like, and can be read by a device. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: electronic device 110: storage unit
120: input unit 130: output unit
140: lane information generation unit 150: lane location information generation unit
160: augmented reality providing unit 170: control unit
180: communication unit 190: sensing unit
195: power supply

Claims (12)

A method for controlling an electronic device, comprising:
receiving a user input requesting route guidance;
generating a route guide line based on destination information corresponding to the request;
performing correction of the generated path guide line by reflecting the radius of the driving trajectory of the actual driving vehicle;
performing variable three-dimensionalization by varying the height of the corrected route guide line according to a distance from the own vehicle;
generating a route guidance indicator by mapping a texture having displacement according to vehicle speed to the three-dimensional data generated according to the variable three-dimensionalization; and
A control method comprising a; outputting the route guidance indicator to the screen through augmented reality. According to claim 1,
The step of performing the correction of the route guide line,
adding vertices for maintaining a straight line of the path guide line in front of the own vehicle;
adding vertices for curving the curve section to the curve section of the path guide line; and
and generating a path guide line reflecting the radius of the driving trajectory of the actual driving vehicle by using the added vertices. According to claim 1,
The step of performing the variable three-dimensionalization comprises:
generating virtual path guides on both sides of the corrected path guides;
calculating a height value such that height values of vertices included in the corrected path guide line increase in proportion to a distance of the host vehicle; and
and three-dimensionalizing the vertices of the path guide line from which the height value is calculated and the vertices included in each of the virtual path guide lines through polygons. delete According to claim 1,
performing calibration for estimating a camera parameter corresponding to the camera from the captured image taken by the camera; and
It further comprises; generating a virtual 3D space from the image taken by the camera based on the camera parameter;
The output step is
Control method characterized in that the output by combining the generated route guidance indicator in the generated virtual 3D space. In an electronic device,
an input unit for receiving a user input requesting route guidance;
a route guide line generator for generating a route guide line based on the destination information corresponding to the request;
a route guidance indicator generator for generating a route guide indicator for route guidance in augmented reality using the generated route guide line; and
Including; a display unit for outputting the generated route guidance indicator on the screen through augmented reality;
The route guidance indicator generating unit,
a path guide line processing unit that corrects the generated path guide line by reflecting the radius of the driving trajectory of the actual driving vehicle;
a path guide line stereoscopic unit configured to change the height of the corrected path guide line according to a distance from the host vehicle to perform variable stereoscopicization; and
and a texture mapping unit to generate a route guidance indicator by mapping a texture having a displacement according to a vehicle speed to the stereoscopic data generated according to the variable three-dimensionalization. 7. The method of claim 6,
The route guide line processing unit,
Add vertices to maintain the straight line of the route guide line in front of the own vehicle, add vertices for curvature of the curve section to the curve section of the route guide line, and use the added vertices to reflect the radius of the driving trajectory of the actual driving vehicle Electronic device, characterized in that for generating. 7. The method of claim 6,
The route guide line three-dimensionalization unit,
Virtual path guide lines are generated on both sides of the corrected path guide line, height values of vertices included in the corrected path guide line are calculated to increase in proportion to the distance of the host vehicle, and the height value is calculated path guide The electronic device of claim 1, wherein three-dimensionalization is performed on vertices of a line and vertices included in each of the virtual path guide lines through polygons. delete 7. The method of claim 6,
a calibration unit that performs calibration for estimating a camera parameter corresponding to the camera from the captured image taken by the camera; and
Further comprising; a 3D space generating unit for generating a virtual 3D space from the image captured by the camera based on the camera parameters;
The display unit,
An electronic device characterized in that the generated virtual 3D space is output by combining the generated route guidance indicator. A computer-readable recording medium in which a program for executing the method according to any one of claims 1 to 3 and 5 is recorded. A program stored in a computer-readable recording medium comprising a program code for executing the method according to any one of claims 1 to 3 and 5.

Images