KR20210010935A - Electronic apparatus, control method of electronic apparatus and computer readable recording medium - Google Patents

Abstract

Disclosed is a method for controlling an electronic device for generating an indicator for driving-related guidance that assists in driving a vehicle and displaying the generated indicator on a predetermined area of a road guidance screen. According to the present invention, the control method comprises the steps of: generating information for performing driving-related guidance that assists driving of a vehicle; generating an indicator for driving-related guidance using the generated information; and overlapping the generated indicator on a predetermined area of a road guidance screen.

Description

Electronic device, control method of electronic device, and computer-readable recording medium TECHNICAL FIELD [ELECTRONIC APPARATUS, CONTROL METHOD OF ELECTRONIC APPARATUS AND COMPUTER READABLE RECORDING MEDIUM}

The present invention relates to an electronic device, a control method of the electronic device, and a computer-readable recording medium, and more particularly, an electronic device that performs driving-related guidance for assisting the driving of a vehicle, a control method of the electronic device, and a computer-readable recording medium. It relates to recording media.

The most important thing when driving a vehicle is safe driving and the prevention of traffic accidents.To this end, various auxiliary devices that perform the vehicle's attitude control and function control of vehicle components, and safety devices such as seat belts and airbags are installed on the vehicle. have.

In addition, recently, devices such as a black box that are located in a vehicle and store driving images of the vehicle and data transmitted from various sensors are being provided in vehicles to identify the cause of an accident in the vehicle when an accident occurs. Portable terminals such as smartphones and tablets can also be equipped with a black box or a navigation application, and thus are used as such a vehicle device.

However, currently, the utilization of driving images in such vehicle devices is low. More specifically, even if a driving image of the vehicle is acquired through a vision sensor such as a camera currently mounted on the vehicle, the electronic device of the vehicle simply displays and transmits the driving image, or only generates simple surrounding notification information such as whether or not a lane is departed.

In addition, a HUD (Head Up Display) or an augmented reality interface has also been proposed as an emerging electronic device of a vehicle, but the utilization of the driving image of the vehicle is limited to simple display or simple notification information generation.

It was conceived to solve the above-described problems of the present invention, and an object of the present invention is an electronic device that generates an indicator for driving-related guidance that assists the driving of a vehicle, and displays the generated indicator in a predetermined area of a road guidance screen , To provide a method for controlling an electronic device and a computer-readable recording medium.

A method of controlling an electronic device according to an embodiment of the present invention for achieving the above object comprises the steps of generating information for performing a driving-related guide that assists driving of a vehicle, and the driving by using the generated information. And generating an indicator for a related guide, and displaying the generated indicator overlaid on a predetermined area of a road guide screen.

In addition, the generating of the information for performing the driving-related guide includes signal type information of a traffic light, lane information on a lane in which the vehicle is located, location information on a lane in which the vehicle is located, and another vehicle located in front of the vehicle. Information for performing driving-related guidance including at least one of distance information and route information for guiding the road may be generated.

In addition, the step of determining an item requiring guidance among lane departure guidance, forward vehicle collision prevention guidance, signal change guidance of a traffic light, lane change guidance, and lane departure guidance using the generated information Can include.

In addition, the route guidance screen may be an Augmented Reality (AR) route guidance screen.

In addition, the indicator may include a first area for performing the front vehicle collision prevention guidance, a second area located on the left and right sides of the first area, and a second area for performing the lane departure guidance, and located at an upper end of the first and second areas and the A third area for performing signal change guidance, and a fourth area passing through the second area and performing change guidance for the lane may be included.

In addition, the indicator includes a first area for performing the front vehicle collision prevention guidance, a third area positioned at an upper end of the first area for performing the signal change guidance, and a fourth area for performing the lane change guidance. And, the second area for guiding the lane departure may be located in the lane portion of the augmented reality road guidance screen.

In addition, the generating of the indicator may generate an indicator in which an area corresponding to the item requiring guidance is activated among the plurality of areas.

The method may further include determining a vanishing point from image data related to driving of the vehicle, and determining a predetermined area in which the indicator is to be displayed on the augmented reality road guide screen based on the determined vanishing point.

In addition, in the determining of a predetermined area on which the indicator is to be displayed, an area located above the determined vanishing point may be determined as a predetermined area on which the indicator is to be displayed.

Meanwhile, an electronic device according to an embodiment of the present invention for achieving the above object includes a display unit for displaying a screen, and a guide-related information generation unit for generating information for performing a driving-related guide that assists driving of a vehicle. And an indicator generating unit generating an indicator for driving-related guidance using the generated information, and a control unit controlling the display unit to overlap and display the generated indicator on a predetermined area of a road guidance screen.

In addition, the guide-related information generation unit includes signal type information of a traffic light, lane information on a lane in which the vehicle is located, location information on a lane in which the vehicle is located, distance information from another vehicle located in front of the vehicle, and the road guidance Information for performing driving-related guidance including at least one of the route information for may be generated.

In addition, the guide-related information generation unit may use the generated information to guide the lane departure to assist the driving of the vehicle, guide the front vehicle collision prevention, guide the signal change of the traffic light, guide the lane change, and guide the lane departure guide. Items can be decided.

In addition, the route guidance screen may be an Augmented Reality (AR) route guidance screen.

In addition, the indicator may include a first area for performing the front vehicle collision prevention guidance, a second area located on the left and right sides of the first area, and a second area for performing the lane departure guidance, and located at an upper end of the first and second areas and the A third area for performing signal change guidance, and a fourth area passing through the second area and performing change guidance for the lane may be included.

In addition, the indicator includes a first area for performing the front vehicle collision prevention guidance, a third area positioned at an upper end of the first area for performing the signal change guidance, and a fourth area for performing the lane change guidance. And, the second area for guiding the lane departure may be located in the lane portion of the augmented reality road guidance screen.

In addition, the indicator generator may generate an indicator in which an area corresponding to an item requiring guidance among the plurality of areas is activated.

In addition, the indicator generator may determine a predetermined area in which the indicator is to be displayed on the augmented reality road guidance screen based on a vanishing point determined from image data related to driving of the vehicle.

In addition, the indicator generator may determine an area located above a predetermined distance from the determined vanishing point as a predetermined area in which the indicator is to be displayed.

Meanwhile, in a computer-readable recording medium according to an embodiment of the present invention for achieving the above object, a program code for executing the above-described control method in a computer may be recorded.

According to various embodiments of the present disclosure described above, driving related guidance such as lane departure guidance, front vehicle collision prevention guidance, signal change guidance of a traffic light, lane change guidance, and lane departure guidance may be performed. Accordingly, the driver's safe driving and convenience for the vehicle can be promoted.

Further, according to various embodiments of the present disclosure described above, by designing the positions of each of the plurality of guide areas within the indicator to correspond to the road situation, it is possible to provide guidance to the driver in a more intuitive manner.

In addition, according to various embodiments of the present disclosure, by superimposing an indicator for driving-related guidance on an augmented reality (AR) route guidance screen, guidance may be provided to the driver in a more intuitive manner.

In addition, according to various embodiments of the present invention described above, the position of the indicator displayed on the augmented reality (AR) route guidance screen is positioned in an area located above a predetermined distance from the vanishing point, thereby guiding the driver in a more intuitive manner. Can provide.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention.
2 is a block diagram specifically showing a guide-related information generation unit according to an embodiment of the present invention.
3 is a diagram illustrating a system network connected to an electronic device according to an embodiment of the present invention.
4 is a flowchart illustrating an operation of generating signal type information according to an embodiment of the present invention.
5 is a diagram illustrating a method of determining ROI image data and a stop signal corresponding to a stop signal according to an embodiment of the present invention.
6 is a flowchart illustrating a method of generating lane information according to an embodiment of the present invention.
7 is a diagram illustrating a lane type ROI binarization and one-dimensional mapping in a gray image according to an embodiment of the present invention.
8 is a detailed flowchart illustrating a method of generating location information on a lane according to an embodiment of the present invention.
9 is a diagram illustrating a difference determination table according to an embodiment of the present invention.
10 is a detailed flowchart illustrating a method of controlling an electronic device according to an embodiment of the present invention.
11 is a detailed flowchart illustrating a method of controlling an electronic device according to an embodiment of the present invention.
12 is a diagram illustrating an indicator according to an embodiment of the present invention.
13 is a diagram illustrating an indicator for a signal change guide and a front vehicle collision prevention guide according to an embodiment of the present invention.
14 is a diagram illustrating an indicator for guiding a front vehicle collision prevention according to an embodiment of the present invention.
15 is a diagram illustrating an indicator for guiding a lane change according to an embodiment of the present invention.
16 is a diagram illustrating an indicator for a lane departure prevention guidance and a front vehicle collision prevention guidance according to an embodiment of the present invention.
17 is a diagram illustrating a route guidance screen according to an embodiment of the present invention.
18 is a diagram illustrating a road guidance screen according to another embodiment of the present invention.
19 is a diagram illustrating an implementation form when a camera and an electronic device are separated according to an embodiment of the present invention.
20 is a diagram illustrating an implementation form when a camera and an electronic device are integrated according to an embodiment of the present invention.
21 is a diagram illustrating an implementation form using a head-up display (HUD) and an electronic device according to an embodiment of the present invention.

The following content merely illustrates the principles of the present invention. Therefore, those skilled in the art can implement the principles of the present invention and invent various devices included in the concept and scope of the present invention, although not clearly described or illustrated herein. In addition, all conditional terms and examples listed in this specification are, in principle, intended to be clearly intended only for the purpose of making the concept of the present invention understood, and should be understood as not limiting to the embodiments and states specifically listed as described above. do.

In addition, it is to be understood that all detailed descriptions listing specific embodiments as well as principles, aspects and embodiments of the present invention are intended to include structural and functional equivalents of these matters. It should also be understood that these equivalents include not only currently known equivalents, but also equivalents to be developed in the future, that is, all devices invented to perform the same function regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing a conceptual perspective of exemplary circuits embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudocodes, etc. are understood to represent various processes performed by a computer or processor, whether or not the computer or processor is clearly depicted and that can be represented substantially in a computer-readable medium. Should be.

The functions of the various elements shown in the figures, including a processor or functional block represented by a similar concept, may be provided by the use of dedicated hardware as well as hardware having the ability to execute software in association with appropriate software. When provided by a processor, the function may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

In addition, the explicit use of terms presented as processor, control, or similar concepts should not be interpreted exclusively by referring to hardware capable of executing software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM, and non-volatile memory. Other commonly used hardware may also be included.

In the claims of this specification, components expressed as means for performing the functions described in the detailed description include all types of software including, for example, a combination of circuit elements or firmware/microcode that perform the above functions. It is intended to include all methods of performing a function to perform the function, and is combined with suitable circuitry for executing the software to perform the function. Since the invention defined by these claims is combined with the functions provided by the various enumerated means and combined with the manner required by the claims, any means capable of providing the above functions are equivalent to those conceived from this specification. It should be understood as.

The above-described objects, features, and advantages will become more apparent through the following detailed description in connection with the accompanying drawings, whereby those of ordinary skill in the technical field to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention. Referring to FIG. 1, the electronic device 100 includes a storage unit 110, an input unit 120, an output unit 130, a guide-related information generation unit 140, an indicator generation unit 150, an augmented reality providing unit ( 160), the control unit 170, the communication unit 180, and all or part of the sensing unit 190.

Here, the electronic device 100 includes a smart phone, a tablet computer, a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a smart glass, a project glass, and a navigation system that can provide driving-related guidance to the vehicle driver. navigation), black-box, and the like.

Here, the driving state of the vehicle may include various states in which the vehicle is being driven by the driver, such as a stopped state of the vehicle, a driving state of the vehicle, and a parking state of the vehicle.

Driving-related guidance may include various guidance to assist the driver of the vehicle, such as directions, lane departure guidance, signal guidance, signal change guidance, forward vehicle collision prevention guidance, lane change guidance, lane guidance, and lane change guidance. have.

Here, the route guidance is an augmented reality route guidance that performs route guidance by combining various information such as a user's location and direction with an image photographed in front of a driving vehicle, and 2-Dimensional (2-D) or 3-D (3- A 2D (2-Dimensional) or 3D (3-Dimensional) road guidance may be included in which various information such as a user's location and direction is combined with the map data of Dimensional). Here, the route guidance may be interpreted as a concept including not only a case where a user rides on a vehicle and drives, but also a case where a user walks or runs and moves.

In addition, the lane departure guidance may be a guide to whether a vehicle being driven has deviated from a lane.

In addition, the front vehicle collision prevention guide may be a guide to prevent a collision with the vehicle in front when the distance between the vehicle stopped or the vehicle located in front of the vehicle being driven is within a predetermined distance. Here, the front vehicle collision prevention guide may be a concept including a front vehicle departure guide for guiding whether a vehicle located in front of the vehicle being stopped starts.

Further, the signal guidance may be guiding a signal state of a traffic light located in front of the vehicle being driven, for example, a stop signal, a straight signal, a left turn signal, a right turn signal, and the like. Here, the color and shape corresponding to each signal may be different for each country. In Korea, a stop signal may be a red circle, a straight signal may be a blue circle, a left turn signal may be a blue left arrow, and a right turn signal may be a blue right arrow.

In addition, the signal change guidance may inform that a signal state of a traffic light located in front of the driving vehicle has changed. For example, when a stop signal is changed to a straight signal, it may be guided.

In addition, the lane change guidance may guide a change from a lane in which a vehicle is located to another lane in order to guide a route to a destination.

Also, the lane guidance may be guiding a lane in which the vehicle is currently located.

Such a driving-related image enabling the provision of various guides may be captured in real time by a camera mounted toward the front of the vehicle. Here, the camera may be a camera formed integrally with the electronic device 100 mounted on the vehicle to photograph the front of the vehicle. In this case, the camera may be integrally formed with a smart phone, a navigation system, or a black box, and the electronic device 100 may receive an image captured by the integrally formed camera.

As another example, the camera may be a camera that is mounted on a vehicle separately from the electronic device 100 to photograph the front of the vehicle. In this case, the camera may be a separate black box mounted toward the front of the vehicle, and the electronic device 100 receives a photographed image through wired/wireless communication with a separately mounted black box, or captures a photographed image of the black box. When a storage medium to be stored is inserted into the electronic device 100, the electronic device 100 may receive a captured image.

Hereinafter, the electronic device 100 according to an embodiment of the present invention will be described in more detail based on the above description.

The storage unit 110 functions to store various data and applications required for the operation of the electronic device 100. In particular, the storage unit 110 may store data necessary for the operation of the electronic device 100, for example, an OS, a route search application, and map data. In addition, the storage unit 110 may store data generated by an operation of the electronic device 100, for example, searched route data, and a received image. In addition, the storage unit 110 may store location relationship information of a plurality of signals included in a traffic light, a lane determination table, and the like.

Here, the storage unit 110 includes a random access memory (RAM), a flash memory, a read only memory (ROM), an erasable programmable ROM (EPROM), an electronically erasable and programmable ROM (EEPROM), a register, a hard disk, a removable disk, and a memory. It can be implemented not only as a built-in storage such as a card or USIM (Universal Subscriber Identity Module), but also as a removable storage such as a USB memory.

The input unit 120 functions to convert a physical input from the outside of the electronic device 100 into a specific electrical signal. Here, the input unit 120 may include all or part of the user input unit 121 and the microphone unit 123.

The user input unit 121 may receive a user input such as a touch or a push operation. Here, the user input boop 120 may be implemented using at least one of various button types, a touch sensor receiving a touch input, and a proximity sensor receiving an approaching motion.

The microphone unit 123 may receive a user's voice and a sound generated inside and outside the vehicle.

The output unit 130 is a device that outputs data of the electronic device 100. Here, the output unit 130 may include all or part of the display unit 131 and the audio output unit 133.

The display unit 131 is a device that outputs data that can be visually recognized by the electronic device 100. The display unit 131 may be implemented as a display unit provided on the front of the housing of the electronic device 100. Here, the display unit 131 is integrally formed with the electronic device 100 to output visual recognition data, and is installed separately from the electronic device 100 such as a head up display (HUD) to output visual recognition data. May be.

The audio output unit 133 is a device that outputs data that can be audibly recognized by the electronic device 100. The audio output unit 133 may be implemented as a speaker that expresses sound data to be informed to the user of the electronic device 100.

The communication unit 180 may be provided to allow the electronic device 100 to communicate with other devices. The communication unit 180 includes all or part of a location data unit 181, a wireless internet unit 183, a broadcast transmission/reception unit 185, a mobile communication unit 186, a short range communication unit 187, and a wired communication unit 189 It may include.

The location data unit 181 is a device that acquires location data through a Global Navigation Satellite System (GNSS). GNSS refers to a navigation system capable of calculating the location of a receiving terminal using radio signals received from satellites. Specific examples of GNSS include GPS (Global Positioning System), Galileo, GLONASS (Global Orbiting Navigational Satellite System), COMPASS, IRNSS (Indian Regional Navigational Satellite System), QZSS (Quasi-Zenith Satellite System), etc. I can. The location data unit 181 of the electronic device 100 according to an embodiment of the present invention may obtain location data by receiving a GNSS signal serving in an area where the electronic device 100 is used.

The wireless internet unit 183 is a device that acquires or transmits data by accessing the wireless internet. The wireless Internet that can be accessed through the wireless Internet unit 183 may be a wireless LAN (WLAN), a wireless broadband (Wibro), a world interoperability for microwave access (Wimax), a high speed downlink packet access (HSDPA), or the like.

The broadcast transmission/reception unit 185 is a device that transmits and receives broadcast signals through various broadcast systems. Broadcast systems that can transmit and receive through the broadcast transmission/reception unit 185 include Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), Media Foeward Link Only (MediaFLO), Digital Video Broadcast Handheld (DVBH), and ISDBT ( Integrated Services Digital Broadcast Tereestrial). Broadcast signals transmitted and received through the broadcast transmission/reception unit 185 may include traffic data, life data, and the like.

The mobile communication unit 186 may access and communicate with a mobile communication network according to various mobile communication standards such as 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), and Long Term Evoloution (LTE).

The short-range communication unit 187 is a device for short-range communication. As described above, the short-range communication unit 187 includes Bluetooth, Radio Frequency Idntification (RFID), infrared communication (IrDA, Infraed Data Association), Ultra WidBand (UWB), ZigBee, Near Field Communication (NFC), Wi You can communicate through -Fi.

The wired communication unit 189 is an interface device capable of connecting the electronic device 100 to another device by wire. The wired communication unit 119 may be a USB module capable of communicating through a USB port.

The communication unit 180 includes at least one of a location data unit 181, a wireless Internet unit 183, a broadcast transmission/reception unit 185, a mobile communication unit 186, a short range communication unit 187, and a wired communication unit 189 You can communicate with other devices using.

For example, when the electronic device 100 does not include a camera function, an image captured by a vehicle camera such as a black box may be received using at least one of the short-range communication unit 187 and the wired communication unit 189.

As another example, in the case of communication with a plurality of devices, one may communicate with the short-range communication unit 187 and the other may communicate with the wired communication unit 119.

The sensing unit 190 is a device capable of detecting the current state of the electronic device 100. The sensing unit 190 may include a motion sensing unit 191 and all or part of the light sensing unit 193.

The motion sensing unit 191 may detect motion of the electronic device 100 in a 3D space. The motion sensing unit 191 may include a 3-axis geomagnetic sensor and a 3-axis acceleration sensor. By combining the motion data acquired through the motion sensing unit 191 with the position data acquired through the position data unit 191, a more accurate trajectory of the vehicle to which the electronic device 100 is attached may be calculated.

The light sensing unit 193 is a device that measures the ambient illumination of the electronic device 100. The brightness of the display unit 195 may be changed to correspond to the ambient brightness by using the illuminance data acquired through the light sensing unit 193.

The power supply unit 195 is a device that supplies power necessary for the operation of the electronic device 100 or the operation of another device connected to the electronic device 100. The power supply unit 195 may be a device that receives power from an external power source such as a battery or a vehicle built in the electronic device 100. In addition, the power supply unit 195 may be implemented as a wired communication module 119 or a device that is supplied wirelessly according to a type of receiving power.

Meanwhile, the controller 170 controls the overall operation of the electronic device 100. Specifically, the control unit 170 includes a storage unit 110, an input unit 120, an output unit 130, a guide-related information generation unit 140, an indicator generation unit 150, an augmented reality providing unit 160, and a communication unit ( 180), it is possible to control all or part of the sensing unit 190.

In particular, the control unit 170 generates information for the guidance-related information generation unit 140 to perform a driving-related guidance that assists the driving of the vehicle, and the indicator generation unit 150 provides driving-related guidance using the generated information. An indicator for is generated, and the display unit 131 may control the generated indicator to overlap and display the generated indicator on a predetermined area of the road guide screen.

Here, the guide-related information generation unit 140, referring to FIG. 2, a signal type information generation unit 141 that generates signal type information of a traffic light, and a lane information generation unit that generates lane information about a lane in which a vehicle is located. (143), a lane location information generation unit 145 that generates location information of a lane in which the vehicle is located, a route information generation unit 146 that generates path information for guiding a route to a set point, and another vehicle located in front of the vehicle It may include a distance information generating unit 147 that generates distance information of the distance. A method of generating information by the guide-related information generation unit 140 will be described later with reference to FIGS. 4 to 9.

On the other hand, the indicator generation unit 150 uses the information generated by the guidance-related information generation unit 140 to guide the lane departure to assist the driving of the vehicle, guide the front vehicle collision prevention, guide the signal change of the traffic light, guide the lane change, During lane departure guidance, you can decide which items need guidance.

Here, the indicator may have a shape designed to correspond to a road situation in the position of each of the plurality of guide areas within the indicator so that the driver can be guided in a more intuitive way.

As an example, the indicator is a first area for performing front vehicle collision prevention guidance, a second area located on the left and right sides of the first area and performing lane departure guidance, and is located at the top of the first and second areas and performing signal change guidance. A fourth area that penetrates the third area and the second area and performs lane change guidance may be included.

In addition, as another example, the indicator may include a first area for performing front vehicle collision prevention guidance, a third area positioned at the top of the first area and performing signal change guidance, and a fourth area for performing lane change guidance. have. In this case, the second area for lane departure guidance may be located in the lane portion of the augmented reality navigation screen.

In this case, the indicator generator 150 may generate an indicator in which an area corresponding to an item requiring guidance among a plurality of areas included in the indicator is activated.

In addition, the indicator generator 150 may determine a predetermined area in which the indicator is to be displayed on the augmented reality road guidance screen based on the vanishing point determined from the driving-related image data of the vehicle. Here, the indicator generator 150 may extract lanes from a captured image captured by a camera while driving the vehicle, and determine a point at which the extracted lanes cross each other as a vanishing point. In addition, the indicator generator 150 may determine an area located above a predetermined distance from the determined vanishing point as a predetermined area in which the indicator is to be displayed.

Meanwhile, the control unit 170 may control the output unit 130 to perform guidance related to driving of the vehicle. Specifically, the control unit 170 may control the display unit 131 to display the indicator generated by the indicator generating unit 150 overlaid on a predetermined area of the road guide screen. In addition, the control unit 170 may control the audio output unit 133 to output a driving-related guide of the vehicle as a voice.

In addition, the controller 170 may control the electronic device 100 to perform driving-related guidance based on the augmented reality in conjunction with the augmented reality providing unit 160. Here, augmented reality means additional information (e.g., a graphic element indicating a point of interest (POI), a graphic element indicating a path to a destination, etc.) on a screen that contains the real world that the user is actually viewing. It may be a method of visually overlapping and providing. As an example, the augmented reality may be provided through a HUD using a vehicle windshield or an image overlay using a separate image output device, and the augmented reality providing unit 160 may provide a reality image or an interface image overlaid on the glass. Can be created. Through this, an augmented reality navigation or vehicle infotainment system may be implemented.

3 is a diagram illustrating a system network connected to an electronic device according to an embodiment of the present invention. Referring to FIG. 3, an electronic device 100 according to an embodiment of the present invention may be implemented as various devices provided in a vehicle such as a navigation system, a black box, a smart phone, or other vehicle augmented reality interface providing device. It can be connected to a communication network and other electronic devices 61 to 64.

In addition, the electronic device 100 may calculate a current location and a current time zone by interworking with the GPS module according to the radio signal received from the satellite 20.

Each satellite 20 may transmit an L-band frequency having a different frequency band. The electronic device 100 may calculate a current location based on a time taken for the L-band frequency transmitted from each satellite 20 to reach the electronic device 100.

Meanwhile, the electronic device 100 may wirelessly access the network 30 through the control station 40 (ACR), the base station 50 (RAS) through the communication unit 180. When the electronic device 100 is connected to the network 30, data can be exchanged by indirectly connecting with other electronic devices 61 and 62 connected to the network 30.

Meanwhile, the electronic device 100 may indirectly access the network 30 through another device 63 having a communication function. For example, when a module capable of accessing the network 30 is not provided in the electronic device 100, communication with another device 63 having a communication function may be performed through a short-range communication module or the like.

4 is a flowchart illustrating an operation of generating signal type information according to an embodiment of the present invention. Referring to FIG. 4, first, the signal type information generation unit 141 may determine an ROI including a traffic light from driving-related image data (S101). In this case, since the signal type information generation unit 141 is located in an area above the vanishing point, the signal type information generation unit 141 may determine the area above the determined vanishing point as the region of interest. Meanwhile, as another example, the signal type information generation unit 141 may determine a predetermined region from the driving-related image data as the region of interest.

In addition, the signal type information generation unit 141 may generate the ROI image data by converting the determined image data of the ROI based on a preset pixel value (S102). In this case, the signal type information generator 141 may generate ROI image data by applying the determined ROI image data to Equation 1 below.

Here, R, G, B are R, G, and B values of the determined image data of the ROI, K is a preset pixel value that is a reference for transformation, and x, y, and z may be specific coefficients. For example, K may be an average pixel value of 128.

Accordingly, the signal type information generator 141 may generate ROI image data in which the signal region portions of the traffic light are clearly distinguished.

Meanwhile, a color and/or shape corresponding to each of a plurality of signals included in a traffic light may be different for each country. In Korea, a stop signal may be a red circle, a straight signal may be a blue circle, a left turn signal may be a blue left arrow, and a right turn signal may be a blue right arrow.

Accordingly, the above-described preset pixel values and x, y, and z may have different values according to an algorithm.

Meanwhile, the signal type information generator 141 may detect image data of a signal region portion of a traffic light by applying the first region and the second region to the region of interest image data (S103). Specifically, the signal type information generation unit 141 applies a first area having a first area and a second area including the first area and having a second area to the ROI image data, Image data can be detected. Here, the first and second regions may have a shape corresponding to a shape of a traffic light signal. For example, when a signal of a traffic light is a circle, an ellipse, or a square, the first and second regions may be a circle, an ellipse, or a square.

Meanwhile, the signal type information generator 141 may compare the difference between the area pixel value of the second area and the area pixel value of the first area with a preset area pixel value (S104). Here, the preset area pixel value may be changed by reflecting a color and/or shape corresponding to each of a plurality of signals included in the traffic light.

In addition, the signal type information generation unit 141 may generate signal type information of a traffic light such as a traffic light according to the comparison result (S105).

These steps S103, S104, and S105 will be described in detail with reference to FIG. 5.

5 is a diagram illustrating a method of determining ROI image data and a stop signal corresponding to a stop signal according to an embodiment of the present invention. When the traffic light displays a red stop signal, when the signal type information generation unit 141 converts the image data of the ROI including the traffic light based on a preset pixel value, the ROI image as shown in FIG. 5(a) Data 501 can be generated. Here, the red stop signal of the traffic light may be generated as 502 on the ROI image data 501.

In this case, the signal type information generator 141 applies the first region 504 and the second region 503 to the ROI image data 501 to detect the image data of the signal region portion 502 of the traffic light. I can.

In addition, the signal type information generation unit 140 compares the difference between the area pixel value of the second area 503 and the area pixel value of the first area 504 with a preset area pixel value, and provides signal type information of a traffic light such as a traffic light. Can be created. As an example, when the difference between the area pixel value of the second area 503 and the area pixel value of the first area 504 is smaller than a preset area pixel value (ie, FIG. 5(b) In the case of ), the area pixel value of the first region 504 is large) and signal type information may be determined as a stop signal.

Meanwhile, although omitted in FIG. 5, determination of a turn signal such as a straight signal, a left turn, and a right turn may be performed similarly to the above-described method. For example, the signal type information generator 140 may determine the rotation signal using the above-described method and/or the shape of image data (eg, arrow shape) of the signal area portion of the traffic light.

According to the method of generating signal type information as shown in FIG. 5, the signal type information is determined in a state in which the image data of the region of interest is leveled based on a preset pixel value, thereby improving the image processing speed. It is possible to quickly determine the signal type information in a driving state where the driving, driving, and stopping are changing each time.

Meanwhile, the method of generating signal type information as illustrated in FIG. 5 is only an example of the present invention, but is not limited thereto. Accordingly, various signal type information generation methods may be used, such as detecting a red signal corresponding to a stop signal from driving-related image data, which is a color image, or determining signal type information by detecting a blue signal corresponding to a straight signal. .

Meanwhile, according to an embodiment of the present invention, when it is determined that the signal type information is a stop signal, the signal type information generation unit 141 provides a start signal and a rotation signal such as a left turn/right turn in the region of interest image data. The first area and the second area can be positioned in advance. In this case, the signal type information generation unit 141 may pre-locate the first region and the second region to the position using positional relationship information of a plurality of signals stored in the storage unit 110. In this case, by increasing the image processing speed, it is possible to more quickly determine a change from a stop signal to a straight signal or a turn signal such as a left turn/right turn.

6 is a detailed flowchart illustrating a method of generating lane information according to an embodiment of the present invention. Referring to FIG. 6, first, the lane information generation unit 143 may convert color image data into a gray image (S201) and detect a lane area from the converted gray image (S202).

Specifically, the lane information generation unit 143 may extract a region for detecting a lane from the captured driving-related image. In addition, since the lane information generator 143 has difficulty in detecting a lane when a part of the road is affected by a shadow, in order to minimize the shadow influence, the light source correction for the original image may be performed in advance.

In addition, the lane information generation unit 143 may detect an area in which a lane can exist as a lane area according to a predetermined camera position or a camera installation angle. For example, the lane information generation unit 143 may determine a lane area based on a position where a lane can start. In addition, the lane information generator 143 may estimate the starting position of the lane area and the length of the lane area as the width of the lane in the driving-related image (the maximum width between the left and right lane areas) and the viewing angle of the camera.

In addition, the lane information generator 143 may convert a gray image corresponding to the lane detection region into an edge image, and detect the lane region based on a straight line position extracted from the converted edge image. More specifically, the driving-related image may be converted into an edge image through various known algorithms, and the edge image may include edges representing a plurality of straight lines. In this case, the lane information generation unit 143 may recognize the position of the detected straight line as a lane. In addition, the lane information generation unit 143 may determine a lane area based on a position of a straight line having a constant lane width with respect to a driving direction of a vehicle among a plurality of straight line candidates.

Thereafter, when the lane area is detected, the lane information generation unit 143 may set the lane type interest area based on the lane area (S203). Specifically, when a lane area is detected, the lane information generation unit 143 may set a lane type region of interest (ROI) based on the detected lane area. The lane type ROI may mean a portion of a driving-related image including a lane for determining the line type and color of the lane and a predetermined area around the lane.

After that, the lane information generation unit 143 performs binarization on the lane type region of interest (S204), maps the binarized partial gray image to a one-dimensional region (S205), and uses at least one of visual continuity and speed. Thus, the line type can be identified (S206).

The lane information generator 143 may extract a partial gray image of a lane type ROI from the converted gray image and perform binarization on the partial gray image. The reference value for binarization may be determined as an average luminance value of the partial gray image of the ROI. Accordingly, the lane information generation unit 143 can clearly distinguish only a portion determined as a lane from the partial gray image.

In addition, the lane information generation unit 143 may map each line (left and right) identified in the binary partial gray image to a one-dimensional region. And, by analyzing the pattern of each line mapped to the one-dimensional area, the line type can be identified.

More specifically, FIG. 7 shows the binarization and one-dimensional mapping of a lane type ROI in a gray image.

As shown in FIG. 7, when binarization is performed on a lane type ROI, a binarized image 700 may be obtained. In the binarized image 700, a portion indicated by white may be identified as a lane, and other portions may be identified as black.

In addition, each line identified in the binarized image 700 may be mapped to a one-dimensional region. The lane information generation unit 143 may easily determine a line type by using the image 710 mapped to the 1D area.

For example, the lane information generator 143 may determine whether a dotted line or a solid line is based on a starting point and a length characteristic of each line mapped in one dimension. In addition, the lane information generation unit 143 may determine whether a dotted line or a solid line is made using the continuity and speed of each line mapped in one dimension. In addition, the lane information generator 143 may first determine a dotted line or a solid line according to the starting point and length characteristics, and then secondly determine the dotted line or a solid line using time continuity and speed.

More specifically, the lane information generation unit 143 may first determine whether a dotted line or a solid line by comparing and determining the line length according to the starting point position of each line. In this case, the lane information generation unit 143 may determine whether there is a dotted line or a solid line with only one image frame.

In addition, the lane information generation unit 143 may more clearly determine whether each line is a dotted line or a solid line depending on whether each line is continuously formed over time. For example, the lane information generation unit 143 may preset a degree of continuity according to a line movement speed in an image, and determine a continuity degree as a dotted line when the continuity of each line is lower than a preset continuity degree value.

Accordingly, according to an exemplary embodiment of the present invention, it is possible to determine a final line type by pre-distinguishing whether a dotted line or a solid line is performed through one frame and verifying it through successive frames.

Thereafter, the lane information generation unit 143 may detect the color of the line portion whose type is identified from the original image data that is color (S207).

The lane information generator 143 may analyze a color image, detect a color of a portion corresponding to a line whose type has been previously identified, and classify it. For example, the lane information generation unit 140 may classify the detected color into white, yellow, or blue.

Thereafter, the lane information generation unit 143 may generate lane information corresponding to a lane in which the vehicle is located based on the identified line type and the classified color (S208).

8 is a detailed flowchart illustrating a method of generating location information on a lane according to an embodiment of the present invention. Referring to FIG. 8, the lane location information generation unit 145 may obtain lane information of a road on which a vehicle is located from map data (S301). Here, the lane information of the road may be information about the lane of the road where the vehicle being driven is currently located, and may include information about the number of lanes of the road where the vehicle is located. In addition, road lane information is obtained from map data stored in the storage unit 110 in the electronic device 100, or from an external map database (DB) separate from the electronic device 100, or other electronic devices 100 Can be obtained from

In addition, the lane location information generation unit 145 may determine whether the vehicle is located in the first lane (first lane) or the last lane (end lane) of the road by using the generated lane information (S302). Specifically, the lane location information generation unit 145 determines whether the vehicle is located in the first lane or the last lane of the road by applying the lane information corresponding to the lane where the vehicle is located to the lane determination table as shown in FIG. 9(a). can do.

That is, the lane determination table may include a primary lane and an end lane determined according to a country, a left line type and color, and a right line type and color. Here, the lane determination table as shown in FIG. 9A may be an example, and may be set to different values for each country or situation according to the setting.

Meanwhile, when the vehicle is located in the first or last lane, the lane location information generation unit 145 may reflect information on the lane of the road to generate the lane location information in which the vehicle is located (S303). For example, if it is determined that the vehicle is located in the last lane, the lane location information generation unit 145 may generate the lane location information as N lanes. In addition, when the number of lanes corresponding to road lane information is 5, N lanes may be generated as 5 lanes by reflecting this.

In addition, when the lane where the vehicle is located is changed to a lane located between the first or last lane according to the change of the lane of the vehicle, the lane location information generation unit 145 may update the generated lane location information with the changed lane location information. Yes (S304). In this case, the lane location information generation unit 145 may determine whether to deviate from the lane using the lane information, and determine whether to change the lane based on this. For example, when it is determined that the vehicle is located in the 5th lane and changed to the 1st lane to the left lane, for example, the lane location information generation unit 145 may reflect this and update the lane location information from the 5th lane to the 4th lane.

In addition, when the lane where the vehicle is located is changed to the first or last lane from the lane located between the lane according to the change of the lane of the vehicle, the lane location information generation unit 145 may re-acquire information about the lane of the road where the vehicle is located ( S305). In addition, the location information of the lane where the vehicle is located may be regenerated by reflecting the reacquired lane information (S306). For example, if it is determined that the vehicle is located in the 4th lane and changed to the 1st lane to the right lane, the vehicle has moved to the 5th lane, which is the last lane set before, and information about the lane of the road where the vehicle is currently located can be obtained. In addition, when the acquired lane information is the fourth lane, the location information of the lane where the vehicle is located may be recreated as the fourth lane.

Meanwhile, the method of generating lane location information according to an embodiment of the present invention is not limited to the above-described FIG. 8. Accordingly, the above-described order may be partially changed according to other embodiments. For example, the step of acquiring lane information of a road on which the vehicle is located may be performed in step S304. In this case, when the vehicle is located in the first or last lane, the lane location information generator 145 may generate location information of the lane where the vehicle is located (S303). For example, if it is determined that the vehicle is located in the last lane, the lane location information generation unit 145 may generate the lane location information as N lanes.

In addition, when the lane where the vehicle is located is changed to a lane located between the first or last lane according to the change of the lane of the vehicle, the lane location information generation unit 145 uses the generated lane location information and the obtained lane information of the road. Thus, it is possible to update the vehicle location information (S304). For example, when it is determined that the vehicle is located in the N lane corresponding to the last lane and has changed to the first lane to the left lane, the lane location information generation unit 145 reflects the number of lane information N = 5 in the N-1 lane. Lane location information can be updated to four lanes.

Further, according to another embodiment of the present invention, the lane location information generation unit 145 applies lane information corresponding to the lane where the vehicle is located to the lane determination table as shown in FIG. 9(b), so that the vehicle is the first lane of the road, It is possible to determine whether it is located in the central lane or the end lane. However, in this case, in a situation in which there are multiple central lanes of the road (for example, when the number of lanes is 4 or more), it is not possible to accurately determine the lane in which the vehicle is located among the plurality of central lanes. Preferably, the same method as in FIG. 8 may be used.

Meanwhile, the route information generator 146 may generate route information for guiding a route to a point set by a driver of the vehicle using map data, traffic flow-related information, and the like.

In addition, the distance information generation unit 147 may generate distance information from another vehicle located in front of the vehicle. For example, the distance information generation unit 147 determines a region of interest including the vehicle in front from the driving-related image data of the vehicle, and uses the image data of the determined region of interest to obtain distance information from another vehicle located in front of the vehicle. Can be generated. As another example, the distance information generation unit 147 receives sensing information generated from a distance detection sensor installed in front of the vehicle, for example, an infrared sensor, an ultrasonic sensor, etc., and generates distance information from another vehicle located in front of the vehicle. can do.

Accordingly, the guidance-related information generation unit 140 provides information on signal types of traffic lights, lane information on the lane where the vehicle is located, location information on the lane where the vehicle is located, distance information from other vehicles located in front of the vehicle, and directions. Information for performing driving-related guidance including at least one of route information for may be generated.

10 is a detailed flowchart illustrating a method of controlling an electronic device according to an embodiment of the present invention. Referring to FIG. 10, first, the guide-related information generation unit 140 may generate information for performing a driving-related guide that assists in driving the vehicle (S401).

In addition, the guide-related information generation unit 140 uses the generated information to guide the lane departure to assist the driving of the vehicle, the vehicle collision prevention guidance in front, the signal change guidance of the traffic light, the lane change guidance, the lane departure guidance. The item can be determined (S402). For convenience of explanation, a signal change guide will be described as an example.

When the signal type information is changed from a stop signal to a left turn signal in a state in which the route information of the vehicle stopped at the intersection is in a straight direction, the guidance-related information generator 140 may determine that the signal change guidance is unnecessary. Alternatively, when the generated signal type information is changed from a stop signal to a straight signal in a state in which the route information of the vehicle stopped at the intersection is in a straight direction, the guidance-related information generation unit 140 may determine that signal change guidance is necessary. have.

In addition, a lane change guidance will be described as another example.

In a state in which the route information of the vehicle being driven is a left turn after a certain distance, and the lane location information is not a left turn lane, the guidance-related information generation unit 140 may determine that lane change guidance is necessary. Alternatively, in a state in which the route information of the vehicle being driven is a left turn after a certain distance, and the lane location information is a left turn lane, the guidance-related information generation unit 140 may determine that lane change guidance is unnecessary.

In addition, the indicator generation unit 150 may generate an indicator in which an area corresponding to an item requiring guidance among a plurality of areas included in the indicator is activated (S403). In more detail, the indicator generator 150 may generate an indicator in which an area corresponding to an item requiring guidance is activated and an area corresponding to an item which does not require guidance is deactivated.

In addition, the display unit 131 may overlap and display the generated indicator on a predetermined area of the road guide screen (S404). In more detail, the display unit 131 may display an augmented reality road guide screen in which the generated indicator is superimposed on an image photographed in front of the driving vehicle in connection with the augmented reality providing unit 160. In addition, the display unit 131 displays a 2D (2-Dimensional) or 3D (3-Dimensional) route guidance screen that superimposes the generated indicator on the 2D (2-Dimensional) or 3D (3-Dimensional) map data. can do.

11 is a detailed flowchart illustrating a method of controlling an electronic device according to an embodiment of the present invention. Referring to FIG. 11, first, the guide-related information generation unit 140 may generate information for performing a driving-related guide that assists in driving the vehicle (S501).

In addition, the indicator generator 150 may generate an indicator for driving-related guidance by using the generated information (S502).

In addition, the indicator generator 150 may determine a vanishing point from image data related to driving of the vehicle (S503). In more detail, the indicator generator 150 may extract lanes from a captured image captured by a camera while driving a vehicle, and determine a point at which the extracted lanes cross each other as a vanishing point.

In addition, the indicator generation unit 150 may determine a predetermined area in which the indicator is to be displayed on the augmented reality road guidance screen based on the vanishing point determined from the driving-related image data of the vehicle (S504). Specifically, the indicator generator 150 may determine an area located above a predetermined distance from the determined vanishing point as a predetermined area in which the indicator is to be displayed. In other words, according to augmented reality, which displays additional information on a screen containing the real world that the user is actually seeing, an image corresponding to an actual road or a vehicle in front is displayed in the lower area of the vanishing point, but the upper area corresponds to an actual traffic light. Except for the video that is used, the video required while driving is not displayed. Accordingly, the indicator generator 150 may determine an area located above a predetermined distance from the determined vanishing point as a predetermined area in which the indicator is to be displayed.

In addition, the display unit 131 may superimpose and display the generated indicator on a predetermined area of the augmented reality navigation screen (S505).

12 is a diagram illustrating an indicator according to an embodiment of the present invention. 12 is a diagram showing an indicator in a state in which all of the plurality of guide areas are deactivated. The indicator 1200 is a first area 1230 that performs front vehicle collision prevention guidance, located on the left and right sides of the first area, and guides lane departure A second area (1220-1, 1220-2) performing a signal, a third area (1210-1, 1210-2, 1210-3) located at the top of the first and second areas and performing signal change guidance, It may include fourth regions 1241-1 and 1240-2 penetrating the second region and guiding the lane change.

That is, the positions of each of the plurality of guide areas within the indicator may be designed to correspond to actual road conditions.

Meanwhile, it is not limited to the shape of the indicator as shown in FIG. 12 described above, and according to another example, one or more of the first to fourth regions may be omitted, and according to another example, the first region In addition to the fourth area, a guide area such as a lane guide or the like for guiding the number of lanes in which the vehicle is currently located may be added.

Meanwhile, when an item requiring guidance is determined among the plurality of guides, the indicator generating unit 150 may generate an indicator in which an area corresponding to an item requiring guidance among the plurality of areas is activated. This will be described in detail with reference to FIGS. 13 to 16.

13 is a diagram illustrating an indicator for a signal change guide and a front vehicle collision prevention guide according to an embodiment of the present invention. Specifically, when the distance information from the vehicle located in front of the vehicle is greater than the preset distance and the signal type information is a stop signal, the indicator generation unit 150, as shown in FIG. 13 (a), An indicator in which the first area 1230 is activated with a color indicating sufficientness and the stop signal area 1210-1 in the third area for guiding the signal change is activated may be generated.

And, when the signal state of the traffic light is changed from the stop signal to the straight signal, the indicator generating unit 150, as shown in Fig. 13 (b), the straight signal area 1210-3 of the third area performing the signal change guidance You can create an indicator that activates.

14 is a diagram illustrating an indicator for guiding a front vehicle collision prevention according to an embodiment of the present invention. Specifically, when the distance information of the vehicle located in front of the vehicle is smaller than the preset distance, the indicator generating unit 150 is a first area with a color guiding the proximity of the vehicle ahead, as shown in FIG. 14(a). You can create an indicator that activates (1230).

And, when the distance information of the vehicle located in front of the vehicle is very close, the indicator generating unit 150, as shown in Fig. 14 (b), the first area 1230 with a color indicating that the distance is very close to the vehicle in front. You can create an indicator that activates ).

15 is a diagram illustrating an indicator for guiding a lane change according to an embodiment of the present invention. Specifically, when the vehicle needs to change the lane from the current lane to the left lane, the indicator generation unit 150, as shown in Fig. 15(a), the left lane change guidance area 1240-of the fourth area guiding the change to the left lane. You can create an indicator that activates 1).

And, when the vehicle needs to change the lane from the current lane to the right lane, the indicator generation unit 150, as shown in Fig. 15(b), the right lane change guidance area 1240-of the fourth area guiding the change to the right lane. You can create an indicator that activates 2).

16 is a diagram illustrating an indicator for a lane departure prevention guidance and a front vehicle collision prevention guidance according to an embodiment of the present invention. Specifically, when the distance information with the vehicle located in front of the vehicle is greater than the preset distance and the vehicle deviates from the left lane, the indicator generating unit 150, as shown in FIG. An indicator in which the first area 1230 is activated with a color indicating sufficientness, and the left lane departure guide area 1220-1 of the second area for performing the lane departure guide is activated may be generated.

And, when the distance information of the vehicle located in front of the vehicle is greater than the preset distance, and the vehicle deviates from the right lane, the indicator generating unit 150 has a sufficient distance to the vehicle in front, as shown in FIG. 16(b). An indicator of activating the first area 1230 with a color guiding the vehicle and activating the right lane departure guide area 1220-2 of the second area performing the lane departure guidance may be generated.

17 is a diagram illustrating a route guidance screen according to an embodiment of the present invention. Referring to FIG. 17, the road guide screen includes an augmented reality road guide screen 1710 and a 2-dimensional (2D) or 3D (3D) augmented reality road guide screen 1710 that overlaps and displays an indicator 1200 generated on an image photographed in front of a driving vehicle. A 2-Dimensional (2-Dimensional) or 3-Dimensional (3D) route guidance screen 1720 may be displayed on the map data of -Dimensional).

In addition, the indicator 1710 may be displayed in an area within the augmented reality road guide screen 1710 located above a predetermined distance from a vanishing point, which is a point at which the lanes extend and cross each other.

18 is a diagram illustrating a road guidance screen according to another embodiment of the present invention. Referring to FIG. 18, an indicator 1300 according to another embodiment of the present invention includes a first area for performing forward collision prevention guidance, a third area for guiding a signal change, and a lane change guidance located at the top of the first area. It may include a fourth area to perform.

In this case, the second area 1830 for lane departure guidance may be located in a lane portion of the augmented reality navigation screen 1810. That is, when the vehicle deviates from the left lane, the left lane portion of the lane where the vehicle is located may be activated on the augmented reality navigation screen 1810, and when the vehicle deviates from the right lane, the augmented reality road guidance screen 1810 ), the right lane portion of the lane where the vehicle is located may be activated.

19 is a diagram illustrating an implementation form when a camera and an electronic device are separated according to an embodiment of the present invention. Referring to FIG. 19, a vehicle black box 200 provided separately from the vehicle navigation 100 may constitute a system according to an embodiment of the present invention using a wired/wireless communication method.

The vehicle navigation 100 may include a display unit 145 provided on the front of the navigation housing 191, a navigation operation key 193, and a navigation microphone 195.

The vehicle black box 200 may acquire vehicle data while driving and while stopping. In other words, it is possible to capture an image while the vehicle is driving, as well as to capture an image even when the vehicle is stopped. The quality of the image acquired through the vehicle black box 200 may be constant or may be changed. For example, by increasing the image quality before and after the occurrence of an accident, and lowering the image quality in a normal case, it is possible to store essential images while minimizing required storage space.

The vehicle black box 200 may include a black box camera 222, a black box microphone 224, and an attachment part 281.

Meanwhile, in FIG. 19, it is shown that the vehicle black box 200 provided separately from the vehicle navigation 100 is connected through a wired/wireless communication method, but the vehicle navigation 100 and the vehicle black box 200 have a wired/wireless communication method. It may not be connected through. In this case, when a storage medium for storing the captured image of the black box 200 is inserted into the electronic device 100, the electronic device 100 may receive the captured image. Meanwhile, the vehicle black box 200 may have the function of the vehicle navigation 100, or the vehicle navigation 100 may be integrated with a camera. This will be described in detail with reference to FIG. 13.

20 is a diagram illustrating an implementation form when a camera and an electronic device are integrated according to an embodiment of the present invention. Referring to FIG. 20, when the electronic device includes a camera function, a user may mount the electronic device so that the camera part of the electronic device photographs the front of the vehicle and the display part of the electronic device can be recognized by the user. Accordingly, a system according to an embodiment of the present invention can be implemented.

21 is a diagram illustrating an implementation form using a head-up display (HUD) and an electronic device according to an embodiment of the present invention. Referring to FIG. 21, the electronic device may display an augmented reality guide screen on the head-up display through wired/wireless communication with the head-up display.

Meanwhile, the method for controlling an electronic device according to various embodiments of the present disclosure described above may be implemented as a program code and provided to each server or devices while being stored in various non-transitory computer readable media. have.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, and ROM.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Claims (27)

In a computer readable storage medium storing one or more programs,
The one or more programs,
When executed by at least one processor of an electronic device, contained within the vehicle being moved, with a display,
By superimposing on the screen of the environment in front of the vehicle, a visual object for indicating a path to which the vehicle is to be moved is displayed through the display, wherein the visual object includes the vehicle and the vehicle Including first data for indicating a distance between positions to perform a turn in order to move along a path and second data for indicating a type of the turn,
In response to identifying that the distance between the vehicle and another vehicle in front of the vehicle is less than a specified distance while displaying the visual object including the first data and the second data in an overlapping on the screen , The visual object further including third data for instructing a forward collision warning is displayed through the display in an overlapping manner on the screen,
In response to identifying that the vehicle has at least partially deviated from the lane corresponding to the display position of the visual object while displaying the visual object including the first data and the second data by being superimposed on the screen, the To display through the display by superimposing on the screen another visual object for indicating a lane departure warning next to the visual object,
A computer-readable storage medium containing instructions that cause the electronic device.
The method according to claim 1, wherein the screen,
It is displayed through the display together with another screen for an electronic map having a different view distinguished from the view provided through the screen,
When the one or more programs are executed by the at least one processor of the electronic device,
While displaying the visual object including the first data and the second data through the display by overlapping on the screen, a partial area of the other screen corresponding to the path is highlighted and displayed,
A computer-readable storage medium containing instructions that further cause the electronic device.
The method according to claim 2, wherein the other screen,
A computer-readable storage medium displayed next to the screen.
The method of claim 3, wherein the one or more programs are executed by the at least one processor of the electronic device,
To change the state of the first data in the visual object displayed through the display by overlapping on the screen, according to a change in the distance between the vehicle and the position at which the turn is to be performed,
A computer-readable storage medium containing instructions that further cause the electronic device.
The method according to claim 4, wherein the visual object displayed in an overlapping manner on the screen,
A computer-readable storage medium extending toward the vehicle from a vanishing point of lanes constituting the lane.
The method of claim 5, wherein the one or more programs are executed by the at least one processor of the electronic device,
Recognizing the lanes constituting the lane from an image acquired through a camera associated with the electronic device and used to display the screen,
Identifying the display position of the visual object based on the vanishing point of the lanes,
Superimposed on the screen to display the visual object through the display at the identified display position,
A computer-readable storage medium containing instructions that cause the electronic device.
The method of claim 4, wherein the visual object,
A computer-readable storage medium superimposed on the screen by completely covering the lane within the screen.
The method of claim 4, wherein the one or more programs are executed by the at least one processor of the electronic device,
While displaying the visual object including the first data and the second data in superimposition on the screen, identifying that the vehicle requires changing the lane to another lane to perform the turn In response, another visual object extending from the lane in the screen to the other lane in the screen is displayed through the display in an overlapping manner on the screen,
A computer-readable storage medium containing instructions that further cause the electronic device.
The method of claim 4, wherein the first data,
Consists of text,
The second data,
Computer-readable storage medium consisting of arrows.
In a method for operating an electronic device with a display, contained in a vehicle,
Displaying a visual object through the display to indicate a path to which the vehicle is to be moved by superimposing on the screen of the environment in front of the vehicle, wherein the visual object is the vehicle and the vehicle First data for indicating a distance between positions at which a turn should be performed to move along the path and second data for indicating the type of turn,
In response to identifying that the distance between the vehicle and another vehicle in front of the vehicle is less than a specified distance while displaying the visual object including the first data and the second data in an overlapping on the screen , Displaying the visual object further including third data for instructing a forward collision warning through the display in an overlapping manner on the screen,
In response to identifying that the vehicle has at least partially deviated from the lane corresponding to the display position of the visual object while displaying the visual object including the first data and the second data by being superimposed on the screen, the And displaying through the display another visual object for indicating a lane departure warning next to the visual object in an overlapping manner on the screen.
The method of claim 10, wherein the screen,
It is displayed through the display together with another screen for an electronic map having a different view distinguished from the view provided through the screen,
The above method,
An operation of highlighting and displaying a partial area of the other screen corresponding to the path while the visual object including the first data and the second data is displayed through the display by being superimposed on the screen How to further include.
The method of claim 11, wherein the other screen,
How to be displayed next to the screen.
The method of claim 12,
According to a change in the distance between the vehicle and the position at which the turn is to be performed (according to), changing the state of the first data in the visual object displayed through the display by superimposing on the screen How to.
The method of claim 13, wherein the visual object displayed in an overlapping manner on the screen,
And extending from a vanishing point of lanes constituting the lane toward the vehicle.
The method of claim 14, wherein the operation of displaying the visual object in an overlapping manner on the screen,
Recognizing the lanes constituting the lane from an image acquired through a camera related to the electronic device and used to display the screen,
An operation of identifying a display position of the visual object based on the vanishing point of the lanes,
And displaying the visual object through the display at the identified display position by superimposing on the screen.
The method of claim 13, wherein the visual object,
Superimposed on the screen by completely covering the lane within the screen.
The method of claim 13,
While displaying the visual object including the first data and the second data in superimposition on the screen, identifying that the vehicle requires changing the lane to another lane to perform the turn In response, the method further comprising displaying another visual object extending from the lane in the screen to the other lane in the screen in an overlapping manner on the screen through the display.
The method of claim 13, wherein the first data,
Consists of text,
The second data,
How it is made up of arrows.
In the electronic device included in the vehicle,
A memory configured to store instructions;
display; And
A processor, wherein the processor, when executing the instructions,
By superimposing on the screen of the environment in front of the vehicle, a visual object for indicating a path to which the vehicle is to be moved is displayed through the display, wherein the visual object includes the vehicle and the vehicle Including first data for indicating a distance between positions to perform a turn in order to move along a path and second data for indicating a type of the turn,
In response to identifying that the distance between the vehicle and another vehicle in front of the vehicle is less than a specified distance while displaying the visual object including the first data and the second data in an overlapping on the screen , Displaying the visual object further comprising third data for instructing a forward collision warning through the display in an overlapping manner on the screen,
In response to identifying that the vehicle has at least partially deviated from the lane corresponding to the display position of the visual object while displaying the visual object including the first data and the second data by being superimposed on the screen, the To display through the display another visual object for indicating a lane departure warning next to the visual object in superimposition on the screen,
Electronic device configured.
The method of claim 19, wherein the screen,
It is displayed through the display together with another screen for an electronic map having a different view distinguished from the view provided through the screen,
The processor, when executing the instructions,
While displaying the visual object including the first data and the second data through the display by overlapping on the screen, a partial area of the other screen corresponding to the path is highlighted and displayed,
Electronic device further configured.
The method of claim 20, wherein the other screen,
An electronic device displayed next to the screen.
The method of claim 21, wherein the processor, when executing the instructions,
To change the state of the first data in the visual object displayed through the display by overlapping on the screen, according to a change in the distance between the vehicle and the position at which the turn is to be performed,
Electronic device further configured.
The method of claim 22, wherein the visual object displayed in an overlapping manner on the screen,
An electronic device extending toward the vehicle from a vanishing point of lanes constituting the lane.
The method of claim 23, wherein the processor, when executing the instructions,
Recognizing the lanes constituting the lane from an image acquired through a camera associated with the electronic device and used to display the screen,
Identifying the display position of the visual object based on the vanishing point of the lanes,
Superimposed on the screen to display the visual object through the display at the identified display position,
Electronic device configured.
The method of claim 22, wherein the visual object,
The electronic device, which is superimposed on the screen by completely covering the lane within the screen.
The method of claim 22, wherein the processor, when executing the instructions,
While displaying the visual object including the first data and the second data in superimposition on the screen, identifying that the vehicle requires changing the lane to another lane to perform the turn In response, another visual object extending from the lane in the screen to the other lane in the screen is displayed through the display in an overlapping manner on the screen,
Electronic device further configured.
The method of claim 22, wherein the first data,
Consists of text,
The second data,
Electronic device consisting of arrows.

Images