KR20150064767A - Electronic apparatus and control method thereof - Google Patents

Abstract

A control method of an electronic device is disclosed. The control method comprises steps of: comparing a pixel value of a certain area included in each of a plurality of frames of a captured image photographed in a camera, and calculating an angle with an axis in direction of progress of a user; and combining a path guiding object, reflecting the calculated angle, to the captured image and performing a path guide on augmented reality.

Description

ELECTRONIC APPARATUS AND CONTROL METHOD THEREOF FIELD OF THE INVENTION [0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic device and a control method thereof, and more particularly, to an electronic device for performing route guidance on an augmented reality and a control method thereof.

Various types of electronic devices are being developed and distributed due to the development of electronic technologies. In particular, recently, a technique of providing geographical information in the form of Augmented Reality (AR) in which additional information such as computer graphics (CG) and characters are displayed on a photographed image photographed in real time has been introduced There is a bar.

According to this augmented reality technique, additional information (for example, a graphical element indicating a point of interest (POI), a graphical element indicating a path to a destination, etc.) is displayed on a screen containing a real world that the user actually sees, It is possible to provide the geographical information in a more intuitive manner to the user.

Accordingly, the conventional navigation terminal can perform the route guidance by synthesizing the expression information related to the map data on the image of the front of the running vehicle by applying the augmented reality technology.

However, when the vehicle speed is low, the conventional navigation terminal frequently loses the accurate position and direction of the vehicle due to the GPS error, and as a result, the user has a problem of incorrectly guiding the route to the desired destination .

Particularly, in such a case, there is a problem that the graphical element indicating the route from the augmented reality to the destination guides in a direction completely different from the direction in which the vehicle should travel.

Recently, studies have been actively carried out to reduce these problems.

For example, studies have been actively conducted to reduce errors in position and orientation by attaching various sensors (acceleration sensors, gyroscopes, etc.) to a navigation terminal or using a hardware DR (dead reckoning) device.

However, even if the additional configuration of the hardware is used, the above-described errors in the position and the direction still occur, and the increase in costs and restrictions on the installation are largely affected.

Another example is to use the DR information based on the CAN communication supported by the vehicle, but it is difficult to connect to the terminal because it is supported only in the advanced vehicle or the protocol (communication protocol) is different.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned needs, and it is an object of the present invention to provide a route guiding object which is capable of reflecting on the route guiding object the rotation angle of the moving image of a user (or a vehicle aboard the user) And to provide an electronic device and a control method thereof capable of performing path guidance accurately.

According to another aspect of the present invention, there is provided a method of controlling an electronic device, comprising: comparing a pixel value of a specific region included in each of a plurality of frames of a photographed image taken by a camera, And a step of performing a route guidance on the augmented reality by combining the path guide object reflecting the calculated angle with the shot image.

According to another aspect of the present invention, there is provided an electronic device including a storage unit storing a pixel value of a specific area included in each of a plurality of frames of an image taken by a camera, And a control unit for performing a route guidance on the augmented reality by coupling the route guidance object reflecting the calculated angle to the photographed image.

According to another aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program code for performing an electronic device control method according to an embodiment of the present invention, Comparing the pixel values of the specific region included in each of the plurality of frames of the augmented reality image to calculate the angle with respect to the axis of the user's progress direction, and combining the path guide object reflecting the calculated angle with the shot image, And performing guidance.

According to various embodiments of the present invention described above, route guidance on an augmented reality requiring high accuracy can be accurately performed.

In addition, according to various embodiments of the present invention described above, since the photographed image photographed by the camera is used, path guidance on the augmented reality can be accurately performed without additional hardware configuration.

In addition, when the camera mounted on the black box is used, route guidance on the augmented reality can be accurately performed even in a low-end terminal equipped with only a simple GPS.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention.
2 is a block diagram illustrating an augmented reality providing unit according to an embodiment of the present invention.
3 is a diagram illustrating a method of determining a planar region according to an embodiment of the present invention.
4 is a diagram illustrating a specific area according to an embodiment of the present invention.
5 is a diagram showing the effect of a control method according to an embodiment of the present invention.
6 is a flowchart illustrating a control method according to an embodiment of the present invention.
7 is a diagram illustrating a system for providing an augmented reality using a black box and navigation according to an embodiment of the present invention.
8 is a diagram illustrating an augmented reality providing system using an electronic device including a camera function according to an embodiment of the present invention.
FIG. 9 is a diagram illustrating a head-up display (HUD) and an augmented reality providing system using an electronic device according to an embodiment of the present invention.

The following merely illustrates the principles of the invention. Thus, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or shown herein, embody the principles of the invention and are included in the concept and scope of the invention. Furthermore, all of the conditional terms and embodiments listed herein are, in principle, only intended for the purpose of enabling understanding of the concepts of the present invention, and are not to be construed as limited to such specifically recited embodiments and conditions do.

It is also to be understood that the detailed description, as well as the principles, aspects and embodiments of the invention, as well as specific embodiments thereof, are intended to cover structural and functional equivalents thereof. It is also to be understood that such equivalents include all elements contemplated to perform the same function irrespective of the currently known equivalents as well as the equivalents to be developed in the future, i.e., the structure.

Thus, for example, it should be understood that the block diagrams herein represent conceptual views of exemplary circuits embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudo code, and the like are representative of various processes that may be substantially represented on a computer-readable medium and executed by a computer or processor, whether or not the computer or processor is explicitly shown .

The functions of the various elements shown in the figures, including the functional blocks depicted in the processor or similar concept, may be provided by use of dedicated hardware as well as hardware capable of executing software in connection with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

Also, the explicit use of terms such as processor, control, or similar concepts should not be interpreted exclusively as hardware capable of running software, and may be used without limitation as a digital signal processor (DSP) (ROM), random access memory (RAM), and non-volatile memory. Other hardware may also be included.

In the claims hereof, the elements represented as means for performing the functions described in the detailed description include all types of software including, for example, a combination of circuit elements performing the function or firmware / microcode etc. , And is coupled with appropriate circuitry to execute the software to perform the function. It is to be understood that the invention defined by the appended claims is not to be construed as encompassing any means capable of providing such functionality, as the functions provided by the various listed means are combined and combined with the manner in which the claims require .

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, in which: There will be. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention. 1, an electronic device 100 includes a communication unit 110, an input unit 120, a sensing unit 130, an output unit 140, a storage unit 150, a power source unit 160, Including all or part of them.

The electronic device 100 may be a smart phone, a tablet computer, a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a smart glass, a project glass, A black box, and the like.

The electronic device 100 may include an augmented reality view mode and a general mode as a route guidance method. Such route guidance can be interpreted not only as a case where the user rides on the vehicle but also includes a route guidance when the user moves or jumps.

Here, the augmented reality view mode may mean a mode in which a route guidance is performed by combining various information such as a user's position, direction, and the like, on an image of a forward moving vehicle.

Such a front photographing image of the running vehicle can be photographed from the camera. Here, the camera may be a camera built in the electronic device 100, or may be a camera installed separately from the electronic device 100 and photographing the front of the vehicle. For example, it may be a smart phone, a navigation device, a camera built in a black box, or a separate camera installed toward the front of the vehicle.

In addition, the general mode may refer to a mode for performing route guidance by combining various information such as a user's location, direction, and the like to 2D (2-Dimensional) or 3D (3-Dimensional) map data.

The communication unit 110 may be provided for the electronic device 100 to communicate with other devices. The communication unit 110 may include all or a part of the location data module 111, the wireless Internet module 113, the broadcast transmission / reception module 115, the short distance communication module 117, and the wired communication module 119 have.

The position data module 111 is a device for acquiring position data through a Global Navigation Satellite System (GNSS). GNSS means a navigation system capable of calculating the position of a receiving terminal using a radio signal received from a satellite. A specific example of GNSS is a GPS (Global Positioning System), a Galileo, a GLONASS (Global Orbiting Navigational Satellite System), a COMPASS, an Indian Regional Navigational Satellite System (IRNSS), a Quasi-Zenith Satellite System . The location data module 111 of the electronic device 100 according to an embodiment of the present invention may receive the GNSS signal serviced in the area where the electronic device 100 is used to obtain location data.

The wireless Internet module 113 is a device that accesses the wireless Internet and acquires or transmits data. The wireless Internet that can be accessed through the wireless Internet module 113 may be a wireless LAN (WLAN), a wireless broadband (WIBRO), a world interoperability for microwave access (WIMAX), or a high speed downlink packet access (HSDPA).

The broadcast transmission / reception module 115 is a device for transmitting and receiving broadcast signals through various broadcasting systems. The broadcasting system that can transmit and receive through the broadcasting transmission and reception module 115 may be a digital multimedia broadcasting terrestrial (DMBT), a digital multimedia broadcasting satellite (DMBS), a media forwarding link (MediaFLO), a digital video broadcasting handheld (DVBH) Integrated Services Digital Broadcast Tereestrial). The broadcast signal transmitted / received through the broadcast transmission / reception module 115 may include traffic data, living data, and the like.

The short-range communication module 117 is a device for short-range communication. The short range communication module 117 can communicate via Bluetooth, Radio Frequency Identification (RFID), Infra Data Association (IrDA), Ultra WidBand (UWB), ZigBee, or the like as described above.

The wired communication module 119 is an interface device capable of connecting the electronic device 100 with other devices by wire. The wired communication module 119 may be a USB module that can communicate through a USB port.

The electronic device 100 according to one embodiment of the present invention and another embodiment can communicate with other devices via the short-range communication module 117 or the wired communication module 119 as needed. Further, when communicating with a plurality of devices, it is possible that one communicates with the short-range communication module 117 and the other communicates with the wired communication module 119. [ Accordingly, in the case of the electronic device not including the camera function according to the embodiment of the present invention, it is possible to receive the photographed image photographed in the black box through the communication unit 110, Image can be received.

The input unit 120 is a device that converts a physical input from the outside of the electronic device 100 into a specific electrical signal. The input unit 120 may include all or some of the user input module 121 and the microphone module 123.

The user input module 121 is an input device that a user can input through touch, push operation, or the like. Here, the user input module 120 may be implemented using at least one of various types of buttons, a touch sensor for receiving a touch input, and a proximity sensor for receiving a motion to be accessed.

The microphone module 123 is a device for receiving user's voice and sound generated from the inside and the outside of the vehicle.

The sensing unit 130 is a device that can sense the current state of the electronic device 100. The sensing unit 130 may include all or a part of the motion sensing module 131 and the optical sensing module 133.

The motion sensing module 131 may sense movement of the electronic device 100 in a three-dimensional space. The motion sensing module 131 may include a three-axis geomagnetic sensor and a three-axis acceleration sensor. The motion data obtained through the motion sensing module 131 can be combined with the position data acquired through the position data module 111 to calculate a more accurate trajectory of the vehicle to which the electronic device 100 is attached.

The optical sensing module 133 is a device for measuring the ambient illuminance of the electronic device 100. The brightness of the display unit 145 can be changed to correspond to the ambient brightness using the illumination data acquired through the optical sensing module 133. [

The output unit 140 is a device for outputting data of the electronic device 100. The output unit 140 may include all or a part of the display module 141 and the audio output module 143.

The display module 141 is a device that outputs data that the electronic device 100 can visually recognize. The display module 141 may be implemented as a display unit provided on the front surface of the housing of the electronic device 100. Here, the display module 141 may be integrally formed with the electronic device 100 to output visual recognition data, and may be installed separately from the electronic device 100, such as a head up display (HUD), to output visual recognition data It is possible.

The audio output module 143 is a device that the electronic device 100 outputs data that can be audibly recognized. The audio output module 143 may be implemented as a speaker for expressing sound data to be notified to the user of the electronic device 100.

The storage unit 150 stores data necessary for the operation of the electronic device 100 and data generated by the operation. The storage unit 150 may be a memory built into the electronic device 100 or a removable memory. The data required for the operation of the electronic device 100 may include an OS, a route search application, a map, and the like. Further, the data generated by the operation of the electronic device 100 may be the searched path data, the received video, and the like.

The power supply unit 160 is a device that supplies power necessary for the operation of the electronic device 100 or the operation of another device connected to the electronic device 100. [ The power source unit 160 may be a battery built in the electronic device 100, or a device that receives power from an external power source such as a vehicle. Also, the power supply unit 160 may be implemented as a wired communication module 119 according to the type of the power supply, or may be implemented as a wirelessly supplied device.

The control unit 170 is a device for outputting a control signal for controlling various operations of the electronic device 100. Further, the control unit 170 may output a control signal for controlling another device connected to the electronic device 100. [

Meanwhile, the electronic device 100 according to an embodiment of the present invention may further include an augmented reality providing unit 180 for providing an augmented reality view mode. This will be described in detail with reference to FIG.

2 is a block diagram illustrating an augmented reality providing unit according to an embodiment of the present invention. 2, the augmented reality providing unit 180 includes a calibration unit 181, a 3D space generating unit 182, a plane determining unit 183, an area setting unit 184, an angle calculating unit 185, And may include all or a portion of portion 186.

The calibration unit 181 may perform calibration for estimating a camera parameter corresponding to the camera from the photographed image photographed by the camera. Here, the camera parameter may be a parameter that constitutes a camera matrix, which is information indicating a relation in which a real space is formed on a photograph.

The 3D generating unit 182 can generate a virtual 3D space based on the photographed image photographed by the camera. Specifically, the 3D generation unit 182 acquires depth information from the image captured by the camera based on the camera parameters estimated by the calibration unit 181, and generates a virtual image based on the acquired depth information and the captured image 3D space can be created.

The plane determining unit 183 can determine the 2D plane using the photographed image photographed by the camera. Specifically, the plane determining unit 183 can detect the vanishing point using the photographed image taken by the camera, and determine the plane connecting the detected vanishing point, the upper left vertex of the photographed image, and the upper right vertex of the photographed image. Here, detection of the vanishing point can be performed by extracting lane information included in the photographed image as an example.

The 2D plane determination method of the plane determining unit 183 will be described in detail with reference to FIG.

3 is a diagram illustrating a method of determining a planar region according to an embodiment of the present invention.

The plane determining unit 183 extracts the lane information 310 and the lane information 320 from the photographed image and extends the extracted lane information 310 and 320 to form a vanishing point, (330) can be detected. The plane determining unit 183 can detect the upper left vertex 340 of the photographed image and the upper right vertex 350 of the photographed image.

The plane determining unit 183 can calculate a linear equation between the vanishing point 330 and the upper left vertex 340 of the photographed image and a linear equation between the vanishing point 330 and the upper right vertex 350 of the photographed image.

The plane determining unit 183 calculates the linear equations between the vanishing point 330 and the upper left vertex 340 of the photographed image and the linear equations between the vanishing point 330 and the upper right vertex 350 of the photographed image, It is possible to determine the plane 360 formed by the upper side of the second plane as a 2D plane.

On the other hand, in the case of analyzing the image of the determined plane to calculate the rotation angle of the vehicle during driving, the plane determined according to the operation of the above-mentioned planar determination section 183 is different from other vehicle, structure, It can be an area that can minimize the influence of an object, a building, and the like.

The area setting unit 184 can set a specific area so that a specific area is located within the plane determined by the plane determining unit 183. [

Here, the specific area may be an area for calculating the angle with respect to the traveling direction axis of the vehicle through comparison of the pixel values of the specific area included in each of the plurality of frames of the photographed image.

The specific area will be described in detail with reference to FIG.

4 is a diagram illustrating a specific area according to an embodiment of the present invention. Referring to FIG. 4, the specific region includes a first region 410 positioned above a predetermined distance from a vanishing point, a second region 430 located near a top of the photographed image and located on a predetermined distance from the first region, A third region 420 located on the left side of the second region and a fourth region 440 located on the right side of the second region that are close to the left upper end vertex, .

Here, the specific region may include at least two of the first to fourth regions. In particular, preferably, the specific region may be four or more. This number of specific areas can be set automatically or manually by a tradeoff relationship between data processing speed and accuracy.

In FIG. 4, the shape of the specific region is a rectangle, but the present invention is not limited thereto and can be implemented in various forms.

When the angle of view of the camera is large, the area setting unit 184 sets the size of the predetermined area 410, 420, 430 or 440 to a large value because the moving value of the camera is relatively large. On the other hand, Since the moving value with respect to an object is relatively small, the size of the predetermined areas 410, 420, 430, and 440 can be set small.

On the other hand, the angle calculating unit 185 can calculate the angle with respect to the traveling direction axis of the vehicle by comparing the pixel values of the specific area included in each of the plurality of frames of the photographed image photographed by the camera.

For example, the angle calculating unit 185 compares a pixel value of a specific region of a first frame of a photographed image photographed by a camera with a pixel value of a specific region of a second frame, which is a frame after the first frame, The angle with the axis can be calculated.

Here, the pixel value of the specific area of the first frame and the pixel value of the specific area of the second frame may be stored in the storage unit 150.

In this case, the angle calculating unit 185 can determine the traveling direction of the vehicle by comparing the pixel value of the specific area of the first frame with the pixel value of the specific area of the second frame, which is the frame after the first frame. Specifically, when the third region 420, the second region 430, and the fourth region 440 move to the left as a result of the comparison, the angle calculating unit 185 can determine that the vehicle is rotating in the rightward direction. When the third region 420, the second region 430, and the fourth region 440 move to the right as a result of the comparison, the angle calculating unit 185 can determine that the vehicle is rotating in the leftward direction.

However, the determination of the traveling direction is merely an example, and the present invention is not limited thereto. That is, in the case of the case where the vehicle runs under the high-priced road, even if the vehicle rotates left / right, the condition of the specific region described above may not be established. Therefore, in this case, the angle calculating section 185 can determine the running direction of the vehicle in accordance with the comparison result of the first area. That is, the angle calculating unit 185 can determine the traveling direction of the vehicle by reading the traveling direction determination results for various cases from the storage unit 150. [

Then, the angle calculating unit 185 can calculate the number of moving pixels in the specific area using the comparison result. Specifically, the angle calculating unit 185 may calculate the number of pixels moved in a specific region according to the progression from the first frame to the second frame through the template matching technique.

The angle calculating unit 185 can extract an angle of view from the camera parameters calculated by the calibration unit 181. [

The angle calculating unit 185 can calculate the angle with respect to the traveling direction axis of the vehicle using the traveling direction, the number of pixels, and the angle of view. For example, the angle calculating unit 185 may calculate the angle by applying the above-described traveling direction, the number of pixels, and the angle of view to the arctan value.

On the other hand, the mapping unit 186 can combine the route guidance object with the taken image photographed by the camera. Specifically, the mapping unit 186 may combine the path guide object reflecting the angle calculated by the angle calculating unit 185 into the virtual 3D space generated by the 3D generator 181.

Here, the information object may be an object that displays information related to a road on which the vehicle is traveling or a road on which the vehicle is traveling. As an example, the information object may include information indicating that a specific place has been left by 000M, information indicating a place of caution such as a crosswalk, speeding camera position information, and a path display object performing path display.

For example, when the vehicle rotates, the position of the overspeed camera on the augmented reality is changed. The mapping unit 186 corrects the position of the overspeed camera by reflecting the angle calculated by the angle calculation unit 185, Lt; / RTI >

As another example, the mapping unit 186 may combine the path display object reflecting the angle calculated by the angle calculation unit 185 into the photographed image. This will be described in detail with reference to FIG.

According to this operation, the controller 170 controls the augmented reality providing unit 180 to combine the information object reflecting the calculated angle with the photographed image to perform route guidance on the augmented reality.

5 is a diagram showing the effect of a control method according to an embodiment of the present invention. Referring to FIG. 5, the screens 520 and 540 are cases in which the control method according to an embodiment of the present invention is not applied, and the screens 510 and 530 are cases in which the control method according to an embodiment of the present invention is applied. The screen 510 and the screen 520 are the augmented reality route guidance screen when route guidance is performed from the same position to the same destination and the screen 530 and the screen 540 are displayed on the same position In which the route guide is performed to the same destination.

Referring to the screens 510, 520, 530, and 540, when the vehicle rotates more in the course of turning on the curve road, the route guidance direction to the destination of the path display objects 511, 521, 531, and 541 on the augmented reality changes.

In this case, referring to screens 520 and 540, which are cases in which the control method according to an embodiment of the present invention is not applied, the path display object 541 incorrectly reflects the route guidance direction to the destination. That is, when the vehicle is rotated 540 at a second angle by increasing the rotation angle during the course of running 520 at the first angle on the curve road, if the control method according to the embodiment of the present invention is not applied, The path display object 541 incorrectly reflects the direction of the route guidance to the destination by judging that the change of the angle is not recognized and that the route is running at the first angle.

However, referring to screens 510 and 530 in which a control method according to an embodiment of the present invention is applied, the path display objects 511 and 531 accurately reflect a route guidance direction to a destination. That is, when the vehicle turns on the curve road at a first angle (510) during the running and rotates at a second angle (530), the rotation angle is calculated through the rotation angle calculation using the pixel value of the specific area The route display object 531 correctly determines the direction of the route guidance to the destination and determines that the route is being traveled 520 at the second angle.

On the other hand, the route guidance screens 510 and 530 can display the route guidance screen on the augmented reality and the route guidance screens 550 and 560 on the map together.

Further, the route guidance screens 510 and 530 may further display identifiers 512 and 532 indicating the magnitude of the rotation angle with respect to the traveling direction axis. Here, the identifiers 512 and 532 may have a shape corresponding to the magnitude of the rotation angle with respect to the moving direction axis. For example, in the case where the rotation angle with respect to the moving direction axis is small, such as the identifier 512, it may be displayed in the form of a short line, and when the rotation angle with respect to the moving direction axis is large like the identifier 532, .

Meanwhile, the process of calculating the angle of the augmented reality providing unit 180 with respect to the axis of the forward direction of the vehicle according to an embodiment of the present invention may be performed when the moving speed of the vehicle is equal to or less than a predetermined speed. Accordingly, the above-described calculation process can be performed only in the low-speed region where the attitude control error of the vehicle is large.

6 is a flowchart illustrating a control method according to an embodiment of the present invention. Referring to FIG. 6, a camera parameter may be calculated by performing a calibration on a camera (S601).

Then, an angle of view can be extracted from the calculated camera parameters (S602).

Then, it can be determined whether the moving speed of the user is less than a predetermined speed (S603).

If the moving speed of the user exceeds the predetermined speed (S603: N), the step to be described later may not proceed.

If the moving speed of the user is equal to or less than the preset speed (Y in S603), the process to be described later can be performed.

First, the pixel values of the specific region included in each of the plurality of frames of the photographed image photographed by the camera can be compared (S604).

Then, the direction of the user's progress can be determined using the comparison result (S605).

Then, the number of moving pixels in a specific area can be calculated using the comparison result (S606).

Then, the angle between the user and the axis of the progress direction can be calculated using the progress direction, the number of pixels, and the angle of view (S607).

The path guide object reflecting the calculated angle may be combined with the photographed image to perform route guidance on the augmented reality (S608).

7 is a diagram illustrating a system for providing an augmented reality using a black box and navigation according to an embodiment of the present invention. Referring to FIG. 7, a vehicular black box 200 provided separately from the navigation system 100 for a vehicle may configure an augmented reality providing system according to an embodiment of the present invention using a wired / wireless communication system.

The vehicle navigation system 100 may include a display unit 145 provided on the front surface of the navigation housing 191, a navigation operation key 193, and a navigation microphone 195.

The vehicle black box 200 can acquire the data of the vehicle during driving and stopping. That is, it is possible not only to shoot a moving image of a vehicle, but also to shoot an image even when the vehicle is stationary. The image quality of the image obtained through the vehicle black box 200 may be constant or changed. For example, it is possible to store a core image while minimizing a required storage space by increasing the image quality before and after occurrence of an accident, and lowering the image quality in a normal case.

The vehicle black box 200 may include a black box camera 222, a black box microphone 224, and an attaching portion 281. [

7, the black box 200 for a vehicle provided separately from the navigation device 100 for a vehicle is connected via a wire / wireless communication method. However, when the black box 200 for a vehicle has the function of the navigation device 100 for a vehicle, It is also possible that the vehicle navigation system 100 is integrated with a camera. This will be described in detail with reference to FIG.

8 is a diagram illustrating an augmented reality providing system using an electronic device including a camera function according to an embodiment of the present invention. Referring to Fig. 8, when an electronic device includes a camera function, a user can photograph an electronic device such that a camera portion of the electronic device photographs the front of the vehicle and a display portion of the electronic device is user-recognizable. Accordingly, the augmented reality providing system according to the embodiment of the present invention can be implemented.

FIG. 9 is a diagram illustrating a head-up display (HUD) and an augmented reality providing system using an electronic device according to an embodiment of the present invention. Referring to FIG. 9, the electronic device can display the augmented reality route guidance screen on the head-up display through the head-up display and wired / wireless communication.

Meanwhile, the method of controlling an electronic device according to various embodiments of the present invention described above may be implemented in program code and provided to each server or devices in a state stored in various non-transitory computer readable media have.

A non-transitory readable medium is a medium that stores data for a short period of time, such as a register, cache, memory, etc., but semi-permanently stores data and is readable by the apparatus. In particular, the various applications or programs described above may be stored on non-volatile readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM,

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

Claims (18)

A method of controlling an electronic device,
Comparing the pixel values of the specific region included in each of the plurality of frames of the photographed image photographed by the camera and calculating an angle with respect to the axis of the user's progress direction; And
And performing path guidance on the augmented reality by combining the path guide object reflecting the calculated angle with the photographed image. The method according to claim 1,
Detecting a vanishing point using the photographed image photographed by the camera; And
And determining a plane connecting the detected vanishing point, the upper left vertex of the captured image, and the upper right vertex of the captured image. 3. The method of claim 2,
And setting the specific region so that the specific region is located within the determined plane. The method of claim 3,
The specific region may be,
A first region positioned above a predetermined distance from the vanishing point,
A second region located near the top of the photographed image and positioned above a predetermined distance from the first region,
A third region located near the left upper end vertex and located on the left side of the second region,
And a fourth region located closer to the right upper end vertex and located to the right of the second region. The method according to claim 1,
Further comprising: calibrating the camera to calculate a camera parameter,
Wherein the calculating the angle comprises:
Determining a traveling direction of the user using the comparison result;
Calculating the number of moving pixels in the specific area using the comparison result;
Extracting an angle of view from the calculated camera parameters; And
And calculating an angle of the user with respect to a moving direction axis using the moving direction, the number of pixels, and the angle of view. The method according to claim 1,
Wherein the calculating the angle comprises:
And when the moving speed of the user is less than a predetermined speed. 6. The method of claim 5,
And generating a virtual 3-dimensional (3D) space for the photographed image based on the camera parameters,
The step of performing the route guidance may include:
Wherein the path guidance is performed on the augmented reality by combining the path guide object reflecting the calculated angle in the virtual 3D space with the captured image. The method according to claim 1,
The step of performing the route guidance may include:
A route guidance screen on the augmented reality and a route guidance screen on the map are displayed together. In an electronic device,
A storage unit for storing pixel values of a specific area included in each of a plurality of frames of the photographed image photographed by the camera;
An angle calculating unit for comparing the stored pixel values and calculating an angle with respect to a moving direction axis of the user;
And a controller for performing a route guidance on the augmented reality by combining the route guidance object reflecting the calculated angles with the photographed image. 10. The method of claim 9,
And a plane determination unit for detecting a vanishing point using the photographed image taken by the camera and determining a plane connecting the detected vanishing point, the upper left vertex of the photographed image, and the upper right vertex of the photographed image, . 11. The method of claim 10,
And a region setting unit configured to set the specific region so that the specific region is located within the determined plane. 12. The method of claim 11,
The specific region may be,
A first region positioned above a predetermined distance from the vanishing point,
A second region located near the top of the photographed image and positioned above a predetermined distance from the first region,
A third region located near the left upper end vertex and located on the left side of the second region,
And a fourth region located closer to the upper right vertex and located to the right of the second region. 10. The method of claim 9,
Further comprising: a calibration unit for calibrating the camera to calculate camera parameters,
The angle calculating unit may calculate,
Determining a traveling direction of the user using the comparison result,
Calculating a number of moving pixels in the specific area using the comparison result,
Extracting an angle of view from the calculated camera parameters,
And calculates the angle of the user with respect to the traveling direction axis using the traveling direction, the number of pixels, and the angle of view. 10. The method of claim 9,
The angle calculating unit may calculate,
And when the moving speed of the user is equal to or less than a predetermined speed. 14. The method of claim 13,
And a 3D space generating unit for generating a virtual 3-dimensional space for the photographed image based on the camera parameters,
Wherein,
Wherein the path guidance is performed on the augmented reality by combining the path guide object reflecting the calculated angle in the virtual 3D space with the captured image. 10. The method of claim 9,
The camera comprises:
A camera provided integrally with the electronic device, or a black box provided separately from the electronic device and transmitting the captured image through communication with the electronic device. 10. The method of claim 9,
Wherein,
A path guide on the augmented reality through a display unit integrated with the electronic device or a route guidance on an augmented reality through a head up display (HUD) provided separately from the electronic device . A computer-readable recording medium having recorded thereon a program code for performing a control method of an electronic device,
In the control method,
Comparing the pixel values of the specific region included in each of the plurality of frames of the photographed image photographed by the camera and calculating an angle with respect to the axis of the user's progress direction; And
And performing path guidance on the augmented reality by combining the path guide object reflecting the calculated angle with the photographed image.

Images