KR102299081B1 - Electronic apparatus and control method thereof - Google Patents

Abstract

A method of controlling an electronic device is disclosed. The control method includes calculating an angle with a user's moving direction axis by comparing pixel values of a specific region included in each of a plurality of frames of a captured image captured by a camera, and photographing a route guide object reflecting the calculated angle and performing route guidance on augmented reality by combining it with an image.

Description

ELECTRONIC APPARATUS AND CONTROL METHOD THEREOF

The present invention relates to an electronic device and a control method thereof, and more particularly, to an electronic device for performing route guidance in augmented reality and a control method thereof.

With the development of electronic technology, various types of electronic devices are being developed and distributed. In particular, recently, a technology for providing geographic information in the form of Augmented Reality (AR) that synthesizes and displays additional information such as computer graphics (CG) and text with a captured image taken in real time has been introduced. there is a bar

According to this augmented reality technology, additional information (eg, a graphic element indicating a point of interest (POI), a graphic element indicating a route to a destination, etc.) It is possible to provide geographical information in a more intuitive way to users because it is possible to provide them visually.

Accordingly, the conventional navigation terminal was able to perform route guidance by synthesizing expression information related to map data with an image photographed in front of a driving vehicle by applying augmented reality technology.

However, when the vehicle speed is low, the conventional navigation terminal frequently loses the correct position and direction of the vehicle due to a GPS error. .

In particular, in this case, there is a problem in that the graphic element indicating the route to the destination on the augmented reality guides the vehicle in a direction completely different from the direction in which the vehicle should travel.

Accordingly, recently, research to reduce these problems has been actively conducted.

For example, research to reduce the above-described position and direction errors by attaching various sensors (accelerometers, gyroscopes, etc.) to a navigation terminal or using a hardware dead reckoning (DR) device is being actively conducted.

However, even if the additional configuration of the hardware is used, the above-described position and orientation errors still occur, and accordingly, cost increase and restrictions on installation are greatly affected.

As another example, there is a method of using DR information based on CAN communication supported by the vehicle, but this is supported only in high-end vehicles or since the protocols (communication protocols) are all different, there is a problem in that connection with the terminal is quite difficult.

The present invention has been devised in response to the above-mentioned necessity, and an object of the present invention is to reflect the rotation angle of the moving image of the user (or the vehicle in which the user is riding) calculated using the captured image in the route guide object, so that it is better in augmented reality. An object of the present invention is to provide an electronic device capable of accurately performing route guidance and a control method thereof.

According to an embodiment of the present invention for achieving the above object, there is provided a method for controlling an electronic device by comparing pixel values of a specific region included in each of a plurality of frames of a captured image captured by a camera to compare the user's moving direction axis and the calculating the angle of , and performing route guidance on augmented reality by combining a route guide object reflecting the calculated angle with the captured image.

Meanwhile, in an electronic device according to an embodiment of the present invention for achieving the above object, a storage unit for storing pixel values of a specific region included in each of a plurality of frames of the captured image in a camera, and storing the stored pixel values and an angle calculator for calculating an angle with the user's traveling direction axis by comparison, and a controller for performing route guidance on augmented reality by combining a route guide object reflecting the calculated angle with the captured image.

Meanwhile, in a computer-readable recording medium recording a program code for performing a control method of an electronic device according to an embodiment of the present invention for achieving the above object, the control method includes: Comparing pixel values of a specific region included in each of a plurality of frames of and performing guidance.

According to various embodiments of the present invention described above, it is possible to accurately perform route guidance in augmented reality requiring a high degree of accuracy.

In addition, according to the above-described various embodiments of the present invention, since a captured image captured by a camera is used, route guidance in augmented reality can be accurately performed without additional hardware configuration.

In addition, in the case of using a camera mounted on a black box, route guidance in augmented reality can be accurately performed even in a low-end terminal equipped with only a simple GPS.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention.
2 is a block diagram illustrating an augmented reality providing unit according to an embodiment of the present invention.
3 is a diagram illustrating a method of determining a planar area according to an embodiment of the present invention.
4 is a view showing a specific area according to an embodiment of the present invention.
5 is a diagram illustrating an effect of a control method according to an embodiment of the present invention.
6 is a flowchart illustrating a control method according to an embodiment of the present invention.
7 is a diagram illustrating an augmented reality providing system using a black box and navigation according to an embodiment of the present invention.
8 is a diagram illustrating an augmented reality providing system using an electronic device including a camera function according to an embodiment of the present invention.
9 is a diagram illustrating an augmented reality providing system using a head-up display (HUD) and an electronic device according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art will be able to devise various devices which, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. In addition, it should be understood that all conditional terms and examples listed herein are, in principle, expressly intended only for the purpose of understanding the inventive concept and are not limited to the specifically enumerated embodiments and states as such. do.

Further, it is to be understood that all detailed description reciting specific embodiments, as well as principles, aspects, and embodiments of the present invention, are intended to include structural and functional equivalents thereof. It is also to be understood that such equivalents include not only currently known equivalents, but also equivalents developed in the future, i.e., all devices invented to perform the same function, regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of illustrative circuitry embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc. may be tangibly embodied on computer-readable media and be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly shown. should be

The functions of the various elements shown in the figures including a processor or functional blocks represented by similar concepts may be provided by the use of dedicated hardware as well as hardware having the ability to execute software in association with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, clear use of terms presented as processor, control, or similar concepts should not be construed as exclusively referring to hardware having the ability to execute software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM (RAM) and non-volatile memory. Other common hardware may also be included.

In the claims of the present specification, a component expressed as a means for performing the function described in the detailed description includes, for example, a combination of circuit elements that perform the function or software in any form including firmware/microcode, etc. It is intended to include all methods of performing the functions of the device, coupled with suitable circuitry for executing the software to perform the functions. Since the present invention defined by these claims is combined with the functions provided by the various enumerated means and combined in a manner required by the claims, any means capable of providing the functions are equivalent to those contemplated from the present specification. should be understood as

The above-described objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in the description of the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an electronic device according to an embodiment of the present invention. Referring to FIG. 1 , an electronic device 100 includes a communication unit 110 , an input unit 120 , a sensing unit 130 , an output unit 140 , a storage unit 150 , a power supply unit 160 , and a control unit 170 . includes all or part of it.

Here, the electronic device 100 includes a smart phone, a tablet computer, a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), smart glasses, project glasses, navigation, etc. capable of guiding a route to a specific point. It may be implemented in various devices such as a black box.

As such, the electronic device 100 may include an augmented reality view mode and a normal mode as a route guidance method. Such route guidance may be interpreted as a concept including route guidance not only when the user travels in a vehicle, but also when the user walks or jumps.

Here, the augmented reality view mode may refer to a mode for performing route guidance by combining various types of information such as a user's location and direction with an image captured in front of a driving vehicle.

Such a front-captured image of the driving vehicle may be captured from the camera. Here, the camera may be a camera built into the electronic device 100 or a camera installed separately from the electronic device 100 to photograph the front of the vehicle. For example, it may be a camera built into a smart phone, a navigation system, or a black box, or a separate camera installed toward the front of the vehicle.

In addition, the normal mode may refer to a mode in which route guidance is performed by combining various types of information such as a user's location and direction with 2D (2-dimensional) or 3D (3-dimensional) map data.

The communication unit 110 may be provided for the electronic device 100 to communicate with other devices. The communication unit 110 may include all or part of the location data module 111 , the wireless Internet module 113 , the broadcast transmission/reception module 115 , the short-range communication module 117 , and the wired communication module 119 . have.

The location data module 111 is a device for obtaining location data through a Global Navigation Satellite System (GNSS). GNSS refers to a navigation system capable of calculating the position of a receiving terminal using a radio signal received from an artificial satellite. Specific examples of GNSS include GPS (Global Positioning System), Galileo, GLONASS (Global Orbiting Navigational Satellite System), COMPASS, IRNSS (Indian Regional Navigational Satellite System), QZSS (Quasi-Zenith Satellite System), etc. can The location data module 111 of the electronic device 100 according to an embodiment of the present invention may obtain location data by receiving a GNSS signal serviced in a region where the electronic device 100 is used.

The wireless Internet module 113 is a device for acquiring or transmitting data by accessing the wireless Internet. The wireless Internet accessible through the wireless Internet module 113 may be a wireless LAN (WLAN), a wireless broadband (Wibro), a world interoperability for microwave access (Wimax), a high speed downlink packet access (HSDPA), or the like.

The broadcast transmission/reception module 115 is a device for transmitting and receiving broadcast signals through various broadcast systems. Broadcast systems that can transmit and receive through the broadcast transmission/reception module 115 include Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), Media Foeward Link Only (MediaFLO), Digital Video Broadcast Handheld (DVBH), and ISDBT (Digital Multimedia Broadcasting Terrestrial). Integrated Services Digital Broadcast Tereestrial) and the like. The broadcast signal transmitted and received through the broadcast transmission/reception module 115 may include traffic data, life data, and the like.

The short-range communication module 117 is a device for short-range communication. As described above, the short-range communication module 117 may communicate through Bluetooth, Radio Frequency Idntification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like.

The wired communication module 119 is an interface device that can connect the electronic device 100 to another device by wire. The wired communication module 119 may be a USB module capable of communicating through a USB port.

The electronic device 100 according to an embodiment and another embodiment of the present invention may communicate with other devices through the short-range communication module 117 or the wired communication module 119 as necessary. Furthermore, when communicating with a plurality of devices, one of them communicates with the short-range communication module 117 and the other communicates with the wired communication module 119 . Accordingly, in the case of an electronic device that does not include a camera function according to an embodiment of the present invention, a photographed image photographed in the black box is received through the communication unit 110 or photographed from a separate camera installed toward the front of the vehicle. video can be received.

The input unit 120 is a device that converts a physical input from the outside of the electronic device 100 into a specific electrical signal. The input unit 120 may include a user input module 121 and all or a part of the microphone module 123 .

The user input module 121 is an input device through which a user can input through a touch, a push operation, or the like. Here, the user input module 120 may be implemented using at least one of various button shapes, a touch sensor for receiving a touch input, and a proximity sensor for receiving an approaching motion.

The microphone module 123 is a device for receiving a user's voice and a sound generated inside or outside the vehicle.

The sensing unit 130 is a device capable of detecting the current state of the electronic device 100 . The sensing unit 130 may include a motion sensing module 131 and all or a part of the light sensing module 133 .

The motion sensing module 131 may detect a motion of the electronic device 100 in a 3D space. The motion sensing module 131 may include a 3-axis geomagnetic sensor and a 3-axis acceleration sensor. A more accurate trajectory of the vehicle to which the electronic device 100 is attached may be calculated by combining the motion data acquired through the motion sensing module 131 with the position data acquired through the position data module 111 .

The light sensing module 133 is a device for measuring ambient illuminance of the electronic device 100 . By using the illuminance data acquired through the light sensing module 133 , the brightness of the display unit 145 may be changed to correspond to the ambient brightness.

The output unit 140 is a device that outputs data of the electronic device 100 . The output unit 140 may include all or part of the display module 141 and the audio output module 143 .

The display module 141 is a device that outputs data that can be visually recognized by the electronic device 100 . The display module 141 may be implemented as a display unit provided on the front surface of the housing of the electronic device 100 . Here, the display module 141 may be integrally formed with the electronic device 100 to output visual recognition data, and may be installed separately from the electronic device 100 such as a Head Up Display (HUD) to output visual recognition data. may be

The audio output module 143 is a device for outputting data that can be audibly recognized by the electronic device 100 . The audio output module 143 may be implemented as a speaker expressing sound of data to be notified to the user of the electronic device 100 .

The storage unit 150 is a device for storing data required for the operation of the electronic device 100 and data generated by the operation. The storage unit 150 may be a memory built into the electronic device 100 or a removable memory. Data necessary for the operation of the electronic device 100 may include an OS, a route search application, a map, and the like. In addition, the data generated by the operation of the electronic device 100 may be searched route data, a received image, or the like.

The power supply unit 160 is a device for supplying power necessary for the operation of the electronic apparatus 100 or the operation of another device connected to the electronic apparatus 100 . The power supply unit 160 may be a device that receives power from an external power source such as a battery built in the electronic device 100 or a vehicle. In addition, the power supply unit 160 may be implemented as a wired communication module 119 or a wirelessly supplied device depending on the form of receiving power.

The controller 170 is a device that outputs a control signal for controlling various operations of the electronic device 100 . Furthermore, the controller 170 may output a control signal for controlling another device connected to the electronic apparatus 100 .

Meanwhile, the electronic device 100 according to an embodiment of the present invention may further include an augmented reality providing unit 180 for providing an augmented reality view mode. This will be described in detail with reference to FIG. 2 .

2 is a block diagram illustrating an augmented reality providing unit according to an embodiment of the present invention. Referring to FIG. 2 , the augmented reality providing unit 180 includes a calibration unit 181 , a 3D space generation unit 182 , a plane determination unit 183 , a region setting unit 184 , an angle calculation unit 185 , and a mapping unit. It may include all or part of portion 186 .

The calibrator 181 may perform calibration for estimating a camera parameter corresponding to a camera from a captured image captured by the camera. Here, the camera parameter may be a parameter constituting a camera matrix, which is information indicating a relationship between an actual space and a photograph.

The 3D generator 182 may generate a virtual 3D space based on the captured image captured by the camera. Specifically, the 3D generating unit 182 obtains depth information from an image photographed by the camera based on the camera parameter estimated by the calibrator 181, and virtual based on the acquired depth information and the photographed image. You can create 3D space.

The plane determiner 183 may determine a 2D plane by using the captured image captured by the camera. Specifically, the plane determiner 183 may detect a vanishing point using a captured image captured by the camera, and determine a plane connecting the detected vanishing point, the upper left vertex of the captured image, and the upper right vertex of the captured image. Here, the detection of the vanishing point may be performed, for example, by extracting lane information included in the captured image.

Such a 2D plane determination method of the plane determining unit 183 will be described in detail with reference to FIG. 3 .

3 is a diagram illustrating a method of determining a planar area according to an embodiment of the present invention.

For example, the plane determining unit 183 extracts the lane information 310 and 320 from the captured image, and extends the extracted lane information 310 and 320 to intersect with each other, a vanishing point. (330) can be detected. In addition, the plane determiner 183 may detect the upper left vertex 340 of the captured image and the upper right vertex 350 of the captured image.

In addition, the plane determiner 183 may calculate a linear equation between the vanishing point 330 and the upper left vertex 340 of the captured image, and a linear equation between the vanishing point 330 and the upper right vertex 350 of the captured image.

And, the plane determining unit 183 is a linear equation between the vanishing point 330 and the upper left vertex 340 of the captured image, and the linear equation between the vanishing point 330 and the upper right vertex 350 of the captured image and the image 300 A plane 360 formed by the upper side of may be determined as a 2D plane.

On the other hand, the plane determined according to the operation of the above-described plane determining unit 183 is another vehicle, structure, and terrain that can be photographed while driving when the image of the determined plane is analyzed to calculate the rotation angle of the vehicle in motion. It may be an area in which the influence of features, buildings, etc. can be minimized.

The region setting unit 184 may set a specific region so that the specific region is located within the plane determined by the plane determining unit 183 .

Here, the specific region may be a region for calculating an angle with the traveling direction axis of the vehicle by comparing pixel values of the specific region included in each of a plurality of frames of the captured image.

Such a specific region will be described in detail with reference to FIG. 4 .

4 is a diagram illustrating a specific area according to an embodiment of the present invention. Referring to FIG. 4 , the specific region is 'a first region 410 located above a preset distance from the vanishing point', 'a second region 430 close to the top of the captured image and located above a preset distance from the first region' , 'a third region 420 close to the upper left vertex and located on the left side of the second region' and 'a fourth region 440 close to the upper right vertex and located on the right side of the second region' can

Here, the specific region may include at least two or more of the first to fourth regions. In particular, preferably, the specific region may be four or more. The number of such a specific region may be set automatically or manually according to a trade-off relationship between data processing speed and accuracy.

Meanwhile, in FIG. 4 , the shape of a specific region is a rectangle, but the present invention is not limited thereto and may be implemented in various shapes.

In addition, when the angle of view of the camera is wide, the area setting unit 184 sets the size of a certain area (410, 420, 430, 440) large because the movement value for an object located far away is relatively large, whereas when the angle of view of the camera is narrow, it is located far Since the movement value of the object is relatively small, the size of the predetermined regions 410 , 420 , 430 , and 440 may be set to be small.

Meanwhile, the angle calculator 185 may calculate an angle with the axis of the traveling direction of the vehicle by comparing pixel values of a specific region included in each of a plurality of frames of a captured image captured by the camera.

For example, the angle calculator 185 compares pixel values of a specific region of a first frame of a captured image captured by the camera with a pixel value of a specific region of a second frame that is a frame after the first frame in the driving direction of the vehicle. The angle with the axis can be calculated.

Here, the pixel value of the specific region of the first frame and the pixel value of the specific region of the second frame may be stored in the storage unit 150 .

In this case, the angle calculator 185 may determine the traveling direction of the vehicle by comparing the pixel value of the specific region of the first frame with the pixel value of the specific region of the second frame that is the frame after the first frame. Specifically, as a result of the comparison, when the third region 420 , the second region 430 , and the fourth region 440 move to the left, the angle calculator 185 may determine that the vehicle is rotating in the right direction. Also, as a result of the comparison, when the third region 420 , the second region 430 , and the fourth region 440 move to the right, the angle calculator 185 may determine that the vehicle is rotating in the left direction.

However, this determination of the driving direction is only an example, and is not limited thereto. That is, in an example such as when the vehicle passes under an overpass, the condition of the specific region may not be satisfied even if the vehicle rotates left/right. Accordingly, in this case, the angle calculator 185 may determine the driving direction of the vehicle according to the comparison result of the first region. That is, the angle calculator 185 may determine the driving direction of the vehicle by reading the driving direction determination results for various cases from the storage unit 150 .

In addition, the angle calculator 185 may calculate the number of moving pixels in a specific area by using the comparison result. In more detail, the angle calculator 185 may calculate the number of pixels moved in a specific area according to the progress from the first frame to the second frame through the template matching technique.

In addition, the angle calculator 185 may extract an angle of view from the camera parameter calculated by the calibrator 181 .

In addition, the angle calculator 185 may calculate an angle with the traveling direction axis of the vehicle using the traveling direction, the number of pixels, and the angle of view. For example, the angle calculating unit 185 may calculate the angle by applying the above-described traveling direction, the number of pixels, and the angle of view to the arctan value.

Meanwhile, the mapping unit 186 may combine the route guide object with the captured image captured by the camera. Specifically, the mapping unit 186 may combine the path guide object reflecting the angle calculated by the angle calculating unit 185 to the virtual 3D space generated by the 3D generating unit 181 .

Here, the information object may be an object that displays information related to a route on which the vehicle is traveling or a road on which the vehicle is traveling. As an example, the information object may include information indicating that 000M is left to a specific place, information indicating a place of interest such as a crosswalk, speed camera location information, and a path display object performing path display.

For example, when the vehicle rotates, the position of the speed camera changes in augmented reality, and the mapping unit 186 reflects the angle calculated by the angle calculation unit 185 to correct the position of the speed camera to capture the image. can be coupled to

As another example, the mapping unit 186 may combine the path display object reflecting the angle calculated by the angle calculating unit 185 to the captured image. This will be described in detail with reference to FIG. 5 .

Meanwhile, according to this operation, the controller 170 may control the augmented reality providing unit 180 to perform route guidance on the augmented reality by combining the information object reflecting the calculated angle with the captured image.

5 is a diagram illustrating an effect of a control method according to an embodiment of the present invention. Referring to FIG. 5 , screens 520 and 540 are cases to which the control method according to an embodiment of the present invention is not applied, and screens 510 and 530 are cases to which the control method according to an embodiment of the present invention is applied. In addition, the screens 510 and 520 are augmented reality route guidance screens when route guidance is performed from the same location to the same destination, and the screens 530 and 540 are the same location after the vehicle moves a little. It is an augmented reality route guidance screen when route guidance is performed to the same destination.

Referring to screens 510 , 520 , 530 , and 540 , when the vehicle rotates larger while rotating on a curved road, the route guidance direction of the route display objects 511 , 521 , 531 , and 541 to the destination in the augmented reality is changed.

In this case, referring to screens 520 and 540 , which are cases in which the control method according to an embodiment of the present invention is not applied, the route display object 541 incorrectly reflects the route guidance direction to the destination. That is, when the vehicle rotates at a second angle by increasing the rotation angle while driving 520 at a first angle on a curve road 540, when the control method according to an embodiment of the present invention is not applied, the vehicle rotates. The change in angle is not recognized, and it is determined that the vehicle is rotating at the first angle, and the route display object 541 incorrectly reflects the route guidance direction to the destination.

However, referring to the screens 510 and 530 that are cases to which the control method according to an embodiment of the present invention is applied, the route display objects 511 and 531 accurately reflect the route guidance direction to the destination. That is, when the vehicle rotates at a first angle (510) while the vehicle is rotating at a first angle on a curved road (510) and rotates at a second angle (530), the change of the rotation angle is calculated by calculating the rotation angle using the pixel value of a specific area. It is immediately recognized, and it is determined that it is rotating at a second angle (520), and the route display object 531 accurately reflects the route guidance direction to the destination.

Meanwhile, the route guidance screens 510 and 530 may display the route guidance screen on the augmented reality and the route guidance screens 550 and 560 on the map together.

Also, the route guidance screens 510 and 530 may further display identifiers 512 and 532 indicating the magnitude of the rotation angle relative to the traveling direction axis. Here, the identifiers 512 and 532 may have a shape corresponding to the magnitude of the rotation angle relative to the moving direction axis. For example, when the rotation angle relative to the traveling direction axis is small as in the identifier 512, it may be displayed in the form of a short line, and when the rotation angle relative to the traveling direction axis is large as in the identifier 532, it may be displayed in the form of a long line. can

Meanwhile, the process of calculating the angle with the traveling direction axis of the vehicle of the augmented reality providing unit 180 according to an embodiment of the present invention may be performed when the moving speed of the vehicle is less than or equal to a preset speed. Accordingly, the above-described calculation process may be performed only in a low-speed region in which the vehicle attitude control error is large.

6 is a flowchart illustrating a control method according to an embodiment of the present invention. Referring to FIG. 6 , camera parameters may be calculated by first performing calibration on the camera ( S601 ).

Then, an angle of view may be extracted from the calculated camera parameter ( S602 ).

Then, it may be determined whether the user's moving speed is less than or equal to a preset speed (S603).

If the user's moving speed exceeds the preset speed (S603:N), the steps to be described later may not proceed.

If the moving speed of the user is less than or equal to the preset speed (S603:Y), a step to be described later may be performed.

First, pixel values of a specific region included in each of a plurality of frames of a captured image captured by the camera may be compared ( S604 ).

Then, the user's moving direction may be determined using the comparison result ( S605 ).

Then, the number of moving pixels in a specific area may be calculated using the comparison result ( S606 ).

Then, an angle with the user's moving direction axis may be calculated using the moving direction, the number of pixels, and the angle of view ( S607 ).

Then, the route guidance object reflecting the calculated angle may be combined with the captured image to perform route guidance on the augmented reality (S608).

7 is a diagram illustrating an augmented reality providing system using a black box and navigation according to an embodiment of the present invention. Referring to FIG. 7 , a vehicle black box 200 provided separately from the vehicle navigation system 100 may configure an augmented reality providing system according to an embodiment of the present invention using a wired/wireless communication method.

The vehicle navigation device 100 may include a display unit 145 provided on the front surface of the navigation housing 191 , a navigation operation key 193 , and a navigation microphone 195 .

The vehicle black box 200 may acquire vehicle data while driving and stopping. That is, it is possible to capture an image while the vehicle is driving, as well as capture an image even when the vehicle is stopped. The quality of the image acquired through the vehicle black box 200 may be constant or may be changed. For example, by increasing the image quality before and after the occurrence of an accident, and lowering the image quality in a normal case, it is possible to store essential images while minimizing the required storage space.

The vehicle black box 200 may include a black box camera 222 , a black box microphone 224 , and an attachment unit 281 .

On the other hand, although FIG. 7 shows that the vehicle black box 200 provided separately from the vehicle navigation device 100 is connected through a wired/wireless communication method, the vehicle black box 200 has the function of the vehicle navigation system 100, or It is also possible that the vehicle navigation device 100 is integrated with a camera. This will be described in detail with reference to FIG. 8 .

8 is a diagram illustrating an augmented reality providing system using an electronic device including a camera function according to an embodiment of the present invention. Referring to FIG. 8 , when the electronic device includes a camera function, the user may mount the electronic device so that the camera part of the electronic device photographs the front of the vehicle and the display part of the electronic device can be recognized by the user. Accordingly, it is possible to implement the augmented reality providing system according to an embodiment of the present invention.

9 is a diagram illustrating an augmented reality providing system using a head-up display (HUD) and an electronic device according to an embodiment of the present invention. Referring to FIG. 9 , the electronic device may display an augmented reality route guidance screen on the heads-up display through wired/wireless communication with the heads-up display.

Meanwhile, the above-described method for controlling an electronic device according to various embodiments of the present invention may be implemented as a program code and provided to each server or device while stored in various non-transitory computer readable media. have.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims Various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Claims (18)

A method for controlling an electronic device, comprising:
comparing pixel values of a specific region included in each of a plurality of frames of a captured image captured by a camera;
calculating the number of moving pixels in the specific area according to the comparison;
determining a moving direction of the vehicle according to the comparison;
calculating an angle change between an axis of a previous traveling direction of the vehicle and an axis of a current traveling direction of the vehicle using the traveling direction, the number of moving pixels, and the angle of view of the camera; and
A control method comprising a; correcting the position of the guide object according to the calculated angle change, combining it with the captured image, and performing augmented reality guidance based on the combined image. According to claim 1,
detecting a vanishing point using the captured image taken by the camera; and
determining a plane connecting the detected vanishing point, the upper left vertex of the captured image, and the upper right vertex of the captured image. 3. The method of claim 2,
and setting the specific area so that the specific area is located within the determined plane. 4. The method of claim 3,
The specific area is
a first area located above a predetermined distance from the vanishing point;
a second region close to the upper end of the captured image and positioned above a preset distance from the first region;
a third region adjacent to the upper left vertex and located to the left of the second region;
and at least two or more of a fourth region adjacent to the upper right vertex and positioned to the right of the second region. According to claim 1,
Calculating the camera parameters by performing a calibration (Calibration) on the camera; further comprising,
Calculating the angle change comprises:
and extracting an angle of view from the calculated camera parameter. According to claim 1,
Calculating the angle change comprises:
A control method, characterized in that it is performed when the moving speed of the vehicle is less than or equal to a preset speed. 6. The method of claim 5,
Creating a virtual 3D (3-dimensional) space for the captured image based on the camera parameter; further comprising,
The step of performing the augmented reality guidance,
A control method characterized in that the augmented reality guidance is performed by combining a guidance object reflecting the calculated angle change in the virtual 3D space with the captured image. According to claim 1,
The step of performing the augmented reality guidance,
A control method comprising displaying a route guidance screen on the augmented reality and a route guidance screen on a map together. In an electronic device,
a storage unit for storing pixel values of a specific region included in each of a plurality of frames of a captured image captured by the camera;
Comparing the stored pixel values, calculating the number of moving pixels in the specific area according to the comparison, determining the traveling direction of the vehicle according to the comparison, and determining the traveling direction, the number of moving pixels, and the angle of view of the camera an angle calculating unit for calculating an angle change between an axis of a previous traveling direction of the vehicle and an axis of a current traveling direction of the vehicle using the present invention; and
The electronic device comprising a; a control unit that corrects the position of the guide object according to the calculated angle change, combines it with the captured image, and performs augmented reality guidance based on the combined image. 10. The method of claim 9,
A plane determining unit for detecting a vanishing point using the captured image captured by the camera, and determining a plane connecting the detected vanishing point, the upper left vertex of the captured image, and the upper right vertex of the captured image; further comprising Electronic device, characterized in that. 11. The method of claim 10,
and a region setting unit configured to set the specific region so that the specific region is located within the determined plane. 12. The method of claim 11,
The specific area is
a first area located above a predetermined distance from the vanishing point;
a second region close to the upper end of the captured image and positioned above a preset distance from the first region;
a third region adjacent to the upper left vertex and located to the left of the second region;
and at least two or more of a fourth area adjacent to the upper right vertex and positioned to the right of the second area. 10. The method of claim 9,
Further comprising; a calibration unit for calculating the camera parameters by performing a calibration (Calibration) for the camera,
The angle calculation unit,
and extracting an angle of view from the calculated camera parameter. 10. The method of claim 9,
The angle calculation unit,
The electronic device characterized in that the electronic device is performed when the moving speed of the vehicle is less than or equal to a preset speed. 14. The method of claim 13,
Further comprising; a 3D space generator for generating a virtual 3D (3-dimensional) space for the captured image based on the camera parameter;
The control unit is
The electronic device characterized in that the augmented reality guidance is performed by combining a guidance object reflecting the calculated angle change in the virtual 3D space with the captured image. 10. The method of claim 9,
The camera is
The electronic device according to claim 1, wherein the camera is provided integrally with the electronic device, or a black box provided separately from the electronic device to transmit a photographed image through communication with the electronic device. 10. The method of claim 9,
The control unit is
Performing route guidance in augmented reality through a display unit provided integrally with the electronic device, or performing path guidance in augmented reality through a Head Up Display (HUD) provided separately from the electronic device characterized by an electronic device. In the computer-readable recording medium recorded with a program code for performing a control method of an electronic device,
The control method is
comparing pixel values of a specific region included in each of a plurality of frames of a captured image captured by a camera;
calculating the number of moving pixels in the specific area according to the comparison;
determining a moving direction of the vehicle according to the comparison;
calculating an angle change between an axis of a previous traveling direction of the vehicle and an axis of a current traveling direction of the vehicle using the traveling direction, the number of moving pixels, and the angle of view of the camera; and
Comprising: Compensating the position of the guide object according to the calculated angle change, combining the captured image, and performing augmented reality guidance based on the combined image; Computer-readable recording medium comprising a.

Images