KR20190120122A - Method and system for navigation using video call - Google Patents

Abstract

According to one embodiment, a computer-implemented road guide method can comprise: a step of transmitting the video data based on a current location, which is inputted in a video call screen of a terminal, to a counterpart terminal during a video call; a step of receiving instruction information for road guidance on the basis of the video data from the counterpart terminal on which the video data is displayed on the video call screen; a step of generating road guidance information by mapping the instruction information to video data of the terminal; and a step of guiding the road guidance information according to the location information of the terminal. Therefore, the present invention is capable of providing a movement route which can conveniently be grasped.

Description

Direction method and system using video call {METHOD AND SYSTEM FOR NAVIGATION USING VIDEO CALL}

The description below relates to a method and system for guiding directions information during a video call.

As communication technology develops, video communication services are provided not only for portable communication devices such as mobile phones but also for large electronic devices such as TVs. When the video call service is attempted by the calling terminal, the call attempt message is transmitted to the exchange through the base station / base station controller, and the exchange acquires the routing information to the destination terminal through the home location register and receives the incoming call through the destination exchange. Perform session setup with the terminal. According to this process, the calling terminal and the called terminal can perform a video call while looking at each other's faces through transmission and reception of audio and video.

However, the video call cannot see the video being viewed by the other party, nor can it check the video transmitted by the other party. Accordingly, in the case of querying or guiding a route during a video call, it is inconvenient to explain the route only by voice.

Republic of Korea Patent Publication 10-2016-0086560, Republic of Korea Patent Publication 10-2012-0008400,

A method and system for providing augmented reality based road guidance information during a video call between terminals may be provided.

A method and system for guiding road guidance information generated based on location information of a terminal during a video call between terminals may be provided.

The computer-based road guidance method may include transmitting current location-based video data input to a video call screen of a terminal to a counterpart terminal during a video call; Receiving indication information for road guidance based on the video data from a counterpart terminal on which the video data is displayed on a video call screen; Generating road guide information by mapping the indication information to image data of the terminal; And guiding the road guide information according to the location information of the terminal.

The step of guiding the road guide information according to the location information of the terminal comprises: guiding augmented reality-based road guide information generated by mapping the indication information to image data of the terminal according to the location information of the terminal. It may include.

The generating of the road guidance information by mapping the indication information to the image data of the terminal may include generating road guidance information by mapping a layer including the indication information to the image data corresponding to a position where the indication information is input. It may include the step.

Receiving the indication information for the guidance based on the video data from the counterpart terminal in which the video data is displayed on the video call screen, the user terminal from the counterpart terminal as the user interface for inputting the indication information is provided to the counterpart terminal; And receiving indication information input through an interface.

Receiving direction information based on the video data from the counterpart terminal on which the video data is displayed on the video call screen, receiving the guide information related to a specific event input to the video data from the counterpart terminal. It may include.

The step of guiding the road guide information according to the location information of the terminal may include providing guide information related to the road guide information according to the location information of the terminal.

The step of guiding the road guide information according to the location information of the terminal may include continuously updating the road guide information based on the changed location information of the terminal as the location information of the terminal is changed.

The step of guiding the road guide information according to the location information of the terminal may include the content as the road guide information according to the location information of the terminal as the content including the destination route of the terminal is uploaded by the counterpart terminal. Reproducing.

The step of guiding the road guide information according to the location information of the terminal may include generating an alarm related to the departure when detecting that the departure of the terminal occurs based on the road guidance information. .

The step of guiding the road guide information according to the location information of the terminal may include transmitting and receiving a message between the terminal and the counterpart terminal.

The step of guiding the road guide information according to the location information of the terminal may include displaying a moving route moving according to the location information of the terminal.

Transmitting the current location-based video data input to the video call screen of the terminal to the counterpart terminal during the video call, transmitting the current location-based video data input to the video call screen to the other terminal according to the camera mode It may include a step.

A computer program stored in a computer readable storage medium for executing the way guidance method comprises the steps of: transmitting video data based on the current location inputted on the video call screen of the terminal to the counterpart terminal during the video call; Receiving indication information for driving directions based on the video data from a counterpart terminal on which the video data is displayed on a video call screen; Generating road guide information by mapping the indication information to image data of the terminal; And guiding the road guide information according to the location information of the terminal.

The route guidance server may include a location information determiner configured to determine current location information of the terminal during a video call; An additional information obtaining unit which obtains additional information associated with indication information for way guidance input based on the video data from the counterpart terminal as the video data of the terminal is displayed on the video call screen of the counterpart terminal; And an additional information providing unit for providing the terminal with additional information associated with the indication information to the image data.

The additional information providing unit may generate additional augmented reality-based road guidance information by mapping additional information associated with the indication information in the terminal to the image data.

The road guidance server may further include a map information transfer unit for transmitting map information including current location information of the terminal to a video call screen of the counterpart terminal.

The additional information obtaining unit may generate guide information based on the current location information of the terminal and the indication information.

The additional information obtaining unit may set a departure range for detecting departure of the terminal, and generate an alarm message by determining whether the terminal is leaving based on the departure range.

The additional information providing unit may generate a layer including additional information associated with the indication information and transmit the generated layer to the terminal.

The map information transfer unit may provide a moving path according to the location information of the terminal to the video call screen of the counterpart terminal based on the road guide information displayed on the video call screen of the terminal.

By displaying the road guidance information on the video call screen of the terminal during the video call between the terminal can easily determine the movement route.

The road guidance information according to the location information of the terminal is guided to the video call screen of the terminal during the video call between the terminals to easily and accurately reach the destination.

1 is a diagram illustrating an example of a network environment according to an embodiment.
2A is a block diagram illustrating an internal configuration of a terminal and a server according to an embodiment.
2B is a block diagram illustrating an example of components that may be included in a processor of a terminal according to an embodiment.
3 is a flowchart illustrating an example of a method for guiding a terminal according to an embodiment.
4 is a block diagram illustrating an example of a component that may be included in a processor of a server according to an exemplary embodiment.
5 is a diagram for describing input of video data on a video call screen of a terminal according to an exemplary embodiment.
6 and 7 are diagrams for describing an example of inputting indication information in a terminal according to an exemplary embodiment.
8 to 11 are diagrams for explaining that the road guidance information is provided in the terminal according to an embodiment.
12 is a flowchart illustrating a method of providing road guidance information according to an embodiment.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an example of a network environment according to an embodiment.

The network environment of FIG. 1 illustrates an example including a terminal 110, a server 100, and a network 120. 1 is an example for describing the present invention, and the number of terminals and the number of servers are not limited as in FIG. 1.

The terminal 110 may be a fixed terminal or a mobile terminal implemented with a computer device. Examples of the terminal 110 include a smart phone, a mobile phone, navigation, a computer, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, a wearable device, and an HMD ( Head mounted Display). The terminal 110 may communicate with other terminals and / or the server 100 through the network 120 using a wireless or wired communication scheme.

The communication method is not limited and may include not only a communication method using a communication network (eg, a mobile communication network, a wired internet, a wireless internet, a broadcasting network) that the network 120 may include, but also a short range wireless communication between devices. For example, the network 120 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). And one or more of networks such as the Internet. In addition, the network 120 may include any one or more of network topologies including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network. It is not limited.

The server 100 may be implemented as a computer device or a plurality of computer devices that communicate with the terminal 110 through the network 120 to provide commands, codes, files, contents, services, and the like. For example, the server 100 may provide a file for installing an application to the terminal 110 connected through the network 120. In this case, the terminal 110 may install an application using a file provided from the server 100. In addition, a service or content provided by the server 100 by accessing the server 100 under the control of an operating system (OS) included in the terminal 110 or at least one program (for example, a browser or the installed application). Can be provided. For example, when the terminal 110 transmits a service request message to the server 100 through the network 120 under the control of the application, the server 100 transmits a code corresponding to the service request message to the terminal 110. The terminal 110 may provide content to a user by configuring and displaying a screen according to a code under the control of an application.

Hereinafter, a description will be given in detail of a technique for providing at least one terminal 110 with road guidance information during a video call.

2A is a block diagram illustrating an internal configuration of a terminal and a server according to an embodiment.

Although FIG. 2A illustrates an internal configuration of one terminal and one server, the same or similar internal components may be applied to other terminals or other servers that may be included in the network environment described with reference to FIG. 1.

The terminal 110 may include, for example, a processor 200, at least one application 201, a memory 202, a communication module 203, an input / output interface 204, and a camera 206.

The server 200 may include a processor 211, a memory 212, and a communication module 213. The memories 202 and 212 are computer-readable recording media and may include permanent mass storage devices such as random access memory (RAM), read only memory (ROM), and disk drives. In addition, the memory 202 or 212 may store an operating system or at least one program code (for example, a code installed in the terminal 110 and driven for the browser or the above-described application). These software components may be loaded from a computer readable recording medium separate from the memories 202 and 212. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, and the like. In other embodiments, software components may be loaded into the memory 202, 212 via the communication module 203, 213 rather than a computer readable recording medium. For example, at least one program is a program installed by files provided by a file distribution system (for example, the server 100 described above) through the network 120 for distributing installation files of developers or applications (for example, May be loaded into the memory 202 or 212 based on the above-described application).

Processors 200 and 211 may be configured to process instructions of a computer program by performing basic arithmetic, logic and input / output operations. Instructions may be provided to the processors 200, 211 by the memory 202, 212 or the communication modules 203, 213. For example, processors 200 and 211 may be configured to execute instructions received in accordance with program codes stored in recording devices such as memories 202 and 212.

The communication module 203 or 213 may provide a function for the terminal 110 and the road guidance server 100 to communicate with each other through the network 120, and may provide a function for communicating with another terminal or another server. Can be. For example, a request generated by the processor 200 of the terminal 110 according to a program code stored in a recording device such as a memory 202 may be generated through a network 120 through the network 120 under the control of the communication module 203. 100). Conversely, control signals, commands, contents, files, and the like provided according to the control of the processor 211 of the server 100 are transmitted through the communication module 213 and the network 120 to the communication module 203 of the terminal 110. It may be received by the terminal 110 through. For example, a control signal or command of the road guidance server 100 received through the communication module 203 may be transmitted to the processor 200 or the memory 202, and the content or file may be transmitted to the terminal 110. This may be stored as a storage medium that may further include.

The input / output interface 204 may be a means for interfacing with the input / output device. For example, the input device may include a device such as a keyboard or mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the input / output interface 204 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. More specifically, the processor 200 of the terminal 110 may include a service screen configured by using data provided by the road guidance server 100 or another terminal in processing a command of a computer program loaded in the memory 202. Content may be displayed on display 205 via input / output interface 204.

In addition, in other embodiments, the terminal 110 and the server 100 may include more components than the components of FIG. 2. However, it is not necessary to clearly show most of the prior art components. For example, the terminal 110 may be implemented to include at least some of the input / output devices or further include other components such as a transceiver, a global positioning system (GPS) module, a camera 206, various sensors, a database, and the like. You may. More specifically, when the terminal 110 is a smart phone, the acceleration sensor or gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, a vibrator for vibration, and the like generally included in the smart phone Various components may be implemented to be further included in the terminal 110.

2B is a block diagram illustrating an example of a component that may be included in a processor of a terminal according to an exemplary embodiment. FIG. 3 is a flowchart illustrating an example of a method for guiding a terminal according to an embodiment.

The processor 200 of the terminal 110 may include a location-based image data transmitter 210, a road guidance instruction information receiver 220, a guide generator 230, and a route guide 240. The components of the processor 200 may be representations of different functions performed by the processor 200 according to a control command provided by a program code stored in the terminal 110. The processor 200 and the components of the processor 200 may control the terminal 110 to perform steps 310 to 340 included in the way guidance method of FIG. 3. In this case, the processor 200 and the components of the processor 200 may be implemented to execute instructions according to code of an operating system included in a memory and code of at least one program.

The processor 200 may load a program code stored in a file of a program for the way guidance method into a memory. For example, when a program is executed in the terminal 110, the processor may control the terminal to load program code from a file of a program into a memory under control of an operating system. In this case, each of the processor 200 and the location-based image data transmitter 210, the road guidance instruction information receiver 220, the guide generator 230, and the path guide 240 included in the processor 200 may be stored in a memory. It may be different functional representations of the processor 200 for executing instructions of a corresponding portion of the loaded program code to execute steps 310-340 thereafter.

In operation 310, the location-based video data transmitter 210 may transmit the current location-based video data input to the video call screen of the terminal to the counterpart terminal during the video call.

In operation 320, the direction guidance instruction information receiver 220 may receive direction information for route guidance based on the image data from the counterpart terminal on which the video data is displayed on the video call screen. For example, a user interface for inputting indication information to the counterpart terminal may be provided. The indication information input through the user interface may be received from the counterpart terminal. Alternatively, the indication information may be received from the counterpart terminal through an event input to the video data displayed on the video call screen of the counterpart terminal.

In operation 330, the guide generator 230 may generate road guide information by mapping the indication information to image data of the terminal. The guide generation unit 230 may generate road guide information by mapping a layer including the indication information to image data corresponding to a position where the indication information is input. In this case, the layer including the indication information may include location information of the indication information input from the image data. The guide generation unit 230 may generate augmented reality-based road guide information by mapping a layer including the indication information to image data of the terminal. In addition, when the guide generation unit 230 receives additional information related to the indication information from the server, the guide generation unit 230 may generate road guide information by mapping a layer including the additional information to image data displayed on the terminal.

In operation 340, the route guide unit 340 may guide the road guide information according to the location information of the terminal. In this case, as the location information of the terminal is changed, the route guide unit 340 may update the road guide information based on the changed location information of the terminal, and may provide updated road guidance information. The route guide 340 may provide guide information related to the road guide information according to the location information of the terminal. For example, the route guide 340 may notify in which direction to move in the road guide information for the current location of the terminal. The route guide 340 may display the guide information as text data on the video call screen of the terminal and provide the guide information as voice data. In addition, the route guide 340 may display the movement route moving along the position information of the terminal on the video call screen of the terminal.

4 is a block diagram illustrating an example of a component that may be included in a processor of a server according to an exemplary embodiment.

The processor 211 of the road guidance server 100 may include a location information determiner 410, an additional information acquirer 420, an additional information provider 430, and a map information transfer unit 440. The components of the processor 211 may be representations of different functions performed by the processor 211 according to a control command provided by the program code stored in the guidance server 100. The processor 211 and the components of the processor 211 may be implemented to execute instructions according to code of an operating system included in the memory and code of at least one program.

The processor 211 may load program code stored in a file of a program for the way guidance method into a memory. For example, when a program is executed in the terminal 110, the processor may control the terminal to load program code from a file of a program into a memory under control of an operating system. In this case, each of the processor 211 and the location information determining unit 410, the additional information obtaining unit 420, the additional information providing unit 430, and the map information transmitting unit 440 included in the processor 211 are loaded into a memory. Different functional representations of the processor 211 executing instructions of the corresponding portion of the programmed code.

The location information determiner 410 may determine the current location information of the terminal during the video call. For example, the location information determination unit 410 may determine the current location information of the terminal based on the video data input to the video call screen of the terminal. More specifically, the location information determiner 410 may determine the location information of the terminal based on metadata included in the image data acquired from the camera of the terminal. The image data may include metadata such as size information of the image, date information on which the image is captured, and location information on which the image is captured. As another example, the location information determiner 410 may determine whether the terminal is indoor or outdoor based on building information and road information through image data input from the camera of the terminal. The location information determination unit 410 may determine the current location of the terminal from text information or image information included in building information (eg, business name, telephone number, etc.) and road information (eg, road name address) included in the image data. Information can be estimated. As another example, the location information determiner 410 may estimate the current location information of the terminal using the GPS information of the terminal.

The additional information obtaining unit 420 may obtain additional information associated with the indication information for the guidance input based on the video data from the counterpart terminal as the video data of the terminal is displayed on the video call screen of the counterpart terminal. In this case, the additional information may be information that can be additionally provided in relation to the indication information. For example, the additional information obtaining unit 420 may generate guide information based on current location information and indication information of the terminal. The additional information obtaining unit 420 may generate guide information to guide the road guide information based on the current location information of the terminal. In this case, the guide information may mean providing a guide message based on the current location information of the terminal, such as navigation. In addition, the additional information obtaining unit 420 sets a departure range for detecting departure of the terminal based on the road guide information guided by the terminal, and generates an alarm message by determining the departure of the terminal based on the departure range. Can be. For example, the additional information obtaining unit 420 may generate an alert message by determining that the terminal has deviated when the terminal leaves 5M or more while moving the route based on the road guide information guided to the terminal.

The additional information provider 430 may provide additional information associated with the indication information to the terminal to map the additional information to the image data. The additional information provider 430 may provide additional information associated with the indication information to the terminal in order to map the image information currently displayed on the terminal. For example, the additional information provider 430 may generate a layer including additional information associated with the indication information and transmit the generated layer to the terminal. The layer including additional information provided to the terminal from the additional information providing unit 430 may be mapped to image data currently displayed on the terminal to generate road guidance information. In this case, the road guidance information may be generated based on the augmented reality.

The map information transmitter 440 may transfer map information including current location information of the terminal to a video call screen of the counterpart terminal. For example, the map information transfer unit 440 may transmit the map information from the map server to the video call screen of the counterpart terminal through map information including the current location information of the terminal, and map the video call screen of the counterpart terminal through the map API. Information can be provided. The map information transmitting unit 440 may also display a moving path (for example, a path in which the terminal moves) according to the location information of the terminal based on the road guidance information displayed on the video call screen of the terminal. Can provide. The counterpart terminal may check the movement path of the terminal displayed on the video call screen.

5 is a diagram for describing input of video data on a video call screen of a terminal according to an exemplary embodiment.

At least one terminal and at least one counterpart terminal may be in a video call. In this case, the video call may generally be a video call between one terminal and the other terminal, it is also possible to make a video call between a plurality of terminals. For example, when a video call is performed through a messenger application, a video call between a plurality of terminals existing in a chat room of the messenger application is supported. In this case, the terminals may operate a front camera for each terminal for a video call. Accordingly, the face of the user is displayed on each terminal, so that the video call can be performed facing each other.

The terminal 110 may query the other terminal for a path during a video call with the other terminal. The terminal 110 may query a location that the other terminal already knows. For example, the terminal 110 may be moving to a place where the other terminal is located, or may be a place that the other terminal has visited. The front camera mode for executing the front camera operated in the terminal 110 may be switched to the rear camera mode for executing the rear camera 510. The rear camera 510 may operate in the terminal 110.

As the rear camera 510 is operated in the terminal 110, image data based on the current location where the terminal 110 is currently located may be captured. The terminal 110 may acquire video data based on the current location input on the video call screen of the terminal 110 during the video call. For example, the image data input through the rear camera 510 of the terminal 110 may include a street view including roads and buildings existing around the terminal 110. The current location-based video data input to the video call screen of the terminal 110 may be transmitted to the counterpart terminal.

The terminal 110 may include a terminal for querying a path to another terminal and a terminal for providing a road guide for the queried path. At this time, each terminal may query the route according to the situation, it may provide a road guide. Hereinafter, the terminal providing the route guidance for the inquired route is divided into 110a and the terminal inquiring the route to another terminal will be described as 110b.

6 and 7 are diagrams for describing an example of inputting indication information in a terminal according to an exemplary embodiment.

6 and 7 illustrate a method of inputting indication information from the counterpart terminal 110a in which video data is displayed on the video call screen 600.

The current location-based video data input to the video call screen of the terminal may be transmitted to the counterpart terminal 110a. Video data may be displayed on the video call screen 600 of the counterpart terminal 110a. Video data and map information may be displayed on the video call screen 600 of the counterpart terminal 110a. In this case, map information 610 including the current location information 640 of the terminal may be provided from the server.

The user 630 may input indication information into the video data displayed on the video call screen 600 of the counterpart terminal 110a. In this case, the screen of the counterpart terminal 110a may be configured as a touch screen capable of inputting a touch. The counterpart terminal 110a may recognize the indication information input from the user 630.

Referring to FIG. 6, the user 630 may input indication information through a specific event (eg, touch and drag) based on the video data displayed on the video call screen 600 of the counterpart terminal 110a. . In this case, the specific event is set to identify the command input from the user as the indication information. For example, the specific event may correspond to a touch and a drag. In addition, a specific event for identifying with indication information may be changed according to the user's setting. For example, the user may input indication information including the direction information through the touch and drag.

For example, the user 630 may input indication information into the image data based on the map information 610 including the current location information 640 of the terminal displayed on the video call screen 600 of the counterpart terminal 110a. have. The user 630 may input a touch for inputting indication information on the video call screen 600 of the counterpart terminal 110a, and the completion point 622 to end the touch at the start point 620 where the touch is input. The instruction information can be input by dragging 621.

As another example, referring to FIG. 7, user interfaces 710 to 713 for inputting indication information on the video call screen 600 of the counterpart terminal 110a may be provided. The user selects the user interface 710 to 713 from the video data of the video call screen 600 of the counterpart terminal based on the map information including the location information of the terminal displayed on the video call screen 600 of the counterpart terminal 110a. Can be. In this case, the user interface may include direction information for up, down, left, and right. The user interface 720 selected by the user may be displayed on the video data displayed on the video call screen 600 of the counterpart terminal 110a. The user may modify the user interface displayed on the video data of the video call screen 600 of the counterpart terminal 110a.

As such, the terminal may generate road guidance information based on the current location information of the user terminal in response to the indication information input from the user on the video call screen 600 of the counterpart terminal. In other words, since the user cannot accurately select the current location information 640 displayed in the map information of the counterpart terminal on the video call screen 600 of the counterpart terminal, the terminal is input to the video data displayed in the video call of the counterpart terminal. The directions information is generated based on the current location information 640 of the terminal.

8 to 11 are diagrams for explaining that the road guidance information is provided in the terminal according to an embodiment.

The terminal 110b may generate road guide information by mapping the indication information input from the counterpart terminal to the image data of the terminal. The terminal 110b may generate the road guide information 810 by mapping the layer including the indication information to the image data corresponding to the position where the indication information is input. In addition, the guide information may be generated based on the indication information input from the counterpart terminal in the server and the location information of the terminal. As the terminal 110b receives the guide information from the server, the terminal 110b generates the road guide information 810 by mapping the guide information to the image data.

The terminal 110b may guide the road guide information 810 according to the location information of the terminal. In this case, the terminal 110b may guide the road guide information 810 to the augmented reality-based road guide information, and may be guided to the virtual reality-based road guide information according to the terminal 110b.

The road guide information 810 and the guide information 820 may be provided on the video call screen 600 of the terminal 110b. When the terminal 110b is moving based on the road guidance information, the terminal 110b may provide the guide information 820 based on the location information of the terminal. For example, as the terminal moves based on the indication information 810, the terminal may provide guide information 820 such as 'turn right in front of 20M' based on the current location information currently present. In this case, the guide information 820 may be displayed as a message on the video call screen 600 of the terminal 110b and may also be output as a voice message.

According to an exemplary embodiment, the terminal may not only guide the route through the road guide information but also provide the guide information 820 together to provide the user with an easier route.

Meanwhile, since the terminal 110b and the counterpart terminal are performing the video call, the video data including the moving path of the terminal 110b may be displayed on the video call screen of the counterpart terminal. The counterpart terminal may input a command through the video call to the video data and the movement path displayed on the video call screen. For example, if the other terminal inputs a message such as 'when the user turns right to the right' with respect to the moving path in which the terminal 110b is moving, the input message may be delivered to the terminal 110b.

Referring to FIG. 9, as content including a destination route of the terminal 110b is uploaded, the content may be reproduced in the terminal and provided as road guide information. For example, the terminal 110b may generate road guidance information based on the acquired route information registered in the app 900. For example, when a user first arrives at a new destination, there is a difficulty in reaching the destination. Accordingly, the road guide information that can reach the destination through the app 900 can be provided. In more detail, as route information is input from other users who have visited a new travel destination, the route guidance server may generate the contents including the route guidance information by collecting the route information. The terminal 110b may guide the road guide information by playing the content 910 according to the location information of the terminal 110b. For example, the content 910 including the augmented reality-based road guide information according to the location information of the terminal may be played on the terminal 110b. At this time, the content corresponding to the location information of the user is reproduced so that the user can more easily navigate to the destination.

Alternatively, the terminal 110b may be provided with the content 910 including the route information by another user who has visited a new travel destination or a user who exists at the destination, and the content 910 according to the location information of the terminal 110b. By replaying, the guidance information can be guided.

Referring to FIG. 10, the current location information 1020 of the terminal 110b may be displayed on the video call screen 600 of the terminal 110b. The terminal 110b may determine whether the current location information 1020 of the terminal 110b is departed based on the departure range 1000 set in the road guidance information 1010. When the terminal 110b is out of the departure range 100, the terminal 110b may receive an alert message from the road guidance server. An alarm message 1030 may be displayed on the video call screen 600 of the terminal 110b, and an alarm (for example, vibration) may be generated on the terminal 110b. For example, the route guidance server may set a departure range 1000 for detecting departure of the terminal based on the indication information input from the counterpart terminal. The road guidance server may generate an alarm message by determining whether the terminal 110b has moved away based on the set departure range 1000. The route guidance server may transmit the alert message to the terminal 110b. As another example, by providing the current location information 1020 of the terminal 110b to the terminal 110b and the counterpart terminal, the moving path according to the route guidance information of the terminal 110b is a video call of the terminal 110b and the counterpart terminal. It can be displayed on the screen. When the terminal 110b is out of the departure range 100, the road guidance server may transmit an alert message to the counterpart terminal. An alarm may be generated from the counterpart terminal to the terminal 110b based on the current location information 1020 of the terminal 110b displayed on the video call screen of the counterpart terminal.

Referring to FIG. 11, the directions server may provide a messaging environment 1100 while guiding directions on a video call screen of the terminal 110b. As the video call is performed between the terminal 110b and the counterpart terminal 110a, the road guide information may be guided according to the location information of the terminal on the video call screen 600 of the terminal 110b. In this case, a user interface for inputting a message 1110 on the video call screen 600 of the terminal 110b and the counterpart terminal 110a may be provided. For example, as the message 1110, 'When arrive?' Is input on the video call screen of the counterpart terminal 110a, the counterpart terminal is displayed on the video call screen of the terminal 110b and the video call screen of the counterpart terminal 110a. The message 1110 input from 110a may be displayed. The terminal 110b may input a response message to the message 1110 input from the counterpart terminal 110a through the video call screen of the terminal. The response message input to the video call screen of the terminal 110b may be displayed on the video call screen of the terminal 110b and the counterpart terminal 110a. In this way, while receiving the directions through the video call screen of the terminal can transmit and receive the message 1110.

12 is a flowchart illustrating a method of providing road guidance information according to an embodiment.

A method of guiding road guidance information through operations of the first terminal 1210, the second terminal 1230, and the server 100 will be described.

The first terminal 1210 may request a video call to the second terminal 1230. The second terminal 1230 may accept the video call in response to the video call request requested from the first terminal 1210. Through this, a video call may be connected between the first terminal 1210 and the second terminal 1230 to perform a video call.

When the first terminal 1210 and the second terminal 1230 are in a video call 1211, the video call screen of the first terminal 1210 may acquire image data input through the rear camera of the first terminal 1210. Can be. Herein, the rear camera mode among the at least one camera mode (for example, the front camera mode for executing the front camera or the rear camera mode for executing the rear camera) of the first terminal 1210 is described as an example. However, the present invention is not limited to the rear camera mode, and may be video data input from the first terminal 1210 to 1211 during a video call. In this case, the image data may be image data based on the current location of the terminal. The first terminal 1210 may transmit image data input to the video call screen of the first terminal to the second terminal 1230 (1212). In this case, although the path is queried from the first terminal 1210 in FIG. 12, it is not limited to the first terminal 1210.

The second terminal 1230 may receive image data input to the video call screen of the first terminal. In this case, for example, the server 100 may determine location information of the first terminal as receiving the video data inputted on the video call screen of the first terminal 1210 (1213). As another example, the server 100 may determine the location information of the first terminal 1210 by referring to the GPS information of the first terminal 1210. As another example, the server 100 may determine the location information of the first terminal as receiving the GPS information of the first terminal 1210 and the video data inputted on the video call screen of the first terminal 1210. . The server 100 may transmit map information including the current location information of the first terminal to the second terminal 1230 (1214). Accordingly, image data and map information transmitted from the first terminal 1210 may be displayed on the video call screen of the second terminal 1230 (1215).

The second terminal 1230 may input indication information for driving directions based on the image data (1216). The second terminal 1230 may transfer the indication information input to the image data to the first terminal 1210. The first terminal 1210 may receive the indication information input from the second terminal 1230. In this case, the server 100 may obtain additional information based on the indication information input from the second terminal 1230 (1218). For example, the server 100 may obtain additional information through a separate server based on the indication information. Alternatively, the server 100 may generate additional information by extracting information stored in the database based on the indication information. The server 100 may transfer the obtained additional information to the first terminal 1210 (1219).

The first terminal 1210 generates road guidance information by mapping the indication information received from the second terminal 1230 and the additional information received from the server 100 to image data currently displayed on the first terminal 1210. It may be done (1220). For example, the first terminal 1210 overlaps the layer including the indication information and the layer including the additional information with the image data based on the position at which the indication information is input and the position information of the first terminal 1210. Can be mapped. In this case, the augmented reality-based road guidance information may be generated.

The first terminal 1210 may guide the road guide information according to the location information (1221). For example, as the user of the first terminal 1210 moves, the user may guide the road guide information according to the location information of the first terminal 1210.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments are, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable gate arrays (FPGAs). Can be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. It can be embodied in. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine code, such as produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (22)

In a computer program stored in a computer readable storage medium for executing the directions method,
The way guidance is,
Transmitting image data input from at least one camera of a terminal to a counterpart terminal according to a camera mode that is operated during a video call, wherein the camera mode includes at least one of a front camera mode and a rear camera mode;
From the counterpart terminal on which the video data is displayed on the video call screen, the indication information input on the displayed video data through a user interface provided on the counterpart terminal together with a position where the indication information is input on the video call screen. Receiving;
The image that is currently input to the layer including the indication information by mapping the indication information to image data currently input through the camera according to the camera mode based on a position where the indication information is input on the video call screen. Generating road guidance information by mapping the data;
Generating guide information related to the road guidance information according to the current location information of the terminal; And
Displaying the road guide information and the guide information on a video call screen of the terminal.
And a computer program stored in a computer readable storage medium. The method of claim 1,
The transmitting step,
As the path is inquired to the counterpart terminal through the voice data during the video call, the front camera mode for executing the front camera during the video call is switched to the rear camera mode for executing the rear camera, and the converted rear camera mode is Transmitting the input image data to the counterpart terminal;
Including, a computer program. The method of claim 1,
The transmitting step,
Transmitting the video data to which the road information or the building information including the face of the user displayed on the video call screen is input to the counterpart terminal as the front camera mode is operated.
Including, a computer program. The method of claim 1,
The transmitting step,
Transmitting video data including road information or building information input to a video call screen of the terminal as video data based on a current location;
Including, a computer program. The method of claim 1,
The receiving step,
Receiving indication information input through a user interface including direction information for up, down, left, and right provided on the counterpart terminal;
Including, a computer program. The method of claim 1,
The transmitting step,
Transmitting location information of the terminal when the image data is input to the counterpart terminal;
Including,
The receiving step,
Receiving indication information input on the displayed video data from a counterpart terminal whose map information including the location information of the terminal is displayed on a video call screen together with the video data;
Including, a computer program. The method of claim 1,
The way guidance is,
Providing a user interface for inputting a message to be transmitted to the counterpart terminal on the video call screen;
The computer program further comprising. The method of claim 1,
The way guidance is,
Providing a moving path of the terminal moving according to the road guidance information on the video call screen, and if it is detected that the departure of the terminal occurs based on the road guidance information, generating an alarm associated with the departure
The computer program further comprising. In the computer-guided way,
Transmitting image data input from at least one camera of a terminal to a counterpart terminal according to a camera mode that is operated during a video call, wherein the camera mode includes at least one of a front camera mode and a rear camera mode;
From the counterpart terminal on which the video data is displayed on the video call screen, the indication information input on the displayed video data through a user interface provided on the counterpart terminal together with a position where the indication information is input on the video call screen. Receiving;
The image that is currently input to the layer including the indication information by mapping the indication information to image data currently input through the camera according to the camera mode based on a position where the indication information is input on the video call screen. Generating road guidance information by mapping the data;
Generating guide information related to the road guidance information according to the current location information of the terminal; And
Displaying the road guide information and the guide information on a video call screen of the terminal.
Including, directions. The method of claim 9,
The transmitting step,
As the path is inquired to the counterpart terminal through the voice data during the video call, the front camera mode for executing the front camera during the video call is switched to the rear camera mode for executing the rear camera, and the converted rear camera mode is used. Transmitting the input image data to the counterpart terminal;
Including, directions. The method of claim 9,
The transmitting step,
Transmitting the video data to which the road information or the building information including the face of the user displayed on the video call screen is input to the counterpart terminal as the front camera mode is operated.
Including, directions. The method of claim 9,
The transmitting step,
Transmitting video data including road information or building information input to a video call screen of the terminal as video data based on a current location;
Including, directions. The method of claim 9,
The receiving step,
Receiving indication information input through a user interface including direction information for up, down, left, and right provided on the counterpart terminal;
Including, directions. The method of claim 9,
The transmitting step,
Transmitting location information of the terminal when the image data is input to the counterpart terminal;
Including,
The receiving step,
Receiving indication information input on the displayed video data from a counterpart terminal whose map information including location information of the terminal is displayed on a video call screen together with the video data;
Including, directions. The method of claim 9,
The way guidance is,
Providing a user interface for inputting a message to be transmitted to the counterpart terminal on the video call screen;
Further comprising, directions. The method of claim 9,
The way guidance is,
Providing a moving path of the terminal moving according to the road guidance information on the video call screen, and if it is detected that the departure of the terminal occurs based on the road guidance information, generating an alarm associated with the departure
Further comprising, directions. In the navigation server,
A location information determination unit to determine current location information of the terminal during a video call;
When the image data input from at least one camera of the terminal is displayed on the video call screen of the counterpart terminal according to the camera mode of the terminal, the counterpart terminal on the counterpart terminal is provided through the user interface provided on the counterpart terminal. An additional information obtaining unit which receives the indication information input on the image data displayed on the terminal and generates guide information based on the current position information of the terminal and the indication information, thereby obtaining additional information associated with the received indication information; And
Additional information providing unit for providing additional information associated with the obtained indication information to the terminal
Including, directions server. The method of claim 17,
The additional information providing unit,
Generate a layer including additional information associated with the indication information, and the layer including the generated additional information is mapped to image data input through the camera of the terminal to generate augmented reality-based road guidance information. And a layer including additional information associated with the information to the terminal. The method of claim 17,
The location information determination unit,
It is determined whether the terminal is indoors or outdoors based on the video data including the building information or the road information input on the video call screen of the terminal, and the text information or image information included in the building information and the road information. Directions server, to estimate the current location information. The method of claim 17,
The additional information providing unit,
A route guidance server for setting a departure range for detecting the departure of the terminal, and determining whether to leave the terminal based on the set departure range, to generate a route message. The method of claim 17,
Map information transfer unit for transmitting the map information including the current location information of the terminal to the video call screen of the counterpart terminal
Further comprising, directions server. The method of claim 21,
The map information transfer unit,
The route guidance server, based on the route guidance information displayed on the video call screen of the terminal to provide a moving path according to the location information of the terminal to the video call screen of the counterpart terminal.

Images