KR102225175B1 - Method and system for navigation using video call - Google Patents

Abstract

A method for guiding a route implemented by a computer according to an embodiment includes: transmitting video data based on a current location input to a video call screen of a terminal to a counterpart terminal during a video call; Receiving instruction information for directions based on the video data from a counterpart terminal in which the video data is displayed on a video call screen; Generating directions information by mapping the indication information to image data of the terminal; And guiding the directions information according to the location information of the terminal.

Description

Direction and system using video call {METHOD AND SYSTEM FOR NAVIGATION USING VIDEO CALL}

The following description relates to a method and system for guiding directions information during a video call.

With the development of communication technology, video call services are provided not only to portable communication devices such as mobile phones, but also to large electronic devices such as TVs. In the video call service, when a video call call is attempted from the calling terminal, the call attempt message is transmitted to the exchange through the base station/base station controller. Performs terminal and session setup. According to this process, the calling terminal and the called terminal can perform a video call while viewing each other's faces through transmission and reception of audio and video.

However, in a video call, you cannot see the video that the other party is watching, nor can you check the video that the other party is transmitting. Accordingly, in the case of inquiring about a route or trying to receive guidance during a video call, there is an inconvenience in that the route should be described only by voice.

Korean Patent Application Publication No. 10-2016-0086560, Korean Patent Application Publication No. 10-2012-0008400,

It is possible to provide a method and system for providing route guidance information based on augmented reality during a video call between terminals.

It is possible to provide a method and system for guiding route guidance information generated based on location information of a terminal during a video call between terminals.

The computer-implemented route guidance method includes: transmitting video data based on a current location input to a video call screen of a terminal to a counterpart terminal during a video call; Receiving instruction information for directions based on the video data from a counterpart terminal in which the video data is displayed on a video call screen; Generating directions information by mapping the indication information to image data of the terminal; And guiding the directions information according to the location information of the terminal.

The guiding the directions information according to the location information of the terminal includes: guiding the directions information based on augmented reality generated by mapping the instruction information to the image data of the terminal according to the location information of the terminal It may include.

The step of generating directions information by mapping the indication information to image data of the terminal comprises: mapping a layer including the indication information to the image data corresponding to a location where the indication information is input to generate directions information It may include the step of.

The receiving of instruction information for directions based on the video data from a counterpart terminal in which the video data is displayed on a video call screen includes the user from the counterpart terminal as a user interface for inputting instruction information to the counterpart terminal is provided. It may include the step of receiving the instruction information input through the interface.

The receiving of instruction information for directions based on the video data from a counterpart terminal in which the video data is displayed on a video call screen includes receiving instruction information related to a specific event input to the video data from the counterpart terminal. Can include.

The guiding the directions information according to the location information of the terminal may include providing guide information related to the directions information according to the location information of the terminal.

The guiding the directions information according to the location information of the terminal may include continuously updating the directions information based on the changed location information of the terminal as the location information of the terminal is changed.

In the guiding the directions information according to the location information of the terminal, the content including the destination route of the terminal is uploaded by the counterpart terminal, so that the content is converted to the location information of the terminal as the directions information. It may include the step of regenerating.

The step of guiding the directions information according to the location information of the terminal may include generating an alarm related to the departure when detecting that the departure of the terminal occurs based on the directions information. .

The guiding the directions information according to the location information of the terminal may include transmitting and receiving a message between the terminal and the counterpart terminal.

The step of guiding the directions information according to the location information of the terminal may include displaying a moving path moving according to the location information of the terminal.

The transmitting of the current location-based video data input to the video call screen of the terminal to the other terminal during the video call includes: transmitting the current location-based video data input to the video call screen to the counterpart terminal according to a camera mode. It may include steps.

The computer program stored in a computer-readable storage medium to execute the directions method includes: transmitting video data based on a current location input to a video call screen of a terminal to a counterpart terminal during a video call; Receiving instruction information for directions based on the video data from a counterpart terminal in which the video data is displayed on a video call screen; Generating directions information by mapping the indication information to image data of the terminal; And guiding the directions information according to the location information of the terminal.

The directions server includes: a location information determination unit that determines current location information of a terminal during a video call; An additional information acquisition unit for acquiring additional information related to direction information input based on the video data from the counterpart terminal as the video data of the terminal is displayed on the video call screen of the counterpart terminal; And an additional information providing unit provided to the terminal to map additional information related to the indication information to image data.

The additional information providing unit may generate augmented reality-based route guidance information by mapping additional information related to the indication information in the terminal to the image data.

The directions server may further include a map information delivery unit for transferring map information including current location information of the terminal to a video call screen of the counterpart terminal.

The additional information acquisition unit may generate guide information based on the current location information and the indication information of the terminal.

The additional information acquisition unit may set a departure range for detecting the departure of the terminal, and may generate an alert message by determining whether the terminal leaves the departure range based on the departure range.

The additional information providing unit may generate a layer including additional information related to the indication information and transmit it to the terminal.

The map information delivery unit may provide a moving route according to the location information of the terminal to the video call screen of the counterpart terminal based on directions information displayed on the video call screen of the terminal.

During a video call between terminals, a moving route can be easily identified by displaying directions information on the video call screen of the terminal.

During a video call between terminals, directions information according to the location information of the terminal is guided on the video call screen of the terminal, so that the destination can be easily and accurately reached.

1 is a diagram illustrating an example of a network environment according to an embodiment.
2A is a block diagram illustrating an internal configuration of a terminal and a server according to an embodiment.
2B is a block diagram illustrating an example of components that may be included in a processor of a terminal according to an embodiment.
3 is a flowchart illustrating an example of a method for guiding a way of a terminal according to an embodiment.
4 is a block diagram illustrating an example of components that may be included in a processor of a server according to an embodiment.
5 is a diagram illustrating input of video data to a video call screen of a terminal according to an exemplary embodiment.
6 and 7 are diagrams for explaining an example in which indication information is input from a terminal according to an embodiment.
8 to 11 are diagrams for explaining provision of directions information in a terminal according to an embodiment.
12 is a flowchart illustrating a method of providing directions information according to an exemplary embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an example of a network environment according to an embodiment.

The network environment of FIG. 1 shows an example including a terminal 110, a server 100, and a network 120. This FIG. 1 is an example for explaining the invention, and the number of terminals or servers is not limited as shown in FIG. 1.

The terminal 110 may be a fixed terminal implemented as a computer device or a mobile terminal. For example, the terminal 110 is a smart phone, mobile phone, navigation, computer, notebook, digital broadcasting terminal, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), tablet PC, wearable device, HMD ( Head　mounted　Display). The terminal 110 may communicate with other terminals and/or the server 100 through the network 120 using a wireless or wired communication method.

The communication method is not limited, and short-range wireless communication between devices as well as a communication method using a communication network (for example, a mobile communication network, wired Internet, wireless Internet, and broadcasting network) that the network 120 may include may be included. For example, the network 120 includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). And any one or more of networks such as the Internet. In addition, the network 120 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. Not limited.

The server 100 may be implemented as a computer device or a plurality of computer devices that communicate with the terminal 110 and the network 120 to provide commands, codes, files, contents, services, and the like. For example, the server 100 may provide a file for installing an application to the terminal 110 accessed through the network 120. In this case, the terminal 110 may install an application using a file provided from the server 100. In addition, services or contents provided by the server 100 by accessing the server 100 under the control of an operating system (OS) included in the terminal 110 or at least one program (for example, a browser or the installed application). Can be provided. For example, when the terminal 110 transmits a service request message to the server 100 through the network 120 under the control of the application, the server 100 sends a code corresponding to the service request message to the terminal 110. Transmission may be performed, and the terminal 110 may provide content to a user by configuring and displaying a screen according to a code according to the control of the application.

Hereinafter, a technology in which at least one terminal 110 provides directions information while a video call is being performed will be described in detail.

2A is a block diagram illustrating an internal configuration of a terminal and a server according to an embodiment.

Although the internal configuration of one terminal and one server is described in FIG. 2A, the same or similar internal components may be applied to other terminals or other servers that may be included in the network environment described with reference to FIG. 1.

The terminal 110 may include, for example, a processor 200, at least one application 201, a memory 202, a communication module 203, an input/output interface 204, and a camera 206.

The server 200 may include a processor 211, a memory 212, and a communication module 213. The memories 202 and 212 are computer-readable recording media, and may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive. In addition, the memory 202 and 212 may store an operating system or at least one program code (for example, a browser installed and driven in the terminal 110 or a code for the above-described application). These software components may be loaded from a computer-readable recording medium separate from the memories 202 and 212. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, software components may be loaded into the memories 202 and 212 through communication modules 203 and 213 rather than a computer-readable recording medium. For example, at least one program is a program installed by files provided through the network 120 by a file distribution system (for example, the server 100 described above) for distributing the installation files of developers or applications (example It may be loaded into the memories 202 and 212 based on the above-described application).

The processors 200 and 211 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processors 200 and 211 by the memories 202 and 212 or the communication modules 203 and 213. For example, the processors 200 and 211 may be configured to execute commands received according to program codes stored in a recording device such as the memories 202 and 212.

The communication modules 203 and 213 may provide a function for the terminal 110 and the directions server 100 to communicate with each other through the network 120, and provide a function for communicating with other terminals or other servers. I can. As an example, a request generated by the processor 200 of the terminal 110 according to a program code stored in a recording device such as the memory 202 is transmitted through the network 120 under the control of the communication module 203 through the route guidance server ( 100) can be delivered. Conversely, control signals, commands, contents, files, etc. provided under the control of the processor 211 of the server 100 are transmitted through the communication module 213 and the network 120 to the communication module 203 of the terminal 110 It may be received by the terminal 110 through. For example, control signals or commands of the directions server 100 received through the communication module 203 may be transmitted to the processor 200 or the memory 202, and contents or files may be transmitted to the terminal 110. It may be stored in a storage medium that may further contain this.

The input/output interface 204 may be a means for an interface with an input/output device. For example, the input device may include a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the input/output interface 204 may be a means for interfacing with a device in which input and output functions are integrated into one, such as a touch screen. As a more specific example, the processor 200 of the terminal 110 processes a command of a computer program loaded in the memory 202, a service screen or a service screen configured using data provided by the directions server 100 or other terminals. Content may be displayed on the display 205 through the input/output interface 204.

In addition, in other embodiments, the terminal 110 and the server 100 may include more components than those of FIG. 2. However, there is no need to clearly show most of the prior art components. For example, the terminal 110 may be implemented to include at least some of the input/output devices or further include other components such as a transceiver, a global positioning system (GPS) module, a camera 206, various sensors, and a database. You may. As a more specific example, when the terminal 110 is a smart phone, in general, an acceleration sensor or a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input/output port, a vibrator for vibration, etc. Various components may be implemented to be further included in the terminal 110.

FIG. 2B is a block diagram illustrating an example of components that may be included in a processor of a terminal according to an embodiment, and FIG. 3 is a flowchart illustrating an example of a method for guiding a route of a terminal according to an embodiment.

The processor 200 of the terminal 110 may include a location-based image data transmission unit 210, a directions instruction information reception unit 220, a guide generation unit 230, and a route guide unit 240. The components of the processor 200 may be expressions of different functions performed by the processor 200 according to a control command provided by a program code stored in the terminal 110. The processor 200 and components of the processor 200 may control the terminal 110 to perform steps 310 to 340 included in the directions method of FIG. 3. In this case, the processor 200 and the components of the processor 200 may be implemented to execute an instruction according to the code of the operating system included in the memory and the code of at least one program.

The processor 200 may load a program code stored in a program file for a directions method into a memory. For example, when a program is executed in the terminal 110, the processor may control the terminal to load the program code from the program file into the memory under the control of the operating system. At this time, each of the location-based image data transmission unit 210, the directions instruction information receiving unit 220, the guidance generation unit 230, and the route guidance unit 240 included in the processor 200 and the processor 200 is stored in a memory. These may be different functional expressions of the processor 200 for executing the subsequent steps 310 to 340 by executing an instruction of a corresponding part of the loaded program code.

In step 310, the location-based video data transmission unit 210 may transmit the current location-based video data input to the video call screen of the terminal to the other terminal during the video call.

In step 320, the directions instruction information receiving unit 220 may receive instruction information for directions based on the image data from a counterpart terminal in which the image data is displayed on the video call screen. For example, a user interface for inputting instruction information to a counterpart terminal may be provided. The instruction information input through the user interface may be received from the counterpart terminal. Alternatively, instruction information may be received from the counterpart terminal through an event input to the video data displayed on the video call screen of the counterpart terminal.

In step 330, the guide generation unit 230 may generate directions information by mapping the instruction information to image data of the terminal. The guide generation unit 230 may generate directions information by mapping a layer including the indication information to image data corresponding to a location where the indication information is input. In this case, the layer including the indication information may include location information of the indication information input from the image data. The guide generator 230 may generate augmented reality-based route guide information by mapping a layer including the indication information to image data of the terminal. In addition, when receiving the additional information related to the instruction information from the server, the guide generation unit 230 may map a layer including the additional information to image data displayed on the terminal to generate the directions information.

In step 340, the route guide unit 340 may guide the route guide information according to the location information of the terminal. In this case, the route guide unit 340 may update the directions information based on the changed location information of the terminal as the location information of the terminal is changed, and may provide the updated directions information. The route guide 340 may provide guide information related to route guide information according to the location information of the terminal. For example, the route guide unit 340 may notify which direction to move from the route guide information for the current location of the terminal. The route guide unit 340 may display guide information as text data on a video call screen of the terminal and provide audio data at the same time. In addition, the path guide 340 may display a moving path moving according to the location information of the terminal on the video call screen of the terminal.

4 is a block diagram illustrating an example of components that may be included in a processor of a server according to an embodiment.

The processor 211 of the directions server 100 may include a location information determining unit 410, an additional information obtaining unit 420, an additional information providing unit 430, and a map information transmitting unit 440. Components of the processor 211 may be expressions of different functions performed by the processor 211 according to a control command provided by a program code stored in the directions server 100. The components of the processor 211 and the processor 211 may be implemented to execute an instruction according to the code of the operating system included in the memory and the code of at least one program.

The processor 211 may load a program code stored in a program file for a directions method into a memory. For example, when a program is executed in the terminal 110, the processor may control the terminal to load the program code from the program file into the memory under the control of the operating system. At this time, each of the location information determination unit 410, the additional information acquisition unit 420, the additional information providing unit 430, and the map information delivery unit 440 included in the processor 211 and the processor 211 are loaded into the memory. These may be different functional expressions of the processor 211 executing an instruction of a corresponding part of the program code.

The location information determination unit 410 may determine current location information of the terminal during a video call. For example, the location information determination unit 410 may determine the current location information of the terminal based on image data input to the video call screen of the terminal. More specifically, the location information determination unit 410 may determine the location information of the terminal based on metadata included in the image data obtained from the camera of the terminal. The image data may include metadata such as size information of an image, information on a date when an image is captured, and information on a location where an image is captured. As another example, the location information determination unit 410 may determine whether the terminal is indoors or outdoors based on building information and road information through image data input from a camera of the terminal. The location information determination unit 410 determines the current location of the terminal from text information or image information included in building information (eg, business name, phone number, etc.) and road information (eg, road name address) included in the image data. Information can be estimated. As another example, the location information determination unit 410 may estimate the current location information of the terminal using GPS information of the terminal.

As the image data of the terminal 420 is displayed on the video call screen of the counterpart terminal, the additional information acquisition unit 420 may obtain additional information related to instruction information for directions input from the counterpart terminal based on the video data. In this case, the additional information may be information that can be additionally provided in connection with the indication information. For example, the additional information acquisition unit 420 may generate guide information based on the current location information and indication information of the terminal. The additional information acquisition unit 420 may generate guide information to guide the directions information based on the current location information of the terminal. In this case, the guide information may mean providing a guide message based on the current location information of the terminal, such as navigation. In addition, the additional information acquisition unit 420 sets a departure range for detecting the departure of the terminal based on the directions information guided by the terminal, and determines whether the terminal leaves the departure based on the departure range, and generates an alarm message. I can. For example, the additional information acquisition unit 420 may determine that the terminal has departed and generate an alert message if the terminal deviates more than 5M while moving the path based on the directions information guided to the terminal.

The additional information providing unit 430 may provide the additional information related to the indication information to the terminal in order to map the image data. The additional information providing unit 430 may provide the additional information related to the indication information to the terminal to map the image data currently displayed on the terminal. For example, the additional information providing unit 430 may generate a layer including additional information related to the indication information and transmit it to the terminal. The layer including the additional information provided to the terminal from the additional information providing unit 430 may be mapped to image data currently displayed on the terminal to generate directions information. In this case, the directions information may be generated based on the augmented reality.

The map information delivery unit 440 may transmit map information including current location information of the terminal to the video call screen of the counterpart terminal. For example, the map information delivery unit 440 may transmit the map information including the current location information of the terminal from the map server to the video call screen of the other terminal, and map the video call screen of the other terminal through the map API. You can provide information. The map information delivery unit 440 also provides a moving path (for example, a path on which the terminal is moving) according to the location information of the terminal based on the directions information displayed on the video call screen of the terminal. Can provide. The counterpart terminal can check the moving path of the terminal displayed on the video call screen.

5 is a diagram illustrating input of video data to a video call screen of a terminal according to an exemplary embodiment.

At least one terminal and at least one counterpart terminal may be conducting a video call. In this case, in the video call, in general, a video call may be made between one terminal and another terminal, but a video call may be made between a plurality of terminals. For example, when a video call is performed through a messenger application, a video call between a plurality of terminals existing in a chat room of the messenger application is supported. In this case, the terminals may operate a front camera at each terminal for a video call. Accordingly, the user's face is displayed on each terminal, so that a video call can be performed by facing each other.

The terminal 110 may query the other terminal for a route during a video call with the other terminal. The terminal 110 may query a location that the other terminal already knows. For example, the terminal 110 may be moving to a place where the other terminal is located, or a place where the other terminal has been. The front camera mode for executing the front camera operated on the terminal 110 may be switched to the rear camera mode for executing the rear camera 510. The rear camera 510 may be operated on the terminal 110.

As the rear camera 510 is operated on the terminal 110, image data based on the current location where the terminal 110 is currently located may be photographed. The terminal 110 may acquire image data based on a current location input to the video call screen of the terminal 110 during a video call. For example, the image data input through the rear camera 510 of the terminal 110 may include a street view including roads and buildings existing around the terminal 110. The image data based on the current location input to the video call screen of the terminal 110 may be transmitted to the counterpart terminal.

The terminal 110 may include a terminal that queries another terminal for a route and a terminal that provides directions for the queried route. At this time, each terminal may query a route or provide a route guide according to a situation. Hereinafter, a terminal that provides directions for a queried route is divided into 110a, and a terminal that queries another terminal for a route is divided into 110b.

6 and 7 are diagrams for explaining an example in which indication information is input from a terminal according to an embodiment.

6 and 7 will be described a method of inputting instruction information in the counterpart terminal 110a in which video data is displayed on the video call screen 600.

The video data based on the current location input to the video call screen of the terminal may be transmitted to the counterpart terminal 110a. Image data may be displayed on the video call screen 600 of the counterpart terminal 110a. Image data and map information may be displayed on the video call screen 600 of the counterpart terminal 110a. In this case, the map information 610 including the current location information 640 of the terminal may be provided from the server.

The user 630 may input instruction information into image data displayed on the video call screen 600 of the counterpart terminal 110a. In this case, the screen of the counterpart terminal 110a may be configured as a touch screen capable of inputting a touch. The counterpart terminal 110a may recognize instruction information input from the user 630.

Referring to FIG. 6, the user 630 may input instruction information through a specific event (eg, touch & drag) based on the image data displayed on the video call screen 600 of the counterpart terminal 110a. . In this case, a specific event is set to identify a command input from a user as instruction information, and may correspond to, for example, touch & drag. In addition, a specific event for identification as instruction information may be changed according to a user's setting. For example, the user may input instruction information including direction information through touch and drag.

As an example, the user 630 may input instruction information in the image data based on the map information 610 including the current location information 640 of the terminal displayed on the video call screen 600 of the counterpart terminal 110a. have. The user 630 may input a touch for inputting instruction information on the video call screen 600 of the counterpart terminal 110a, and the end point 622 at which the touch is to be terminated at the starting point 620 where the touch is input. By dragging up to 621, the instruction information can be input.

As another example, referring to FIG. 7, user interfaces 710 to 713 for inputting instruction information on the video call screen 600 of the counterpart terminal 110a may be provided. The user selects the user interfaces 710 to 713 for the video data of the video call screen 600 of the other terminal based on the map information including the location information of the terminal displayed on the video call screen 600 of the other terminal 110a. I can. In this case, the user interface may include direction information for up, down, left, and right. The user interface 720 selected by the user may be displayed on the image data displayed on the video call screen 600 of the counterpart terminal 110a. The user may modify the user interface displayed on the video data of the video call screen 600 of the counterpart terminal 110a.

In this way, as the terminal receives instruction information input on the video call screen 600 of the other terminal from the user, the terminal may generate directions information based on the current location information of the user terminal. In other words, since it is impossible for the user to accurately select the current location information 640 displayed on the map information of the other terminal on the video call screen 600 of the other terminal, the terminal Direction information is generated based on the indication information on the current location information 640 of the terminal.

8 to 11 are diagrams for explaining provision of directions information in a terminal according to an embodiment.

The terminal 110b may generate directions information by mapping the instruction information input from the counterpart terminal to the image data of the terminal. The terminal 110b may generate the directions information 810 by mapping the layer including the indication information to the image data corresponding to the location where the indication information is input. In addition, the guide information may be generated based on the instruction information input from the counterpart terminal and the location information of the terminal in the server. As the terminal 110b receives the guide information from the server, it maps the guide information to the image data to generate the directions information 810.

The terminal 110b may guide the directions information 810 according to the location information of the terminal. At this time, the terminal 110b may guide the directions information 810 as augmented reality-based directions information, and according to the terminal 110b, the terminal 110b may guide the directions information based on virtual reality.

Direction information 810 and guide information 820 may be provided on the video call screen 600 of the terminal 110b. When the terminal 110b is moving based on the directions information, the terminal 110b may provide the guide information 820 based on the location information of the terminal. For example, as the terminal moves based on the directions information 810, guide information 820 such as'Turn right in front of 20M' may be provided based on the current location information that the terminal currently exists. In this case, the guide information 820 may be displayed as a message on the video call screen 600 of the terminal 110b, and may also be output as a voice message.

The terminal according to an embodiment may provide the user with the guide information 820 as well as guiding the route through the route guidance information so that the user can find the route more easily.

Meanwhile, since the terminal 110b and the counterpart terminal are performing a video call, video data including a moving path on which the terminal 110b is moving may be displayed on the video call screen of the counterpart terminal. The other party's terminal may input a command for the video data and the moving path displayed on the video call screen through a video call. For example, if the other terminal inputs a message such as'If you turn right to right, you can find it immediately' for a moving path on which the terminal 110b is moving, the input message may be delivered to the terminal 110b.

Referring to FIG. 9, as content including a destination route of the terminal 110b is uploaded, it may be reproduced on the terminal and provided as directions information. For example, the terminal 110b may generate directions information based on the acquired route information registered in the app 900. For example, when a user first arrives at a new travel destination, it is difficult to reach the destination. Accordingly, directions information capable of reaching the destination through the app 900 may be provided. In more detail, as route information is input from other users who have visited a new travel destination, the route guidance server may aggregate the route information to generate content including route guidance information. The terminal 110b may guide directions information by playing the content 910 according to the location information of the terminal 110b. For example, the content 910 including the augmented reality-based route guidance information according to the location information of the terminal may be played on the terminal 110b. In this case, content corresponding to the user's location information is played, so that the user can more easily find the destination.

Alternatively, the terminal 110b may be provided with the content 910 including route information by another user who has visited a new destination or a user existing at the destination, and the content 910 may be provided according to the location information of the terminal 110b. You can guide directions information by playing back.

Referring to FIG. 10, current location information 1020 of the terminal 110b may be displayed on the video call screen 600 of the terminal 110b. The terminal 110b may determine whether the current location information 1020 of the terminal 110b has deviated based on the departure range 1000 set in the directions information 1010. When the terminal 110b is out of the departure range 100, the terminal 110b may receive an alert message from the directions server. An alert message 1030 may be displayed on the video call screen 600 of the terminal 110b, and an alarm (eg, vibration) may be generated on the terminal 110b. For example, the directions server may set the departure range 1000 for detecting the departure of the terminal based on the instruction information input from the counterpart terminal. The directions server may generate an alert message by determining whether the terminal 110b has deviated based on the set departure range 1000. The directions server may transmit an alert message to the terminal 110b. As another example, by providing the current location information 1020 of the terminal 110b to the terminal 110b and the counterpart terminal, the moving path according to the directions information of the terminal 110b is determined by a video call between the terminal 110b and the counterpart terminal. It can be displayed on the screen. The directions server may transmit an alert message to the counterpart terminal when the terminal 110b is out of the departure range 100. An alarm may be generated from the counterpart terminal to the terminal 110b based on the current location information 1020 of the terminal 110b displayed on the video call screen of the counterpart terminal.

Referring to FIG. 11, the directions server may guide the directions to the video call screen of the terminal 110b and provide a messaging environment 1100 at the same time. As the video call is performed to the terminal 110b and the counterpart terminal 110a, the directions information may be guided on the video call screen 600 of the terminal 110b according to the location information of the terminal. In this case, a user interface for inputting a message 1110 on the video call screen 600 of the terminal 110b and the counterpart terminal 110a may be provided. For example, as the message 1110 of'When will you arrive?' is input on the video call screen of the other terminal 110a, the video call screen of the terminal 110b and the video call screen of the other terminal 110a are displayed. A message 1110 input from 110a may be displayed. The terminal 110b may input a response message to the message 1110 input from the counterpart terminal 110a through the video call screen of the terminal. A response message input on the video call screen of the terminal 110b may be displayed on the video call screen of the terminal 110b and the counterpart terminal 110a. In this way, a message 1110 can be transmitted and received at the same time as directions are provided through the video call screens of the terminals.

12 is a flowchart illustrating a method of providing directions information according to an exemplary embodiment.

A method of guiding directions information through operations of the first terminal 1210, the second terminal 1230, and the server 100 will be described.

The first terminal 1210 may request a video call to the second terminal 1230. The second terminal 1230 may accept the video call in response to the video call request requested from the first terminal 1210. Through this, a video call call may be connected between the first terminal 1210 and the second terminal 1230 to perform a video call.

When the first terminal 1210 and the second terminal 1230 are making a video call (1211), the video call screen of the first terminal 1210 is used to obtain image data input through the rear camera of the first terminal 1210. I can. Here, the rear camera mode is described as an example among at least one camera mode operating in the first terminal 1210 (eg, a front camera mode for executing a front camera or a rear camera mode for executing a rear camera). However, it is not limited to the rear camera mode, and any video data inputted during a video call 1211 from the first terminal 1210 may be used. In this case, the image data may be image data based on the current location of the terminal. The first terminal 1210 may transmit image data input to the video call screen of the first terminal to the second terminal 1230 (1212). In this case, in FIG. 12, although it is described that a path is queried from the first terminal 1210, it is not limited to the first terminal 1210.

The second terminal 1230 may receive image data input to the video call screen of the first terminal. In this case, for example, the server 100 may determine the location information of the first terminal as the video data input to the video call screen of the first terminal 1210 is received (1213). As another example, the server 100 may determine location information of the first terminal 1210 by referring to GPS information of the first terminal 1210. As another example, the server 100 may determine the location information of the first terminal by receiving GPS information of the first terminal 1210 and image data input to the video call screen of the first terminal 1210. . The server 100 may transmit map information including current location information of the first terminal to the second terminal 1230 (1214). Accordingly, the image data and map information transmitted from the first terminal 1210 may be displayed on the video call screen of the second terminal 1230 (1215).

The second terminal 1230 may input instruction information for directions based on the image data (1216). The second terminal 1230 may transmit the instruction information input to the image data to the first terminal 1210. The first terminal 1210 may receive the instruction information input from the second terminal 1230. In this case, the server 100 may obtain additional information based on the instruction information input from the second terminal 1230 (1218). For example, the server 100 may acquire additional information through a separate server based on the indication information. Alternatively, the server 100 may generate additional information by extracting information stored in the database based on the instruction information. The server 100 may transmit the acquired additional information to the first terminal 1210 (1219).

The first terminal 1210 generates directions information by mapping the instruction information received from the second terminal 1230 and the additional information received from the server 100 to the image data currently displayed on the first terminal 1210 Can do it (1220). For example, the first terminal 1210 superimposes the layer including the indication information and the layer including the additional information on the image data based on the location where the indication information is input and the location information of the first terminal 1210, respectively. Can be mapped. In this case, route guidance information based on augmented reality may be generated.

The first terminal 1210 may guide the directions information according to the location information (1221). For example, as the user of the first terminal 1210 moves, directions information according to the location information of the first terminal 1210 is guided.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, such as one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. Further, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or, to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description to those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and those equivalent to the claims also fall within the scope of the claims to be described later.

Claims (22)

In a computer program stored in a computer-readable storage medium to execute the directions method,
The directions above are:
Transmitting image data input from the rear camera to a counterpart terminal according to the rear camera mode of a front camera mode using a front camera or a rear camera mode using a rear camera operated during a video call-the video data is Including an actual road or an actual building photographed through the rear camera -;
The instruction information input on the displayed video data through a user interface provided on the counterpart terminal from the counterpart terminal in which the video data is displayed on the video call screen, together with the location where the instruction information is input on the video call screen Receiving;
Mapping the layer including the indication information to the currently input video data by mapping the indication information to the video data currently input through the rear camera based on the location where the indication information is input on the video call screen. Generating directions information;
Generating guide information related to the directions information according to the current location information of the terminal; And
Displaying the directions information and the guide information on the video call screen of the terminal
Including,
The transmitting step,
Transmitting location information of the terminal when the image data is input to the counterpart terminal
Including,
The receiving step,
A user interface including map information on which the location information of the terminal is displayed and direction information for up, down, left, and right are superimposed on the image data and displayed on the video call screen together with the image data from the counterpart terminal. The input through touching the user interface on the image data Receiving instruction information
Including, a computer program stored in a computer-readable storage medium. The method of claim 1,
The front camera mode is switched to the rear camera mode as a route is queried to the counterpart terminal through audio data during the video call, and the input image data is input through the converted rear camera mode. program. delete delete delete delete The method of claim 1,
The directions above are:
Further providing a user interface for inputting a message to be transmitted to the counterpart terminal on the video call screen
Further comprising a computer program. The method of claim 1,
The directions above are:
Providing a moving path of a terminal moving according to the directions information on the video call screen, and generating an alarm related to the departure when it is detected that departure of the terminal occurs based on the directions information
Further comprising a computer program. In the way guidance method implemented by a computer,
Transmitting image data input from the rear camera to a counterpart terminal according to the rear camera mode of a front camera mode using a front camera or a rear camera mode using a rear camera operated during a video call-the video data is Including an actual road or an actual building photographed through the rear camera -;
The instruction information input on the displayed video data through a user interface provided on the counterpart terminal from the counterpart terminal in which the video data is displayed on the video call screen, together with the location where the instruction information is input on the video call screen Receiving;
Mapping the layer including the indication information to the currently input video data by mapping the indication information to the video data currently input through the rear camera based on the location where the indication information is input on the video call screen. Generating directions information;
Generating guide information related to the directions information according to the current location information of the terminal; And
Displaying the directions information and the guide information on the video call screen of the terminal
Including,
The transmitting step,
Transmitting location information of the terminal when the image data is input to the counterpart terminal
Including,
The receiving step,
A user interface including map information on which the location information of the terminal is displayed and direction information for up, down, left, and right are superimposed on the image data and displayed on the video call screen together with the image data from the counterpart terminal. The input through touching the user interface on the image data Receiving instruction information
Including, directions method. The method of claim 9,
The transmitting step,
The front camera mode is switched to the rear camera mode as a route is queried to the counterpart terminal through audio data during the video call, and the input video data is input through the converted rear camera mode. Guidance method. delete delete delete delete The method of claim 9,
The directions above are:
Further providing a user interface for inputting a message to be transmitted to the counterpart terminal on the video call screen
Including more, directions method. The method of claim 9,
The directions above are:
Providing a moving path of a terminal moving according to the directions information on the video call screen, and generating an alarm related to the departure when it is detected that departure of the terminal occurs based on the directions information
Including more, directions method. In the directions server,
A location information determination unit that determines current location information of the terminal during a video call;
Video data input from the rear camera according to the rear camera mode using the rear camera of the terminal-the video data includes an actual road or an actual building photographed through the rear camera-on the video call screen of the other terminal Instruction information input by touching the user interface on image data displayed on the counterpart terminal from the counterpart terminal when displayed together with a user interface including direction information for up, down, left, and right An additional information acquisition unit for receiving additional information related to the received indication information by generating guide information based on the current location information and the indication information of the terminal; And
Additional information providing unit for providing additional information related to the obtained indication information to the terminal
Including,
The additional information providing unit,
The instruction information so that a layer including additional information related to the instruction information is generated, and the layer including the generated additional information is mapped to image data input through the rear camera to generate augmented reality-based route guidance information Transmitting a layer including additional information associated with the terminal to the terminal,
The directions server,
Transmitting location information of the terminal when the image data is input from the rear camera to the counterpart terminal,
Map information on which the location information of the terminal is displayed is superimposed on the image data displayed by the counterpart terminal to display the map information on the video call screen of the counterpart terminal together with the video data displayed by the counterpart terminal. Map information delivery unit to deliver to the video call screen of
Further comprising a, directions server. delete The method of claim 17,
The location information determination unit,
Determine whether the terminal is indoors or outdoors based on image data including the actual road or real building, and estimate the current location information of the terminal from text information or image information included in the building or road. server. The method of claim 17,
The additional information providing unit,
A route guidance server configured to generate a route message by setting a departure range for detecting departure of the terminal, and determining whether the terminal is leaving based on the set departure range. delete The method of claim 17,
The map information transmission unit,
A route guidance server that provides a moving route according to the location information of the terminal to the video call screen of the counterpart terminal based on route guidance information displayed on the video call screen of the terminal.

Images