KR101136542B1 - Method, terminal device and computer-readable recording medium for providing service for reservation using augmented reality - Google Patents

Abstract

According to an aspect of the present invention, (a) calculating the position information of the portable terminal device, and includes information on the viewing frustum of the portable terminal device corresponding to the input image input by the camera module of the portable terminal device. Calculating posture information, (b) searching for a bookable object included in the viewing frustum with reference to the map data including the location information, and (c) corresponding to the searched bookable object. Displaying an icon together with the input image in the form of Augmented Reality (AR), and (d) when a specific bookable object is selected from the searched bookable objects displayed on the augmented reality, the specific reservation is made. Providing a method comprising providing a user interface for performing a booking related service for a possible object do. According to the present invention, it is convenient because the user can easily access the bookable object provided in the form of augmented reality, and privacy is protected because the reservation related information is automatically deleted in the mobile terminal device. There is an effect that only relevant information can be extracted and transmitted quickly.

Description

METHOOD, TERMINAL DEVICE AND COMPUTER-READABLE RECORDING MEDIUM FOR PROVIDING SERVICE FOR RESERVATION USING AUGMENTED REALITY}

The present invention relates to a method, a terminal device, and a computer-readable recording medium for providing a reservation-related service of a user of a mobile terminal device using augmented reality, and more particularly, based on the position and attitude information of the mobile terminal device. By providing a bookable object such as a hotel around the portable terminal device in the form of Augmented Reality (AR) and by performing a reservation-related service for the user can book the desired object, to improve the user's convenience A method, a terminal device and a computer readable recording medium.

In recent years, with the development of information and communication networks, the sale of goods and the provision of services through wired and wireless communication devices and the Internet are increasing day by day. In particular, if the demand is greater than the supply, the product or service is necessary at the optimal time, or if a large number of participants want to manage the event efficiently by applying for participation in a certain event that is expected, The use of the reservation system (hereinafter referred to as reservation) is becoming commonplace. These reservations are made through mobile phones, personal digital assistants (PDAs), and other telecommunications networks, and are expected to become more active in the future.

However, the conventional mobile based reservation system has the following problems.

First, the conventional mobile-based reservation system has the inconvenience that the user must enter the name of the hotel, restaurant, performance, etc. that you want to make a reservation directly through the mobile terminal and then make a reservation.

Secondly, in the conventional mobile-based reservation system, even after a user accesses a reservation server through a mobile terminal and performs a reservation service, all other reservation information remains on the mobile terminal, so that other users use the mobile terminal. There was a security problem such as the privacy of the user can be exposed.

Third, the conventional mobile-based reservation system proceeds with a reservation service using a standardized web browser included in the mobile terminal. Since the web browser is not a browser designed exclusively for the reservation service, not only the information on the reservation but also the reservation is made. There was a problem that all information that is not related to the service is transmitted to the reservation server. As a result, users who use a reservation service using a mobile terminal have to bear the time delay and inconvenience caused by unnecessary information transmission.

Accordingly, the present applicant can more easily access the bookable object through the portable terminal device from the user's point of view, improve the security of the user's personal information, and support the user to use the reservation related service more quickly. To develop technology.

It is an object of the present invention to solve all the problems described above.

In addition, the present invention, when the user performs a reservation-related services such as reservation, payment, etc. in the terminal device, it is another object to increase the convenience of the user by allowing the user to select a reservation target object on the screen displayed in augmented reality .

Another object of the present invention is to improve personal information security by deleting all the reservation related information in the mobile terminal device after the browser embedded in the terminal device sends the reservation related information to the reservation server. .

In addition, the present invention is to reduce the time required for reservation and payment by allowing the browser embedded in the terminal device not to receive all the general information that can be displayed on the screen, and to receive only the information related to the reservation quickly The purpose.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to an aspect of the present invention, (a) calculating position information of a mobile terminal apparatus using a position information obtaining technique, and corresponding to an input image input by a camera module of the portable terminal apparatus using a sensing technique. Calculating posture information including information on a viewing frustum of the mobile terminal device; (b) searching for a bookable object included in the viewing frustum with reference to the map data including the location information; (c) displaying an icon corresponding to the searched bookable object in the form of Augmented Reality (AR) together with the input image, and (d) the searched bookable object displayed on the augmented reality. If a specific bookable object is selected, the service is used to perform a reservation related service for the specific bookable object. The method comprises the step of providing an interface is provided.

According to another aspect of the present invention, the position information of the portable terminal device is calculated using a position information acquisition technology, and the portable terminal device corresponding to an input image input by the camera module of the portable terminal device using a sensing technology. A position and position calculation unit that calculates attitude information including information about a viewing frustum of the user, and a reservable object that searches for a bookable object included in the viewing frustum with reference to the map data including the location information. A search unit, an augmented reality implementation unit for displaying an icon corresponding to the searched bookable object in the form of Augmented Reality (AR) together with the input image, and the searched bookable object displayed on the augmented reality If a specific bookable object is selected, the specific bookable object is selected. Provided is a terminal device including a reservation service performing unit that provides a user interface for performing a reservation related service.

In addition, there is provided another method, system, and computer readable recording medium for recording a computer program for carrying out the method for implementing the present invention.

According to the present invention, when a user performs a reservation related service such as a reservation or a payment in a mobile portable terminal device, etc., since the user can easily access the bookable object provided in the form of augmented reality, the user's convenience is increased, Even if a reservation is made using the terminal device, the reservation related information is automatically deleted in the portable terminal device, thereby protecting the privacy of the individual, and the browser can extract only the information related to the reservation and receive it quickly. .

1 is a diagram schematically illustrating a configuration of an entire system for providing a reservation related service for a user of a mobile terminal device using augmented reality according to an embodiment of the present invention.
2 is a diagram illustrating an internal configuration of a terminal device 200 according to an embodiment of the present invention.
FIG. 3A illustrates a screen on which a live image (ie, an input image) and information about a bookable object around a location of the terminal device 200 are displayed in the form of augmented reality according to an embodiment of the present invention. 3B is a diagram exemplarily illustrating a screen displaying a user interface for accessing related detailed information when a specific bookable object is selected on the screen of FIG. 3A.
4 is a diagram illustrating a screen for performing a reservation through the reservation service performing unit 250 according to an embodiment of the present invention.

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

DETAILED DESCRIPTION Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

[Preferred Embodiments of the Invention]

Configuration of the entire system

1 is a diagram schematically illustrating a configuration of an entire system for providing a reservation related service for a user of a mobile terminal device using augmented reality according to an embodiment of the present invention.

As shown in FIG. 1, the entire system according to an exemplary embodiment of the present invention may include a communication network 100, a terminal device 200, and an information providing server 300.

First, according to an embodiment of the present invention, the communication network 100 may be configured in any communication mode such as wired and wireless, and may include a mobile communication network, a local area network (LAN), and a metropolitan area network (MAN: Metropolitan). It may be configured with various communication networks such as an area network, a wide area network (WAN), and a satellite communication network. More specifically, the communication network 100 according to the present invention is known World Wide Web (WWW), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA) or Global System for Mobile communications It should be understood that the concept includes all communication networks.

Next, according to an embodiment of the present invention, the terminal device 200 is connected to the input image and the input image input through a photographing means such as a camera (a concept including a portable device equipped with a camera). An augmented reality (AR) image is generated by using related information (ie, information on a reserved object included in the viewing angle of the terminal device 200) and displayed on the screen of the terminal device 200, When the user selects a specific bookable object from among the bookable objects displayed on the screen, a user-friendly browser may be provided to perform a reservation related service for the specific bookable object. The terminal device 200 according to the present invention refers to a digital device including a function for enabling communication after connecting to the communication network 100, and includes a personal computer (for example, a desktop computer, a notebook computer, a tablet). A digital device having a computing capability by mounting a microprocessor such as a computer, a workstation, a PDA, a web pad, a mobile phone, etc. can be adopted as the terminal device 200 according to the present invention. . A detailed internal configuration of the terminal device 200 will be described later.

Meanwhile, according to an embodiment of the present invention, the information providing server 300 may perform a function of providing various types of information in response to a request of the terminal device 200 through the communication network 100. More specifically, the information providing server 300 may include a search engine (not shown) to search for information corresponding to the request of the terminal device 200 and browse the search results by the user of the terminal device 200. You can provide it. For example, the information providing server 300 receives a user information, a desired reservation possible object, information required for reservation and payment, such as a desired reservation time, from the terminal device 200, and a reservation server for performing a service such as reservation and payment. Can be. In addition, the information providing server 300 may include a GPS transmission server for transmitting GPS information indicating the location information of the terminal device 200, or may include an operation server of an Internet search portal site.

Terminal device configuration

Hereinafter, the internal structure of the terminal device 200 performing important functions for the implementation of the present invention and the function of each component will be described.

2 is a diagram illustrating an internal configuration of a terminal device 200 according to an embodiment of the present invention.

2, the terminal device 200 according to an embodiment of the present invention, the input image acquisition unit 210, the position and attitude calculation unit 220, the bookable object search unit 230, augmented reality implementation The unit 240 may include a reservation service execution unit 250, a browser 260, a communication unit 270, and a control unit 280. According to an embodiment of the present invention, the input image acquisition unit 210, the position and attitude calculation unit 220, the bookable object search unit 230, augmented reality implementation unit 240, reservation service performing unit 250 The browser 260, the communication unit 270, and the control unit 280 may be program modules in which at least some of them communicate with the user terminal device 300. Such program modules may be included in the terminal device 200 in the form of an operating system, an application program module, and other program modules, and may be physically stored on various known storage devices. In addition, these program modules may be stored in a remote storage device that can communicate with the terminal device 200. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

First, according to an embodiment of the present invention, the input image acquisition unit 210 may perform a function of acquiring an input image based on the augmented reality implemented by the augmented reality implementation unit 240 to be described later. More specifically, the input image acquisition unit 210 according to an embodiment of the present invention may include a photographing apparatus such as a CCD camera, for example, a preview of the scenery around the user carrying the terminal device 200. function to receive input in real time.

Next, according to an embodiment of the present invention, the position and posture calculating unit 220 may determine which area of the real world the input image acquired by the terminal device 200 corresponds to. A function of calculating the position and posture of the 200 may be performed.

More specifically, the position and attitude calculation unit 220 according to an embodiment of the present invention calculates the position of the terminal device 200 by using a position information acquisition technique such as Global Positioning System (GPS) technology or mobile communication technology. For example, for this purpose, the position and attitude calculation unit 220 may include a predetermined GPS module and a mobile communication module. In addition, the position and posture calculating unit 220 according to an embodiment of the present invention may calculate the posture of the terminal device 200 by using a predetermined sensing means. For example, the position and posture calculating unit 220 is an acceleration sensor for detecting the presence, distance, speed, acceleration, direction, etc. of the movement of the terminal device 200 and the presence, rotation, angular velocity, angular acceleration, direction, etc. of the rotation of the terminal device 200 It can include a gyroscope.

In addition, according to an embodiment of the present invention, the position and posture calculating unit 220 refers to the information on the position, the posture and the viewing angle of the terminal device 200 calculated as described above, I) a function of specifying a field of view of the terminal device 200 corresponding to the field of view of the terminal device 200, that is, the input image acquired by the terminal device 200.

More specifically, according to an embodiment of the present invention, the field of view of the terminal device 200 means a three-dimensional area defined in the real world which is a three-dimensional space, and the terminal device 200 is referred to as a viewpoint. It can be specified with a viewing frustum. Here, the viewing frustum refers to a three-dimensional area included in the field of view of the photographing apparatus when an image is photographed by a photographing apparatus such as a camera or input in a preview state, and is provided in the terminal device 200. The projection center of the photographing means may be defined as a viewpoint, and depending on the type of the photographing lens, an infinite region in the form of a cone or a polygonal cone (a near plane perpendicular to the direction of the eye or a cone or the polygonal cone) or Finite region in the form of a trapezoidal cylinder or trapezoidal polyhedron cut by a far plane.

Next, according to an embodiment of the present invention, the reservation possible object search unit 230 is based on the position and attitude information of the terminal device 200 calculated by the position and attitude calculation unit 220 described above. 200) Search for available reservations around.

More specifically, the bookable object search unit 230 according to an embodiment of the present invention may search for a bookable object by referring to map data centered on the location of the terminal device 200. It is also possible to search only the viewable objects included in the field of view of the terminal device 200, that is, the viewing frustum. Here, the bookable object may be a hotel, a restaurant, a golf course, a theater, a performance hall, or the like. In addition, the map data based on the corresponding location information may be two-dimensional or three-dimensional data, data stored in a memory (not shown) of the terminal device 200, or an external server (not shown) through the communication network 100. It may be received from). In addition, the reservable object search unit 230 may search for detailed information on the reservable object using a search engine (not shown) when searching for the reservable object as described above. In addition, when searching for a bookable object as described above, the bookable object search unit 230 may not provide an object as a search result for an object in which a reservation is full or a reservation deadline is closed and a reservation cannot be made. If provided as a search result, it may indicate that the reservation is not available.

Next, according to an embodiment of the present invention, the augmented reality implementation unit 240 is the input image obtained by the input image acquisition unit 210 and the information on the reservable object retrieved from the reservable object search unit 230 By synthesizing the augmented reality can be implemented through the screen of the terminal device 200. For example, the augmented reality implementation unit 240 according to an embodiment of the present invention may include a bookable object (eg, a hotel, determined to be included in an input image and a viewing frustum of the terminal device 200). Display a predetermined graphic element indicating a point of interest (POI) at a location on the screen of a restaurant, golf course, theater, performance hall, etc., and if at least one of the graphic elements is selected by the user, Detailed information on the point (that is, the subject that can be reserved) may be provided, or a task related to the reservation by the reservation service performing unit 250 which will be described later may be performed.

Here, the graphic element displayed together with the input image according to an embodiment of the present invention may include a brand, a logo of the corresponding bookable object, a pattern landmark of the bookable object, and the like. However, the graphic elements according to the present invention are not necessarily limited to those listed above, and any information that may imply or guide the name, type, or property of the bookable object included in the field of view of the terminal device 200 may be used. It is noted that it can be included in the graphic element according to the invention.

In addition, the augmented reality implementation unit 240 according to an embodiment of the present invention, when the reservation-related service is performed by the reservation service execution unit 250 to be described later, to display a browser 260 necessary to perform the reservation service Can play a role.

FIG. 3A illustrates a screen on which a live image (ie, an input image) and information about a bookable object around a location of the terminal device 200 are displayed in the form of augmented reality according to an embodiment of the present invention. 3B is a diagram exemplarily illustrating a screen displaying a user interface for accessing related detailed information when a specific bookable object is selected on the screen of FIG. 3A.

Referring to FIG. 3A, there is shown a screen in which the Westin Chosun Hotel, the Lotte Hotel, the Shilla Hotel, the Grand Ambassador Hotel, the Grand Hilton Hotel, etc., which are available for reservation, are displayed along with the live-action image around the terminal device 200. In addition, when the user selects the Grand Hilton hotel for hotel reservation on the augmented reality screen of FIG. 3A, detailed information of the Grand Hilton hotel may be accessible as shown in FIG. 3B. If the user actually wants to make a reservation at the Grand Hilton hotel after reading the detailed information of FIG. 3B, the user may click the book button of FIG. 3B to perform a hotel reservation. Reserve operation is performed through.

Meanwhile, according to an embodiment of the present invention, other image information including output image information implemented as augmented reality may be visually displayed through a display unit (not shown) of the terminal device 200. For example, The display unit according to an exemplary embodiment of the present invention may be a flat panel display such as a liquid crystal display (LCD) or an organic light emitting diode (OLED).

Next, according to an embodiment of the present invention, the reservation service performing unit 250 may perform a specific reservation selected by the user among the reservationable objects displayed on the augmented reality screen implemented by the augmented reality implementation unit 240 described above. Can perform reservation-related services for the object.

4 is a diagram illustrating a screen for performing a reservation through the reservation service performing unit 250 according to an embodiment of the present invention.

Referring to FIG. 4, when the user presses the reservation button on the detailed information screen of the Grand Hilton Seoul hotel as shown in FIG. 3B, it can be seen that a user interface for inputting reservation information for the Grand Hilton Seoul hotel is provided. have. For example, referring to FIG. 4, the mini-browser 260 may simply input necessary reservation information such as country, room type, number of rooms, room direction, breakfast availability, check-in, check-out, etc. in which the hotel is located. It can be seen that is provided. In this case, the mini browser 260 may be provided in the form of augmented reality. Here, the browser 260 may serve to transmit the user information and the input reservation information of the terminal device 200 to the reservation server, and automatically transmit the user information and reservation information to the terminal device for personal privacy protection. The operation of deleting within 200 may be performed. For reference, while the general-purpose browser used for web browsing on the terminal device transmits all the information not related to the reservation to the terminal device 200, the browser 260 according to the embodiment of the present invention only the information related to the reservation. It may be transmitted to the terminal device 200. Through this, the browser 260 according to an embodiment of the present invention may perform a task such as reservation and payment faster than the general-purpose browser.

Next, according to an embodiment of the present invention, the communication unit 270 performs a function to enable the terminal device 200 to communicate with an external system such as the information providing server 300.

Finally, according to an embodiment of the present invention, the control unit 280 is the input image acquisition unit 210, the position and attitude calculation unit 220, the reservation object search unit 230, augmented reality implementation unit 240 In addition, the reservation service performing unit 250 performs a function of controlling the flow of data between the browser 260 and the communication unit 270. That is, the controller 280 controls the flow of data from the outside or between the respective components of the terminal device 200, such that the input image acquisition unit 210, the position and attitude calculation unit 220, and the reservationable object search unit are provided. The control unit 230, the augmented reality implementation unit 240, the reservation service execution unit 250, the browser 260, and the communication unit 270 each control to perform a unique function.

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the limited embodiments and drawings, it is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the appended claims, fall within the scope of the spirit of the present invention. I will say.

100: Network
200: terminal device
210: input image acquisition unit
220: position and posture calculation unit
230: Reservation object searchable part
240: augmented reality implementation unit
250: reservation service execution unit
260: browser
270: communication unit
280: control unit
300: informational server

Claims (12)

(a) calculating position information of the mobile terminal apparatus using a position information acquisition technique; Calculating posture information including information about the posture;
(b) searching for a bookable object included in the viewing frustum with reference to the map data including the location information;
(c) displaying an icon corresponding to the searched bookable object in the form of Augmented Reality (AR) together with the input image, and
(d) providing a user interface for performing a reservation related service for the specific bookable object when a specific bookable object is selected from the searched bookable objects displayed on the augmented reality;
Including,
In step (d),
(d1) transmitting user information and reservation information of the portable terminal device from the portable terminal device to a reservation server performing the reservation related service; and
(d2) deleting the user information and the reservation information in the portable terminal device;
Method comprising a. The method of claim 1,
In the step (a),
The location information is calculated using a GPS receiver included in the portable terminal device, and the attitude information is calculated using at least one of an acceleration sensor and a gyroscope. The method of claim 1,
In step (b),
And retrieving a bookable object that is not included in the viewing frustum and exists within a predetermined distance from the location of the portable terminal device. The method of claim 3,
In step (c),
And displaying the information on the additionally searchable bookable object together with the input image in the form of augmented reality. The method of claim 1,
In step (c),
And display the icon in the form of augmented reality at a location on the screen of the corresponding reservable object. delete The method of claim 1,
Step (d1),
Is performed by a browser included in the portable terminal,
And the browser transmits only information related to the reservation related service. The method of claim 7, wherein
And the browser is provided in the form of augmented reality. The method of claim 1,
The icon is
And a graphic element including at least one of a brand, a logo, and a pattern landmark of the bookable object. The method of claim 1,
The bookable object,
A method comprising at least one of a hotel, a restaurant, a golf course, a theater, a performance hall. The location information of the mobile terminal device is calculated using the location information acquisition technology, and the information about the viewing frustum of the mobile terminal device corresponding to the input image input by the camera module of the mobile terminal device is sensed using the sensing technology. Position and posture calculation unit for calculating the posture information to include,
A bookable object search unit for searching for a bookable object included in the viewing frustum with reference to the map data including the location information;
An augmented reality implementation unit for displaying an icon corresponding to the searched bookable object in the form of augmented reality (AR) together with the input image;
A reservation service performing unit that provides a user interface for performing a reservation related service for the specific bookable object when a specific bookable object is selected from the searched bookable objects displayed on the augmented reality.
Including,
The reservation service performing unit,
And transmitting the user information and the reservation information of the portable terminal device from the portable terminal device to a reservation server that performs the reservation related service, and deleting the user information and the reservation information from the portable terminal device. . A computer readable recording medium having recorded thereon a computer program for executing the method according to any one of claims 1 to 5 and 7 to 10.

Images