KR20190000360A - System sign language and sign language translation service using augmented reality translation application - Google Patents

Abstract

The present invention relates to a signboard and billboard translation service system using an augmented reality translation application to attract foreign tourists. The system comprises: a signboard and a billboard including an augmented reality marker; a user terminal used by a user who receives information on the signboard and the billboard; an information provider terminal used by an information provider who provides information on the signboard and the billboard; a server for matching markers of the signboard and the billboard recognized by the user terminal and the information provider terminal and transmitting relevant information to the user terminal and the information provider terminal; a communication unit for transceiving data among the user terminal, the information provider terminal and the server. The server includes a matching data unit for storing the markers of the signboard and the billboard, and unique numbers assigned to the markers and matching the unique numbers with a unique number to which the data is inputted, and matches the unique numbers assigned to the markers of the signboard and the billboard with the unique number to which the data is inputted and translates the information on the signboard and the billboard into a foreign language to provide to the user terminal.

Description

{System sign language and sign language translation service using augmented reality translation application}

The present invention relates to a sign translation service system using an augmented reality translation application, and it is an object of the present invention to provide an application for registering a signboard or signboard in Hangul as augmented reality markers against foreign tourists so that a foreign visitor can insert a signboard, The present invention relates to a sign translation service system using an augmented reality translation application that translates into a foreign language and provides guidance.

Recently, as the number of foreign visitors in Korea is increasing, various facilities should be improved in order to attract tourists.

 With K-pop leading the way, various cultural contents such as drama movies have attracted the attention of young people in the world, and the number of foreign tourists in Korea has surged.

In the 2000s, the number of foreign tourists has been increasing every year.

Because foreigners have a wide variety of nationalities, it was a bit difficult to communicate in common English.

In places where traditional markets or traditional cultural heritage sites are located, most of the merchants who did not get to know English well than young merchants have always had difficulty in communication.

Although it is possible to communicate with each other using a smartphone, there is a limit to that an older generation can utilize a smartphone.

The mobile terminal has recently become capable of augmented reality system due to the improvement of the computing ability of the mobile device such as the portable terminal and the PDA and the development of the wireless network device.

As the functions of the portable terminal are diversified, the portable terminal is implemented in the form of a multimedia device having a combination of functions such as photographing, video shooting, music or video file playback, game, and broadcast reception. It may be considered to improve the structural and / or software portions of the terminal for supporting and increasing the functionality of such a handheld terminal.

In addition, in order to supplement the problems with reference to the above, it is urgent to introduce a method using augmented reality in which information of signs and signboards is translated and foreign tourists can easily read them without helping signs or signboards themselves.

In order to solve the above problems, the present invention proposes an application in which a signboard or a signboard in Korean is registered as an augmented reality marker against a foreign tourist, and when a foreign visitor shines a signboard or a signboard with a camera equipped with a portable terminal, And to provide a sign translation service system using an augmented reality translation application.

In order to solve the above problems, a signboard and a signboard including an augmented reality marker in a signboard and a signboard translation service system using an augmented reality translation application according to an embodiment of the present invention; A user terminal used by a user who receives information of the signboard and the signboard; An information provider terminal used by an information provider providing information of the signboard and the signboard; A server for transmitting information to the user terminal and the information provider terminal by matching unique numbers assigned to the markers of the signboard and the signboard recognized by the user terminal and the information provider terminal, Wherein the server includes a matching data portion for storing a unique number assigned to a marker and a marker of the signboard and the signboard and for matching the unique number with the inputted unique number, And the information of the signboard and the signboard is translated into a foreign language to the user terminal by matching the unique number assigned to the user and the unique number into which the data is inputted.

The server may further include an additional description data part for storing information of the signs and signboards further input by the information provider terminal.

The user terminal may further include a user camera module; A user control unit including a user augmented reality module for recognizing a signboard and a sign marker using an augmented reality, the information provider terminal comprising: an information provider camera module; And an information provider control unit including an information provider augmented reality module for recognizing signs and sign markers using the augmented reality.

The information provider terminal may further include an input unit connected to the signboard and the marker marker by a unique number and inputting a description or a correction, and the information provider terminal transmits the information to the server.

In addition, the unique number is formed of any one of a bar code, a two-dimensional bar code, and a readable symbol.

(A) executing a signboard and signboard translation application using augmented reality from a user terminal; (b) recognizing the signboard and the signboard marker from the camera module of the user terminal; (c) transmitting the recognized signs and sign markers to a server; (d) recognizing the signboard and the signboard marker in the server and matching the unique number assigned to the marker with the unique number to which the data is input; (e) transmitting translation information and related information of the signboard and signboard matched in the server to a user terminal; (f) receiving and displaying translation information and related information of the sign and signboard on a user terminal.

(1) executing a signboard and signboard translation application using an augmented reality from an information provider terminal; (2) recognizing the signboard and the signboard marker from the camera module of the information provider terminal; (3) transmitting the recognized signboard, sign marker, and unique number to the server; and (4) connecting the information provider terminal to the unique number transmitted from the server, And (5) storing additional descriptions and revisions in the unique numbers assigned to the signs, sign markers, and markers in the server.

In addition, the augmented reality translation application stored in the medium may be provided to communicate with the server to implement the method of any one of claims 6 to 7.

The sign translating application using the augmented reality according to the embodiment of the present invention can easily be guided by foreign tourists themselves without receiving local help.

In addition, it can reduce the time consumption of locals and foreign tourists, and can enhance the sense of accomplishment by locating the desired attractions.

In addition, foreign tourists can receive explanations in addition to signs and signs translations.

FIG. 1 is a block diagram showing the configuration of a sign and billboard translation service system using an augmented reality translation application according to an embodiment of the present invention.
2 is a block diagram showing a configuration of a server according to an embodiment of the present invention.
3 is a block diagram illustrating a configuration of a user terminal according to an embodiment of the present invention.
4 is a block diagram illustrating a configuration of an information provider terminal according to an embodiment of the present invention.
5 is a flowchart illustrating a method of providing a signboard and a signboard translation service using an augmented reality translation application according to an embodiment of the present invention.
6 is a flowchart illustrating a method of adding and modifying information of a signboard and a signboard of an information provider according to an embodiment of the present invention.

Hereinafter, the description of the present invention with reference to the drawings is not limited to a specific embodiment, and various transformations can be applied and various embodiments can be made. It is to be understood that the following description covers all changes, equivalents, and alternatives falling within the spirit and scope of the present invention.

In the following description, the terms first, second, and the like are used to describe various components and are not limited to their own meaning, and are used only for the purpose of distinguishing one component from another component.

Like reference numerals used throughout the specification denote like elements.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. It is also to be understood that the terms " comprising, "" comprising, "or" having ", and the like are intended to designate the presence of stated features, integers, And should not be construed to preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted in an ideal or overly formal sense unless expressly defined in the present application Do not.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 6 attached hereto.

1 is a configuration diagram showing the configuration of a signboard and a signboard 10 translation service system using an augmented reality translation application according to an embodiment of the present invention.

1, a signboard and a signboard 10 translation service using an augmented reality translation application may include a signboard and a signboard 10, a server 20, a user terminal 100, and an information provider terminal 200 .

A communication unit (not shown) for transmitting and receiving data between the user terminal 100 and the information provider terminal 200 and the server 20 may be included.

(10) Augmented reality, which is the core technology of the translation service system, will be described as follows. In the popular cartoon [Dragon Ball] There is a device called 'Scouter' that displays the attack power information, relative distance and location in real time. This is a typical use of augmented reality technology.

Many people are confused with Augmented Reality and Virture Reality. Virtual reality uses a virtual image rather than reality in both objects (background) and environment, while augmented reality is a technology that superimposes a three-dimensional virtual image on a real image or background and displays it as a single image.

Augmented reality is also called 'Mixed Reality', and 'Augmented Reality' was introduced to the world for the first time in 1990 when Boeing, an airplane manufacturer, added a virtual image to the airplane assembly process.

As such, augmented reality and virtual reality seem to be similar, but they are clearly distinguished according to whether they are virtual or real. As a computer game, for example, a virtual reality fighting game is a game in which a 'character substituting for me' confronts a 'virtual enemy' in 'virtual space', whereas augmented reality fighting game is a ' In the form of a confrontation with a virtual enemy. Therefore, the augmented reality is more realistic than the virtual reality.

The communication unit (not shown) may include a user terminal 100, an information provider terminal 200, and an information provider terminal 200. The communication terminal (not shown) And provides a path for information transmission / reception between the servers 20.

A data network that enables data transmission and reception between the user terminal 100 and the information provider terminal 200 and the server 20 to provide the augmented reality service to the signboard and the signboard 10. [

For example, the communication unit (not shown) may be an IP network providing high-capacity data transmission / reception service through an Internet protocol, and may be an ALL IP network that is an IP network structure in which different networks are integrated based on IP. Or a wired communication network, a WiBro belonging to a wireless communication network, or a WiFi.

Next, the signboard and signboard 10 may include augmented reality markers, and markers are given unique numbers.

The unique number stores the sign information and the translation information of the signboard 10, and the sign information and the related information can be both translated and provided to the tourist.

That is, since the unique number is stored in the server 20, translation information and related information are also stored in the server, and the server may include a database unit (not shown) including the translation information and related information.

Therefore, it is possible to translate the information of the signs and signs (10) by the unique number assigned to the markers so that it can be easily read by foreign tourists, and furthermore, the time required for locating oneself without the help of a local person is reduced, There may also be advantages in accordance with.

The configuration of the server 20 will be described in detail with reference to FIG. 2. The server 20 may include a matching data portion 21 and an additional description data portion 22.

In addition, the server may include a database unit (not shown) including the translation information and related information.

The signs and signs 10 may include augmented reality markers. In the case of the augmented reality markers, the marker means that the augmented reality is performed on the target with the target.

For example, in augmented reality applications that do not show any response to a general object, a specific cup image is coded in the application, and when the camera illuminates the image, the application recognizes the image in the application, It works.

In other words, in the present invention, an image of translated content is displayed rather than an action.

Here, the matching data part 21 plays a role of recognizing the marker on the signboard and the signboard 10 through the camera modules 110 and 210 constituted in the user terminal 100 and the information provider terminal 200, And the translation information and the related information are transmitted to the user terminal 100 by matching the unique number of the data in which the data is input.

When the marker on the signboard and the signboard 10 is recognized by the information provider terminal 200, the unique number assigned to the marker and the unique number of the inputted data are matched with each other, Provide a number.

The method of providing the unique number can be performed by the information provider terminal 200 through the authentication process and can be accessed by a unique number so that the description and the correction contents can be inputted.

The authentication method can select one of the information related to the information provider stored in the signboard and the information of the signboard (10), and the information related to the information provider can be authenticated by inputting the name, resident registration number, business registration certificate and the like.

Accordingly, the server 20 may further include an information provider authentication unit (not shown).

Finally, the unique numbers transmitted from the matching data unit 21 are matched with each other, thereby providing translation information and other information to the user terminal 100.

In addition, the server 20 may further include a storage unit for storing matched data.

The storage unit may be a flash memory, a hard disk, a multimedia card, a USB, a RAM, a ROM, an SRAM, or an optical disk.

The description of the additional description data section 22 will be described later in connection with the information provider terminal 200 later.

 The user terminal 100 is mainly used by foreign tourists or general tourists, and is used to receive information on signs and signs 10.

3, the user terminal 100 may include a user camera module 110 and a user control unit 120, and the user control unit 120 may include a user augmented reality module 123. Referring to FIG.

The configuration of the user terminal 100 will be described in detail with reference to FIG.

The user camera module 110 may include a lens, a VCM actuator, an IR cut filter, a sensor, an ISP, and a PCB.

The above-described configuration shows a configuration of a camera module embedded in a general mobile device, and thus a detailed description thereof will be omitted.

The user camera module 110 plays an important role in recognizing the markers included in the signboard and the signboard 10. For example, when the augmented reality application is executed like a general QR code recognition, A method of displaying the translation information and other information in a three-dimensional manner can be applied to the real image by recognizing the markers of the signboard and the signboard 10 as the user's camera module 110 .

In addition, the server 20 may also include a foreign language type button.

The foreign language type button is provided to the user terminal 100, and when the user (foreign tourist) selects his or her native language before or after recognizing the camera, the signs and signs 10 written in Hangul can be translated and provided to the user .

Next, the user and information provider control units 120 and 220 may be configured with user and information provider augmented reality modules 123 and 223.

The user and information provider control units 120 and 220 may include at least one of a broadcast receiving module, a mobile communication module, a wireless Internet module, a short distance communication module, and a location information module in addition to the above- And serves as an overall control to allow the terminal (100) and the information provider terminal (200) to request or receive the server (20).

The user and information provider augmented reality modules 123 and 223 may be configured in the user terminal 100 and the information provider terminal 200 and may be a module in which necessary components for applying the augmented reality principles are integrated.

That is, an augmented reality system that realizes realistic environments by matching virtual objects to real environments is composed of interfaces of the augmented reality browsers and related devices for processing images and virtual objects.

The image recognized through the camera is matched with the virtual object through the image processing in the augmented reality browser.

The matched image information is provided to the user and the information provider through a display device such as the user terminal 100 and the information provider terminal 200.

In addition, the augmenting reality module for performing the image processing for generating a translated language on the basis of the recognized image information includes a user and information provider augmented reality module 123 and 223 for recognizing and tracking a marker providing a translated language A tracking module, a rendering module for creation / removal / movement of the translated language, and a measurement module for measuring the distance between the translated language and the sign and the sign 10 and for checking the interference.

The augmented reality application can be driven by the above-described configurations.

The database unit (not shown) of the server 20 may include information that can be translated into various languages of signs and signs 10 installed at domestic tourist places.

In addition, it can be composed of at least one of position information, image information, and voice information associated with the signs and signs 10, and it is possible to maintain a set of data without duplication.

It may also be data that manages storage.

The database portion (not shown) can provide real-time accessibility, continuous evolution, simultaneous sharing, content-based reference, and data logical independence.

Referring to FIG. 4 showing the configuration of the information provider terminal 200, the configuration and configuration of the information provider terminal 200 are not limited to the user terminal 100, and the information provider terminal 200 may have the same configuration. Therefore, the description of the configuration of the information provider camera module 210 and the information provider control section 220, which is the configuration of the information provider terminal 200, will be omitted.

However, the information provider terminal 200 may include an input unit 227.

The input unit 227 allows the owner of the signboard and the signboard 10, which is the information provider, to directly update or modify the information about the main contents of the signboard and the signboard 10. [

The input method can be input using a keyboard or a touch method configured in the information provider terminal 200, and can be input or stored as an image or voice.

The additional explanation data and the correction contents inputted to the input unit 227 are transmitted to the server 20 and stored as the additional explanation data unit 22 in association with the additional explanation data formed in the server 20, And the identification number of the signboard 10 information data.

A method of providing a signboard and a signboard (10) translation service using an augmented reality translation application according to an embodiment of the present invention can be described with reference to Figs. 5 and 6. Fig.

Referring to FIG. 5, a method for providing a signboard and a signboard 10 translation service using an augmented reality translation application includes the steps of executing a signboard and a signboard translation application using augmented reality from a user terminal (S10) (S30) of recognizing the signboard and the signboard marker, transmitting the recognized signboard and the signboard marker to the server (S30), recognizing the signboard and the signboard marker in the server, (S50) of translating the translated signboard and signboard matched in the server to the user terminal (S50), and transmitting the translation information and related information of the signboard and the signboard to the user terminal (S60) in which information is received and displayed.

In addition, before step S20, the user may further include a step of selecting his / her native language.

The configuration of each step is a list of methods for executing an augmented reality translation application in the user terminal 100.

Referring to FIG. 6, a method of providing a signboard and a signboard 10 translation service using an augmented reality translation application includes a step of executing a signboard and a signboard translation application using an augmented reality from an information provider terminal (S100) (S300). The information provider terminal accesses the unique number transmitted from the server by matching the received identification number with the unique number transmitted from the server (S400) of inputting additional description and correction contents to the server (S400), and further description and correction contents are stored in the unique number assigned to the signboard, the sign marker and the marker in the server (S500) .

The configuration of the above step is a method of translating an augmented reality translation application using a terminal of an information provider as a main body.

Signs and signs using an augmented reality translation application (10) A user and an information provider can easily execute an augmented reality translation application through a translation service providing method.

Step S400 may further include an information provider authentication step before accessing the information provider terminal 200 by a unique number.

The authentication method is omitted in the description of the configuration.

In addition, the identification numbers assigned to the signs and signs 10 and data may be formed of any one of a bar code, a two-dimensional bar code, and a readable symbol.

Step S500

In addition, the configuration and steps of a sign and billboard (10) translation service system and method using an augmented reality translation application according to an embodiment of the present invention describe execution of an augmented reality translation application.

The embodiments of the present invention described above are not implemented only by the apparatus and / or method, but may be implemented through a program for realizing functions corresponding to the configuration of the embodiment of the present invention, a recording medium on which the program is recorded And such an embodiment can be easily implemented by those skilled in the art from the description of the embodiments described above.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

10: Signs and signs
20: Server
21: Matching data section
22: Additional description data section
100: User terminal
110: User camera module
210: Information provider camera module
120:
220: Information provider control unit
123: User Augmented Reality Module
223: Information provider augmented reality module
200: Information provider terminal
227:

Claims (2)

Signs and signs including augmented reality markers;
A user terminal used by a user who receives information of the signboard and the signboard;
An information provider terminal used by an information provider providing information of the signboard and the signboard;
A server for transmitting information to the user terminal and the information provider terminal by matching unique numbers assigned to the markers of the signs and signs recognized by the user terminal and the information provider terminal,
And a communication unit for transmitting and receiving data between the user terminal and the information provider terminal and the server,
The server includes a matching data part for storing a unique number assigned to a marker and a marker of the signboard and the signboard and for matching the unique number with the inputted unique number,
The user terminal
User camera module;
A user control unit including a user augmented reality module for recognizing a signboard and a sign marker using an augmented reality,
The information provider terminal
An information provider camera module;
And an information provider control module including an information provider augmented reality module for recognizing signs and sign markers using the augmented reality,
Wherein the information provider control unit further comprises at least one of a broadcast receiving module, a mobile communication module, a wireless Internet module, a short distance communication module, and a location information module,
The server can select and authenticate one of the information provider related information stored in the signboard and the billboard information, and the information provider related information is information provider authentication that can authenticate by inputting at least one of a name, a resident registration number, Further,
The identification number assigned to the signboard and the marker marker and the unique number to which the data is input are matched to provide the user terminal with the information of the signboard and the signboard translated into a foreign language,
The server further includes a foreign language type button,
The information of the signs and signboards is information that can be translated into various languages of signboards and signboards installed in domestic tourist places, and further includes at least one of location information, image information, and voice information related to signboards and signboards, , ≪ / RTI >
The information provider terminal,
Further comprising an input unit connected with a unique number assigned to the signboard and the signboard marker and capable of inputting description or correction contents using a keyboard or a touch system configured in the information provider terminal or using an image or voice, and,
The server comprises:
Further comprising: an additional description data unit for storing the identification number of the signboard and signboard information data further input by the information provider terminal,
The unique number,
A bar code, a two-dimensional bar code, and a readable symbol.
Recognizing the signboard and the signboard marker from the camera module of the user terminal, transmitting the recognized signboard and the signboard marker to the server, recognizing the signboard and the signboard marker in the server, And transmitting the translation information and the related information of the signboard and the signboard matched in the server to the user terminal, and transmitting the translation information and the related information of the signboard and the signboard to the user terminal Receiving and displaying,
Before transmitting the translation information and related information of the signboard and the signboard matched in the server to the user terminal,
Further comprising receiving a foreign language type through a foreign language type button on the user terminal,
The information of the signs and signboards may be translated into various languages of signboards and signboards installed in domestic tourist places, and the signs and signboard-related information may include at least one of location information, image information, and voice information, Maintains a set of data,
The information provider related information stored in the signboard and the billboard information can be selected and authenticated, and the information provider related information can be authenticated by inputting at least one of the name, the resident registration number, and the business registration certificate to authenticate the information provider Including,
Further comprising the step of storing the signboard and the signboard information data, which are additionally input by the information provider terminal, through a unique number,
The unique number,
A bar code, a two-dimensional bar code, and a readable symbol,
Accessing the information provider terminal with a unique number given to the signboard and the signboard marker, inputting the description or modification contents using a keyboard or a touch method configured in the information provider terminal, or receiving an image or voice input through the information provider terminal Augmented reality translation application stored on media to make it.

Images