KR20170099234A - Device and method for providing augmented reality contents - Google Patents

Device and method for providing augmented reality contents Download PDF

Info

Publication number
KR20170099234A
KR20170099234A KR1020160021356A KR20160021356A KR20170099234A KR 20170099234 A KR20170099234 A KR 20170099234A KR 1020160021356 A KR1020160021356 A KR 1020160021356A KR 20160021356 A KR20160021356 A KR 20160021356A KR 20170099234 A KR20170099234 A KR 20170099234A
Authority
KR
South Korea
Prior art keywords
augmented reality
content
participant
camera
program
Prior art date
Application number
KR1020160021356A
Other languages
Korean (ko)
Inventor
서상현
Original Assignee
한국전자통신연구원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 한국전자통신연구원 filed Critical 한국전자통신연구원
Priority to KR1020160021356A priority Critical patent/KR20170099234A/en
Publication of KR20170099234A publication Critical patent/KR20170099234A/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Tourism & Hospitality (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Economics (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A device for providing augmented reality contents according to the present invention includes: a position measurement module for measuring position information of a camera; A video input module receiving an image photographed by the camera; A communication module for transmitting and receiving data with a content providing server and one or more participant devices; A memory for storing a program for executing the augmented reality contents; And a processor for executing the program, wherein the processor executes the program, receives the content from the content providing server via the communication module, associates the captured video with the content, Recognizes a user of the at least one participant device included in the photographed image based on a body region tracking technique, and displays the recognized user on the play space.

Figure P1020160021356

Description

[0001] DEVICE AND METHOD FOR PROVIDING AUGMENTED REALITY CONTENTS [0002]

The present invention relates to augmented reality content providing device and method.

In the case of the conventional augmented reality technology, images are often served in the form of using images of specific types of images or landmarks as markers. Particularly, due to space constraints, it is difficult for a large number of participants to simultaneously participate in one augmented reality contents There was a problem.

However, due to the emergence of augmented reality markers based on SLAM (Simultaneous Localization and Mapping), the service space of the augmented reality contents is expanded and the technology is developed so that a large number of participants can participate in the verb in one space, More augmented reality services became possible.

Particularly, in case of multiplayer game using the augmented reality technology, participation rate and immersion feeling can be increased by distinguishing real world users participating in the game and users who are not participating in the game but displayed on the screen.

However, most of the technologies map the user 's position by mapping the GPS position and map measured from the smart device. That is, in the related art, since the location information of a participant is displayed on a game map or a real map in a separate window in the form of a point, there is a problem that the user can not intuitively distinguish the participant from the non-participant.

In this regard, Korean Registered Patent No. 10-2014-0060166 (entitled " Method and Apparatus for Playing Augmented Reality Exhibition Contents Using a Portable Terminal ") tracks a background image constituting an exhibition content, Discloses a method and apparatus for displaying augmented reality display contents using a portable terminal capable of reproducing an enhanced screen on a user portable terminal screen by mixing a virtual object image and a virtual object image and performing an interactive operation according to a user's action.

The embodiment of the present invention constructs a play space using the SLAM technique and the point cloud technique and tracks the position of the camera in real time based on the SLAM technique and the point cloud technique so that the position of the camera of the user participating in the play space can be precisely tracked And a method for providing the augmented reality contents.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to a first aspect of the present invention, there is provided a device for providing augmented reality contents, comprising: a position measurement module for measuring position information of a camera; A video input module receiving an image photographed by the camera; A communication module for transmitting and receiving data with a content providing server and one or more participant devices; A memory for storing a program for executing the augmented reality contents; And a processor for executing the program.

At this time, as the program is executed, the processor receives content from the content providing server through the communication module, recognizes a play space by associating the captured video with the content, ) Technique for recognizing a user of the at least one participant device included in the photographed image and displaying the recognized user on the play space.

According to a second aspect of the present invention, there is provided a method of providing an augmented reality content in a augmented reality content providing device, comprising: receiving a content from a content providing server; Recognizing a play space by associating an image photographed by a camera with the content; Recognizing a user of one or more participant devices included in the photographed image based on a body region tracking technique; And displaying the recognized user on the play space.

In this case, the step of recognizing the play space recognizes the play space based on the position information of the camera, and the position information of the camera is calculated in a three-dimensional coordinate system based on the local coordinate system of the play space.

According to any one of the above objects of the present invention, state information of a user participating in an augmented reality content can be more accurately reproduced by using camera position information of another user projected in two dimensions and an image captured by a camera Can be displayed.

1 is a view for explaining an augmented reality contents providing system according to an embodiment of the present invention.
2 is a block diagram of an augmented reality content providing device according to an embodiment of the present invention.
3A is an exemplary diagram illustrating a process of recognizing a participant by the augmented reality content providing device according to an embodiment of the present invention.
FIG. 3B is a diagram illustrating a process of displaying a participant who is recognized by an augmented reality content providing device according to an exemplary embodiment of the present invention.
4 is a flowchart of a method of providing an augmented reality contents according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like numbers refer to like elements throughout.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions in the embodiments of the present invention, which may vary depending on the intention of the user, the intention or the custom of the operator. Therefore, the definition should be based on the contents throughout this specification.

Hereinafter, a device and a method for providing an augmented reality content according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view for explaining an augmented reality contents providing system according to an embodiment of the present invention.

The augmented reality content providing system 1 according to an embodiment of the present invention includes an augmented reality content providing device 100, one or more participant devices 200, and a content providing server 300. [

At this time, the constituent elements of the augmented reality contents providing system 1 shown in Fig. 1 can be connected through a network. The network refers to a connection structure in which information can be exchanged between each node such as terminals and servers. One example of such a network is a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) , A WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth (Bluetooth), a wireless LAN (Local Area Network) Networks, satellite broadcast networks, analog broadcast networks, Digital Multimedia Broadcasting (DMB) networks, and the like.

The augmented reality contents providing device 100 recognizes the play space by associating the content provided from the content providing server 300 with the image photographed by the camera 10 and displays the position of the participant device 200 located nearby And display the participant corresponding to the participant device 200 on the play space.

At this time, the augmented reality contents providing device 100 may divide the participant and the non-participant to visualize the progress information of the content together and provide it to the user.

The contents providing server 300 provides the augmented reality contents to the augmented reality contents providing device 100 and the participant device 200. [

At this time, the augmented reality content is a content such as a game, a map, etc. composed of at least one of a moving image, an image and a sound, and the type thereof is not limited to this, and various contents providing the augmented reality may be applicable.

The participant device 200 is the same as the augmented reality content providing device 100 and receives the augmented reality content from the content providing server 300 and provides it to the user.

The augmented reality contents providing device 100 and the participant device 200 are connected to a wireless communication device that is guaranteed to be portable and mobility, that is, a PCS (Personal Communication System), a GSM (Global System for Mobile communication), a PDC ), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication) -2000, CDMA (Code Division Multiple Access) -2000, W-CDMA A broadband Internet terminal, and the like.

Hereinafter, the augmented reality contents providing device 100 according to an embodiment of the present invention will be described with reference to Figs. 2, 3a and 3b.

2 is a block diagram of an augmented reality content providing device according to an embodiment of the present invention.

The augmented reality contents providing device 100 according to an exemplary embodiment of the present invention includes a position measurement module 110, a video input module 120, a communication module 130, a memory 140, and a processor 150.

The position measurement module 110 measures the position information of the camera 10. Such a position measurement module 110 may be, for example, a Global Positioning System (GPS), and may further include various means for measuring a position.

The image input module 120 receives an image photographed by the camera 10.

Meanwhile, the augmented reality contents providing device 100 according to an exemplary embodiment of the present invention may receive an image captured by the camera 10, which is a separate component, have. That is, the augmented reality contents providing device 100 according to an embodiment of the present invention may be a device such as a smart phone having the camera 10.

The communication module 130 transmits / receives data to / from the contents providing server 300 that provides the augmented reality contents, and transmits / receives data to / from one or more participant devices 200.

The communication module 130 may include both a wired communication module and a wireless communication module. The wired communication module may be implemented by a power line communication device, a telephone line communication device, a cable home (MoCA), an Ethernet, an IEEE1294, an integrated wired home network, and an RS-485 control device.

In addition, the wireless communication module can be implemented with a wireless LAN (WLAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60 GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology.

In the memory 140, a program for executing the augmented reality contents is stored. Herein, the memory 140 is collectively referred to as a nonvolatile storage device and a volatile storage device which keep the stored information even when power is not supplied.

For example, the memory 140 may be a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid- A magnetic computer storage device such as a NAND flash memory, a hard disk drive (HDD) and the like, and an optical disc drive such as a CD-ROM, a DVD-ROM, etc. .

The processor 150 executes the program stored in the memory 140. [ The processor 150 receives the augmented reality content from the content providing server 300 through the communication module 130 as the program is executed.

Then, the image photographed by the camera 10 is associated with the augmented reality content to recognize the play space.

At this time, the processor 150 can recognize the play space through the SLAM-based camera 10 tracking technique based on the position information of the camera 10. [ Here, the positional information of the camera 10 can be calculated in a three-dimensional coordinate system based on the local coordinate system of the play space. The positional information of the camera 10 calculated by the three-dimensional coordinate system may be defined as a matrix including information on the position and orientation.

When the player 150 recognizes the play space according to the location information of the camera 10, the processor 150 can share the location information of the one or more participant devices 200 and the camera 10 through the communication module 130 .

At this time, the processor 150 may share the progress information of the content with one or more participant devices 200 as well. Here, the progress information of the content may be, for example, game progress information related to enemy / friend, life force, role, level and the like in the case of a game.

The processor 150 then recognizes participants of one or more participant devices 200 included in the photographed image based on body region tracking techniques.

At this time, the human body region tracking technique can be performed based on the position information of the camera 10. [ That is, the human body region tracking technique can be performed based on the current position information of the camera 10, the direction of change between the current position and the position of the camera 10 corresponding to the previous frame, and information on the amount of change.

The augmented reality contents providing device 100 according to an embodiment of the present invention can more accurately and quickly track the human body region by using the human body region tracking technique.

After recognizing the participant based on the human body area tracking technique, the processor 150 displays the recognized participant on the play space.

FIG. 3A is an exemplary diagram illustrating a process of recognizing a participant by the augmented reality content providing device according to an exemplary embodiment of the present invention. FIG. FIG. 3 is a diagram illustrating an example of a process of displaying a display screen; FIG.

The augmented reality contents providing device 100 according to an embodiment of the present invention can project progress information of a recognized participant and contents in two dimensions and display it on the screen.

For example, as shown in FIG. 3A, a first participant P1, a second participant P2, a third participant P3, a fourth participant P4, and a fifth participant P5 are connected to a real object A) is taken and participated.

In this case, the photographed image and the content received from the content providing server 300 correspond to the device of the second participant P2, so that the photographed reality object A as shown in FIG. 3B is augmented reality content A ').

In this case, in the case of the third participant P3 and the fourth participant P4 included in the photographed image, the processor 150 sets a bounding box corresponding to the motion of the participant recognized by the human body tracking technique Can be calculated and displayed.

The processor 150 may also display progress information of the content corresponding to the participant. Here, the progress information of the content may be, for example, game progress information related to enemy / ally, vitality, role, level and the like in the case of a game as described above.

In addition, the processor 150 may display the progress information of the content through a marker such as an exclamation mark, a question mark, or the like. For example, when the content is a game, the ally can be displayed as an exclamation mark, and the enemy group can be displayed as a marker such as a question mark. Accordingly, an embodiment of the present invention enables guidance of progress information of a content to a user in a more detailed manner.

Meanwhile, the processor 150 may display the location information of the user using the upper, lower, right, and left boundaries of the screen in the case of a participant who is off the screen of the augmented reality contents providing device 100. [

As shown in FIG. 3B, in the case of the fifth participant P5, it is possible to display that the fifth participant P5 participates using the boundary of the screen, which is a user not included in the photographed image.

In this case, the size of the labeling of the fifth participant P5 is determined by the three-dimensional coordinate system of the camera 10 of the second participant P2 and the three-dimensional coordinate system of the camera 10 included in the device of the fifth participant P5 Based on the relative distance values.

Also, in case of the participant included in the photographed image, the processor 150 can receive the position information of the camera 10 of each participant through the communication module 130 and display it.

3B, the augmented reality contents providing device 100 of the second participant P2 transmits the three-dimensional coordinate system of the third participant P3 and the fourth participant P4 via the communication module 130, The position information of the camera 10 can be received and projected in two dimensions and displayed on the screen.

2 may be implemented in hardware such as software or an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and may perform predetermined roles can do.

However, 'components' are not meant to be limited to software or hardware, and each component may be configured to reside on an addressable storage medium and configured to play one or more processors.

Thus, by way of example, an element may comprise components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, Routines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

The components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

Hereinafter, a method of providing an augmented reality content in the augmented reality content providing device 100 according to an embodiment of the present invention will be described with reference to Fig.

4 is a flowchart of a method of providing an augmented reality contents according to an embodiment of the present invention.

The method of providing an augmented reality contents according to an exemplary embodiment of the present invention firstly receives a content from a content providing server 300 in step S410 and transmits the video captured by the camera 10 and the content received from the content providing server 300 And recognizes the play space by associating the content (S420).

At this time, the step of recognizing the play space can be recognized based on the position information of the camera 10 calculated in the three-dimensional coordinate system based on the local coordinate system of the play space.

Next, the augmented reality content providing device 100 recognizes a user of one or more participant devices 200 included in the photographed image based on a body region tracking technique (S430) (S440).

On the other hand, in the above description, steps S410 to S440 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. Also, some of the steps may be omitted as necessary, and the order between the steps may be changed.

In addition, the contents already described with respect to the augmented reality contents providing system 1 and the device 100 in Figs. 1 to 3 apply to the augmented reality contents providing method in Fig. 4, even if other contents are omitted.

According to the embodiment of the present invention, by using the image of the real space (outdoor or indoor) photographed by the camera 10 and the position information of the camera 10 of another user projected in two dimensions, The status information of the user participating in the contents can be displayed more precisely.

In addition, it is possible to visualize the information of the participant participating in the augmented reality contents in real time, and it becomes possible to enable participants to participate more intuitively, such as the collaboration between the participants participating in the contents becomes possible.

The augmented reality contents providing method in the augmented reality contents providing device 100 according to an embodiment of the present invention may be performed in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by the computer Can be implemented.

Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media.

In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not to be limited to the details thereof, And various alternatives, modifications, and alterations can be made within a range.

Therefore, the embodiments described in the present invention and the accompanying drawings are intended to illustrate rather than limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and accompanying drawings . The scope of protection of the present invention should be construed according to the claims, and all technical ideas within the scope of equivalents should be interpreted as being included in the scope of the present invention.

10: Camera
100: augmented reality contents providing device
110: Position measurement module
120: image input module
130: Communication module
140: Memory
150: Processor
200: participant device
300: Content providing server

Claims (1)

A device for providing augmented reality content,
A position measurement module for measuring the position information of the camera;
A video input module receiving an image photographed by the camera;
A communication module for transmitting and receiving data with a content providing server and one or more participant devices;
A memory for storing a program for executing the augmented reality contents; And
And a processor for executing the program,
The processor executes the program, receives the content from the content providing server through the communication module, recognizes the play space by associating the photographed video with the content, and performs a human body tracking Recognizes a user of the at least one participant device included in the photographed image, and displays the recognized user on the play space.
KR1020160021356A 2016-02-23 2016-02-23 Device and method for providing augmented reality contents KR20170099234A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160021356A KR20170099234A (en) 2016-02-23 2016-02-23 Device and method for providing augmented reality contents

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160021356A KR20170099234A (en) 2016-02-23 2016-02-23 Device and method for providing augmented reality contents

Publications (1)

Publication Number Publication Date
KR20170099234A true KR20170099234A (en) 2017-08-31

Family

ID=59761152

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160021356A KR20170099234A (en) 2016-02-23 2016-02-23 Device and method for providing augmented reality contents

Country Status (1)

Country Link
KR (1) KR20170099234A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020231215A1 (en) * 2019-05-16 2020-11-19 주식회사 삼십구도씨 Method, system, and non-transitory computer-readable recording medium for providing content comprising augmented reality object by using plurality of devices

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020231215A1 (en) * 2019-05-16 2020-11-19 주식회사 삼십구도씨 Method, system, and non-transitory computer-readable recording medium for providing content comprising augmented reality object by using plurality of devices

Similar Documents

Publication Publication Date Title
CN106982387B (en) Bullet screen display and push method and device and bullet screen application system
CN106803966B (en) Multi-user network live broadcast method and device and electronic equipment thereof
US9566517B2 (en) System and method for visualizing synthetic objects within real-world video clip
US11113887B2 (en) Generating three-dimensional content from two-dimensional images
CN103116451B (en) A kind of virtual character interactive of intelligent terminal, device and system
US20190143193A1 (en) System for providing a virtual exercise place
US20170103664A1 (en) Method and System for Active Learning
CN106850528A (en) Media system and method
CN112312111A (en) Virtual image display method and device, electronic equipment and storage medium
CN112862935A (en) Game character motion processing method and device, storage medium and computer equipment
CN105553821A (en) Game battle matching method and device
CN112933606A (en) Game scene conversion method and device, storage medium and computer equipment
US20130332958A1 (en) Method and system for displaying user selectable picture
US11854130B2 (en) Methods, apparatus, systems, devices, and computer program products for augmenting reality in connection with real world places
CN111833457A (en) Image processing method, apparatus and storage medium
CN107333144A (en) Multichannel picture display process and device based on football race broadcast relay system
CN107564106A (en) Motor activity control method, control device and equipment based on augmented reality
KR20170099234A (en) Device and method for providing augmented reality contents
CN111899349B (en) Model presentation method and device, electronic equipment and computer storage medium
WO2024012405A1 (en) Calibration method and apparatus
CN112788443B (en) Interaction method and system based on optical communication device
US9843642B2 (en) Geo-referencing media content
KR101685505B1 (en) Method for generating 3d image by user's participation and system and method for edutainments service using the same
KR102031760B1 (en) Bumper Car Service Method Using Real-Time Indoor Positioning System and System Therefor
CN107404661A (en) A kind of micro- film video acquisition methods and device