KR101572346B1 - Service system and service method for augmented reality stage, live dance stage and live audition - Google Patents

Abstract

A service system and method for augmented reality stage, live dance stage, and live audition are disclosed. A system for providing services through a space for entertainment comprises at least one three-dimensional projector for projecting at least one three-dimensional object on the space, at least one motion sensor for recognizing movement in the space, And a processor for controlling the at least one three-dimensional projector to project an object onto the space, and calculating a score based on the operation of the user recognized through the at least one motion sensor and the positional information of the three-dimensional object can do.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a service system and a service method for an augmented reality stage, a live dance stage and a live audition,

Embodiments of the present invention relate to augmented reality (AR) stage, live dance stage, and service system and method for live audition.

Augmented reality (AR), a concept that complements the real world with a virtual world, uses a virtual environment created by computer graphics, but the protagonist is a real environment. Computer graphics serve to provide additional information needed for the real world. By overlapping the three-dimensional virtual image on the real image that the user is viewing, it means that the distinction between the real environment and the virtual screen becomes blurred. In addition, chroma-key is a screen synthesis technique for compositing a moving subject on a different screen using color difference, and is often used in programs such as weather forecast, election broadcasting, and history special. In addition to these technologies, there are various techniques using images, such as transmitting generated videos through a network.

However, in the prior art, each of these techniques has existed in a fragmentary manner, and a fusion technique has not been provided for a specific field such as entertainment. In addition, since most existing games use the avatar to play the game, there is a problem that the actual user does not feel like playing the game.

An augmented reality stage that can enhance the user's fun by adding various effects according to the progress of the game or the score according to the song in the surrounding space or the appearance of the user displayed on the screen, , A live dance stage, and a service system and service method for live audition.

A system for providing services through a space for entertainment, comprising: at least one three-dimensional projector for projecting at least one three-dimensional object on the space; At least one motion sensor for recognizing movement within the space; And controlling the at least one three-dimensional projector to project the three-dimensional object onto the space, and calculating a score based on the motion of the user recognized through the at least one motion sensor and the positional information of the three-dimensional object And a processing unit for processing the service information.

According to one aspect, the processing unit controls the at least one three-dimensional projector to further project a three-dimensional background object or a three-dimensional item object at a position associated with the user according to the score, Dimensional projector, or controlling at least one of the three-dimensional projectors to project and change at least one of shape, color and size of the three-dimensional item object.

According to another aspect, the processing unit may control the at least one three-dimensional projector so that the already-projecting three-dimensional background object or three-dimensional item object moves together according to the operation of the recognized user .

According to another aspect, the processing unit controls the at least one three-dimensional projector to project a three-dimensional content object for selection of the content, and recognizes the operation of the user with respect to the projected three- Dimensional projectors to control the at least one three-dimensional projector to project and change the projected three-dimensional content object into another three-dimensional content object, or to recognize the user's selection of the content corresponding to the three- can do.

A method of providing service through a space for entertainment, comprising: controlling at least one three-dimensional projector installed on the space to project a three-dimensional object onto the space; Recognizing an operation of a user using at least one motion sensor that recognizes motion in the space; And calculating the score of the user based on the recognized user's operation and the location information of the 3D object.

A system for providing a service through a space for entertainment, the system comprising: a display unit installed on the space; A photographing unit installed on the space for photographing a user's appearance; A voice input unit installed on the space for receiving voice of the user; At least one motion sensor for recognizing movement within the space; And a control unit that controls the display unit to reproduce a state of the user photographed through the photographing unit, wherein at least one of the voice of the user input through the voice input unit and the operation of the user recognized through the at least one motion sensor And a processing unit for controlling the display unit to add at least one of a background object and an item object at a position associated with a shape of the user to be reproduced and reproduce the additional object.

A method of providing a service through a space for entertainment, comprising the steps of: photographing a user through a photographing unit installed in the space; Controlling a display unit installed on the space to reproduce the photographed state of the user; Receiving voice of the user through a voice input unit installed in the space; Recognizing an operation of the user using at least one motion sensor that recognizes motion within the space; And controlling the display unit to further add at least one object of a background object and an item object at a position associated with a shape of the user to be reproduced based on at least one of the user's voice and the operation of the user A service method is provided.

When a user plays a game or sings a song according to music, various effects can be added to the user's display on the screen or the surrounding space according to the progress of the game or the score according to the song, thereby enhancing the user's enjoyment.

1 is a diagram showing an example of a space configuration for an augmented reality stage and a live dance stage in an embodiment of the present invention.
2 is a diagram illustrating an example in which a user uses a service in an augmented reality stage and a live dance stage in an embodiment of the present invention.
3 is a view for explaining an example of a service utilization process of the augmented reality stage and the live dance stage in an embodiment of the present invention.
4 is a diagram illustrating an example of a space configuration for live audition in an embodiment of the present invention.
5 is a diagram illustrating an example in which a user is provided with a service in a space for live audition in an embodiment of the present invention.
6 is a diagram illustrating an example of a process of using a service for live audition in an embodiment of the present invention.
7 is a block diagram for explaining an example of the internal configuration of a service system for an augmented reality stage and a live dance stage in an embodiment of the present invention.
8 is a flowchart showing an example of a service method for an augmented reality stage and a live dance stage in an embodiment of the present invention.
9 is a block diagram for explaining an example of an internal configuration of a service system for live audition in an embodiment of the present invention.
10 is a flowchart illustrating an example of a service method for live audition, according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention provide a service system and service method for an augmented reality (AR) stage, a live dance stage, and live audition that can be included in an entertainment facility such as a music theme park for domestic / foreign tourists .

1 is a diagram showing an example of a space configuration for an augmented reality stage and a live dance stage in an embodiment of the present invention. 1, the 'L' -shaped space 'may represent the front and bottom surfaces of a space for the augmented reality stage and the live dance stage, and the' 3D projector 'may include a plurality of devices An image can be expressed in a 3D stereoscopic body through a plurality of devices illuminating the image on the front surface and a plurality of devices illuminating the image on the bottom surface. Also, the 'motion sensor' may represent an example of a sensor for recognizing the operation of the user in the space. As described above, in the present embodiment, the augmented reality stage and the live dance stage can mean a space for recognizing a user's operation and providing an image in association with the recognized operation.

For example, in augmented reality stage or live dance stage, a user can select contents such as a desired game or a song to be applied to a game through a user's motion related to at least one image of a bottom surface and a front surface. At this time, the selected content may also be processed according to the user's operation. In one example, a user may perform an action associated with an image associated with at least one of a frontal and a bottom surface, and the service system may calculate the score according to the accuracy of the user's perceived action through the sensor. In this case, the service system can provide the 3D image including the more various effects to the user as the score increases.

If necessary, the content selection may be performed through a key included in the kiosk or a touch screen using a separate kiosk (kiosk) installed inside / outside the space for entertainment.

2 is a diagram illustrating an example in which a user uses a service in an augmented reality stage and a live dance stage in an embodiment of the present invention. In this embodiment, the user wears 3D glasses (for example, 3D shutter glasses) to view the 3D image, and can play a game for performing an operation for touching objects flying to the user in the 3D image. At this time, the service system can calculate the game score by judging whether the user has touched the object correctly, and the higher the score of the game (or the success of touching more objects or successively touching the object) It is possible to provide the user with more various visual effects. For example, the service system may display a 3D item such that a 3D item to decorate a user's surrounding space is projected in an 'L' shaped space, or a 3D item is worn by a user in association with a user's motion It can be projected. In this way, the higher the score of the user's game, the more spectacular the user's peripheral space or the more diverse (or more gorgeous) items directly to the user.

3 is a view for explaining an example of a service utilization process of the augmented reality stage and the live dance stage in an embodiment of the present invention.

Step 1 shows a process of selecting a desired music by the user. For example, the user can touch a user interface displayed as a 3D image on the bottom of the 'L' shaped space 'described with reference to FIG. 1 to select a song desired by the user. At this time, the service system can adjust the image being projected or recognize a song selected by the user, based on the position of the currently projected user interface and the position according to the user's operation. In FIG. 3, although the process 1 is described as an example of selecting a song, it may be a process for selecting a game or contents when various games or various contents are provided.

The process 2 is a process in which a user touches objects flying in order to perform a game and dances. The process of proceeding the game has already been described in detail with reference to FIG.

The process 3 is a process of confirming the score of the game played by the user. The service system can provide the user with not only the current score of the user but also information about the comparison with the game score of the other users, the ranking of the users, and the like.

4 is a diagram illustrating an example of a space configuration for live audition in an embodiment of the present invention. In FIG. 4, 'Chroma-key stage' may mean a space for live audition that can synthesize the original image and the user's view and project it. Such a space may include a screen including a plurality of surfaces for displaying images in various ways such as a 'Screen', a plurality of devices for projecting images on respective surfaces such as 'Projectors', and a 'motion sensor' And may include a sensor for sensing motion. Although not shown in FIG. 4, the space for live audition may further include a voice input unit such as a microphone for receiving a voice of a user, a camera for photographing a user to express the appearance of the user on the screen have.

5 is a diagram illustrating an example in which a user is provided with a service in a space for live audition in an embodiment of the present invention. Unlike the case where the three-dimensional object is displayed to the user in the augmented reality stage and the live dance stage, the live audition captures the user's appearance, displays the captured user's image on the screen, and simultaneously displays the user's voice ) And a score according to the user's action, the user can further display various effects.

Of course, in another embodiment, a live audition may also proceed with a song audition using a three-dimensional object, such as in an augmented reality stage and a live dance stage. In this case, the various effects can be projected directly to the actual user, rather than being displayed on the captured user's shape as shown in FIG.

6 is a diagram illustrating an example of a process of using a service for live audition in an embodiment of the present invention.

 The process 1 may be a process in which the user selects a desired song. At this time, the service system may display a user interface image for selecting a song through a screen installed in a room for live audition. The service system may display information on a plurality of song contents on a screen and provide a function to the user to move or select information on the song contents according to a user's operation. For example, when the user recognizes that the user has moved his hand from left to right via the motion sensor, the service system may move the listed song contents from left to right to change the song content of the target area. As another example, if the user recognizes that the user has moved his hand from top to bottom via the motion sensor, the service system may recognize that the user has selected the song content of the current target area.

The process 2 can be a process in which the user sings and dances. At this time, the service system can calculate the score of the user's song and the score of the user's action. The higher the score of the song and the more the user's action, the more brilliant effects are applied to the captured user's appearance, Can be displayed through. 5, various effects are added to the user's surroundings, and an item such as a wing is added to the user. In addition, various effects can be applied in association with the user's appearance, such as changing the style, costume, etc. of the user and adding a small article or the like.

The process 3 may be a process in which the user confirms the admission and score of the audition. At this time, the service system can provide the user with information such as the user's score, ranking, audition, and so on through the screen. For example, the service system can provide the information to the user by using the image of the famous celebrity photographed in advance.

FIG. 7 is a block diagram for explaining an example of an internal configuration of a service system for an augmented reality stage and a live dance stage in an embodiment of the present invention. In FIG. 8, A stage and a service method for a live dance stage. The service method of FIG. 8 may be performed by the service system 700 of FIG. For example, the steps involved in the service method may be performed by the components that the service system 700 includes. At this time, the service system 700 is a system for providing services through a space for entertainment, and includes at least one three-dimensional projector 710, at least one motion sensor 720, And a processing unit 730. In addition, the service method may include only steps 830 through 850. At this time, at least one of the steps 810, 820, and 860 through 880 may be optionally included in the service method as needed.

In step 810, the processing unit 730 may control at least one three-dimensional projector 710 installed in the space to project a three-dimensional content object for selection of content.

In operation 820, the processing unit 730 recognizes the user's operation on the projected three-dimensional content object, and then converts the projected three-dimensional content object into another three-dimensional content object, 710) or recognize a user's selection of content corresponding to a three-dimensional content object. In addition, the content selection may be performed through a key included in the kiosk or a touch screen using a separate kiosk (kiosk) installed inside / outside the space for entertainment, if necessary.

Examples of the steps 810 to 820 have been described in the process 1 of FIG.

In step 830, the processing unit 730 may control at least one three-dimensional projector 710 to project the three-dimensional object onto space. An example of such at least one three-dimensional projector 710 has been described with reference to the '3D projector' of FIG.

In operation 840, the processing unit 730 may recognize the motion of the user by using at least one motion sensor 720 that recognizes motion in the space. An example of such at least one motion sensor 720 has been described with reference to the 'motion sensor' of FIG.

In step 850, the processing unit 730 may calculate the user's score based on the recognized user's actions and the location information of the 3D object. For example, the processing unit 730 may calculate a user's score according to whether the user has performed a touch on the three-dimensional object or how accurately the three-dimensional object is touched. For example, FIG. 2 illustrates an example of a game in which a plurality of objects move and a user touches a plurality of such three-dimensional objects.

At step 860, the processing unit 730 may control the at least one three-dimensional projector 710 to further project a three-dimensional background object or a three-dimensional item object at a location associated with the user, depending on the score.

At step 870, the processing unit 730 may generate at least one three-dimensional projector 710 to change and project at least one of the shape, color, and size of the already-projecting three-dimensional background object or the three- Can be controlled.

At step 880, the processing unit 730 may control at least one three-dimensional projector 710 to move the already-projecting three-dimensional background object or the three-dimensional item object together according to the recognized user's actions.

Steps 860 through 880 may be steps for providing a more colorful background or loading an item to a user as the score of the user is higher. For example, the processing unit 730 may include at least one three-dimensional projector (not shown) to allow the user to feel as if the three-dimensional item object projected at the location associated with the user is moving along with the user's movement, 710).

FIG. 9 is a block diagram for explaining an example of the internal configuration of a service system for live audition according to an embodiment of the present invention. FIG. 10 is a flowchart illustrating an example of a service method for live audition 1 is a flowchart showing an example. The service method of FIG. 10 can be performed by the service system 900 of FIG. For example, the steps involved in the service method may be performed by the components that the service system 900 includes. 9, the service system 900 includes a display unit 910, a photographing unit 920, a voice input unit 930, And may include at least one motion sensor 940 and a processing unit 950.

In step 1010, the processing unit 950 can take a picture of the user through the photographing unit 920 installed in the space.

In step 1020, the processing unit 950 may control the display unit 910 installed in the space to reproduce the photographed state of the user. In one example, the display portion 910 may be configured to include a plurality of screens (not shown) provided at different angles in front of the space and a plurality of projectors (not shown) projecting images to each of the plurality of screens . An example of such a plurality of screens and a plurality of projectors has been described with reference to FIG.

In step 1030, the processing unit 950 may receive the user's voice through the voice input unit 930 installed in the space. For example, the voice input unit 930 may be installed in a space in the form of a wired or wireless microphone.

In operation 1040, the processing unit 950 may recognize the motion of the user using at least one motion sensor 940 that recognizes motions in the space.

At step 1050, the processing unit 950 adds at least one object of the background object and the item object to a position associated with the appearance of the user being reproduced, based on at least one of the user ' The control unit 910 can be controlled.

For example, step 1050 includes calculating (step) a user's score based on at least one of the input user's voice and the recognized user's action, and the higher the score, the more background objects or item objects (Not shown) for controlling the display unit 910 to further reproduce the content by adding the additional information. These unpublished steps may also be performed by the processing unit 950.

In another embodiment, the service method may include, in addition to the steps 1010 to 1050 shown in FIG. 10, the user's score calculated based on at least one of the input user's voice and the recognized user's action, (Not shown) for controlling the display unit to reproduce at least one of ranking according to the score of the audition and the score of the users along with the image generated in advance in connection with the audition. This unpublished step may be performed by the processing unit 950 after step 1050. [

In addition, in the service system and method for live audition, the appearance of the user reproduced through the display unit 910 and the background object or the item object reproduced in addition to the appearance of the user can be made into one image together . For example, if the user desires, the produced image may be uploaded to a specific site on the Internet as an audition image so that the user may participate in the actual audition. As another example, images uploaded to a specific site may be used for various purposes such as selection of the best image through popular vote of the general public, a user's social network service (SNS), a multimedia messaging service (MMS) Or may be shared with users using e-mail or the like. To this end, the service method may further include a step (not shown) of uploading an image reproduced through the display unit to an external server. At this time, the service system may further include a communication unit (not shown) for communication with an external server. The processing unit 950 can upload the image to the external server using the communication unit.

As described above, according to the embodiments of the present invention, when a user plays a game or sings a song in accordance with music, various effects are produced depending on the progress of the game or the score of the song In addition, the user's enjoyment can be enhanced.

The system described above may be a hardware component, a software component, and / or a device implemented in a combination of hardware components and software components. For example, the systems and components described in the embodiments may be implemented as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) unit, a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (18)

delete delete delete delete A system for providing services through a space for entertainment,
A display unit installed in the space;
A photographing unit installed on the space for photographing a user's appearance;
A voice input unit installed on the space for receiving voice of the user;
At least one motion sensor for recognizing movement within the space; And
Wherein the control unit controls the display unit to reproduce a state of the user photographed through the photographing unit based on at least one of the voice of the user input through the voice input unit and the operation of the user recognized through the at least one motion sensor A processing unit for controlling the display unit to add at least one object of a background object and an item object at a position associated with a shape of the user to be reproduced,
And the service system. 6. The method of claim 5,
Wherein,
Wherein the control unit controls the display unit to calculate the score of the user based on at least one of the input user's voice and the recognized user's action and to add and reproduce more background objects or item objects as the score is higher And the service system. 6. The method of claim 5,
Wherein,
Information on at least one of the score of the user calculated based on at least one of the input user's voice and the recognized user's operation, whether or not the audition is won based on the user's score, And controls the display unit to reproduce the image together with the image generated in advance in connection with the audition. 6. The method of claim 5,
The display unit includes:
A plurality of screens provided at different angles in front of the space; And
A plurality of projectors for projecting images to each of the plurality of screens,
And the service system. 6. The method of claim 5,
Wherein,
And the image reproduced through the display unit is uploaded to an external server. delete delete delete delete In a method for providing a service through a space for entertainment,
Photographing the user through a photographing unit installed in the space;
Controlling a display unit installed on the space to reproduce the photographed state of the user;
Receiving voice of the user through a voice input unit installed in the space;
Recognizing an operation of the user using at least one motion sensor that recognizes motion within the space; And
Controlling the display unit to further add at least one object of a background object and an item object at a position associated with a shape of the user to be reproduced based on at least one of the user's voice and the operation of the user
Gt; service method. ≪ / RTI > 15. The method of claim 14,
Wherein the step of controlling the display unit to further add and reproduce the at least one object comprises:
Calculating a score of the user based on at least one of the input user's voice and the recognized user's action; And
Controlling the display unit to add more background objects or item objects and reproduce more as the score is higher
Gt; service method. ≪ / RTI > 15. The method of claim 14,
Information on at least one of the score of the user calculated based on at least one of the input user's voice and the recognized user's operation, whether or not the audition is won based on the user's score, Controlling the display unit to reproduce the image together with the image generated in advance in connection with the audition
Lt; RTI ID = 0.0 > 1, < / RTI > 15. The method of claim 14,
Wherein the display unit includes a plurality of screens provided at different angles at a front surface of the space and a plurality of projectors for projecting images to each of the plurality of screens. 15. The method of claim 14,
A step of uploading an image reproduced through the display unit to an external server
Lt; RTI ID = 0.0 > 1, < / RTI >

Images