KR101535992B1 - Display system and method with large wall surface screen usage type using user movement reaction based on motion sensor - Google Patents

Abstract

The present invention relates to a large wall screen utilization display system and method using a motion sensor based user movement reaction. According to a first aspect of the present invention, there is provided a display device comprising: a large wall screen for displaying image contents projected by a projector interlocked with a display device; A motion sensor for generating respective user movement response data for at least one or more users in front of the large wall screen; And a display device for receiving user movement reaction data from the motion sensor and transmitting the user movement reaction data to a display server through a communication network; And an infrared ray projection module for outputting infrared light of a predetermined wavelength in a plurality of dots in a forward direction; An infrared camera in which a plurality of dots sense an infrared reflection image reflected by a user and surrounding objects; And extracting an infrared reflection image constituting the infrared reflection image input to the infrared camera on the basis of a predetermined number of feature points through a minutiae processing process and then extracting the position information, A controller for generating user motion reflection data including movement motion information for each motion element (head, body, arms, and legs) in real time; And a motion sensor based on the movement response of the user.
The second aspect of the present invention also provides a first aspect of the present invention in which a motion sensor detects a user's motion for at least one user in front of a large wall screen by using an infrared dot projection module to output light of an infrared wavelength in a plurality of dot- step; A second step of performing detection of an infrared reflection image reflected by the user and surrounding objects through the infrared camera; And extracting a user form through a minutia point processing process for an infrared reflection image forming the infrared reflection image sensed by the motion sensor, extracting a user's location information, movement route information, A third step of generating user motion reflection data including information in real time; The present invention provides a display method using a large wall screen using a motion sensor based user movement reaction.
Accordingly, the present invention can be applied to a wide space and is applicable to an interactive content capable of user movement based operation, thereby providing an effect that can be used for multiple motion content based on user's motion.
According to the present invention, when applied as an advertisement, a PR and a guidance system according to a space to be installed, a message is outputted according to a movement of a user passing through a corresponding space and an area, In addition, it is possible to participate in a game based on a character that is responsive to the movement of each of a plurality of people at the same time, unlike the case of playing with a small number of persons in each home, and thus, it can be utilized in various kinds of image contents fields.

Description

TECHNICAL FIELD [0001] The present invention relates to a display system and a method for using a large wall screen using a motion sensor based user movement reaction,

The present invention relates to a large-sized wall screen utilizing display system and method using a motion sensor based user movement reaction. More specifically, the present invention can be applied to a wide space as well as a narrow space, The present invention relates to a display system and a method for using a large wall screen using a motion sensor based user movement reaction for utilizing available interactive image contents.

Recent developments in technology and the spread and widespread use of smart devices such as smart phones and smart pads have intimately influenced the intuitive user interface and various types of convenience devices based on touch screens in our lives.

As a result, users are demanding new things in a new and changing environment, and are easy to adapt and use.

In accordance with this environmental change, the present applicant previously presented an interactive game platform using a multi-touch method in a large-sized wall environment (name of the invention: interactive screen content display system using a large-sized wall, Patent Application No. 10-2013-0004994) In addition, we have also presented an advertising platform (the invention is an interactive display method of a large wall screen adaptable to viewing position by an infrared sensor).

However, such a game platform and an advertisement platform are suitable for presenting an intuitive UI, but the utilization is limited to a game platform field or an advertisement platform field, and there is a limit to expanding to all image content fields, There has been a weakness in the adaptation to movement.

[Related Technical Literature]

Description of the Related Art [0002] Interactive image contents display system on large wall surface (Patent Application No. 10-2013-0004994)

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide an environment for supporting motion sensors for an interactive contents system that can be used in a wide space and adaptable to multi- And to provide a display system and method for a large-sized wall screen utilizing a motion sensor-based user movement reaction, which can be used for adaptable image contents for user's movement.

In addition, the present invention is to provide a display system and method for a large wall screen utilizing a motion sensor based user movement reaction applicable to various systems such as a game, an advertisement, a publicity and a guidance system according to a space to be installed.

However, the objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a large-sized wall screen utilizing display system using a motion sensor-based user movement reaction according to an embodiment of the present invention includes a large-sized wall screen for displaying image contents projected by a projector interlocked with a display device; A motion sensor for generating respective user movement response data for at least one or more users in front of the large wall screen; And a display device for receiving user movement reaction data from the motion sensor and transmitting the user movement reaction data to a display server through a communication network; And an infrared ray projection module for outputting infrared light of a predetermined wavelength in a plurality of dots in a forward direction; An infrared camera in which a plurality of dots sense an infrared reflection image reflected by a user and surrounding objects; And extracting an infrared reflection image constituting the infrared reflection image input to the infrared camera on the basis of a predetermined number of feature points through a minutiae processing process and then extracting the position information, A controller for generating user motion reflection data including movement motion information for each motion element (head, body, arms, and legs) in real time; .

The motion sensor may be a general camera that captures a user in front of the user, recognizes each user, and reflects the image content projected through the projector to be displayed on the large wall screen. .

In addition, the display server displays position information, movement path information, and motion information of each user (head, body, arms, and legs) recognized by the display device except for unnecessary data among user motion reflection data received in real time It is preferable that only the operation information is received from the display device through the communication network.

Also, the display server collects location information, movement route information, and movement operation information of each user (head, body, arms, and legs) for the entire environment based on the data received from the display device It is preferable to perform the image content conversion on the image content source currently being projected on the projector based on the collected information and to return it so that the projection of the image content converted into the large wall screen is performed by the projector .

To achieve the above object, there is provided a method of displaying a large wall screen using a motion sensor based user movement reaction according to an exemplary embodiment of the present invention, the method comprising: detecting motion of each user for at least one user in front of a large- A first step of outputting light of an infrared wavelength in a plurality of dot shapes using an infrared dot projection module; A second step of performing detection of an infrared reflection image reflected by the user and surrounding objects through the infrared camera; And extracting a user form through a minutia point processing process for an infrared reflection image forming the infrared reflection image sensed by the motion sensor, extracting a user's location information, movement route information, A third step of generating user motion reflection data including information in real time; .

In this case, the present invention may include a fourth step in which the motion sensor is finally transmitted to the display server via the display device through wired or wireless data communication after the third step; And a fifth step of allowing the display server to reflect the received user motion reflection data and return the converted image content to the display device in real time and to project the converted image content on a large wall screen through a projector under the control of the display device. Is further preferably performed.

The large-sized wall screen utilizing display system and method using the motion sensor-based user movement reaction according to an embodiment of the present invention can be applied to a wide range of users, Based video content of the present invention.

In addition, a large-sized wall screen utilizing display system and method using a motion sensor based user movement reaction according to another embodiment of the present invention can be applied to a user who passes through a space and a region when it is applied as an advertisement, The message is output according to the movement of the player, and the player can move the game according to the passing route. In addition, when the game contents are combined, a plurality of people react to each player's movements It can be applied to various kinds of image contents field.

FIG. 1 is a perspective view showing a concept of a motion reaction by the motion sensor 20 in a large wall screen utilizing display system using a motion sensor based user movement reaction according to an embodiment of the present invention.
2 is a front perspective view illustrating a structure between a large wall screen 10, a motion sensor 20, and a projector 30 in a large wall screen utilizing display system using a motion sensor based user movement reaction according to an embodiment of the present invention. to be.
3 is a flowchart illustrating a process of transmitting data from a motion sensor 20 to a display device 40 and a display server 60 in a large wall screen utilizing display system using a motion sensor based user movement reaction according to an embodiment of the present invention. FIG.
4 is a flowchart illustrating a method of displaying a large-sized screen using a motion sensor-based user movement reaction according to an exemplary embodiment of the present invention.
5 is a perspective view illustrating a concept of a motion reaction by the motion sensor 20 in a large wall screen utilizing display system using a motion sensor based user movement reaction according to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of preferred embodiments of the present invention will be given with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In the present specification, when any one element 'transmits' data or signals to another element, the element can transmit the data or signal directly to the other element, and through at least one other element Data or signal can be transmitted to another component.

FIG. 1 is a perspective view showing a concept of a motion reaction by the motion sensor 20 in a large wall screen utilizing display system using a motion sensor based user movement reaction according to an embodiment of the present invention. 2 is a front perspective view illustrating a structure between a large wall screen 10, a motion sensor 20, and a projector 30 in a large wall screen utilizing display system using a motion sensor based user movement reaction according to an embodiment of the present invention. to be.

3 illustrates a process of transmitting data from the motion sensor 20 to the display device 40 and the display server 60 in a large-sized wall screen utilizing display system using a motion sensor-based user movement reaction according to an embodiment of the present invention FIG.

1 to 3, a large wall screen utilizing display system using a motion sensor based user movement reaction includes a large wall screen 10, a motion sensor 20, a projector 30, a display device 40, a communication network 50 and a display server 60.

The large wall screen 10 displays image contents projected by the projector 30 interlocked with the display device 40. The motion sensor 20 detects user movement reaction data in front of the large wall screen 10 , The display device 40 receives user movement reaction data from the motion sensor 20.

3, the motion sensor 20 is embedded in a large-sized wall screen 10 as shown in FIG. 2 and includes an infrared camera 21, a general camera 22, and an infrared dot projection module 23 as core components And further includes a control unit 24 and a transmission unit 25 so as to recognize multiple user movement responses moving on the front face of the large wall screen 10 based on the respective components and generate data, Provides a large wall screen environment.

The infrared dot projection module 23 outputs light of an infrared wavelength forward in a plurality of dots, and a plurality of dots are reflected by the user and surrounding objects and detected by the infrared camera 21 as an infrared reflection image. do. Accordingly, the controller 24 recognizes the depth information of the user and the surrounding background based on the infrared reflection image input to the infrared camera 21 based on the object recognition.

More specifically, the control unit 24 extracts a user-recognized shape based on a preset number of feature points (a total of 20) through the feature point processing for the infrared reflection image forming the infrared reflection image, Based on the user's position information, movement route information, and movement motion information for each motion element (head, body, both arms and legs) based on the shape.

On the other hand, the general camera 22 photographs a user in front, recognizes one user and / or a plurality of users, reflects the image contents projected through the projector 20 to be displayed on the large-sized screen 10 Function. Here, an example of the image contents realized by recognizing a plurality of users may be an example of the game of the game contents, or may be the user interactive advertisement contents or the guiding contents.

The display device 40 is provided with the same number of motion sensors 20 as the motion sensors 20 as shown in FIG. 3 to establish an environment that recognizes all user movements in a wide area, such as a large- Or wireless data lines.

The display device 40 can be formed of a general personal computer (PC) and receives user motion reflection data from the transmission unit 25 of the connected motion sensor 20 in real time.

Then, the display device 40 extracts only the user's position information, movement route information, and movement operation information per the operation elements (head, body, arms, and legs) except for unnecessary data among the user movement reflection data, To the server 60 via the communication network 50 in real time.

Accordingly, the display server 60 displays location information of each user, route information, and information of each operation element (such as head, body, and body) based on the data received from each display device 40 corresponding to the client, Both of the arms and legs) motion operation information, and performs the image content conversion on the image content source currently projected on the projector 30 based on the information. The converted image contents are transmitted again to the display device 40 through the communication network 50 in real time so that the projection onto the large wall screen 10 is performed by the display device 40 under the control of the projector 30.

Meanwhile, the communication network 50 according to the present invention is a communication network that is a high-speed period network of a large communication network capable of a large-capacity, long-distance voice and data service, and may be a next generation wired and wireless network for providing Internet or high-speed multimedia service. When the communication network 50 is a mobile communication network, it may be a synchronous mobile communication network or an asynchronous mobile communication network. As an embodiment of the asynchronous mobile communication network, a WCDMA (Wideband Code Division Multiple Access) communication network is exemplified. In this case, although not shown in the drawings, the communication network 50 may include a Radio Network Controller (RNC). Meanwhile, although the WCDMA network is exemplified, it may be a next generation communication network such as a 3G LTE network or a 4G network, or an IP network based on other IP. The communication network 50 transmits signals and data between the display device 40, the display server 60, and other systems.

Accordingly, users of a large wall screen utilizing display system utilizing a motion sensor based user movement reaction can be provided with a motion sensor 20, which is located in front of a large wall screen 10 having a built-in motion sensor 20, The process of image contents implemented in a large wall screen utilizing display system using a sensor based user movement reaction can be performed.

When the video content corresponds to the game content, the user watches the video outputted to the large-sized screen 10, moves his / her arm, selects a menu implemented in a preset area of the video, You can participate in the game by moving your body, arms and legs. In addition, when the image content corresponds to the user interactive interactive content or the guiding content, the user can recognize the movement of the user by simply passing in front of the large screen 10 without any designated movement, The reaction of the type suitable for the operation is performed.

Referring to FIG. 2, the system according to the present invention uses a motion sensor-based user movement reaction based on user movement reflection data, and displays a large number of users A plurality of motion sensors 20 suitable for the corresponding region are applied so as to recognize the movement of the user, thereby recognizing movement paths of the users and movements of the body, limbs, etc., .

4 is a flowchart illustrating a method of displaying a large-sized screen using a motion sensor-based user movement reaction according to an exemplary embodiment of the present invention. 1 to 4, the motion sensor 20 outputs infrared light in a plurality of dot shapes using an infrared dot projection module 23 for user movement detection in front of the large-sized screen 10 S11).

After step S11, the motion sensor 20 performs detection of the infrared reflection image reflected by the user and the surrounding objects using a plurality of dots outputted in step S11 using the infrared camera 21 (S12).

After step S12, the motion sensor 20 extracts the user's shape through a minutia point process for an infrared reflection image forming an infrared reflection image (S13). More specifically, the control unit 24 of the motion sensor 20 extracts a user-recognized shape based on a predetermined number of minutiae points through a minutiae processing process on an infrared reflection image constituting an infrared reflection image. To this end, the motion sensor 20 may further include an own storage unit (not shown) for storing user type information in advance or may control the transmission unit 25 to transmit the user information It is possible to receive and extract the type information.

After step S13, the motion sensor 20 generates user motion reflection data including user's position information, movement route information, and motion operation information per motion element based on the user form extracted in step S13 (S14).

After step S14 the motion sensor 20 transmits the user motion reflection data to the display device 40 via wired or wireless data communication so that the display device 40 (S15).

After step S15, the display server 60 returns the converted image contents reflecting the received user motion reflection data to the display device 40, and controls the display device 40 via the projector 30 under the control of the display device 40 And is projected onto the large-sized screen 10 (S16).

5 is a perspective view illustrating a concept of a motion reaction by the motion sensor 20 in a large wall screen utilizing display system using a motion sensor based user movement reaction according to another embodiment of the present invention. Referring to FIG. 5, the motion sensor 20 is formed as an external shape of the large-sized screen 10, unlike FIG. 1, and is driven on the same principle as FIG.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also implemented in the form of a carrier wave (for example, transmission over the Internet) .

The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers skilled in the art to which the present invention pertains.

As described above, preferred embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention , And are not intended to limit the scope of the present invention. It is to be understood by those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

10: Large wall screen
20: Motion sensor
21: Infrared camera
22: Normal camera
23: Infrared dot projection module
24:
25:
30: Projector
40: Display device
50: Network
60: Display server

Claims (6)

A large wall screen for displaying image contents projected by a projector interlocked with a display device; A motion sensor for generating respective user movement response data for at least one or more users in front of the large wall screen; And a display device for receiving user movement reaction data from the motion sensor and transmitting the user movement reaction data to a display server through a communication network; 1. A large wall screen utilizing display system using a motion sensor based user movement reaction, the motion sensor comprising:
An infrared dot projection module for forwardly outputting light of an infrared wavelength in a plurality of dots;
An infrared camera in which a plurality of dots sense an infrared reflection image reflected by a user and surrounding objects; And
Extracting an infrared reflection image constituting an infrared reflection image input to the infrared camera on the basis of a predetermined number of feature points through a minutia point processing process and then extracting the position information of the user based on the extracted user form, A controller for generating user motion reflection data including movement motion information for each motion element (head, body, both arms and legs) in real time; Wherein the motion sensor is based on a motion sensor.
The motion sensor according to claim 1,
A general camera for photographing a user in front of the user, recognizing each user, reflecting the image contents projected through the projector to be displayed on the large wall screen; Wherein the controller is further configured to control the motion sensor based on the movement of the user.
delete delete A first step of the motion sensor outputting light of an infrared wavelength in the form of a plurality of dots using an infrared dot projection module for each user's motion detection for at least one user in front of a large wall screen;
A second step of performing detection of an infrared reflection image reflected by the user and surrounding objects through the infrared camera; And
The motion sensor extracts a user form through a minutia point process for an infrared reflection image forming a detected infrared reflection image, and then extracts the user's position, movement route information, and movement operation information A third step of generating user motion reflection data including the motion data in real time; Wherein the motion sensor is based on a motion sensor.
The method of claim 5, further comprising, after the third step,
A fourth step of finally transmitting the motion sensor to a display server via a display device through wired or wireless data communication; And
The display server reflecting the received user motion reflection data and returning the converted image content to the display device in real time and projecting the converted image content to the large screen through the projector according to the control of the display device; The method of claim 1, further comprising the steps of:

Images