KR20150115302A - Apparatus for reproducing and authoring contents - Google Patents

Abstract

A content playback apparatus according to the present invention includes a motion recognizing unit which recognizes a motion of a user, a storage unit in which acoustic data is stored, an acoustic data selecting unit which selects the stored acoustic data based on a result of the motion recognition, an acoustic data correcting unit which corrects the selected acoustic data based on the result of the motion recognition, and an output unit which outputs the corrected acoustic data, wherein the acoustic data correcting unit corrects the acoustic data on the basis of at least one among a type of the selected acoustic data, a direction and an arrival time of a sound.

Description

[0001] APPARATUS FOR REPRODUCING AND AUTHORING CONTENTS [0002]

The present invention relates to a content reproducing and authoring apparatus.

Augmented Reality is a networked environment in which virtual objects created by a computer on a physical object in the real world are displayed in an overlapping form. Also, the augmented reality corresponds to a field of image synthesis that combines all or a part of a virtual image to an image of a real space through a predetermined marker or an identifier.

On the other hand, the augmented reality technology has been developed to supplement or improve the reality image using virtual images, and has been developed so that a computer device can be attached to a human body or can be mounted on a portable mobile terminal.

Such augmented reality technology has the advantage that it can maximize real feeling and immersion feeling in comparison with the existing virtual reality in that a user can see a real environment and view a virtual environment or information.

Such augmented reality can be realized by using a transparent display window in a bathroom mirror, a window, and a living room wall in a future indoor environment as a large transparent display device installed on a wall recently develops. In particular, the smart media wall that changes in conjunction with the contents of the display wall inside the room itself is likely to be available in the future ubiquitous residential environment.

In such a future indoor environment, a gesture is expected to be useful as a noncontact interface method of a user when using augmented reality technology. In order to control an augmented reality technique, a gesture corresponding sound effect is automatically reproduced A method is needed.

In this regard, Korean Patent Registration No. 10-0912264 (entitled " method and system for generating a user-responsive augmented image) discloses a technique for generating an augmented image adaptively responding to a change in a situation of a real space or a virtual space .

However, the above prior art can introduce the agent concept in the augmented reality to determine the acoustic sample to be reproduced based on the current situation information, but does not consider the directionality of the acoustic sample to be reproduced, There is a problem.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the conventional art, and it is an object of the present invention to provide a sound reproducing apparatus and a sound reproducing method which select sounds corresponding to a user's operation, And to provide a content reproducing and authoring device that outputs sound corresponding to a user's operation by correcting the sound in consideration of the user.

According to a first aspect of the present invention, there is provided a content reproducing apparatus including an operation recognizing unit for recognizing an operation of a user, a storage unit for storing sound data, An acoustic data correction unit for correcting the selected acoustic data based on the operation recognition result and an output unit for outputting the corrected acoustic data, And corrects the sound data based on at least one of the type of sound data, the direction of sound, and the arrival time.

According to a second aspect of the present invention, there is provided a content authoring apparatus comprising an operation recognizing unit for recognizing an operation of a user, a storage unit for storing sound data, an output unit for outputting the sound data, A sound data correction unit for correcting the selected sound data based on the result of the motion recognition, a sound data correction unit for outputting the corrected sound data, Wherein the acoustic data corrector corrects the acoustic data based on at least one of the type of the selected acoustic data, the direction of the acoustic and the arrival time.

According to any of the embodiments of the present invention described above, the sound corresponding to the user's operation can be outputted by correcting the previously stored sound data based on the operation of the user.

In addition, it is possible to output realistic and immersive sound to the user's position by correcting the sound data in consideration of the type and direction of the sound to be reproduced and the time to reach the user position.

In addition, it is possible to newly author content corresponding to the user's action by selecting and correcting the sound data corresponding to the user's action and storing the selected sound data.

1 is a block diagram of a content reproducing apparatus according to an embodiment of the present invention.
2 is a block diagram of a content authoring apparatus according to an embodiment of the present invention.
3 and 4 are views for explaining an example of a content reproducing apparatus or a content authoring apparatus according to the present invention.
5 is a flowchart of a content reproduction method according to an embodiment of the present invention.
6 is a flowchart of a content authoring method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

1 is a block diagram of a content reproduction apparatus 100 according to an embodiment of the present invention.

The content reproduction apparatus 100 includes an operation recognition unit 110, a storage unit 120, a sound data selection unit 130, a sound data correction unit 140, and an output unit 150.

The motion recognition unit 110 recognizes the motion of the user. The motion recognition unit 110 may be a motion sensor, a vision-based sensor, or a plurality of cameras. In addition, the motion recognition unit 110 can recognize the operation of the user by infrared transmission / reception.

At this time, the motion recognition unit 110 may measure the distance to the output unit 150 from which the sound is output based on the position of the user. The measured time is measured based on the measured distance, and the measured arrival time can be used in the sound data correction unit 140 as described below.

On the other hand, the motion recognition unit 110 can recognize the motion of the user based on the type of the motion of the user and generate the motion recognition result based thereon. In addition, the motion recognition unit 110 may generate the motion recognition result by recognizing the motion based on the speed of the motion of the user. The motion recognizing unit 110 recognizes a predetermined motion at a predetermined position and outputs a predetermined sound. Instead of the motion recognizing unit 110, motion of various users in a virtual reality in which a position of a user or a space where a user is located is variable, Can be recognized.

The storage unit 120 stores sound data. That is, data of various sound effects that can be generated according to a user's operation is stored. For example, it stores basic acoustic data about the sound that occurs when a baseball is thrown, as well as sound data that can occur when throwing a basketball or throwing a boomerang.

The sound data selection unit 130 selects the sound data stored in the storage unit 120 based on the result of the operation recognition according to the user's operation. For example, when a user takes an action of throwing a ball, the player selects sound data generated when the player throws the ball, and transmits the sound data to the sound data correction unit 140 to be described later.

The sound data correction unit 140 corrects the sound data selected by the sound data selection unit 130 based on the result of the operation recognition. At this time, it is possible to correct the sound data based on at least one of the kind of sound data selected at the time of correction, the direction of sound and the arrival time at which sound is transmitted from the output unit 150 to the user.

For example, if a user takes a throwing action, the user may take action to throw the ball slowly or throw it quickly. In this case, the sound data correction unit 140 can correct based on the type of sound generated when the baseball is thrown, and can also correct the sound direction, that is, whether the sound is unidirectional or surround direction have. In addition, it is possible to correct the sound by adjusting the velocity feeling when the ball is blown in consideration of the time of arrival of sound from the output unit 150 to the user.

On the other hand, the acoustic data correction unit 140 corrects the acoustic data based on the theoretical and experimental equations related to the type of acoustic data, the direction and arrival time of the acoustic sound, the distance between the user and each speaker, And it is also possible to correct the sound data by reflecting the influence of the mutual relation of the plurality of speakers.

The output unit 150 outputs the thus corrected sound data. The output unit 150 may include a plurality of speakers. The output unit 150 may output the corrected sound data through the plurality of speakers by reflecting the direction or arrival time of the sound. As a result, the sound is moved from left to right, or sound speed is output through a plurality of speakers, so that the user can provide more realistic and spaceful sound.

The motion recognition unit 110 or the output unit 150 described above may be included in a structure installed in a space where the user is located. In other words, it may be inserted on a structure such as a display device in which the entire wall surface is a display window, such as a media wall. However, it is not necessarily installed on the wall, and may be installed in various forms, such as a column, inserted into a specific structure or a separate display device.

2 is a block diagram of a content authoring apparatus 200 according to an embodiment of the present invention.

A content authoring apparatus 200 according to the present invention includes an operation recognition unit 210, a storage unit 220, a sound data selection unit 230, a sound data correction unit 240, a content authoring unit 250, 260).

The motion recognition unit 210 recognizes the motion of the user. At this time, the motion recognition unit 210 can recognize the motion of the user based on the type of motion of the user or the speed of motion. 1, various types of devices capable of recognizing motion such as a motion sensor, a vision-based sensor, and a camera can be used.

In addition, the motion recognition unit 210 can measure the distance to the output unit 260 from which the sound is output based on the position of the user. The measured arrival time may be used in the acoustic data corrector 240, as described below, based on the distance thus measured.

The storage unit 220 stores sound data. That is, data of various sound effects that can be generated according to a user's operation is stored. Also, the storage unit 220 may output the corrected sound data and store the outputted sound data, as described below. And stores the authored one content by storing the outputted sound data.

The output unit 260 outputs sound data, as well as the output unit 260 may output the corrected sound data, or may output sounds included in the finally authored content. In addition, when the sound corresponding to the action of the user is selected, the selected sound data can be output. By outputting various types of sound data as described above, the user can check whether or not the sound corresponding to the operation is outputted and the sound corresponding to the operation is corrected and output, and the sound corresponding to the finally generated content is output . The output unit 260 may include a plurality of speakers. The output unit 260 may output the corrected sound data through a plurality of speakers by reflecting the direction or arrival time of the sound.

The sound data selection unit 230 selects the sound data stored in the storage unit 220 based on the result of the operation recognition according to the user's operation. For example, when the user takes an action of throwing a ball, the player selects the sound data generated when the player throws the ball, and transmits the sound data to the sound data correction unit 240 to be described later.

The sound data correction unit 240 corrects the sound data selected by the sound data selection unit 230 based on the result of the operation recognition. At this time, it is possible to correct the sound data based on at least one of the kind of sound data selected at the time of correction, the direction of sound and the arrival time at which the sound is transmitted from the output unit 260 to the user.

The content authoring unit 250 outputs the corrected sound data, and writes the content by storing the outputted sound data. For example, when the user takes the first operation and the second operation, the acoustic data corresponding to the first operation and the second operation is selected, and the acoustic data corresponding to the speed and the type of the first and second operations are reflected, When the data is corrected, the content authoring unit 250 outputs the corrected sound data and stores the corrected sound data to author contents including the sound data according to the first operation and the second operation. Meanwhile, the authored content can be stored in the storage unit 220, and the user can further modify the sound data included in the authored content by further performing an operation.

The operation recognition unit 210 or the output unit 260 may be included in a structure installed in a space where the user is located, as described with reference to FIG. The description of the various examples is described with reference to FIG. 1, and a detailed description thereof will be omitted.

1 and 2 may include a cache, a ROM, a PROM (Programmable ROM), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a Flash memory Or a storage medium such as a hard disk drive (HDD), a CD-ROM, or a non-volatile memory device such as a random access memory (RAM) .

3 and 4 are views for explaining an example of the content reproducing apparatus 100 or the content authoring apparatus 200 according to the present invention.

The user takes his or her intended action in the virtual space in which the content reproducing apparatus 100 or the content authoring apparatus 200 is installed. Accordingly, the motion recognition units 110 and 210 receive the distance between the user and the output units 150 and 260 according to the position of the user through the motion recognition device such as a camera, and the motion of the user. In addition, parameters such as the type, direction, and distance of sound are applied to the sound data in consideration of the contents to be reproduced through the sound data correction units 140 and 240 so that sound can be reproduced naturally at the user's position. Accordingly, the predetermined sound is output based on the operation of the user through the output units 150 and 260.

At this time, the direction of sound can be indicated by using a speaker of at least four channels or more. For example, in a virtual space, when a user throws an object such as a baseball ball, the direction and speed of the ball are calculated based on the trajectory of the user's arm and the velocity per frame. In addition, the type of sound to be reproduced varies depending on the combination of the speed of the ball and the colliding object. That is, when you throw a baseball and when you throw a basketball, the type of sound that is played varies.

Another example is to play content related to sound that occurs when a boomerang is thrown. When the user takes a boomerang, he or she can play the surround sound so that the user can feel the boomerang return from the user's unidirectional or multi-channel speaker based on the user's swing speed.

5 is a flowchart of a content reproduction method according to an embodiment of the present invention.

In the content reproduction method according to the present invention, first, the operation of the user is recognized (S510). Here, the motion recognition device for recognizing the motion of the user may be a motion sensor, a vision-based sensor, or a plurality of cameras. In addition, the operation of the user can be recognized by infrared transmission / reception.

At this time, the step of recognizing the operation may measure the distance of the sound output based on the position of the user. Based on the distance thus measured, the time at which the acoustic data reaches the user can be measured, and the measured time can be used to correct the acoustic data.

In addition, the step of recognizing the action may recognize the action of the user based on the kind of the action of the user or the speed of the action of the user, and may generate the action recognition result based thereon.

Next, the sound data corresponding to the user's operation is selected from the sound data samples stored based on the user's operation (S520). The pre-stored acoustic data samples may have various actions that the user may take, including, for example, acoustic data according to the texture of the object that occurs when throwing a baseball, sound that may occur when throwing a basketball or throwing a boomerang Samples of data may be stored.

The thus stored sound data is selected based on the motion recognition result. For example, when an action of throwing a baseball is taken, an acoustic data generated when a player throws a baseball is selected in consideration of a user's arm motion.

Next, the sound data selected based on the operation recognition result according to the user's operation is corrected (S530). For example, when sound data generated when throwing a baseball is selected, based on the selected sound data, the velocity of the ball, the direction of movement, etc. are corrected by correcting the sound data by analyzing the motion speed of the user and the trajectory of the arm Lt; / RTI > As such, the step of calibrating the acoustic data may correct the acoustic data based on at least one of the type of acoustic data, the direction of the acoustic and the time at which the acoustic reaches the user from the output speaker.

On the other hand, the acoustic data can be corrected according to the frequency band based on the type of acoustic data, the direction and arrival time of the sound, the distance between the user and each speaker, and the theoretical and experimental equations related to the physical attenuation of the acoustic signal. In addition, it is also possible to correct the sound data by reflecting the influence of the mutual relation of the plurality of speakers.

Next, the corrected sound data is output (S540). In order to output the corrected sound data, a plurality of speakers may be installed in a space where the user is located. As a result, the sound is moved from the left to the right, or the sound speed is outputted through the plurality of speakers, thereby providing the user with realistic and space-sensitive sound.

6 is a flowchart of a content authoring method according to an embodiment of the present invention.

In the content authoring method according to the present invention, first, the operation of the user is recognized (S610). Various types of devices capable of recognizing motion, such as a motion sensor, a vision-based sensor, and the like, as described with reference to FIG. 1, may be used as an apparatus for recognizing a user's operation.

On the other hand, it is possible to measure the distance to the speaker on which the sound is output based on the position of the user by using the device for recognizing the motion. Based on the distance thus measured, the time at which the acoustic data reaches the user can be measured.

Next, the sound data corresponding to the user's operation is selected from the sound data sample stored based on the user's operation (S620). The description of the step of selecting the sound data has been described with reference to FIG. 1, FIG. 2, and FIG. 5, so that detailed description thereof will be omitted.

Next, the sound data selected based on the operation of the user is corrected (S630). At this time, it is possible to correct the sound data based on at least one of the kind of the selected sound data at the time of correction, the direction of the sound, and the arrival time for the sound to be transmitted from the speaker to the user.

Next, the corrected sound data is outputted through the speaker (S640), and the contents are written by storing the outputted sound data (S650). At this time, the authored content may be stored in the storage means, and when the user further performs the operation, the user may modify the sound data included in the authored content.

In addition, it is also possible to output the corrected sound data through the speaker or to output the sound data included in the finally authored content. In addition to this, when the sound corresponding to the operation of the user is selected, it may also output the selected sound data. By outputting various kinds of sound data as described above, the user can check whether or not the sound corresponding to the operation is outputted and the sound is corrected and outputted according to the operation, Quot;

1 and 2 according to an embodiment of the present invention refer to a hardware component such as software or an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), and a predetermined role Lt; / RTI >

However, 'components' are not meant to be limited to software or hardware, and each component may be configured to reside on an addressable storage medium and configured to play one or more processors.

Thus, by way of example, an element may comprise components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, Routines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

The components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: Content reproduction apparatus 200:
110, 210: operation recognition unit 120, 220:
130, 230: Acoustic data selection unit 140, 240: Acoustic data correction unit
150, 260: output unit 250: content authoring unit

Claims (8)

In the content reproduction apparatus,
An operation recognition unit for recognizing the operation of the user,
A storage unit for storing sound data,
An acoustic data selector for selecting the stored acoustic data based on the operation recognition result,
An acoustic data corrector for correcting the selected acoustic data based on the operation recognition result,
And an output unit for outputting the corrected sound data,
The sound data correction unit may include:
And corrects the sound data based on at least one of the type of the selected sound data, the direction of the sound and the arrival time. The method according to claim 1,
Wherein the operation recognition unit or the output unit is included in a structure installed in a space in which the user is located. The method according to claim 1,
Wherein the motion recognition unit measures the distance between the user and the output unit based on the position of the user,
And the arrival time is calculated based on a distance between the user and the output unit. The method according to claim 1,
Wherein the motion recognizing section recognizes the operation of the user on the basis of the kind of the operation of the user or the speed of the operation. A content authoring apparatus comprising:
An operation recognition unit for recognizing the operation of the user,
A storage unit for storing sound data,
An output unit for outputting the sound data,
An acoustic data selector for selecting the stored acoustic data based on the operation recognition result,
An acoustic data corrector for correcting the selected acoustic data based on the operation recognition result,
And a content authoring unit for outputting the corrected sound data and for authoring the contents by storing the outputted sound data,
The sound data correction unit may include:
And corrects the sound data based on at least one of the type of the selected sound data, the direction of the sound and the arrival time. 6. The method of claim 5,
Wherein the operation recognition unit or the output unit is included in a structure installed in a space in which the user is located. 6. The method of claim 5,
Wherein the output unit includes a plurality of speakers,
And the sound data reflecting the direction or arrival time of the sound is output through the plurality of speakers. 6. The method of claim 5,
Wherein the operation recognizing unit recognizes the operation of the user on the basis of the kind of the operation of the user or the speed of the operation.

Images