KR20180012166A - Story-telling system for changing 3 dimension character into 3 dimension avatar - Google Patents

Abstract

The present invention relates to a three-dimensional avatar-based speaker changeable storytelling system. The purpose of the present invention is to replace a three-dimensional character of three-dimensionally displayed storytelling with a three-dimensional avatar and enable the three-dimensional avatar to be reproduced with the same motion as that of the three-dimensional character. According to an embodiment of the present invention, the three-dimensional avatar-based speaker changeable storytelling system includes: an object volume forming part forming the volume of an object by generating a triangle based on mesh information of a three-dimensional character; an object appearance generating part generating the appearance of the object by adding texture information to a material generated through the object volume forming part; an object frame generating part inserting bone information into the material generated through the object appearance generating part; an object movement ratio setting part setting a movement ratio at which the material follows a moving bone when the material is applied to a corresponding animation based on vertex weight information; and an object animation applying part mapping an existing three-dimensional character with a new three-dimensional avatar by applying the corresponding animation to the rigging information and the material with the movement ratio set through the object movement ratio setting part.

Description

{STORY-TELLING SYSTEM FOR CHANGING 3 DIMENSION CHARACTER INTO 3 DIMENSION AVATAR}

In the embodiment of the present invention, a 3D avatar-based speaker-changeable storytelling system for replacing a 3D character, which is a character of storytelling displayed in 3D, with a 3D avatar and playing the replaced avatar in a motion like a 3D character, .

In the case of a conventional 3D animation, a specific animation scenario includes a 3D background, a 3D character, animation, and voice information. Based on this information, the 3D animation is reproduced through the display terminal.

In a conventional 3D character-based storytelling system, when a user desires to change a 3D character, the 3D character can be replaced using 3D character information provided through a distributor of the 3D animation.

Japanese Patent Application Laid-Open No. 10-2009-0126450 (published on December 09, 2009)

In an embodiment of the present invention, in an animation reproduction based on motion capture, a 3D character, which is a character of story telling displayed in 3D, is replaced with a 3D avatar, and the replaced avatar is replaced with a 3D character A 3D avatar-based talker-changing storytelling system that can reproduce a 3D avatar.

The 3D avatar-based speaker-altered storytelling system according to an embodiment of the present invention may include a 3D storytelling system that includes a mesh, a bone, a rigging texture, and a vertex weight information included in an animation scenario A 3D character information including the 3D information as one meta information, and a system for changing an existing 3D character to a new 3D avatar based on the animation information. The system creates a triangle based on the mesh information, An object volume implementing unit for performing the object creation process; An object contour generation unit for generating an contour of an object by adding the texture information to a material generated through the object volume implementation unit; An object skeleton generation unit for inserting the main information into the result generated through the object outer shape generation unit; An object movement ratio setting unit for setting a movement ratio of the result when the bone moves when applying the result generated through the object skeleton generating unit to the animation based on the vertex weight information; And an object animation application unit for mapping the existing 3D character to a new 3D avatar by applying the animation to the result set with the movement ratio and the rigging information through the object movement rate setting unit.

The apparatus may further include an animation player for playing the animation on the basis of the scenario including the 3D character information and the animation information.

The 3D avatar manager may further include a 3D avatar manager for generating and managing information about the 3D avatar.

A meta information loader for loading the meta information from a scenario selected by the user among scenarios of the animation player; A character replacement setting unit for receiving a change request of a 3D character from a user and setting a 3D character to be replaced in the meta information loaded through the meta information loader unit; And a meta information arrangement unit for arranging the meta information about the 3D character set through the character replacement setting unit so that the meta information can be referred to in correspondence with each item of the 3D avatar information generated through the 3D avatar manager.

The 3D avatar loader may further include a 3D avatar loader for applying the 3D avatar information changed through the object animation applying unit to the 3D character of the animation being reproduced through the animation player in real time.

According to the embodiment of the present invention, in the animation reproduction based on the motion capture, the 3D character, which is the character of the story telling displayed in 3D, is replaced with the 3D avatar, and the replaced avatar is replaced with the 3D character It is possible to provide a 3D avatar-based talker-changing storytelling system that can be reproduced in motion.

FIG. 1 is a block diagram illustrating an overall configuration of a 3D avatar-based speaker-changing storytelling system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a process of replacing a 3D avatar with a component of a 3D avatar-based speaker-changing storytelling system.
FIG. 3 shows that the time-dependent data B and the background data C are not affected by replacing the 3D avatar DATA A 'with the existing 3D avatar DATA A in order to apply a new 3D avatar to the 3D avatar, the time data and the background data. In which only the avatar is replaced and displayed without the need to regenerate the avatar.
4 is a flowchart illustrating a schematic process of replacing a 3D avatar in a 3D avatar-based speaker-changing storytelling system according to an embodiment of the present invention.
FIG. 5 is a flowchart illustrating a method for replacing an avatar in a 3D avatar-based speaker-changing storytelling system according to an exemplary embodiment of the present invention.
FIG. 6 is a diagram illustrating an example of a 3D character before replacement and a 3D avatar after replacement in a 3D avatar-based speaker changing type storytelling system according to an embodiment of the present invention.
FIGS. 7 to 10 are views schematically showing a technique of retargeting a character, a rigging, a viewpoint, and an animation so that the replaced avatar takes the same motion as a character without being awkward.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

While the present invention has been described in connection with certain embodiments, it is obvious that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the invention. It is to be understood, however, that the invention is not intended to be limited to the particular forms disclosed, but on the contrary, is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

In the drawings, the same reference numerals are used for the same reference numerals, and in particular, the numerals of the tens and the digits of the digits, the digits of the tens, the digits of the digits and the alphabets are the same, Members referred to by reference numerals can be identified as members corresponding to these standards.

In the drawings, the components are expressed by exaggeratingly larger (or thicker) or smaller (or thinner) in size or thickness in consideration of the convenience of understanding, etc. However, since the protection scope of the present invention is limited It should not be interpreted.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the term " comprising " or " consisting of ", or the like, refers to the presence of a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

FIG. 1 is a block diagram showing an entire configuration of a 3D avatar-based speaker-altering storytelling system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a process of replacing a 3D avatar with a 3D avatar- And FIG.

The 3D avatar-based speaker-altered storytelling system 1000 according to an exemplary embodiment of the present invention may include a mesh, a bone, a rigging texture, and a vertex weight included in an existing animation scenario, weight information as one meta information, and the animation information, the existing 3D character can be changed to a new 3D avatar. The animation information may include animation background data, time data, and the like.

Referring to FIGS. 1 and 2, a 3D avatar-based speaker-changing story telling system 1000 according to an embodiment of the present invention includes a meta information loader unit 100, a character replacement setting unit 200, The object moving part 300, the object volume implementing part 400, the object contour generating part 500, the object skeleton generating part 600, the object moving ratio setting part 700, the object animation applying part 800 and the 3D avatar loader 900). ≪ / RTI >

In addition, the 3D avatar-based speaker-changing story telling system 1000 according to the embodiment of the present invention may include at least one of the animation player 10 and the 3D avatar manager 20.

The meta information loader unit 100 may load meta information from a scenario selected by the user among a plurality of animation scenarios provided in the animation reproducer 10. [ Here, the animation player 10 may be means for playing an animation based on a scenario including 3D character information and animation information, respectively. The animation player 10 may include a plurality of scenarios (scenarios 1, 2 and 3) each including 3D character information and animation information as shown in FIG. 2, and an animation reproduction module for reproducing the scenarios .

The character replacement setting unit 200 may receive a 3D character change request from a user and set a 3D character to be replaced in the meta information loaded through the meta information loader unit 100. [

The meta information arrangement unit 300 can refer to the meta information about the 3D character set through the character replacement setting unit 200 in correspondence with each item of the 3D avatar information generated through the 3D avatar management unit 20 Can be deployed. That is, a mesh of a 3D avatar information to be changed, such as a mesh, a bone, a rigging texture, and a vertex weight, which are meta information about a 3D character, ), Rigging texture, and vertex weight information items, so as to refer to the meta information of the 3D character.

The object volume implementing unit 400 may implement a volume of an object by generating a triangle based on the mesh information of the 3D character set through the meta information arrangement unit 300. [ In this case, the object may mean an object in the process in which the final result is replaced with a 3D avatar.

The object contour generation unit 500 may generate a contour of the object by attaching a texture to the resultant material generated through the object volume implementer 400. [

The object skeleton generating unit 600 may insert skeleton information into a resultant material generated through the object contour generating unit to form a skeleton for the corresponding material.

The object moving rate setting unit 700 sets the object movement rate when the bone moves when applying the resultant material generated through the object skeleton generation unit 600 based on the vertex weight information to the corresponding animation You can set the movement rate that the material follows. That is, it is possible to set as a ratio how well the 3D avatar's bone will follow the movement of the 3D character.

The object animation applying unit 800 applies a corresponding animation (an animation scenario designated by the user) to the material and rigging information for which the bone movement rate is set through the object movement rate setting unit 700 , And can be replaced by mapping an existing 3D character to a new 3D avatar.

The 3D avatar loader 900 maps the 3D avatar information changed through the object animation applying unit 800 to the 3D character of the animation being played back through the animation player 10 in real time and displays it in the existing animation You can replace your 3D character with a new 3D avatar.

The 3D avatar manager 20 is a means for generating and managing information on the 3D avatar, and provides the 3D avatar information generated through the 3D avatar generator to the 3D avatar loader 900 as shown in FIG. 2 , The corresponding 3D avatar information and animation scenario can be stored in a separate repository through wired / wireless communication network.

FIG. 3 shows that the time-dependent data B and the background data C are not affected by replacing the 3D avatar DATA A 'with the existing 3D avatar DATA A in order to apply a new 3D avatar to the 3D avatar, the time data and the background data. In which only the avatar is replaced and displayed without the need to regenerate the avatar.

As shown in FIG. 3, the existing animation scenario includes 3D characters DATA A (Mesh, Texture, Bone Info, Vertex Weight), time dependent data B (operation data, voice DATA) and background DATA C If the 3D character is changed by applying the 3D avatar DATA A 'in the process of playing back the image, the time dependent data B (operation data, voice data) and background data C (mesh, texture) And can be displayed and replaced with a 3D avatar only without having to newly generate an image D as a result.

4 is a flowchart illustrating a schematic process of replacing a 3D avatar in a 3D avatar-based speaker-changing storytelling system according to an embodiment of the present invention.

As shown in FIG. 4, a 3D speaker character can be set in the existing animation, and the set 3D speaker character can be referred to the 3D avatar.

Then, 3D character characters can be mapped to 3D avatars and replaced, and storytelling can be performed.

FIG. 5 is a flowchart illustrating a method for replacing an avatar in a 3D avatar-based speaker-changing storytelling system according to an exemplary embodiment of the present invention.

First, the animation scenario can be loaded. At this time, the 3D character information and the animation information can be separately loaded.

In the process of applying and playing the 3D character animation, the 3D character is changed to the 3D avatar desired by the user, and the changed information can be reproduced by applying the 3D character animation again through the 3D avatar real time loader.

FIG. 6 is a diagram illustrating an example of a 3D character before replacement and a 3D avatar after replacement in a 3D avatar-based speaker changing type storytelling system according to an embodiment of the present invention.

FIGS. 7 to 10 are views schematically showing a technique of retargeting a character, a rigging, a viewpoint, and an animation so that the replaced avatar takes the same motion as a character without being awkward.

Referring to FIG. 7, an avatar's instance pointer may be replaced with an instance pointer of content and an avatar through instance retargeting. At this time, by replacing an instance in the game (animation) with an instance of the avatar, the replaced 3D avatar can appear in a scenario in which animation is desired.

Referring to FIG. 8, the avatar's bone information may be replaced with content bonus information through retargeting. At this time, even if the main information (Bone Info) of the animation and the main information (Bone Info) of the avatar are different, it is possible to make the animation correctly run by matching the names.

Referring to FIG. 9, a transform matrix of an avatar is mapped to a transform matrix of a content avatar through a matrix retargeting using a position vector (Vectro) and a rotation quaternion (Quaternion) . At this time, the avatar of the user can be accurately mapped to the avatar of the content.

Referring to FIG. 10, the Bone Transform Matrix may be mapped to a Bone Transform Matrix through interpolation of transform data over time. At this time, it is possible to correctly map the corresponding animation in a time period.

The 3D avatar-based speaker-changing storytelling system according to this embodiment performs story-telling through animation reproduction based on 3D character information and animation information included in the animation scenario selected by the user, In the course of storytelling, a plurality of 3D speaker characters can be changed to a 3D avatar.

As described above, in this embodiment, the 3D character, which is the character of the story telling displayed in 3D, is replaced with the 3D avatar, and the replaced 3D avatar can be reproduced in the same motion as the existing 3D character without awkwardness.

The above description is only one embodiment for implementing the 3D avatar-based talker changing storytelling system according to the present invention, and the present invention is not limited to the above-described embodiment, It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

1000: 3D avatar based narrative storytelling system
100: Meta information loader section
200: Character replacement setting unit
300: Meta information arrangement unit
400: object volume implementation unit
500: Object contour generation unit
600: object skeleton generation unit
700: object moving ratio setting unit
800: Object animation application section
900: 3D Avatar Loader
10: Animation player
20: 3D avatar manager

Claims (5)

Based on the 3D character information including the mesh, the bone, the rigging texture, and the vertex weight information included in the animation scenario as one meta information, and the animation information, A system for changing a 3D character to a new 3D avatar,
An object volume implementation unit for generating a triangle based on the mesh information to implement a volume of the object;
An object contour generation unit for generating an contour of an object by adding the texture information to a material generated through the object volume implementation unit;
An object skeleton generation unit for inserting the main information into the result generated through the object outer shape generation unit;
An object movement ratio setting unit for setting a movement ratio of the result when the bone moves when applying the result generated through the object skeleton generating unit to the animation based on the vertex weight information; And
And an object animation applying unit for applying an animation to the result set in which the movement ratio is set and the rigging information through the object moving rate setting unit and mapping the existing 3D character to a new 3D avatar, Storytelling system.
The method according to claim 1,
Further comprising an animation player for playing an animation on the basis of a scenario including the 3D character information and the animation information, respectively.
3. The method of claim 2,
Further comprising a 3D avatar manager for generating and managing information about the 3D avatar.
The method of claim 3,
A meta information loader for loading the meta information from a scenario selected by a user during a scenario of the animation player;
A character replacement setting unit for receiving a change request of a 3D character from a user and setting a 3D character to be replaced in the meta information loaded through the meta information loader unit; And
And a meta information arrangement unit for arranging the meta information about the 3D character set through the character replacement setting unit so that the meta information can be referred to in correspondence with each item of the 3D avatar information generated through the 3D avatar manager. Avatar - based speaker - changing storytelling system.
3. The method of claim 2,
And a 3D avatar loader for applying the changed 3D avatar information through the object animation applying unit to the 3D character of the animation being reproduced through the animation player in real time.

Images