KR102343562B1 - Activity character creating method in virtual environment based on sketch character - Google Patents

Abstract

The present invention accurately extracts a two-dimensional figure (character) sketched by a user, and clearly extracts the joint part of the extracted figure so that it is displayed as an optimal natural activity pattern suitable for the two-dimensional figure, making it more realistic and user-friendly. It relates to a sketch-based active character creation method for creating a sketch-based character that can maximize interest in a virtual environment as an active character in a virtual environment. According to the present invention, a sketch drawing reading step of reading the entire image including the sketch drawing drawn on the paper by the user and the background of the sketch drawing; a sketch picture extraction step of extracting only a sketch picture from among all the read images; a sketch drawing type identification step of analyzing the extracted sketch picture to identify and determine the type of the sketch picture; a picture image animating information generating step of generating animating information so that the sketch picture is animated based on the determined type of the sketch picture; There is provided a sketch-based active character creation method comprising: mapping the generated animation information to a sketch figure, and animating the mapped sketch figure; a mapping-animating step.

Description

How to create a sketch-based active character to create a sketch-based character as an active character in a virtual environment {ACTIVITY CHARACTER CREATING METHOD IN VIRTUAL ENVIRONMENT BASED ON SKETCH CHARACTER}

The present invention relates to a sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment, and more particularly, accurately extracts and extracts a two-dimensional figure (character) sketched by a user arbitrarily. Creating a sketch-based character as an active character in a virtual environment that is more realistic and maximizes the user's interest by clearly extracting the joint part of the picture and displaying it as an optimal natural activity pattern suitable for the two-dimensional picture It relates to a sketch-based activity character creation method for

This invention is the result of research conducted as a cultural technology R&D project supported by the Korea Creative Content Agency (task number: 1375026811, research project name: augmented reality animation and mapping technology development for creative art interactive content of edutainment)

Currently, due to the development of media and Internet-related technologies, a content implementation method for composing a web page in which various moving images are implemented is developed and implemented so as to be convenient for computer users and to enhance the use effect of computer users. It is increasing the interest and satisfaction of computer users by providing information by composing the contents in a form that includes dynamic images instead of the form.

The use of services using the Internet is now increasing in fields such as e-commerce, entertainment, and distance education, and accordingly, methods for searching and providing necessary content information for each field of use are being developed.

In particular, with the recent development of electrical and electronic technology and information and communication technology and the popular spread of various smart devices including smartphones and smart pads, various types of convenience devices based on intuitive interfaces and touch screens have a close influence on daily life. are giving Accordingly, users are demanding newer things in the newly changing media display environment, while using new convenience devices with quick adaptability.

In relation to such a media display environment, in order to provide a variety of media display environments according to user manipulation, image content such as characters or games is advantageous rather than general image content.

However, video content reflecting real-time immediate interactions of users is generally limited to smart devices such as smartphones or smart pads having a small screen size.

In addition, in the prior art, since the character of the video content generates the body movement and facial expression of the character by pre-production, the character has a limitation in that only a passive character function that directs the movement according to a standardized pattern, that is, a predetermined scenario, is possible, and the user There was a problem in that there was a limit to arousing the interest of

Accordingly, there is a need to develop content that allows users to actively participate and arouse interest in a media display environment that is developing day by day, and is reflected in the character drawn by the user as it is, and can implement natural movement suitable for the character.

Republic of Korea Patent Publication No. 10-0301962 (2001.09.26. Announcement) Republic of Korea Patent Publication No. 10-0915084 (2009.09.02. Announcement) Korean Patent Publication No. 10-2002-0049384 (published on June 26, 2002) Korean Patent Publication No. 10-2014-0061951 (published on May 22, 2014) Korean Patent Publication No. 10-2002-0086306 (published on Aug. 1, 2002)

Therefore, the present invention for solving the above-mentioned conventional problems accurately extracts a two-dimensional figure (character) sketched by a user, and clearly extracts the activity skeleton and joint part of the extracted figure to obtain a corresponding two-dimensional figure. The purpose is to provide a sketch-based active character creation method to create a sketch-based character as an active character in a virtual environment, which is more realistic and maximizes user interest by displaying the optimal natural activity pattern suitable for have.

The problems to be solved of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to one aspect of the present invention for achieving the above objects and other features of the present invention, a sketch drawing reading step of reading the entire image including the sketch figure drawn by the user on the paper and the background of the sketch figure; a sketch picture extraction step of extracting only a sketch picture from among all the read images; a sketch drawing type identification step of analyzing the extracted sketch picture to identify and determine the type of the sketch picture; a picture image animating information generating step of generating animating information so that the sketch picture is animated based on the determined type of the sketch picture; There is provided a sketch-based active character creation method comprising: mapping the generated animation information to a sketch figure, and animating the mapped sketch figure; a mapping-animating step.

In the present invention, the step of reading the sketch picture is read through a photographing means including an image sensor, and the step of extracting the sketch picture uses the dimensional transformation of the HSV color space in the read image to convert the color of the picture to binary. However, it is characterized in that it is extracted using the following RGB to HSV color space conversion formula.

(Where, H = Hue, S = Saturation, V = Brightness, and RGB is Red Cyan)

In the present invention, the step of identifying the type of sketch drawing is made to understand by comparing the DB image of the image database built in advance with the binary (Binary) and extracted sketch drawing image by deep learning, , The step of generating the information for animating includes a mesh generation step of generating a mesh based on the binary (Binary) and extracted sketch picture image, and the binary (Binary) and extracted sketch picture image. It may include a skeleton-joint point generation step of generating a skeleton and a joint point as a target, and a mesh skinning step of applying the final skeleton to which the generated skeleton and joint point are applied to the generated mesh.

In the present invention, the mesh generation step is made to generate a character mesh to which a polygon mesh is applied using a Delaunay triangulation technique, and the bone-joint point generation step includes all pixels of the binarized sketch figure. , the skeleton is extracted by removing pixels that are not satisfied with the set pixel inspection value by examining the surrounding pixels centered on one pixel, It is characterized in that it is made to extract a plurality of joints based on the number of binarized pixels.

In the present invention, it is preferable to further include a joint point subdivision process of subdividing between a plurality of joints extracted in the bone-joint point generation step.

In the present invention, the joint point segmentation process determines whether there is a pixel connected to the periphery around the current pixel, and if there is no pixel connected to the periphery with respect to the current pixel, it is displayed as a segment point, If there is a neighboring pixel with respect to the current pixel, it is determined whether it is a different feature point, and if it is a different feature point, the process is displayed as the dividing point. It is determined whether or not it is greater than the segment length, and when the movement distance is greater than a predefined segment length, the division point display and movement distance are initialized, and when the movement distance is smaller than the predefined segment length, movement It is characterized in that it is made to erase one pixel.

In the present invention, the mesh skinning step may be made to map the subdivided joint points and the skeleton including the subdivided joint points to the character mesh generated in the mesh generating step as a final skeleton.

In the present invention, in the mesh skinning step, information for mapping animation in which a weight is given to a predetermined joint point to define which position of the vertex of the mesh to move based on the joint point of the vertex of the mesh. can be made to create

In the present invention, in the weighting, the distance between any vertex Vi and all joint points B among all vertices V is obtained using the Euclidean Distance formula, and then a straight line arranged in the order of the shortest distance. A set D (NearBoneList) is obtained, and the n joints closest to the NearBoneList, which is the set of straight lines D, are connected with a random vertex (Vi) and weighted, but the closer the joint to the random vertex, the greater the weight is given. , the mapping-animating step may be performed to perform an animation mapping based on the mapping animating information.

According to the sketch-based active character creation method for creating a sketch-based character as an active character in a virtual environment according to the present invention, the user accurately recognizes and extracts the character sketched by the user, and the active skeleton and joint part optimally suitable for the character It has the effect of maximizing the user's interest and immersion by allowing them to be realized as realistic and natural active characters in a virtual environment by extracting them and giving them activity.

In addition, the present invention does not require a paper equipped with a marker for recognition for the recognition of 2D pictures, any paper can be used, and it can be applied to various media contents such as games, entertainment, education, publicity, advertisement, and exhibition. This has the effect of securing product competitiveness by being able to compose more rich and diverse media content.

Effects of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a flowchart illustrating a sketch-based active character creation process for generating a sketch-based character as an active character in a virtual environment according to the present invention.
2 is a diagram showing the color of a picture by using a dimensional transformation of the HSV color space in the sketch picture extraction step of the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. ) is a diagram showing before (A) and after (B) for an example of extraction.
3 is a diagram showing an outline of the architecture of Inception v3 Neural Networks produced by Google.
4 is an exemplary diagram illustrating an example of a figure type identification step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention.
5 is a drawing result to which a polygon mesh is applied through the mesh generation process of the animation information generation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. , (A) is a picture reflecting the Delaunay triangulation technique, and (B) is a picture after removing the outline triangle.
6 is a method of thinning by removing pixels for bone extraction in the bone-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention; It is a drawing for explaining.
7 is a thinning algorithm in the binarized sketch figure (A) in the bone-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. It is a diagram showing an example of the skeleton (B) of the sketch figure extracted through the
8 is an example of pixel inspection for joint extraction and reference pixel P1 in the skeleton-joint point generation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. It is a diagram illustrating a function result value.
9 is a diagram illustrating a multi-example of a figure in which joint points are extracted according to the result value according to FIG. 8 .
10 is an example of a figure to which joint segmentation is applied through the joint segmentation process in the skeleton-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention; to be.
11 is a flowchart illustrating a joint subdivision process in the bone-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention.

Additional objects, features and advantages of the present invention may be more clearly understood from the following detailed description and accompanying drawings.

Prior to the detailed description of the present invention, the present invention can make various changes and can have various embodiments, and the examples described below and shown in the drawings are not intended to limit the present invention to specific embodiments. No, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

In addition, terms such as "...unit", "...unit", "...module", etc. described in the specification mean a unit that processes at least one function or operation, which includes hardware or software or hardware and It can be implemented by a combination of software.

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

In addition, throughout this specification, when a step is located “on” or “before” another step, this means not only a case in which a step is in a direct time-series relationship with another step, but also a step of mixing after each step and Likewise, the order of two stages includes the same rights as in the case of an indirect time series relationship in which the time series order can be changed.

Hereinafter, a sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. It is assumed that each step and technique described below is implemented through a known programming language software (eg, C++, etc.).

First, a sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention will be described in detail with reference to FIG. 1 . 1 is a flowchart illustrating a sketch-based active character creation process for generating a sketch-based character as an active character in a virtual environment according to the present invention.

A sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention, as shown in FIG. ') and a sketch picture reading step of reading the entire image including the background of the sketch picture (S100); a sketch picture extraction step (S200) of extracting only a sketch picture from among the entire images read in the sketch picture reading step (S100); A sketch figure identification step (S300) of analyzing the sketch figure extracted in the sketch figure extraction step (S200) to identify and determine the type of the sketch figure; a picture image animating information generation step of generating animating information so that the sketch picture (real picture) is animated based on the type of sketch picture determined in the sketch picture type identification step (S300) (S400); According to the status information (status) of the contents of the virtual environment to which the animation information generated in the animation information creation step (S400) is mapped to a sketch figure (real figure), and to which the mapped sketch figure (real figure) is to be applied and a mapping-animating step (S500) of animating the mapped sketch figure.

Each of the above steps will be described in detail.

Sketch drawing reading step (S100)

The sketch picture reading step (S100) reads the background and sketch picture of the paper as image data through a camera image sensor, etc. At this time, if the sketch picture image is recognized in the sketch picture recognition area defined in advance, the sketch picture It is made to be read as image data for the entire image including the paper and drawing paper. In other words, the sketch figure reading step ( S100 ) reads image data for the entire image including the sketch figure and the drawing paper.

In addition, the reading of the image data in the sketch drawing reading step (S100) is a camera image sensor, for example, a device that detects the strength and weakness of an optical image and converts it into digital image data (image data), and a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or CCD image sensor (Charge Coupled Device) may be made of a photographing means including.

Sketch figure extraction step (S200)

In the sketch figure extraction step (S200), only the sketch figure (picture image data) is extracted by dividing the boundary outline between the white drawing paper and the sketch figure in order to extract only the sketch figure (figure image data) from the read image data.

2 is a diagram showing the color of a picture by using a dimensional transformation of the HSV color space in the sketch picture extraction step of the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. ) is a diagram showing before (A) and after (B) for an example of extraction.

Specifically, the sketch picture extraction step (S200) extracts only the sketch picture (picture image data) drawn by the user from the picture image data (video) input through a webcam, etc., and the present invention provides the following RGB to HSV color space Using the conversion formula, the picture image data input as RGB values is dimensionally transformed in the HSV color space, and the color of the picture is binarized or binary and extracted.

In other words, the sketch figure extraction step S200 converts the input RGB pixel value into HSV using the above RGB to HSV conversion formula. Here, H = hue, S = saturation, and V = brightness.

Among the pixel values converted to HSV in this way, a pixel with an S value lower than the specified saturation threshold and a V value higher than the specified brightness threshold can be viewed as a white area. As shown, a binarized image result (a binarized sketch picture image) can be obtained.

In addition, in the sketch figure extraction step (S200), a binarized image result (a binarized sketch figure image) may be obtained by recognizing the edge of the figure and extracting it using an edge extraction method that separates the sketch figure based on the outline.

Sketch figure type identification step (S300)

The sketch figure identification step (S300) compares the DB image of the image database built in advance with the extracted sketch figure image, analyzes the pattern to determine the closest DB image, and uses the DB image to determine the type of sketch figure. (eg, people, dogs, cats, snakes, etc.).

This step of identifying the type of sketch drawing ( S300 ) can be executed by deep learning using a machine learning engine such as TensorFlow.

Specifically, the step of identifying the type of sketch figure ( S300 ) will be described with reference to FIGS. 3 and 4 . 3 is a diagram showing the outline of the architecture of Inception v3 Neural Networks produced by Google, and FIG. 4 is a sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. It is an example diagram showing an example of the step of identifying the type of picture to be used.

The sketch figure type identification step (S300) is, as shown in FIG. 3, a sketch figure image extracted using a deep learning technique (eg, TensorFlow) by capturing features based on the sketch figure image extracted first. It is made to recognize whether it is an object.

Here, recognition and picture type analysis and inference use the proven inception model among deep learning models, and the data set (image database) to be used for picture inference is registered through the retraining process.

As such, the extracted sketch figure image is compared with the sketch figure inference data set (image database) using the above deep learning model to infer what the actual sketch figure (figure image data) is, and this inferred information Information for animating that the corresponding sketch figure image can be activated in the next step, the sketch figure image creation step (S400), using the mesh and skeleton-joint data of (that is, the image of the inferred image database) will create

Here, each of the image data of the data set includes mesh data and bone-joint data corresponding thereto.

Information generation step for animation (S400)

The animation information generation step (S400) is a mesh generation step of generating a mesh (mesh data) based on the image (binary image) of the image database identified in the sketch figure type identification step (S300) (S410), and the skeleton-joint point generation step (S420) of generating a skeleton and joint points based on the image (binary image) of the image database identified in the sketch figure type identification step (S300) (S420), and the mesh and A mesh skinning step (S430) of applying the skeleton generated in the bone-joint point generation step (S420) and the final skeleton to which the joint point is applied to the mesh generated in the mesh generation step (S410) is included.

In the mesh generation step (S410), the mesh is generated using a Delaunay triangulation technique to generate a polygon mesh.

Here, the Delaunay triangulation technique is a method of dividing by making the minimum value of the interior angles of triangles maximize (close to an equilateral triangle) when dividing points on a plane by connecting them to a triangle.

To apply this Delaunay triangulation, an arbitrary vertex or point is inserted into the extracted sketch picture area, and then the area is divided into triangles by applying the Delaunay triangulation technique. At this time, there are cases where triangles are connected out of the drawing area, which is not an area corresponding to the polygon mesh of the sketch figure, so it will be removed. 5 is a drawing result to which a polygon mesh is applied through the mesh generation process of the animation information generation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. , (A) is a picture reflecting the Delaunay triangulation technique, and (B) is a picture after removing the outline triangle.

The skeleton-joint point creation step (S420) is to extract and create a skeleton from the image (binary image) of the image database identified in the sketch figure type identification step (S300), and to extract and create joint points from the created skeleton is done

That is, the skeleton-joint point creation step (S420) is thinned from the image of the image database (the extracted binarized (binary) sketch figure, the binarized image) previously identified in the sketch figure type identification step (S300). An algorithm is applied to extract the skeleton. At this time, thinning is an algorithm that extracts a line with a thickness of 1 from an input image. By using this algorithm, a skeleton for a sketch figure (picture data of a binarized image) can be extracted. For example, among the thinning algorithms, using the Zhang Suen algorithm, it can be thinned quickly through parallel operation. In other words, the Zhang Suen algorithm, a parallel processing algorithm, is used to maintain the target image and perform a fast thinning operation.

Specifically, in the bone-joint point creation step (S420), the bone extraction generation is based on which pixel for all pixels of the binarized (binary) sketch figure, and the surrounding pixels are referenced to match specific conditions. It is judged for all pixels whether or not it is not, and a thinned binarized picture is obtained while removing pixels that meet specific conditions.

For example, in relation to the specific condition, it will be described with reference to the following pixel removal condition equations and FIGS. 6 and 7 . 6 is a method of thinning by removing pixels for bone extraction in the bone-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention; 7 is a sketch figure binarized in the skeleton-joint point creation step included in the sketch-based active character creation method for creating a sketch-based character as an active character in a virtual environment according to the present invention It is a diagram showing an example of the skeleton of a sketch figure extracted through the thinning algorithm.

P1 is the center pixel, and P2 to P9 are the peripheral pixels of P1. The function A(P1) is a function that outputs the number of pixels connected in the order of 0 and 1 by examining the pixels around P1 in a clockwise direction. The function B(P1) is a function that outputs the number of neighboring pixels.

The first 2<=B(P1)<= 6 condition is a condition that removes pixels that are not surrounded or at the end. The second condition of A(P1) = 1 is to check whether a pixel is a connected pixel and remove the unconnected point. Third, P2*P4*P6 = 0 and P4*P6*P8 = 0 are conditions to inspect and remove corner pixels.

If the above process is performed for all pixels based on one central pixel, the binarized picture as shown in FIG. 7(A) is extracted as a thinned picture (frame of figure) as shown in FIG. 7(B) . The skeleton extracted in this way becomes the reference skeleton of the animation.

8 is an example and criteria for pixel inspection for joint extraction in the skeleton-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention. It is a view exemplifying the function result value of the pixel P1, and FIG. 9 is a view showing an example of a figure in which joint points are extracted according to the result value according to FIG.

As described above, in order to move the sketch figure centering on the skeleton extracted by the pixel removal method in the bone-joint point creation step (S420), it is necessary to define the joint and determine the joint. The determination of such a joint goes through two stages. The first is the definition of the end point and the joint point, and the second is the segmentation process between the end point and the joint point.

First, to determine the joint, the number of the order in which the white pixel and the black pixel appear is calculated by examining the surrounding pixels in a clockwise direction based on the pixels in the skeleton as shown in FIG. 8 . When the calculated result value F(P1) is 1, P1 is defined as the end point, and when F(P1) is greater than or equal to 3, P1 is defined as the joint point.

In addition, in order to move the entire figure, the vector value from the joint point to the end point is used to extend and redefine the end point on the outline of the sketch figure. Through this process, a result is derived as shown in the example of FIG. 9 . In FIG. 9 , the red dot is the end point, and the green dot is the joint point. (A) of Figure 9 is an example of a figure defining an end point and a joint point, (B) is an example of a figure in which the end point and joint point are substituted into the figure mesh, (C) is a vector value from the joint point to the end point. This is an example of a figure redefined by extending the end point to the outline of a sketch figure.

Subsequently, the subdivision process between the end point and the joint point will be described with reference to FIGS. 10 and 11 . 10 is an example of a figure to which joint segmentation is applied through the joint segmentation process in the skeleton-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention; 11 is a flowchart showing a joint subdivision process in the bone-joint point creation step included in the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention.

When the skeleton is moved only with the joint points and end points defined through the above process, there is an awkward movement. To solve this problem, a process of subdividing the joint points for the skeleton between the end point and the joint point is performed for natural movement.

Specifically, in the articulation point segmentation process, as shown in FIG. 10 , the feature point pixel is the current pixel, and it is determined whether there is a pixel connected to the periphery of the current pixel.

If there is no adjacent pixel with respect to the current pixel, it is displayed as a segment point. And if there is a pixel connected to the current pixel, it is determined whether it is a different feature point, and if it is a different feature point, the process proceeds to display as the dividing point. It is determined whether it is greater than the segment length.

When the moving distance is greater than the predefined segment length, the division point display and moving distance are initialized. When the moving distance is smaller than the predefined segment length, the moved pixel is deleted.

This joint point segmentation process is executed for all bones (skeletal pixels) between the end point and the viewpoint point, and when the joint point segmentation process goes through, the final animation skeleton result in which the joints are subdivided as shown in FIG. 11 can be obtained. there will be

Next, in the mesh skinning step (S430), the final skeleton of the pixel skeleton and joint point defined and determined as described above is applied to the character mesh to move based on the joint point of the vertex at a certain position in the vertex of the mesh. It is made to define a weight at the joint point.

The mesh skinning step ( S430 ) targets all vertices V . Find the distance between any vertex Vi and all joint points B among V using the Euclidean Distance formula, and then obtain a straight line D set (NearBoneList) sorted in the order of the shortest distance. At this time, any straight line Di becomes a range that does not deviate from the area of the figure. The following equation is an example of the pseudo code of the process, and it is the shortest distance set pseudo code.

In the NearBoneList, which is the set of straight lines D obtained through the above process, the n closest joints are connected to the vertex Vi and weighted, but the closer the joint to the vertex, the greater the weight. Here, close joint B0weight + B1weight … + Bnweight = 1.

In the above weight, the weight for the joint point close to the vertex is determined by the following formula.

Through this process, as a result, joint points have relative weight values for all vertices, and the positions of the vertices change according to the position and weight value of the joint points connected to them.

Mapping-Animating step (S500)

Through each of the above steps, the mesh, skeleton, and joint point information is mapped to the sketch figure (picture image) drawn by the user. Accordingly, the mapping-animating step (S500) is based on the mapped animating information. Animation mapping is performed on the action of the actual picture.

For example, in the case of a rabbit, it is implemented so that it can move close to the actual rabbit's movement by the signal 'move' by mapping the joint and skeletal information in the picture depicting the movement of the rabbit. It is made to apply the animation according to the status information (status) of the contents of the virtual reality.

According to the sketch-based active character creation method for generating a sketch-based character as an active character in a virtual environment according to the present invention as described above, accurately recognizes and extracts the character sketched by the user, and optimizes the character By extracting suitable activity skeletons and joints and giving them activity, they can be realized as realistic and natural active characters in a virtual environment, thereby maximizing the user's interest and immersion, and a recognition marker for recognizing 2D pictures is provided. No paper is required, any paper can be used, and it can be applied to various media contents such as games, entertainment, education, public relations, advertisements, and exhibitions. There are advantages to being able to

Although the above-described embodiments have been described with reference to the limited drawings, a person skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

The embodiments described in this specification and the accompanying drawings are merely illustrative of some of the technical ideas included in the present invention. Accordingly, since the embodiments disclosed in the present specification are for explanation rather than limitation of the technical spirit of the present invention, it is obvious that the scope of the technical spirit of the present invention is not limited by these embodiments. Modifications and specific embodiments that can be easily inferred by those skilled in the art within the scope of the technical idea included in the specification and drawings of the present invention should be interpreted as being included in the scope of the present invention.

S100: Sketch drawing reading stage
S200: Sketch figure extraction step
S300: Step to identify types of sketches
S400: Steps to Generate Information for Animating
S410: mesh generation step
S420: Skeleton-joint point creation step
S430: mesh skinning step
S500: Mapping-Animating Phase

Claims (9)

A sketch picture reading step of reading the entire image including the sketch picture drawn by the user on the paper and the background of the sketch picture;
a sketch picture extraction step of extracting only a sketch picture from among all the read images;
a sketch drawing type identification step of analyzing the extracted sketch picture to identify and determine the type of the sketch picture;
a picture image animating information generating step of generating animating information so that the sketch picture is animated based on the determined type of the sketch picture;
a mapping-animating step of mapping the generated animation information to a sketch figure, and animating the mapped sketch figure;
The sketch drawing reading step is read through a photographing means including an image sensor,
In the step of extracting the sketch picture, the color of the picture is binary (binary) extracted using the dimensional transformation of the HSV color space from the read image, and extracted using the RGB to HSV color space conversion formula below,

(Where, H = Hue, S = Saturation, V = Brightness, and RGB is Red Green Blue)
In the step of identifying the type of sketch drawing, the DB image of the image database built in advance and the extracted sketch drawing image are compared and executed by deep learning to identify,
The animation information generation step includes a mesh generation step of generating a mesh based on the binary (binary) and extracted sketch picture image, and the binary (Binary) and extracted sketch picture image. A skeleton-joint point generation step of generating a skeleton and a joint point as a target, and a mesh skinning step of applying the final skeleton to which the generated skeleton and joint point are applied to the generated mesh,
The mesh generation step is made to generate a character mesh to which a polygon mesh is applied using a Delaunay triangulation technique,
In the bone-joint point generation step, the skeleton is extracted by removing pixels that are not satisfied with the set pixel inspection value by examining surrounding pixels around one pixel for all pixels of the binary sketch figure, and the extraction It is made to extract a plurality of joints based on the set number of black-and-white binary pixels by examining surrounding pixels around one pixel in the skeleton,
Further comprising a joint point subdivision process of subdividing between a plurality of joints extracted in the bone-joint point generation step,
The joint point segmentation process is,
Determines whether there are connected pixels around the current pixel,
If there is no pixel connected to the periphery of the current pixel, it is displayed as a segment point,
If there is a neighboring pixel with respect to the current pixel, it is determined whether it is a different feature point, and if it is a different feature point, the process is displayed as the dividing point. Determining whether it is greater than the segment length,
When the movement distance is greater than the predefined segment length, the division point display and the movement distance are initialized, and when the movement distance is smaller than the predefined segment length, the moved pixel is deleted.
How to create a sketch-based active character.
delete delete delete delete delete According to claim 1,
The mesh skinning step
The subdivided joint points and the skeleton including the subdivided joint points are used as final skeletons to map to the character mesh generated in the mesh generation step.
How to create a sketch-based active character.
8. The method of claim 7,
The mesh skinning step
It is characterized in that in the vertex of the mesh, information for mapping animation is generated in which a weight is given to a predetermined joint point to define a movement based on a joint point of a vertex of a certain position.
How to create a sketch-based active character.
9. The method of claim 8,
The weighting is a set of straight lines D arranged in the order of the shortest distance after obtaining the distance between any vertex (Vi) and all joint points B among all vertices (V) using the Euclidean Distance formula (NearBoneList) , and connect the n closest joints to an arbitrary vertex (Vi) in the NearBoneList, which is the set of straight lines D, and give weights, but the closer the joint to the arbitrary vertices, the greater the weight,
The mapping-animating step is characterized in that based on the information for mapping animating, animation mapping is performed.
How to create a sketch-based active character.

Images