KR20130059035A

KR20130059035A - Apparatus and method for completing automatically scenario continuity using sketch, and system for making animation using thereof

Info

Publication number: KR20130059035A
Application number: KR1020110125117A
Authority: KR
Inventors: 윤상민
Original assignee: 연세대학교 산학협력단
Priority date: 2011-11-28
Filing date: 2011-11-28
Publication date: 2013-06-05

Abstract

PURPOSE: An automatic completion apparatus of a scenario continuity by using a sketch, a method thereof, and an animation production system using the same are provided to solve a communication problem between an actor and a production crew by searching a 3D model of 'the sketch of the scenario continuity which is drawn on a multi touch device' and automatically completing a 3D scenario continuity. CONSTITUTION: A sketch feature vector extractor(120) extracts a sketch feature vector from a sketch which is inputted for a scenario continuity by a multi touch unit(110). A 3D model search unit(130) searches a 3D model which corresponds to the sketch from a database(300) based on the sketch feature vector. A continuity completion unit(140) completes the scenario continuity by making the searched 3D model correspond to the sketch. [Reference numerals] (110) Multi touch unit; (120) Sketch feature vector extractor; (130) 3D model search unit; (140) Continuity completion unit; (200) Network; (300) Database

Description

Apparatus and method for completing automatically scenario continuity using sketch, and system for making animation using approximately}

The present invention relates to an automatic completion device and method of a scenario conti using a sketch, and to an animation production system using the same. More specifically, the user automatically inputs a sketch drawn by multi-touch to input 3D model information corresponding to the sketch. The present invention relates to an autocompletion apparatus and method of a scenario conti using a sketch for automatically completing or producing a scenario conti and animation by searching a database, and an animation production system using the same.

Scenario's continuity is common by drawing sketches as a preliminary work for filmmaking, but communication between writers, directors, actors and staff can be disrupted due to the limitations of simple man-drawn sketches.

Recently, devices that process a variety of information such as personal computers (PCs), personal digital assistants (PDAs), notebook PCs, portable multimedia players (PMPs), mobile phones, and the like have been developed. In order to process specific information in an information processing device, an input device is essential. In addition to devices such as a keyboard and a mouse, an input device capable of inputting a user's multi-touch input such as a touch screen and a touch pad is becoming popular. For example, smart phones, tablet PCs, and other terminals with multi-touch functions are rapidly spreading. Consider a technique for automatically completing scenario contents with such a multi-touch interface device.

Therefore, by using the sketch inputted by the user's multi-touch on the device provided with the multi-touch function, the user can search for various 3D model information desired by the user, and introduce the technology to automatically complete the scenario cont with the retrieved 3D model information. The need for good communication between writers, directors, actors and staff due to their limitations. Furthermore, it is also possible to easily create a 3D animation using the searched 3D model information by searching for various 3D model information desired by the user using the sketch input by the multi-touch.

The present invention is to solve the above problems, the automatic completion device of the scenario continuity using the sketch to automatically complete the scenario continuity (scenario continuity) using the searched 3D model from the scenario continuity drawn in the sketch And a method.

Another object of the present invention is to provide an animation production system using sketches to search for a 3D model corresponding to a sketch drawn on a multi-touch screen and to produce a 3D animation based on the retrieved 3D model.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to one or more embodiments of the present invention, a device for automatically completing a scenario conti using a sketch comprises: a multi-touch unit capable of multi-touch input; A sketch feature vector extracting unit extracting a sketch feature vector from the sketch inputted by the multi-touch unit for the scenario continuous; A 3D model retrieval unit for retrieving a 3D model corresponding to the sketch from a database in which the 3D model is stored based on the sketch feature vector; And a continuity completion unit for completing the scenario continuation by mapping the retrieved 3D model to a sketch.

According to an aspect of the present invention, a method for automatically completing a scenario conti using a sketch according to an embodiment of the present invention may be performed by retrieving 3D model (3-Dimensional Model) information from a database based on a sketch drawn by multi-touch. A method for automatically completing, the method comprising: inputting a sketch by multi-touch; Retrieving a 3D model similar to the sketch from a database; And automatically completing a scenario context based on the retrieved 3D model.

An animation production system using sketches according to an embodiment of the present invention for achieving the above object, the system for automatically producing animation using the sketch, a multi-touch interface capable of multi-touch input; A sketch feature vector extractor for extracting a sketch feature vector from the sketch input to the multi-touch interface; A 3D model searcher for searching for a 3D model corresponding to the sketch from a database in which 3D model information is stored based on the sketch feature vector; And an animation producer for producing an animation based on the retrieved 3D model.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, the artist, director, actor, and staff members due to the limitation of simple sketches drawn by searching for the 3D model of the sketch drawn on the multi-touch device and automatically completing the 3D scenario cont based on the sketch This can be done smoothly by solving communication problems that may occur between them.

In addition to searching for 3D models of sketches drawn on multi-touch devices and completing 3D scenario contents based on them, you can easily create 3D animations using the retrieved 3D model information, thereby reducing the cost of producing and producing animations. You can save time.

1 is a block diagram for explaining the structure and concept of the automatic completion device of the scenario conti using the sketch according to an embodiment of the present invention.
2 is a diagram illustrating a process of searching for a 3D model based on a sketch.
3 is a diagram illustrating an embodiment of a scenario continuation drawn on a multi-touch screen.
4 is a block diagram of a 3D model search unit of an automatic completion device of a scenario conti using a sketch according to an embodiment of the present invention.
5 is a diagram illustrating an example of a series of 3D model retrieval processes in an auto-completion apparatus of scenario conti using sketch according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating examples of boundary contours and suggestive contours.
FIG. 7 illustrates an example of 14 3D model images and 14 suggestive outline images projected from the 3D model according to an exemplary embodiment of the present invention.
FIG. 8A shows the eigenvalues and eigenvectors of the projected dark outline of the 3D model, FIG. 8B shows an ellipsoid representation of the projected dark outline of the 3D model, and FIG. 8C illustrates the dark outline of FIG. 8B. A histogram quantized into 18 directional bins is shown.
9 is a flowchart illustrating an automatic completion method of a scenario conti using a sketch according to an embodiment of the present invention.
10 and 11 are detailed flowcharts of an automatic completion method of a scenario conti using a sketch according to an embodiment of the present invention.
12 is a block diagram illustrating a structure and a concept of an animation production system using sketches according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used in a sense that can be commonly understood by those skilled in the art. Also, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram illustrating a structure and a concept of an automatic completion device of a scenario conti using a sketch according to an embodiment of the present invention, Figure 2 is a view showing a process of searching for a 3D model based on the sketch. .

An automatic completion device of a scenario conti using a sketch according to an embodiment of the present invention (hereinafter referred to as an "automatic completion device of a scenario conti", 100) is an apparatus for automatically completing a scenario conti using a sketch. Based on the sketch feature vector extracting unit 120 and the sketch feature vector, the sketch feature vector extractor 120 extracts the sketch feature vector from the sketch inputted for the scenario continuity by the multi-touch unit 110 and the multi-touch unit 110. The 3D model search unit 130 may search for a 3D model corresponding to the sketch from the database 300 in which the 3D model is stored, and the conti completion unit 140 may complete the scenario cont by matching the searched 3D model to the sketch.

In addition, since the automatic completion apparatus 100 of the scenario contiguous has to search for a 3D model corresponding to the input sketch (3-Dimensional Model), the auto complete device 100 may connect to the database 300 in which the 3D model information is stored. As the amount of 3D model information increases, the quality of the 3D scenario conti can be improved. Therefore, the database 300 is preferably connected to the auto complete device 100 of the scenario conti via the network 200. Of course, it will be apparent that the database 300 may be configured inside the auto complete device 100 of the scenario conti. The network 200 may be a wired as well as a wireless network. Since the database 300 may be routed by a web server (not shown) so that the auto complete device 100 of the scenario conti may retrieve the 3D model stored in the database 300, the auto complete device 100 of the scenario conti may be HTML. You may have a web browser (Netscape, Internet Explorer, etc.) that can display the contents of a web page, such as XML or XML.

Here, the sketch means all the information that the user inputs by the multi-touch unit 110 and draws by hand. Since the information input through the multi-touch unit 110 from the user corresponds to a sketch or graffiti, characters, figures, symbols, symbols (symbols), etc. may be recognized from the sketch or graffiti.

The multi-touch unit 110 may include a multi-touch screen 105 because a multi-touch input should be made. That is, a sketch may be drawn on the multi-touch screen 105. At the same time, the multi-touch screen 105 is also where the retrieved 3D model corresponding to the sketch is displayed. Therefore, the multi-touch unit 110 may perform the role of the output interface as well as the function of the input interface. In other words, the multi-touch screen 105 is both an input module and an output module. Thus, the sketch drawn by the user is displayed on the multi-touch screen 105, and the article information retrieved based on the sketch is displayed on the multi-touch screen 105 of the multi-touch unit 110.

When a user touches the multi-touch screen 105, the touch result is displayed directly on the screen 105. Therefore, it is possible to sketch on the multi-touch screen 105 using a user's finger, stylus pen 20, or the like. In order to draw an accurate sketch, it is preferable to use the stylus pen 20 etc. rather than a human finger.

The sketch feature vector extractor 120 extracts a sketch feature vector from the sketch inputted by the multi-touch unit 110 for scenario contiguous operation. There are several techniques for extracting sketch feature vectors, but as an example, Histogram of Oriented Gradients (HOG) algorithm is presented. That is, the sketch feature vector is extracted by applying the HOG algorithm to the input sketch information. The specific application of the HOG algorithm will be described later.

The 3D model search unit 130 substantially performs a search function. Here, the 3D model corresponding to the sketch is retrieved from the database 300 in which the 3D model is stored based on the sketch feature vector. For example, when the plane shape is sketched 25 on the multi-touch screen 105, the 3D model search unit 130 searches for the 3D model 35 of the plane corresponding to the plane sketch 25 to search the multi-touch screen. And display on 105.

The continuity completion unit 140 completes the scenario continuation by matching the retrieved 3D model to the sketch. That is, the most suitable 3D model among the various 3D models retrieved by the 3D model search unit 130 is replaced with the sketch to convert the 2D sketch input into the result of the 3D model to complete the 3D scenario content.

In FIG. 2, when the plane shape is sketched 25 using a stylus pen 20 or the like on the multi-touch screen 105 included in the multi-touch unit 110 of the auto complete device 100 of the scenario conti, sketch The feature vector extractor 120 extracts the feature vector from the plane sketch 25 to recognize the plane. Thereafter, the 3D model search unit 130 searches the 3D models of the various planes from the database 300 to find a plane 3D model 35 having a high similarity, and the conti completion unit 140 relies on the plane sketch 25. To replace the 3D conti. Of course, even though many sketches are drawn on the multi-touch screen 105, the sketch feature vector extractor 120 recognizes the difference between the various sketches and converts all the 2D input sketches to the 3D output model.

FIG. 3 is a diagram illustrating an embodiment of a scenario conti drawn on a multi-touch screen, and FIG. 4 is a block diagram of a 3D model search unit of an autocompletion apparatus of a scenario conti using a sketch according to an embodiment of the present invention.

As shown in FIG. 3, in scenario scenarios, various scenes are drawn and backgrounds and objects are expressed together in one scene. Therefore, quick search and classification of 3D models are important. To this end, the 3D model search unit 130 may include a dark outline extraction module 131, a feature vector extraction module 133, a similarity measurement module 135, a search module 137, and a classification module 139. have.

A plurality of Suggestive Contours are projected from the 3D model information stored in the database 300 through the dark outline contour extraction module 131, and the HOG algorithm is applied to the plurality of dark outlines through the feature vector extraction module 133. Extracts the respective dark outline feature vectors from the plurality of dark outlines, measures the similarity between the sketch feature vectors and the respective dark outline feature vectors through the similarity measurement module 145, and uses the search module 147. The matching 3D model corresponding to the sketch may be searched based on the similarity. In addition, the classification module 149 may classify the 3D model by layers by applying a compression sensing algorithm to the 3D model stored in the database 300.

Hereinafter, the details of application of a histogram of oriented gradients (HOG) algorithm, feature vector extraction, similarity measurement, and compression sensing algorithm application will be described.

FIG. 5 is a diagram illustrating an example of a series of 3D model retrieval processes in an autocompletion apparatus of a scenario conti using sketch according to an embodiment of the present invention, and FIG. 6 is a boundary contour and a suggestive contour. FIG. 7 is a diagram illustrating 3D model images and 14 implicit contour images projected from the 3D model according to an embodiment of the present invention. 8A is a diagram showing the eigenvalues and eigenvectors of the projected dark outline of the 3D model, FIG. 8B is an ellipsoid representation of the projected dark outline of the 3D model, and FIG. 8C is an allusion of FIG. 8B. It is a figure which shows the histogram of which the contour was quantized into 18 direction bins.

In FIG. 5, when the sketch 62 is drawn on the multi-touch screen 105 of the multi-touch unit 110, the sketch feature vector extractor 120 extracts a sketch feature vector that is a feature vector for the sketch. The sketch feature vector is a means for determining similarity together with the dark outline feature vector extracted from the 3D model 72.

Then, a plurality of suggestive contours projected from a plurality of directions are extracted from the 3D model information 72 stored in the database 300 through the dark outline contour module 131, and the feature vector extraction module 133 is extracted. The respective dark outline feature vectors are extracted from the plurality of dark outlines. After that, the similarity measurement module 135 measures the similarity between the sketch feature vector and each of the dark outline feature vectors, and the search module 137 searches for a matching 3D model based on the similarity, and then corresponds to the 3D corresponding to the sketch. Complete the scenario content using a suitable 3D model that informs the model or replaces the input sketch 62. Since the 3D model may have various shapes depending on the viewing direction, it is necessary to extract an appropriate number of dark outlines of the 3D model.

In general, the contour of a shape is expressed through a line, and only the outside of the shape is displayed as a boundary contour, and a 3D dimension of the shape is called a suggestive contour. In FIG. 6, the left side represents the human-shaped boundary outline 82, and the right side represents the human-shaped dark outline 84.

In FIG. 7, 3D model information 72 represented in three dimensions may obtain a plurality of dark outlines 84 projected from various directions in the 3D model information 72. The dark outline 84 is composed of lines that clearly display the parts visible from the surface of the object, and further includes a line therein to display the three-dimensional shape. Therefore, the dark outline outlines the three-dimensional shape more appropriately than the boundary outline.

It is necessary to extract an appropriate number of dark outlines of the 3D model. For example, it is preferable to select 14 projections from the 3D model information 72 in the same space. FIG. 7 shows 14 implicit outlines 84 of 3D model information 72 viewed from 14 directions. Here, six orthographic projections and eight isometric projections are shown.

It is possible to extract the dark outline feature vectors from the 14 dark outlines. Extraction of the feature vector may be performed in a domain of Diffusion Tensor Fields. Diffusion tensor fields were first introduced in the field of medical image processing to measure moisture diffusion in tissues. This uses water to diffuse faster along the fiber than in the vertical direction of the fiber.

A tensor is a physical quantity composed of two vector combinations and has a number of components multiplied by each vector component. The secondary tensor field of the two-dimensional symmetry is expressed by Equation 1 below.

Here, Txy = Tyx because it is symmetric.

The characteristic equation for Equation 1 is expressed by Equation 2 below.

From here,

Is an identity matrix,

Is the eigenvalue of the tensor,

Is a normalized eigenvector. The two eigenvectors are orthogonal to each other.

In this case, the tensor in each pixel is represented by an ellipsoid and the major axis length is proportional to the eigenvalue.

In Fig. 8A, the eigenvalues λ1 and λ2 and the eigenvectors e1 and e2 in various ellipses 40 along the projection direction are represented. Also shown in FIG. 8B is an ellipsoid representation of the dark outline projected from one direction of the 3D model.

The eigenvalues λ1 and λ2 and the eigenvectors e1 and e2 derived from the diffuse tensor field provide the main size and direction of each pixel of the dark outline. Thus, the eigenvalues λ1 and λ2 and the eigenvectors e1 and e2 from the diffuse tensor field are used to calculate the feature vectors of the implicit contours of the 3D model information, and also to calculate the feature vectors of the user-drawn sketch. Used for.

The sketch drawn by the user is called Query Image Ic and the dark outline image derived from the 3D model is Is. Here, Ic and Is are normalized at a fixed size. Then, the feature vectors based on the histogram can be calculated, which is called Hc and Hs. Feature vectors and similarity are derived according to the following steps.

1. The size of the ellipsoid representation of each contour pixel

, And directions

Extract Here, an ellipse is defined by the eigenvalues and the eigenvectors from the analysis of the implicit contour in the topology space of the diffusion tensor field.

2. Size corresponding to the above direction

Quantize into n orientation bins weighted by 8C shows a histogram 50 quantized in 18 direction bins. That is, the gradient intensity of the histogram 50 can be expressed in one direction bins of 20 degrees in the 360 degree panorama. In FIG. 8C, the dark outline shown in FIG. 8B is quantized into 18-way bins. The quantized direction is extracted from the direction of the principal axis of the ellipsoid representation of the dark outline. The main direction of the ellipsoid model is determined by the eigenvectors. In FIG. 8C the height of histogram 50 represents the gradient intensity of the pixel.

3. The feature vectors Hs and Hc of the histogram of orientation are normalized to unit length vectors by the sum of all entries.

4. Similarity between query image for sketch and one view image of 3D model

Is given by Equation 3 below.

The similarity value is between 0 and 1, and when Ic and Is are the same, the similarity value is 1.

5. Projection of 3D models from multiple directions is possible for user-drawn sketch-based 3D model retrieval. For example, as shown in FIG. 6, projection can be performed in 14 directions. The similarity with the sketch feature vector can be obtained using the dark outline feature vectors in each direction, and the largest similarity can be determined from this.

For efficiency of retrieving 3D models from sketches, 3D models can be classified using Compressive Sensing (CS). Compression sensing (CS) is a combination of compression and sensing processes and is an emerging field in which signals are compressed at a greatly reduced rate as they are sampled.

Set of Implicit Contour Feature Vectors for N Directions from All 3D Models

May be represented by Equation 4 below.

Hs is a Dx1 column vector, where D is the number of bins.

And a predictor matrix

Since has linearity, it can be converted to Equation 5 below.

Where M is less than N.

Given a sufficient number of known samples obtained from training HOG features, a label of the feature vector Hq of an unknown sketched query image can be classified from the query model.

silver

This is one of the labels of the classes in the database.

For each class of,

The process can be represented as a characteristic function selected from feature vectors associated with each class of 3D model.

The distance from the trained features can be measured to estimate an unknown Hq to rank a set of 3D models and modeled from the sum of mean-squared-errors (MSE) across all labels.

The distance Sj from one of the classes is given by the following equation.

Thus, the 3D models in the database 300 are efficiently classified and retrieved according to Sj, which is the ratio of similarities between classes.

9 is a flowchart illustrating an automatic completion method of a scenario conti using a sketch according to an embodiment of the present invention, and FIGS. 10 and 11 are detailed flowcharts of an automatic completion method of a scenario conti using a sketch according to an embodiment of the present invention. to be.

According to an embodiment of the present invention, the automatic completion of the scenario conti using the sketch, based on the sketch drawn by the multi-touch retrieves the 3D model (3-Dimensional Model) information from the database 300 automatically completes the scenario conti It is.

To this end, a sketch is input by multi-touch (S910), a 3D model similar to the sketch is searched from the database 300 (S920), and the searched 3D model is replaced with the input sketch to automatically complete the scenario contents. (S930). The sketch is input by an interface such as a touch screen or a touch pad, and the user may draw a sketch by touching it with a human finger. However, it is preferable to input the sketch using the stylus pen 20 to draw a sketch accurately and quickly.

3D model information retrieved based on sketches should be constructed in advance. In FIG. 10, after the DB construction is performed to store the 3D model information in the database 300 (S905), the user inputs a sketch by multi-touch (S910) and retrieves a 3D model similar to the sketch from the database 300. In operation S920, a series of processes are performed to automatically complete the scenario contents by replacing the searched 3D model with the input sketch (S930).

Also, in order to recognize the sketch accurately and quickly, the sketch feature vector may be extracted (S915), and the sketch recognition information may be derived based on the sketch feature vector. Here, in order to extract a sketch feature vector from the sketch, the feature vector may be extracted by applying a histogram of oriented gradients (HOG) algorithm. Histogram of Oriented Gradients (HOG) algorithm has been described above and will be omitted.

When the sketch is input by multi-touch (S910), the 3D model similar to the sketch is searched from the database 300 (S920), and the searched 3D model is replaced with the input sketch to automatically complete the scenario contents. (S930) Save the used searched 3D model in the device 100 or in the database 300 (S935), and then utilizes the stored search results, through which users can complete the effect of faster 3D scenario conti Can be provided to

In addition, as described above, in order to extract the feature vector from the sketch or the feature vector from the 3D model information, a Histogram of Oriented Gradients (HOG) algorithm is applied. Extraction of feature vectors can be performed in the domain of Diffusion Tensor Fields, and the main size and direction of each pixel are derived from the eigenvalues and eigenvectors derived from the diffusion tensor fields. That is, the eigenvalues and eigenvectors from the diffusion tensor field are used to calculate the feature vectors of the implicit contours of the 3D model information, and are also used to calculate the feature vectors of the sketches drawn by the user. Then, the gradient intensity and the gradient direction of the luminance with respect to the plurality of directions of the image are calculated, and the frequency vector distribution of the gradient intensity of the luminance with respect to the calculated gradient direction is represented as a histogram to extract the feature vector.

In addition, various feature vectors according to the direction of the 3D model may be extracted. In FIG. 11, after the user inputs a sketch by multi-touch (S910), extracts a feature vector from the sketch (S915), and searches for a 3D model similar to the sketch from the database 300 (S920), 3D of various objects is displayed. The model exists, and several 3D models may exist on the same object. For example, even if you sketch and enter a car, there are many 3D models for the type or model of the car. Therefore, it is necessary to improve the efficiency of searching by classifying 3D models by layers. For example, a vehicle may be classified into an airplane, a ship, a car, and the like, and the vehicle may be reclassified according to a car, a truck, a bus, and the like.

That is, the 3D model is different for each object, even if the same object may have a number of model shapes, it is desirable to increase the efficiency of the search by classifying the 3D model by layer by applying a compression sensing technique for the efficiency of the search. In FIG. 11, it is preferable to classify the various 3D models by layers by applying a compression sensing algorithm to various types of 3D models (S922). Thereafter, a plurality of Suggestive Contours projected from a plurality of directions are extracted (S923), and respective dark contour feature vectors are extracted from the plurality of Suggestive contours by applying a HOG algorithm (S925), and the sketch is performed. The degree of similarity between the feature vector and each of the dark outline contour vectors is measured (S927), and based on the similarity, the 3D model corresponding to the sketch input by the user is searched to automatically complete the scenario contiguous based on the similarity (S930). ).

On the other hand, in-store item search method using a sketch according to an embodiment of the present invention can be implemented as a module by software and hardware, the above-described embodiments of the present invention can be written as a program that can be executed in a computer The present invention may be implemented in a general-purpose computer for operating the program using a computer-readable recording medium. The computer-readable recording medium is implemented in the form of a carrier wave such as a ROM, a floppy disk, a magnetic medium such as a hard disk, an optical medium such as a CD or a DVD, and a transmission through the Internet. In addition, the computer-readable recording medium may be distributed to a network-connected computer system so that computer-readable codes may be stored and executed in a distributed manner.

12 is a block diagram illustrating a structure and a concept of an animation production system using sketches according to an embodiment of the present invention.

An animation production system using a sketch according to an embodiment of the present invention (hereinafter, referred to as an “animation production system” 500) includes a multi-touch interface 520, a sketch feature vector extractor 540, a 3D model finder 560, and animation. A maker 580.

Since the animation production system 500 needs to search for a 3D model corresponding to the input sketch (3-Dimensional Model), the animation production system 500 may access the database 300 in which the 3D model information is stored. It is preferable that the database 300 is connected to the animation production system 500 through the network 200, but it will be obvious that the database 300 may be configured inside the animation production system 500. Since the database 300 may be routed by a web server (not shown) so that the animation production system 500 may retrieve a 3D model stored in the database 300, the animation production system 500 may be configured to display a web page such as HTML or XML. You may have a web browser (Netscape, Internet Explorer, etc.) that can display content. Therefore, the animation production system 500 may produce animation using a 3D model that can be searched on the Internet, which is a sea of information.

The multi-touch interface 520 is a place where multi-touch input is possible, and serves as an interface for inputting a sketch. This performs a similar function to the multi-touch unit 110 of the auto complete device 100 of the scenario continuation described above, the multi-touch interface 520 may also include a multi-touch screen.

The sketch feature vector extractor 540 extracts a sketch feature vector from the sketch input to the multi-touch interface 520, and extracts the sketch feature vector by applying a histogram of oriented gradients (HOG) algorithm. Since the HOG algorithm has been described above, the detailed description thereof will be omitted.

The 3D model finder 560 searches for a 3D model corresponding to the sketch from the database 300 in which 3-Dimensional Model information is stored, based on the sketch feature vector. Since various 3D models are stored in the database 300, and the 3D model is searched by comparing the sketch and the 3D model, the 3D model finder 560 includes the dark outline extraction unit 561, the feature vector extracting unit 563, and the similarity. The measurement unit 565, the 3D model retrieval unit 567, and the classification unit 569 may be provided.

That is, the 3D model finder 560 may include a dark outline contour extracting unit 561 for extracting a plurality of suggestive contours projected from the 3D model information stored in the database 300, and apply a HOG algorithm to the plurality of dark outlines. A feature vector extracting unit 563 which extracts each dark outline feature vector from the plurality of dark outlines, a similarity measuring unit 565 for measuring similarity between the sketch feature vector and the respective dark outline feature vectors; And a 3D model search unit 567 for searching for a matching 3D model corresponding to the sketch based on the similarity, and efficiently searching for a 3D model matching the input sketch using a feature vector and a HOG algorithm. Can be.

In addition, the 3D model finder 560 further includes a classification unit 569 for classifying the 3D model by layers, and applies a compression sensing algorithm to the 3D model through the classification unit 569 to apply the 3D model. Classify according to similarity, so you can quickly search for 3D models that match the sketch.

Here, the specific algorithm applied to the 3D model searcher 560 is similar to the 3D model searcher 130 described above, and thus will be omitted.

The animation maker 580 creates an animation based on the retrieved 3D model. The animation maker 580 connects the retrieved 3D models to connect the completed still images to make a video. That is, when a 3D model corresponding to the input sketch is searched, the 3D model may be replaced with the sketch, and the substituted image may be connected to the video to complete the animation.

The automatic completion device 100 and / or the method of the scenario continuity replaces the 3D model corresponding to the input sketch to complete the continuation in the form of a monograph, and the animation production system 500 corresponds to the 3D model corresponding to the input sketch. By converting this into a video, the animation is completed in the form of a cartoon film.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

100: Scenario Conti's AutoComplete Device
110: multi-touch unit 120: sketch feature vector extraction unit
130: 3D model search unit 140: Conti complete unit
200: network 300: database
500: Sketching Animation System
520: multi-touch interface 540: sketch feature vector extractor
560: 3D Model Browser 580: Animation Maker

Claims

In the device that automatically completes the scenario content using the sketch,
A multi-touch unit capable of multi-touch input;
A sketch feature vector extracting unit extracting a sketch feature vector from the sketch inputted by the multi-touch unit for the scenario continuous;
A 3D model retrieval unit for retrieving a 3D model corresponding to the sketch from a database in which the 3D model is stored based on the sketch feature vector; And
And a continuity completion unit for completing the scenario continuation by mapping the retrieved 3D model to a sketch.

The method of claim 1,
The sketch includes all the information drawn by the user input to the multi-touch unit, the automatic completion device of the scenario conti using the sketch.

The method of claim 1,
The multi-touch unit includes a multi-touch screen, the automatic completion device of the scenario conti using a sketch.

The method of claim 1,
The sketch feature vector extractor extracts the sketch feature vector by applying a histogram of oriented gradients (HOG) algorithm.

The method of claim 1,
The 3D model search unit,
An implicit contour extraction module for extracting a plurality of suggestive contours projected from the 3D model information stored in the database;
A feature vector extraction module for applying a HOG algorithm to the plurality of dark outlines to extract respective dark outline feature vectors from the plurality of dark outlines;
A similarity measurement module for measuring a similarity between the sketch feature vector and each of the dark outline feature vectors; And
And a search module for searching for a matching 3D model corresponding to the sketch based on the similarity.

6. The method of claim 5,
The 3D model retrieval unit further includes a classification module for classifying the 3D models by layers by applying a compression sensing algorithm to the 3D models stored in the database.

In the method to automatically complete the scenario content by retrieving 3D model information (3-Dimensional Model) from the database based on the multi-touch sketch,
Inputting the sketch by multi-touch;
Retrieving a 3D model similar to the sketch from a database; And
Comprising the step of automatically completing the scenarios based on the searched 3D model, automatic completion method of the scenarios using the sketch.

8. The method of claim 7,
And storing the 3D model information in a database.

8. The method of claim 7,
The inputting of the sketch may include inputting using the user's finger or a stylus pen.

8. The method of claim 7,
And extracting a sketch feature vector from the sketch.

The method of claim 10,
The extracting of the feature vector may include extracting the feature vector by applying a histogram of oriented gradients (HOG) algorithm.

The method of claim 10,
Searching for the 3D model,
Extracting a plurality of Suggestive Contours from the 3D model information;
Extracting respective dark outline feature vectors from the plurality of dark outlines by applying a HOG algorithm;
Measuring a similarity between the sketch feature vector and each of the dark outline feature vectors; And
And searching for the 3D model based on the similarity.

13. The method of claim 12,
And classifying the 3D model by layers by applying a compression sensing algorithm to the 3D model.

8. The method of claim 7,
And storing the retrieved 3D model information separately.

In a system that automatically creates animations using sketches,
A multi-touch interface capable of multi-touch input;
A sketch feature vector extractor for extracting a sketch feature vector from the sketch input to the multi-touch interface;
A 3D model searcher for searching for a 3D model corresponding to the sketch from a database in which 3D model information is stored based on the sketch feature vector; And
An animation production system using a sketch, including an animation producer for producing an animation based on the retrieved 3D model.

16. The method of claim 15,
The multi-touch interface includes a multi-touch screen, animation production system using a sketch.

16. The method of claim 15,
The sketch feature vector extractor extracts the sketch feature vector by applying a histogram of oriented gradients (HOG) algorithm.

16. The method of claim 15,
The 3D model finder,
A dark outline contour extracting unit for extracting a plurality of suggestive contours projected from the 3D model information stored in the database;
A feature vector extracting unit for extracting respective dark outline feature vectors from the plurality of dark outlines by applying a HOG algorithm to the plurality of dark outlines;
A similarity measuring unit measuring a similarity between the sketch feature vector and each of the dark outline feature vectors; And
And a 3D model search unit for searching for a matching 3D model corresponding to the sketch based on the similarity.

19. The method of claim 18,
The 3D model finder further comprises a classification unit for classifying the 3D model by layer by applying a compression sensing (Compressive Sensing) algorithm to the 3D model, animation production system using a sketch.