JP5526239B2 - Method, device, and computer-readable medium for generating digital pictures - Google Patents

Description

  Embodiments relate to the field of computer graphics and scientific visualization. By way of example, embodiments relate to methods such as extracting a curve for a three-dimensional (3D) object, generating a digital picture showing a 3D object based on the extracted curve, and stylized animation to convey the 3D object.

  Computer graphics has become a common medium for science, art and communication. Computer graphics has long been defined as a quest in creating realistic images. Research in this area has been very successful. Computers can now produce images that are so real that they cannot be distinguished from photographs. However, in so many aspects non-realistic images are more effective and preferred. Experience shows that very often the artistic drawing or painting of the same scene proves to be both more efficient in communicating information and more satisfying in the visual experience than photography. This is mainly due to the abstraction of various artistic styles. In machine manuals and medical texts, there are more examples where illustrations are widely used instead of photographs to explore inside complex structures and shapes. Under the rule of “detailed as needed, as simple as possible”, conventional illustrations are particularly efficient in communicating information. Also, non-photorealistic images are characteristically more stylish than photos, while focusing on specific features of the shape, omitting extraneous details, exposing subtle attributes and hidden parts. It can also be seen that it has many advantages such as using selection to classify complex scenes and simplifying the shape to prevent visual overload. In recent years, much research has been done in the field of non-photorealistic rendering.

  As one of the most important techniques of traditional art and illustration, line drawing can represent large amounts of information in a highly abstract and relatively concise manner. Computer generated line drawing has been developed since the beginning of computer graphics. In recent years, computer-generated line drawing has been greatly improved as a result of improvements in non-photorealistic rendering techniques.

  Extensive research has been carried out in computer-generated line drawing for 3D object shapes, inspired by the effect of information transmission in conventional line drawing and the beauty of perceptual experience. Appel proposed a method for the extraction of silhouettes and sharp features in object space, and rendering in image space ("The notion of quantitative and the machine rendering of solids", 1967, the second annual event of the 22nd annual conference. Record). Gooch et al. Introduced an illustration system based on interactive technology ("Interactive technical illustration", 1999 Symposium on Interactive 3D graphics). Hertzmann and Zorin proposed an algorithm for extracting silhouettes on smooth surfaces and automatically generating line drawings ("Illustrating smooth surfaces" Sigma Graph 2000 proceedings). To convey the internal structure and complexity of 3D shapes, Kalnins et al. Proposed a method for drawing wrinkles in the WYSIWYG NPR drawing system ("WYSIWYG NPR: Drawing strokes on 3d models" siggraph 2002 ), Interrante et al. Used ridge lines and valley lines to enhance the transparent skin surface ("Enhancing transformant skin surfaces and valley lines" IEEE Visualization 1995). Wilson and Ma described a method of expressing geometric complexity using pen and ink style line drawing (the minutes of “Representing complexity in computer-generated pen-and-ink illustrations” NPAR 2004). Ni et al. Focused on multi-scale line drawing from 3D mesh and presented a method for controlling the size of shape features shown in computer-generated line drawing depending on the view ("Multi-scale line drawings from 3D meshes ”(2006 minutes on Symposium on Interactive 3D graphics).

  In a state-of-the-art computer-generated line drawing system, many categories of feature lines are widely used. The boundary line is one of these feature lines found in a 3D model with an open surface. An equi-ray is a line of constant illumination on the surface that is also a shading boundary between toon shading regions. The self-intersection line is where the model surfaces intersect. Of these, contours, implied contours, ridges, and valleys are the most representative feature lines in recent years. A contour is usually defined as the visible part of a silhouette, and is a line whose surface leaves the viewer and disappears. The contour shows the most powerful clue when distinguishing between the model and the background. The implicit contour proposed by DeCarlo et al. (The minutes of “Suggestive controls for conveying shape” Sigma 2003) is considered an extension of the actual contour of the surface. An implied contour is a line drawn on a clearly visible part of the surface, and the original contour can be seen by changing the viewpoint slightly, and the implied contour clearly conveys the shape. Implied contours are the precursors of view-dependent feature lines that are particularly important in animation sequences. Ridge and valley lines, also called crest lines, are curves on the surface along which the surface curves sharply. These lines are one of the most powerful descriptors for shape variation, widely used in CAD / CAM (Computer Aided Design and Computer Aided Manufacturing) applications.

  Although these are the latest feature lines, none of these lines alone can capture all visually important features. Since the contour cannot capture the structure and complexity within the shape, only the contour is very limited. Implicit contours cannot show prominent features of convex areas, and white ridge / valley lines are fixed on the surface and are not a natural line drawing in animation, and do not depend on gazes that appear more like surface marks It is a characteristic line.

  Thus, using an improved method of extracting curves for 3D objects that can increase the perceptual consistency of digital pictures and give the user freedom in achieving the desired line drawing, It is desirable to generate a digital picture of a 3D object.

  It is also desirable to extend the generation of a digital picture of a 3D object to the generation of a digital picture video sequence using dynamic settings relating to the stylization of the digital picture video sequence. In other words, it would be desirable to provide a method of generating animation using digital picture coherence. In this context, animation refers to a video sequence of digital pictures. Stylization may refer to the presentation of a digital picture, or sequence of digital pictures, in a style or stylistic pattern, rather than by nature or convention. For example, styling can be a property of a non-photorealistic application. In this context, digital picture coherence may refer to the consistency of styling between different digital pictures in a video sequence.

  In one embodiment, a method is provided for generating a digital picture, the method comprising receiving a three-dimensional representation for each of a plurality of curves, each curve at least partly representing a shape of a three-dimensional object. Representatively receiving, using the three-dimensional information provided by the three-dimensional representation, grouping a plurality of curves into at least one curve group based on the three-dimensional representation of the curve, and the curve Associating each curve group with a picture element based on the curve of the group, and using the picture element to form a digital picture.

  According to other embodiments, devices and computer-readable media according to the above-described method are provided.

  It should be noted that the embodiments recited in the claims subordinate to the independent method claims apply equally to corresponding devices and computer-readable media where applicable.

  The drawings are not necessarily drawn to scale, but instead are enlarged as a whole in illustrating the principles of the various embodiments. In the following description, various embodiments will be described with reference to the following drawings.

FIG. 3 is a diagram illustrating the definition of PEL on a surface in one embodiment. (A)-(d) is a figure which shows the effect of PEL. (A)-(d) is a figure which shows the effect of the main light and auxiliary light with respect to 3D object for obtaining PEL. (A)-(f) is a figure which shows adjustment of the local auxiliary light with respect to 3D object. (A) the vertex _{(x 1,} y _1), a diagram showing an example of a (x _{2, y} 2), and _{(x 3, y 3),} (b) is a diagram showing a surface coordinate system is there. (A)-(f) is a figure of PEL extracted about a plurality of different 3D objects in a plurality of different viewpoints. FIG. 3 illustrates a process for generating a video sequence comprising a plurality of digital pictures in one embodiment. FIG. 3 illustrates a method in one embodiment. FIG. 2 illustrates a device in one embodiment. It is a figure which shows the judgment of the continuity of two curves. It is a figure which shows the judgment of the parallelism of two curves. It is a figure which shows judgment of the possibility of two curves which should be connected to a closed curve. (A)-(c) is a figure which shows the curve grouping between several different digital pictures. It is a figure which shows 3D object. It is the figure which expressed the curve extracted from 3D object shown by Fig.14 (a) by one style. It is the figure which expressed the curve extracted from 3D object shown by Fig.14 (a) by another style. (A)-(b) is a figure which shows an example of the judgment of the single curve correspondence between several different digital pictures. 1 is a diagram illustrating a computer according to one embodiment. FIG.

  Edge detection is a well-studied task in computer vision. In a 2D image, an edge point is defined as a point where the gradient magnitude is assumed to be maximum in the gradient direction. From observations in human vision and perception, it can be seen that sudden changes in brightness play a crucial role in representing and recovering 3D information. Also, by observation, in the case of a grayscale image of an illuminated 3D object under general illumination and reflection conditions, the zero crossing of the second directional derivative of the image intensity along the direction of the intensity gradient is It can be seen that it occurs near ridges and valleys. Inspired by edge detection in image processing, in one embodiment, a type of curve, sometimes referred to as a light extreme line (PEL), features local variations in direct illumination to the surface of a 3D object. Defined.

  In one embodiment, the PEL is a set of points on the surface of the 3D object where the illumination variation in the direction of the gradient reaches a local maximum.

  FIG. 1 shows the definition of PEL.

If the smooth surface patch S: DＳR ^2- > R ³ in FIG. 1, the illumination function f: S-> R can be assumed to be C ³ continuity. In this context, R ² refers to a two-dimensional real space, ie, a vector space of real pairs. R ³ may refer to a three-dimensional real space, ie, a real triple vector space. C ^3, the third derivative is present, it means that it is a continuity.

Assuming that an arbitrary point pεS as shown in FIG. 1, n is a surface normal at a point P on the surface patch S. The tangent vectors S _u and S _v define a local coordinate system on the point p of the surface patch S. However,

It is. If S is a regular patch, S _u and S _v are linearly independent and form the basis of the tangent space.

  The gradient of f is

Where f _u and f _v are the partial derivatives of the scalar function f, ie

and

Where E, F and G are the coefficients of the first basic form of S, ie E = (S _u , S _u ), F = (S _u , S _v ) and G = (S _v , S _v ), And (,) is an inner product. Assuming that an arbitrary point pεS as shown in FIG. 1, n is the surface normal at the point P on the surface patch S, and w is f on the tangent space of the point P on the surface patch S. Unit vector of gradients, i.e.

And Then PEL is
D _w || ∇f || = 0 and _{D w D w || ∇f || <} 0 (2)
Can be defined as a set of points that satisfy D _w is referred to as the direction derivative operator. For a scalar function g, D _w g = (∇g, w) is the directional derivative along w. Intuitively speaking, D _w may measure changes in the function f along the direction w.

A global solution with D _w || ∇f || = 0 forms a closed curve or a curve that terminates at a surface boundary. Thus, the surface can be divided into two regions: a region satisfying D _w || ∇f ||> 0 and a region satisfying D _w || ∇f || <0. In one embodiment, PEL corresponds to D _w D _w || ∇f || <0, that is, the portion of the curve where the illumination variation reaches a local maximum. PEL effectively conveys useful information about the shape of the 3D object.

  2A to 2D show a PEL for indicating the shape of the 3D object.

FIG. 2A shows the surface 200 of the 3D object. The dotted line in FIG. 2B indicates a curve that satisfies D _w D _w || ∇f || <0 on the surface 200. The dotted line in FIG. 2C shows a curve that satisfies D _w || ∇f || = 0 and D _w D _w || ∇f || <0 on the surface 200. FIG. 2D shows a region 202 where D _w || ∇f ||> 0 and a region 204 where D _w || ∇f || <0.

Referring to the surface 200 shown in FIG. 2 (a), a global solution with D _w || ∇f || = 0 forms a closed curve or a curve that terminates at a surface boundary. These curves show that the surface 200 is in two regions: a region satisfying D _w || ∇f ||> 0 (colored dark gray in FIG. 2D), and D _w || ∇f. || <0 (colored light gray in FIG. 2D). PEL (indicated by the curve indicated by the dotted line in FIG. 2 (b)) corresponds to D _w D _w || ∇f || <0, ie the part of the curve where the illumination variation reaches a local maximum To do. The curve indicated by the dotted line in FIG. 2C corresponds to a point satisfying D _w || ∇f || = 0 and D _w D _w || ∇f || <0, and is useful for the shape of the 3D object. Don't pass on information. Note that the curves on the hidden surface are not shown in (b) and (c).

  In one embodiment, the illumination of the 3D surface may be entirely user controllable. In other words, in one embodiment, the user can achieve the desired illumination by manipulating light freely.

  3D shaped illumination depends on the lighting conditions. Among the various light models available in graphics, the one commonly used is the following Phong specular model.

However, n is the surface normal, 1 light direction, v is the view direction, r is the reflection direction, n _s is greater than glossy constant for the surface is smoother, k _a is an ambient reflection constant , the I _a is the ambient light intensity, k _d is the diffusion reflection constant, I _d is the diffuse light intensity, k _s is the specular reflection constant, the I _s is the specular reflection intensity. In this context, the surface normal at a surface point P can be referred to as a vector perpendicular to the tangent plane to that surface at P. I _amb represents ambient light. In this context, ambient light may refer to illumination that surrounds an object or scene. I _diff represents diffuse light emanating from the surface, which is light that is uniformly directed. I _spec represents specular reflection light. In this context, specular light may refer to light that reflects more strongly along and around the reflection vector from the surface. Since equation (3) is a function of the light direction and the view direction, the corresponding PEL depends on the view and light in one embodiment.

In one embodiment, the PEL conveys the shape of the 3D object by highlighting significant variations in illumination across the 3D surface. For the purpose of presenting a 3D shape with respect to a human perceptual experience, in one embodiment, the light model may emphasize illumination variations that are strongly related to shape variations, and illumination variations, or shapes, that are less related to shape variations. Illumination fluctuations orthogonal to the fluctuations can be suppressed. In the Phong shading model, the ambient light I _amb is constant and does not contribute to illumination fluctuations. The diffused light I _diff is determined by both shape and light. The fluctuation of the diffused light I _diff is greatly affected by the fluctuation of the surface normal. The specular reflected light I _spec is dominated by the view state and the lighting state and can be regarded as the orthogonal coefficient of the 3D surface. In other words, I _spec is not related to the shape of the 3D object. Thus, the specular reflected light I _spec can introduce feature lines that are not very related to the 3D shape and are dispersed for the illustration of the 3D shape. On the other hand, the output factor of the specular reflection light I _spec has a higher calculation load. Thus, in one embodiment, ambient light I _amb and specular reflection light I _spec can be ignored and only diffuse light is considered. Therefore, the following equation is obtained for the lighting model from Equation (3).
I = k _d I _d (n · l) (4)

  Note that the line-of-sight vector v is not involved in equation (4). Thus, the corresponding PEL in one embodiment is independent of view position.

  In another embodiment, view dependent properties may be desirable in real world applications. More importantly, because real-world 3D shapes are complex in geometry and have many details, the criteria for locating feature lines often vary with different parts and are quite subjective. Is. It is therefore desirable to provide the user with more freedom to control the desired illustration. To meet view dependencies, as well as user controllability and performance requirements, in one embodiment, the following light model for PEL is provided:

  The light model used in one embodiment comprises main light. For example, a light source for main light that uses directional light whose light rays are parallel to the line-of-sight vector is set. In other words, the term “main light” is similar to “headlights” that are widely used in vehicle lighting systems. This setting is based on human perceptual experience. Thus, the light moves when the viewpoint changes. As a result, the illumination and the corresponding PEL are view dependent.

  In one embodiment, the light model used further comprises at least one auxiliary light. In other words, an optional spot light can be used to shine particularly intense light on the area specified by the user. The contribution of the auxiliary light to the illumination is local and does not depend on the view position. n · l can be used to define an auxiliary spotlight, but it can be applied locally to the region of interest to the user, thereby giving the user the freedom to control the PEL in different areas Given to.

  3A to 3D show application examples of a light model for extracting PEL in one embodiment. To show the effect of the light model, select a Stanford bunny to be a 3D object.

  FIG. 3 (a) shows that four lights are applied to the Stanford bunny for extraction of the PEL to show the Stanford bunny. The main light # 1 is directional light whose direction matches the camera direction. By using only the main light # 1, the PEL can convey the rough shape well (FIG. 3B).

  However, some important lines for the eyes and feet (as shown in (b)) may be lost. Auxiliary spot lights # 2 and # 3 can then be used to increase the local contrast for the eyes and feet to add more detail (as shown in FIG. 3 (c)). it can. Note that the Stanford Bunny body contains many short feature lines due to the small relief. However, these lines may be less preferred by users. In order to remove these lines, another auxiliary spot light # 4 shown in FIG. 3 (a) can be added to reduce local contrast on the body. As a result shown in FIG. 3 (d), most of the small feature lines are successfully removed.

  More specifically, FIG. 3 (a) shows an example where four lights are used to show a Stanford bunny. Light # 1 is main light that matches the view direction. The purpose of Main Light # 1 is to convey the rough geometry of Stanford Bunny. Lights # 2, # 3 and # 4 are auxiliary lights that can be used to increase or decrease local contrast. For example, in this case, the auxiliary light # 2 is locally applied to the eye area. Auxiliary light # 3 is applied to the foot area, and auxiliary light # 4 is applied to the body area of the Stanford bunny. In one embodiment, auxiliary light is used to increase or decrease the local contrast of a specified area. Thus, PELs can be added or removed according to the local characteristics of the 3D object or according to user requirements.

  FIG. 3B shows a rough geometry of a Stanford bunny by PEL when only the main light # 1 is applied. As can be seen in FIG. 3 (b), some important lines have disappeared, such as lines that can show the eyes and feet in more detail. There are also many small PELs due to some small features of the Stanford bunny, ie the features on the body area, but these PELs cause scattering and convey the shape of the hind leg area Useless.

  FIG. 3C shows the geometry of Stanford Bunny PEL when main light # 1 and auxiliary lights # 2 and # 3 are applied. As can be seen from FIG. 3 (c), the two auxiliary lights # 2 and # 3 are used to increase the contrast for the eyes and feet.

  FIG. 3 (d) shows the Stamford Bunny PEL geometry when additional auxiliary light # 4 is used to reduce the contrast on the body. As a result, better extraction of Stanford Bunny PEL is achieved.

  For simple shapes, it is often possible to achieve the desired illustration using only main light. However, most real-world models have complex geometries and many details that cannot be shown using a single light. Thus, in one embodiment, auxiliary light is used to improve illustration. One advantage of PEL is that the user has full control over the auxiliary light, such as the number of lights, their intensity, position, and direction. Manipulating these parameters manually can be a tedious task. In one embodiment, a method is provided for automatically calculating an optimal setting for auxiliary light to minimize user interaction effort.

In one embodiment, the user can specify several target areas to be improved. Each area is small enough that only one auxiliary light I _aux can be used.

  User specified patch

And In this context, A means that the integration is performed over the entire area and c is the center of gravity of region A.

In one embodiment, the spot light I _aux is the light direction at the position x.

Can be applied at. In one embodiment, the illumination contrast for S ′ with the spot light I _aux is maximized when it is preferred that the user add details. In another embodiment, the illumination contrast for S ′ by the spot light I _aux is minimized when it is desirable for the line to be removed by the user. For example,
To add details,

(5) or to remove details,

(6) can be used. However,

S (u, v) is a given 3D surface patch, d is the distance between the light source and surface S, k _aux is the intensity of the auxiliary light, k _c , k _l and k _q is a coefficient of a quadratic function of the distance d, and n (u, v) is a unit normal vector, that is,

Where l is a unit vector in the light direction.

  Note that the cut-off angle of the spot light can cause illumination discontinuity, which in turn can result in unwanted feature lines. In this context, the cut-off angel may refer to the cone-shaped angle of the spotlight and is associated with the spotlight boundary applied to the surface S of the 3D object.

In order to avoid additional feature lines caused by illumination discontinuities, in one embodiment, the cut-off angle of the spot light can be large enough to cover S ′. Two illumination functions f = I _main and f = I _aux are applied to each point inside the spotlight region (greater than S ′). To calculate the derivative of illumination, f = I _aux is used for any point pεS ′, and any point pεS \ S ′ (p is a point in S, but in S ′ F = I _main is used for (which means not a point). Thus, overlapping the results of two functions in discontinuity ensures local continuity everywhere by switching between the two functions f = I _main and f = I _main + I _aux Can do. Since the PEL is defined using local extrema and the illumination is locally continuous for all points, the auxiliary light can be used to create a SEL PEL illustration without introducing unwanted lines on the boundary. Can be improved.

In one embodiment, the illumination function is set to be f = I _aux for auxiliary light optimization and PEL extraction at S ′. In one embodiment, the optimal auxiliary light is placed in the object space. Therefore, the optimum setting of the auxiliary light is not affected by changes in the main light and various view states. In one embodiment, the optimization process is not real time. In other words, the optimization is performed in 3D space before extracting the curve of the 3D object to show in the 2D image. In one embodiment, once auxiliary light is determined, the auxiliary light is fixed in the object space, which means that the setting of the auxiliary light is not view dependent. Therefore, once the auxiliary light is calculated, it can be used when rendering the PEL for any viewpoint. Thus, further computational overhead can be avoided. More importantly, auxiliary light can improve PEL illustration without causing any incoherence in the animation when the viewpoint changes.

  FIGS. 4 (a)-(f) show an example illustrating the improvement in PEL illustration for the Stanford Bunny eye area using optimal lighting in one embodiment.

  FIG. 4A shows an eye area of the bunny model when only main light is applied. FIG. 4D shows an extracted PEL based on the bunny model shown in FIG. It can be seen that using only the main light does not show the eye area well due to the low contrast around the eyes.

  FIG. 4B shows an eye region of a bunny model to which spot light that covers the eye region and maximizes contrast is applied in addition to the main light. FIG. 4 (e) shows the eye area of the bunny model shown in FIG. 4 (b) using PEL. As can be seen in the figure, the features are emphasized and the corresponding PEL shows the eyes very well.

  FIG. 4C shows an eye region of a bunny model to which spot light that minimizes the local contrast of the eye region is applied in addition to the main light. FIG. 4F shows the eye area of the bunny model shown in FIG. 4C using PEL. As expected, the feature lines are reduced accordingly.

  Note that illumination discontinuities can occur in shaded images (FIGS. 4 (b) and (c)). Ensuring local continuity for PEL extraction as described above achieves an improved PEL illustration desirable to the user of Stanford Bunny's eyes without introducing meaningless feature lines on the boundary (FIG. 4 (e) and (f)). It is also demonstrated that PEL is not extracted from the image space. When using an edge detection algorithm in image space, edges may occur due to the aforementioned discontinuities in illumination, which may be undesirable. This also explains one important difference between PEL and edge detection in image processing. In this context, the object space may be referred to as 3D space. The image space is sometimes referred to as 2D space.

  Hereinafter, the PEL extraction algorithm will be described in more detail.

  The PEL is defined on the entire surface of a 3D object, which can be a triangle mesh, a subdivided surface, or a spline-based surface. As background information, a triangle mesh is a polygon mesh type in computer graphics. A triangle mesh comprises a set of triangles (typically three-dimensional), where the triangles are connected by their common edges or corners. Many graphics software packages and hardware devices can operate more efficiently on triangles grouped into meshes than on a similar number of triangles presented individually. This is because computer graphics typically operate on the corner vertices of a triangle. In this context, since triangle mesh is the most commonly used form of 3D object representation in the movie, gaming industry, and scientific visualization, to demonstrate the efficiency of PEL for shape illustration of 3D objects, An implementation of PEL extraction is given. For 3D objects represented in other forms, many open source and commercial tools are available for use to convert them to a triangle mesh. Therefore, it should be noted that PEL is not limited to 3D objects represented using vertices.

  Given the triangle mesh S and the illumination f on each vertex, the extraction of the PEL on S, in one embodiment, includes several steps described below.

  The first step is a pre-process that is optional. In this step, the normal line n of the surface S is smoothed.

  In the second step, the illumination gradient ∇f for each vertex is calculated.

In the third step, the directional derivative D _w || ∇f || is calculated for each vertex along the gradient direction w.

In the fourth step, zero crossings are detected and those that are local minimums are filtered out, ie D _w D _w || ∇f ||> 0.

  In the fifth step, the zero crossing is tracked to obtain the PEL.

  The preprocessing will be described in more detail below.

  The purpose of pre-processing is to reduce the noise of the lighting. In one embodiment, for the light model for PEL, both the main diffused light and the auxiliary spotlight are functions of n · l. Thus, illumination noise can be mainly caused by geometry noise, ie surface normal. In one embodiment, a bilateral filter is applied to process the vector field n on S. For each vertex ν, its neighborhood can be N (ν). A bilateral filter may be defined as follows:

Where W _c is the parameter δ _c :

Ρ (ν, p) mimics the relative curvature of ν and p, and W _s is the parameter δ _s :

Feature preserving fillers, so that large variations in feature fields are disadvantageous. In practice, it can be seen that this step often serves to improve the robustness of the algorithm for the scanned model described herein. In one embodiment, an intuitive method is used to set the parameters δ _c and δ _s . In one embodiment, the user selects a point on the mesh where the surface is expected to be smoothed, and then a radius r near that point is defined. In one embodiment, the parameter is set as δ _c = r / 2, where δ _s is the standard deviation of ρ (ν, p) in the selected neighborhood.

  The advantage of smoothing the surface normal is that it can be calculated only once, thereby avoiding computational overhead and ensuring that PEL illustrations are achieved in real time for medium size meshes. In another embodiment, the pre-processing can be performed by smoothing the illumination f on the surface, so that an expensive smoothing operation in each frame when the lighting state or view state is changed. Can be caused.

  The calculation of the derivative will be described in more detail below.

  The definition of PEL involves the third derivative of surface illumination, so it is important to calculate the derivative efficiently and robustly. In one embodiment, the per-vertex derivative is calculated by averaging the derivatives of adjacent face units.

As illustrated, an arbitrary scalar function g is defined on the mesh, i.e., g: M-> R, a triangle TεM with vertices V _i εR ³ , and an associated scalar value g _i = f. (V _i ) εR, i = 1,2,3. The plane unit coordinate system is defined in a plane perpendicular to the plane normal. The three vertices T in the plane-based coordinate system are (x _i , y _i ) εR ² , i = 1, ² , 3. Gradient ∇g per plane is

Where d _T = (x ₁ y ₂ −y ₁ x ₂ ) + (x ₂ y ₃ −y ₂ x ₃ ) + (x ₃ y ₁ -y ₃ x ₁₎
It becomes.

  The slope of each vertex is considered a weighted average of the slopes of adjacent faces of that vertex. The vertex coordinate system (U, V) is defined in a plane perpendicular to the vertex normal.

  Note that the vertex coordinate system and the plane coordinate system are often different. In one embodiment, a coordinate system transformation is applied before calculating the contribution of the per-surface derivative to the per-vertex derivative.

FIG. 5A shows vertices (x ₁ , y ₁ ), (x ₂ , y ₂ ), and (x ₃ , y ₃ ). FIG. 5B shows a plane coordinate system.

In one embodiment, the local vertex coordinate system is first rotated to be coplanar with the local surface coordinate system. Next, assuming that the point (u _i , v _i ) = u _i U _i + v _i V _i in the vertex coordinate system, the gradient of each vertex is

Is calculated in the local plane coordinate system. In one embodiment, they are accumulated using weighted averaging after calculating the per-vertex gradient for all of the adjacent faces of the vertex. In one embodiment, the set Voronoi region weighted so that the portion of the area closest to the vertex w _i is used. Ultimately, the per-vertex derivative is

It can be calculated as follows.

  PEL tracking is described in more detail below.

In one embodiment, once the directional derivative of the illumination variation at each mesh vertex v is calculated, it can be ascertained whether the mesh edge [v ₁ , v ₂ ] contains a zero crossing. Let h (ν) = D _w || ∇f (ν) ||. If h (ν ₁ ) · h (ν ₂ ) <0, linear interpolation can be used to approximate the zero crossing on the edge [v ₁ , v ₂ ], where p is the PEL vertex and Can be considered. If two PEL vertices are detected on a triangular edge, a straight line segment can be used to connect the two vertices. Multiple line segments can be connected to form a piecewise linear PEL curve.

  In one embodiment, the intensity of the PEL can be measured by integration of || ∇f || along the line.

  In one embodiment, a threshold value T can be defined to filter out noisy PELs that are less intense than the threshold value T. In one embodiment, this threshold can be specified by the user.

 Volumetric data is widely used in scientific and medical applications. Volume illustrations can be seen as non-photorealistic rendering applied to volume visualization. In one embodiment, PEL can be applied for volume illustration using isosurfaces. An isosurface is a three-dimensional analog of an isocontour. A surface representing a point of constant value (eg, pressure, temperature, velocity, density) within the volume of space. In other words, it is a level set of continuous functions whose domain is 3D space. In medical images, isosurfaces can be used to represent regions of a specific density in a 3D CT scan, thereby allowing visualization of internal organs, bones, or other structures. . Visualizing a volume data set using PEL consists of two stages: an isosurface extraction stage and a rendering stage. Initially, in the pre-processing stage, isosurfaces of interest to the user can be extracted using standard marching cube methods. In the rendering phase, a PEL can be calculated for each isosurface and then synthesized. Also, to further improve volume illustration, stylized shading can be applied to highlight important details and highlight the internal structure of the volume data set.

  FIGS. 6 (a)-(f) show the application of PEL for illustration of a volume measurement data set. FIGS. 6A to 6C show that isosurfaces are extracted and displayed from various viewpoints. FIGS. 6A to 6C show that the information is transmitted by PEL (solid line). 6 (d)-(f) show another set of isosurfaces extracted and displayed from various viewpoints. In order to emphasize the internal structure, in FIGS. 6 (a) to 6 (f), the outer isosurface is shown using only dotted lines.

  Compared to existing feature lines, the definition of PEL using luminance variation ensures a better perceptual match. At the same time, PEL is also more general than existing feature lines because lighting depends on view position, light model, material, texture, and geometric properties. More importantly, PEL allows the user to control the desired illustration more freely by manipulating the above parameters. Although derived from image processing, PEL differs from image space edge detection in that PEL is calculated in object space. Pixel-based representations of feature lines in image space can be less precise due to loss of 3D scene information during rendering and are not suitable for further processing such as styling. In contrast, since the PEL is defined in the object space, the PEL does not have these constraints, so the user can fully control the lighting by manipulating the light, and the user can Help achieve the illustration.

  According to one embodiment, the process of generating a line drawing using PEL can be summarized as follows.

  In the first step, an initial lighting state 3D scene is set. In other words, main light is applied. In the second step, line drawing using PEL is calculated by the system. In other words, PEL is extracted. In the third step, the user operates the lighting globally or locally. For example, the user can add one or more auxiliary lights locally on the 3D object. In a fourth step, an improved line drawing can be calculated using the adjusted light settings. The process then returns to the third step until the desired line drawing is achieved.

  In one embodiment, an effective light setting using main light and auxiliary light is provided. By using a headlight as the main light, an intuitive view using PEL and light-dependent line drawing can be calculated automatically, which is already desirable in many cases. Using incremental lighting operations allows the user to specify more line drawings for more complex scenes. In the present invention, a user-friendly writing operation is provided. Also, the optimal lighting settings can be automatically calculated using a fast linear optimization algorithm.

  In one embodiment, the illustration of 3D objects using feature lines such as PEL can be further applied to dynamic settings for stylized animation using frame coherence based on correspondence from clustering curves. . In this context, the term “frame” refers to a digital picture and an animation is a video sequence of frames or digital pictures. Frame coherence, or digital picture coherence, may refer to the consistency of styling between different digital pictures in a video sequence. Stylization is a characteristic of non-photorealistic applications. Based on the extracted lines, ie, PEL, the shape of the 3D object can be represented using a stylized curve, which changes the wavy, width and texture, or the natural medium It is similar to another method that uses hand-drawn curves. In these applications, the other goal of illustration is not only to achieve a perceptually consistent illustration of the shape, but also to achieve a visually satisfactory line drawing. This is widely used in cartoon animation for television or movies. An important challenge with curve-based stylized animation is to provide inter-frame coherence so that the curve fits smoothly over time as it changes in the animated scene. Current coherence issues are the main challenges of stylized rendering. Particularly problematic are view-dependent feature lines such as PEL that merge, separate, appear or disappear over time when the viewpoint changes.

  In one embodiment, a technique is provided for generating a stylized animation using frame coherence. In one embodiment, frame coherence is based on curve correspondence obtained based on a curve clustering method. Specifically, the animation can be PEL-based. However, the methods described herein are not limited to being applied to PEL, but can be applied to any video sequence of digital pictures in which curves are extracted for 3D objects from 3D space. Note that you get.

  FIG. 7 shows a diagram of a stylized animation process using curvilinear coherence. In this example, the process is based on PEL curve grouping. Note that the stylized animation process is not limited to PEL curves, but can be applied to any sequence of digital pictures in which the curves are extracted for 3D objects from 3D space.

  In process 701, a PEL for a 3D object is extracted for illustration of the 3D object to be presented in a digital picture video sequence.

  In process 702, the user can influence the animation by setting some rules, such as curve grouping.

  In process 703, PEL curve grouping is performed based on the extracted PEL in process 701 and the input from the user in process 702.

  Optionally, in process 704, knowledge-based line drawing is also involved.

  In process 705, PEL curve correspondence is established based on the curve grouping in process 703 and the knowledge-based line drawing in process 704.

  In process 706, a stylized animation, ie, a video sequence of digital pictures, is presented.

  FIG. 8 illustrates a method for generating a digital picture that includes steps 801, 802, 803, and 804, according to one embodiment.

  In one embodiment, the method includes a step 801 for receiving a three-dimensional representation for each of the plurality of curves, each curve at least partially representing the shape of the three-dimensional object. As an example, for a 3D object to be presented in multiple digital pictures of a video sequence, multiple curves, for example PEL, can be extracted to show the shape of the 3D object. Each curve may have corresponding 3D information, such as a position in 3D space and in a digital picture to be presented in a video sequence. Each curve indicates at least a part of the shape of the 3D object. For example, if the 3D object is a person's head, each curve may indicate a part of the person's head, for example, the shape of the left eye.

  In one embodiment, the method uses the 3D information provided by the 3D representation to step 802 for grouping a plurality of curves into at least one curve group based on the 3D representation of the curve. Further included. As an example, for each digital picture, if the curves show the same part, a nearby part, or a similar part of the 3D object, the curves can be grouped together. In this context, a similar part may refer to a part within a predefined range on the 3D object, or a part of a predefined range of 3D object geometry, eg, a change in curvature. For example, if the 3D object is a human head, multiple curves used to show the left eye can be grouped together. As another example, for different digital pictures, multiple curves used to show the same or similar part of a 3D object can be grouped together. For example, if the 3D object is a human head, a plurality of curves indicating the left eye area to be presented in different digital pictures can be grouped together. In a specific example, in the case of curve grouping between different digital pictures, curve group A may include one or more curves from each digital picture. Also, for at least one digital picture of a video sequence, if at least one digital picture does not contain a curve for a part of the 3D object associated with group A, there is no curve grouped in group A. Things can happen.

  In one embodiment, the three-dimensional information is information regarding a three-dimensional path of a curve in a three-dimensional space. For example, the three-dimensional information can define the position or coordinates of each point of the curve in 3D space.

  In one embodiment, the method further includes a step 803 for associating each curve group with a picture element based on the curves of the curve group. As an example, in the case of a curve group for one digital picture, the curve group can be associated with a 3D object, eg, a part of a person's head, eg, a left eye region. As a further example, in the case of curve groups for different pictures, the curve group can be associated with a 3D object to be presented in different digital pictures, ie a part of the human head, ie the left eye region. .

  In one embodiment, the method further includes step 804 for forming a digital picture using the picture elements. For example, if the 3D object is a human head, curve groups can be generated for different picture elements such as left eye region, right eye region, mouth region, and the like. A digital picture can be generated based on curve groups associated with different picture elements. For example, when the user sets the curve group associated with the left eye region to be displayed in a bold line style, the curve group associated with the left eye can be presented as a curve in the thick line mode.

  In one embodiment, in other words, curves or feature lines are first extracted for 3D objects to be represented in different digital pictures of the video sequence. Each curve reflects at least part of the shape of the 3D object. Each curve has its own information in 3D space, such as position or coordinates in 3D space. Each curve also has information about each curve regarding the digital picture of the video sequence to which the curve relates. Curves are grouped according to certain rules based on information about the position of each curve in 3D space. Each curve group can be associated with the same or similar part on a 3D object in one digital picture or in different digital pictures. Each curve group can be associated with a picture element of a 3D object, eg, a left eye region. For each digital picture, at least one curve group associated with one picture element is formed. A digital picture is generated based on picture elements related to the digital picture.

  FIG. 9 shows a device 900 for generating a digital picture. The device 900 includes a receiving unit 901, a grouping unit 902, an association unit 903, and a generating unit 904.

  In one embodiment, the receiving unit 901 is configured to receive a three-dimensional representation for each of the plurality of curves, each curve at least partially representing the shape of the three-dimensional object.

  In one embodiment, the grouping unit 902 is configured to group at least one curve group based on the 3D representation of the curve using 3D information provided by the 3D representation.

  In one embodiment, the association unit 903 is configured to associate each curve group with a picture element based on the curves of the curve group.

  In one embodiment, the generation unit 904 is configured to form a digital picture using picture elements.

  In one embodiment, a computer readable medium having recorded thereon a program adapted to cause a computer processor to perform a method for generating a digital picture is provided. In one embodiment, the computer-readable medium is program code for causing a processor to perform reception of a three-dimensional representation for each of a plurality of curves, each curve at least partially representing the shape of a three-dimensional object. , With code.

  In one embodiment, the computer-readable medium uses a 3D information provided by the 3D representation to the processor to group a plurality of curves into at least one curve group based on the 3D representation of the curve. The program further includes code for executing the program.

  In one embodiment, the computer-readable medium further comprises program code for causing the processor to perform an association of each curve group with a picture element based on the curves of the curve group.

  In one embodiment, the computer-readable medium further comprises program code for causing a processor to perform the formation of a digital picture using picture elements.

  In one embodiment, the digital picture is a digital picture in a video sequence of a plurality of digital pictures. In other words, this digital picture is an animation digital picture including a plurality of digital pictures. In other words, the curve may be grouped across multiple frames of the video sequence, eg, subsequent frames of the video sequence.

  In one embodiment, at least two of the plurality of curves are associated with different digital pictures in a video sequence of digital pictures.

  In one embodiment, the 3D representation of the curve is a set of vertices in 3D space. For example, the three-dimensional representation of the curve can be PEL. As described above, the PEL can be a set of 3D object vertices where the illumination variation in the direction of the gradient reaches a local maximum.

  In one embodiment, these vertices identify points in the three-dimensional space where the illumination of the three-dimensional object changes when the three-dimensional object is viewed from a predetermined viewpoint and illuminated by at least one predetermined light source. . For example, the at least one predetermined light source can be main light. In another example, the at least one predetermined light source may include main light and at least one auxiliary light.

  PEL stroke clustering is described in more detail below.

  In one embodiment, generating a stylized curve based on the curve grouping method provided herein, which is a basic low-level operation of a curve, eg, PEL perceptual grouping. be able to. As a general process, in both human vision and computer vision, low-level feature groupings provide abstract and effective information in understanding complex scenes. For example, grouping of PEL curves can provide a high level of information in finding the curve correspondence between digital pictures. Most conventional work in image processing and computer vision focuses on segmentation and grouping of points and lines. Compared to these tasks, segmentation and grouping of curves is more complex because any curve has more freedom than points and lines.

In one embodiment, curve grouping follows the principle as described below. Compared to point-based image segmentation, curve grouping proves difficult in some basic aspects, such as noise or gaps and curves with different structures at different scales. It is difficult to determine the vicinity of a curve in a curve set and accurately determine the intersection and distance of two arbitrary curves. Specifically considering the grouping of curves for stylized line drawing, in one embodiment:
・ Continuity. The evaluation of the curvilinearity of the curve takes into account the gap distance with respect to the length of the curve and the amount of curvature of the coupling curve.
・ Parallelity. The evaluation of potential parallelism takes into account separation factors between curves and the amount of curve overlap.
・ Proximity. The minimum distance between all points of two curves.
・ Closedness. The possibility of a curve that tends to lead to a closed curve.
A method for finding a cluster of curves focusing on perceptual grouping rules is provided.

  In one embodiment, initially, the curve may be divided into short segments at high curvature locations prior to clustering. For example, the curve may be divided into short segments where the curvature is greater than a predetermined value. Curve segments can then be grouped separately according to a linear combination of continuity, parallelism, proximity and closure ratings.

In one embodiment, several parameter measurements can be used to determine whether two curves should be grouped together within one digital picture or between different digital pictures. . In one embodiment, such parameters are
1.2 Continuity of two curves 2.2 Parallelism of two curves 3. Proximity of two curves; It may include the possibility of two curves to be connected to a closed curve.

  The measured values of the above parameters will be described in detail below.

  Measurements of the parameters of the continuity, parallelism, proximity and the likelihood of two curves to be connected to a closed curve are not limited to the specific examples given below, but continuity, parallelism, proximity Note that, and any other measurement that may reflect the parameters of the potential of the two curves to lead to a closed curve.

In one embodiment, continuity can be calculated based on the length of the coupling curve (l _i ) with respect to the length of the _two input curves (l ₁ , l ₂ ), as well as the curvature of the coupling curve. In one embodiment, a combined curve is generated by sampling the end points of two input curves and then interpolating these points to calculate a B-spline curve that links the two input curves. For example, curve l ₁ has two ends A and B, curve l ₂ has two ends C and D, and end A is compared to the other end B of l ₁ closer to l _2, end C is assumed to closer l ₁ as compared to the other end D of the l _2. The coupling curve l _i can then be generated by sampling the point between the end A of l _{1 and} the end C of l ₂ . The curvature of the bond curve with the maximum curvature (K _max ) of the bond curve can be determined, which refers to the curvature of the point in the bond curve with the maximum curvature. Therefore, continuity is
m _c = l _i / (l ₁ + l ₂ ) * K _max (7)
Can be defined as follows.

In one example, if the value of m _C is less than a user-defined threshold, the two curves are considered continuous.

FIG. 10 shows an example of determining the continuity of _two curves l ₁ and l ₂ , where l ₁ has two end points 1001 and 1002 and l ₂ has two end points 1003 and 1004. In this example, l ₁ endpoint 1001 is closer to curve l ₂ compared to endpoint 1002 and l ₂ endpoint 1003 is closer to curve l ₁ compared to endpoint 1004.

To determine whether curves l ₁ and l ₂ are continuous, a binding curve l _i can be generated. Initially, a combined curve l _i can be generated by selecting sampling points in curve l ₁ near end point 1001 and curve l ₂ near end point 1003. In this example, five sampling points are selected, points 1001 and 1005 to 1008 are selected in the curve l ₁ close to the end point 1001, and points 1003 and 1009 to 1002 are in the curve l ₂ close to the end point 1003. 1012 is selected. Based on the selected sampling points, these sampling points (points 1001, 1003, and 1005-1012) may be interpolated to generate a B-spline curve l _i that links the _two curves l ₁ and l ₂ .

In one embodiment, the value m _C can be calculated based on l ₁ , l ₂ and l _i using equation (7). The calculated m _C can be compared to a predetermined threshold. If m _C is less than the predetermined threshold, curves l ₁ and l ₂ can be considered continuous. Otherwise, curves l ₁ and l ₂ can be considered open.

In one embodiment, the parallelism measurement takes into account the separation factor between the two input curves and the amount of overlap between the two curves. FIG. 11 shows the determination of the parallelism of two curves. For example, two curves l ₁ and l ₂ , each with a corresponding length, can be represented using B-splines, and the set of points is the same parameter value (t = 0.1, 0.2, 0. 3 ... 0.5 ... 1.0 etc.) can be sampled for each curve. These two sets of sampling points can then be interpolated linearly to generate a third set of points. A third set of points, as shown in FIG. 11, between the curves l ₁ and the curve 1 _2, may be used to produce a B-spline intermediate curve length l _i. The intermediate curve may be split into two parts at point 1100 at t = 0.5. The length of the first part l _i1 can be represented by m ₁ and the length of the second part l _i2 can be represented by m ₂ . The distances between the end points of the two curves and the intermediate curve are e ₁ and e ₂ , respectively. In one embodiment, the parallelism is
m _P = l _i / (l ₁ + l ₂ ) * (l−R ₀ )
But can be defined as
R ₀ − (m ₁ + m ₂ ) / (m ₁ + m ₂ + e ₁ + e ₂ )
It is.

In one embodiment, two curves can be considered parallel if the value of m _P is greater than a user-defined threshold.

The proximity of two curves can be defined as the minimum distance between all points of the two curves. For example, if p _i and p _j are arbitrary points on two curves, respectively, the proximity of the two curves is
_{m D = min || p i -p} j ||
Can be defined as the minimum distance between all points of two curves.

In one embodiment, two curves are considered close if the value of m _D is less than a user defined threshold.

The closure m _O can be defined as the possibility of a curve that tends to lead to a closed curve. For example, points are sampled at the ends of two input curves, and then a B-spline curve is calculated that interpolates these points and links the two input curves. For example, a curve l _i are the two ends A and B, it has a curve 1 ₂ two ends C and D, end A, compared to the other end B of l ₁ closer to 1 _second end station C, a end B, as compared to the other end portion a of l _1, it is assumed that the closer to 1 the _second end D. Then, the binding curve _{l i1} between the ends C of the end A and _{l 2} of _{l 1} is generated resulting, another binding curve _{l i2} between the ends D of the end portion B and _{l 2} of _{l 1} Can be generated. The sum of curvatures along the binding curve can then be calculated. In one embodiment, two curves with a large combined curve sum of curvatures m _O can be considered open. For example, if the sum is greater than a user defined threshold, the two curves may be considered open. In one embodiment, two curves may be considered closed if the sum of curvatures m _O is less than a user defined threshold.

FIG. 12 shows whether the _two curves l ₁ and l ₂ are likely to form a closed curve, ie the determination of the possibility of two curves to be connected to the closed curve, where the curve l ₁ has two end points 1201 and 1202. And the curve l ₂ has two end points 1203 and 1204.

In this example, the end point 1201, closer to the end point 1203 of the curve _{l 2} as compared to the other endpoint 1202 of curve _{l 1,} the end point 1202, the curve _{l 2} as compared to the other endpoint 1201 of curve _{l 1} Is closer to the end point 1204. Then, between the end point 1203 of the end point 1201 and the curve _{l 2} curves _{l 1,} binding curve _{l i1} is obtained is generated, between the end point 1204 of the end point 1202 and the curve _{l 2} curves _{l 1,} another binding Curve l _i2 may be generated. For example, a combined curve l _i1 may be generated by first selecting sampling points in curve l ₁ near end point 1201 and curve l ₂ near end point 1203. In this example, the five sampling points each selection, during curve _{l 1} close to the end point 1201, point 1201 and from 1205 to 1208 is selected, during curve _{1 2} close to the end point 1203, point 1203 and 1209～ 1212 is selected. Based on the selected sampling points, these sampling points (points 1201, 1203, and 1205-1212) may be interpolated to generate a B-spline curve l _i1 that links the _two curves l ₁ and l ₂ . Similarly, a combined curve l _i2 may be generated by first selecting sampling points in curve l ₁ near end point 1202 and curve l ₂ near end point 1204. In this example, each of the five sampling points is selected, points 1202 and 1213-1216 are selected in curve l ₁ close to end point 1202, and points 1204 and 1217˜ are in curve l ₂ close to end point 1204. 1220 is selected. Based on the selected sampling points, these sampling points (points 1202, 1204, and 1213-1220) can be interpolated to generate a B-spline curve l _i2 that links the _two curves l ₁ and l ₂ . In one embodiment, the sum of curvatures along the binding curves l _i1 and l _i2 is calculated. In one embodiment, this sum may be compared to a user defined threshold. In one embodiment, two curves may be considered closed if the sum of curvatures m _O is less than a user defined threshold. If the sum is greater than a user defined threshold, the two curves l ₁ and l ₂ can be considered open.

In one embodiment, the grouping of the two curves into the same group depends on the measured values of all the parameters m _C , m _P , m _D , m _O described above. In one embodiment, it is possible to calculate an overall grouping values M _A, using a linear combination. The user can adjust the weight of the linear combination to emphasize certain parameters. In one embodiment, input curves may be grouped into the same group when the overall grouping value is greater than a user defined threshold.

For example, _{M A} is,
M _A = a * m _C + b * m _P + c * m _D + d * m _O (8)
Where a, b, c and d are the weights of m _C , m _P , m _D and m _O , respectively. The threshold value TH may be set to be a default value or may be set by the user. In one example, if M _A is greater than TH, it is determined that the two curves should be in one curve group.

In one example, the weights a, b, c, and d may be set to be default values, or may be determined by a user, for example. In one example, the weights a, b, c and d can be determined according to different enhancements of different parameters m _C , m _P , m _D and m _O , respectively. For example, if only continuity and parallelism are considered important parameters, the corresponding weights a and b can be set to be certain non-zero values, respectively, and the weights c and d are set to zero. Can be set to In another example, to evaluate whether two curves should be grouped together, if only one parameter, namely proximity, is considered important, each corresponding weight c is given a certain And the weights a, b and d can be set to be zero. In a further example, if all parameters of m _C , m _P , m _D and m _O are considered important, the weights a, b, c and d are respectively determined according to the respective importance of each parameter. Can be set to a certain non-zero value.

In one embodiment, a measure of continuity between two curves, a measure of parallelism of two curves, a measure of proximity of two curves, and a measure of the likelihood of two curves to be connected to a closed curve Based on at least one of the two, it is determined whether the two curves are grouped into the same curve group. By way of example, referring to the equation (8) for M _A given above, at least one of the weights a, b, c, and d is a non-zero value, so that M _A is m _C , The determination is based on at least one parameter of m _P , m _D, and m 2 _O.

In one embodiment, a measure of continuity between two curves, a measure of parallelism of two curves, a measure of proximity of two curves, and a measure of the likelihood of two curves to be connected to a closed curve Based on a comparison of at least one of the and a predetermined threshold, it is determined whether the two curves are grouped into the same curve group. For example, referring to Equation (8) of M _A given above, a, b, at least one of the weights of c and d is set to be a non-zero value. Therefore, M _A is determined based on at least one parameter of m _C , m _P , m _D and m _O , and M _A further determines whether the two curves should be in the same curve group. Therefore, it is compared with a predetermined threshold value TH.

In one embodiment, a measure of continuity between two curves, a measure of parallelism of two curves, a measure of proximity of two curves, and a measure of the likelihood of two curves to be connected to a closed curve Based on the comparison of at least two of the two and a predetermined threshold value, it is determined whether the two curves should be grouped into the same curve group. As an example, with reference to Equation (8) of M _A given above, a, b, at least two of the weights of c and d is set to be a non-zero value. Thus, M _A is determined based on at least two parameters of m _C , m _P , m _D and m _O , and M _A further determines whether the two curves should be in the same curve group. Therefore, it is compared with a predetermined threshold value TH.

In one embodiment, the combination is a measure of continuity between two curves, a measure of parallelism of two curves, a measure of proximity of two curves, and the possibility of two curves to lead to a closed curve Is a weighted sum of at least two of the measurements. As an example, the parameters m _C , m _P , m _D and m _O are weighted by a, b, c and d, respectively (Equation (8)). The user can set the weight of each parameter according to the importance of evaluating whether two curves should be grouped into the same curve group. In one embodiment, if curve grouping is not ideal, the user can further adjust the weights for different parameters.

  In one embodiment, the curve grouping is performed both in a single digital picture and between different digital pictures in a video sequence of multiple digital pictures. Curve grouping in a single digital picture may be helpful in finding high-level structures of curves, and grouping between digital pictures is helpful in finding correspondence of digital pictures in a video sequence.

  When operating in 3D space, additional depth dimensions can provide additional information regarding curve grouping.

  In one embodiment, curve groupings between digital pictures are designed with the goal of finding curve correspondences. In animation, the corresponding curves in each digital picture can be grouped into a single group. For example, these curves may be located at similar locations and have similar geometric characteristics. In this context, a similar position may refer to the positions of the curves being within a predefined range from each other. Similar geometric characteristics can refer to the fact that the difference in curvature of the curves can be within a predefined range. Along with these curve group information, curve styles and inter-frame interpolation can be applied to different curve groups.

  FIGS. 13A to 13C show an example of curve grouping related to a plurality of different digital pictures.

  As can be seen in the figure, FIGS. 13 (a)-(c) show the rotating human head in animation with three different digital pictures of the video sequence. In FIG. 13A, curves A and B of the first digital picture show the left eye of the face. In FIG. 13B, which is the second digital picture, only the curve A1 is used to show the left eye. In FIG. 13C, which is the third digital picture, curves A2 and B2 are used to show the left eye. In one embodiment, as a result of curve grouping, two curve groups are determined for the left eye: one group with curves A, A1 and A2, and another group with curves B and B2. obtain. Based on curve grouping, in one embodiment, different styles can be applied to the curves in these two groups. This embodiment can facilitate understanding of the shape of the 3D object and helps achieve visually satisfactory animation using stylized strokes.

In some embodiments, fully automated curve grouping may not guarantee satisfactory grouping, so in one embodiment, user interaction is used to adjust and modify curve grouping during the grouping process. Can be incorporated. For example, the user can adjust the weights of a, b, c and d as shown in equation (8). In another example, the user can adjust the threshold value TH of M _A.

  In one embodiment, curve grouping can provide a high level of information about the structure of a 3D object, thereby allowing semantic information to be incorporated to generate a more sophisticated line drawing. Semantic information can give meaning to a curve, for example, it is the eyes or mouth of an individual's face. Curve grouping can be applied to a wide range of applications, and in the following, the effect of the method of curve grouping will be demonstrated using knowledge-based line drawing and stylized stroke animation.

  Knowledge base line drawing will be described in more detail below.

  From observation, it can be seen that the line drawn by an artist does not always follow the geometry or lighting on the object surface. For example, in hand-drawn line drawing, line exaggeration is often seen for special purposes. This is noticeable in some traditional art styles, such as comics. It is well understood by people because of some common sense about objects represented by line drawing.

  In one embodiment, a method for animated face illustration using curves is provided by incorporating template and semantic information into the drawing process. In one embodiment, the method is based on a grouping of curves extracted for 3D objects and a predefined, strictly constrained domain. In this context, a strictly constrained domain may refer to the requirement that a curve group clearly corresponds to a predefined component. For example, a curve group may include a curve in the area of the left eye region, and the curve group may be matched with a predefined component, ie, a template for the left eye region. In one embodiment, the curve is PEL.

  In one embodiment, a knowledge-based algorithm is provided to incorporate template and semantic information into the drawing process. In one embodiment, after the curves are grouped, each curve group can be matched with a predefined component. For example, a curve group can be matched with a predefined component, i.e., the left eye. Multiple templates of predefined components may already be stored in the system. In one embodiment, a pre-defined component template that best matches a curve group may be selected. In one embodiment, each curve group can be exchanged with a selected template. In one embodiment, the selected template can be further styled, i.e. put into a thick line drawing mode. In one embodiment, the selected template can be transformed and synthesized to fit a digital picture. In this way, a stylish line drawing can be achieved. While these line-drawing curves may not follow the features of the image, overall, the line drawing may still be “correct” and beautiful. This process can be optional, and the pre-defined drawing can have a well-defined curve correspondence, which is useful in applying a stylized curve to the animation.

  In one embodiment, as described above, curve grouping is automatically performed according to rules such as continuity and parallelism.

  In one embodiment, the knowledge-based algorithm operates on all digital pictures in the video sequence.

  In one embodiment, the predefined template database may be constructed for various fields, such as human faces, animals, buildings, and the like.

  In one embodiment, PEL curves are first grouped to represent meaningful components including eyes, nose and mouth. A search in the library can then be performed to find a predefined line drawing to be matched with the stroke cluster. In this process, different matching parameters representing different facial sketches can be used. Finally, PEL stroke clusters can be exchanged for predefined user drawings.

  FIGS. 14A to 14C show examples in which two different styles of line drawing are generated from a predefined library.

  FIG. 14A shows a 3D object. Based on the 3D object shown in FIG. 14A, a curve such as PEL representing the shape of the 3D object can be extracted. FIG. 14B shows an example of the extracted curve in one style of the stroke template. FIG. 14 (c) shows another example of the extracted curve in another style of different stroke templates.

  The curve correspondence will be described in more detail below.

  Finding curves, eg, PEL correspondence and evolution, for 3D objects between digital pictures is important for curve-based stylized animation. It can be applied to smoothly interpolating curves, rendering coherent artistic curves in various styles, and manipulating animation with motion in curve space, and so on. As mentioned above, this is a problematic problem, especially for PEL curves.

  In one embodiment, curve correspondence is established based on curve grouping. In an animation sequence, a set of curves for all digital pictures is extracted and then divided into small groups with curves from different digital pictures. In one embodiment, the point correspondence of a curve in a group can be found by projecting the curve into screen space and finding the closest point in screen space. In this context, screen space may refer to 2D space. The extracted curve of the 3D object in 3D space is projected into 2D space to form a 2D illustration of the 3D object. If grouping is not successful in this way, user interaction can be incorporated as described above, and some rules of this user interaction can be learned and applied to further curve grouping processes.

  In one embodiment, a correspondence is established for each group of digital pictures. In other words, for a digital picture, a group of curves can be determined for a picture element, eg, the left eye. A correspondence is then established for all groups associated with the left eye in different digital pictures of the video sequence.

  In one embodiment, using curve grouping, curve group correspondences are first established, and then within a group, single curve correspondences are determined. By finding a curve correspondence with this hierarchical matching, the effectiveness of the algorithm is improved. As an example, referring to FIG. 13, FIGS. 13 (a)-(c) show three different digital pictures of a human head of a 3D object in a video sequence. Curve group F associated with the left eye of a person's head may be determined to include curves A, B, A1, A2, and B2, and curves A, B, A1, A2, and B2 are all included in the left eye Used to indicate shape. In one embodiment, the curve group F can be further divided into three subgroups. For example, the curve subgroup F1 associated with the left eye in the first frame may be determined to include curves A and B. The curve group F2 associated with the left eye in the second frame may be determined to include the curve A1. The curve group F3 associated with the left eye in the third frame may be determined to include curves A2 and B2. In one embodiment, since the curve subgroups Fl, F2 and F3 are all associated with the component, i.e. the left eye, a correspondence can be established between the curve subgroups Fl, F2 and F3.

  In one embodiment, after correspondence is established for curve subgroups Fl, F2, and F3, the correspondence of a single curve may be determined. For example, for curve subgroups Fl, F2 and F3, curves A, A1 and A2 are at similar positions on the person's head and share similar geometric properties, so Can be determined as corresponding curves. Similarly, curves B and B2 are corresponding curves. In one embodiment, the same style can be applied to the same set of curves corresponding to the entire digital pictures of different video sequences.

  In other words, in one embodiment, curves of different digital pictures associated with the same picture element, i.e. the left eye, can be grouped together. The curve group can then be divided into subgroups, each subgroup corresponding to one digital picture. Furthermore, a single curve correspondence can be established based on the curves in the subgroup. In a further embodiment, point correspondences can be determined further based on a single curve correspondence. Establishing correspondence between different curve subgroups, between single curves in multiple curve subgroups, and between points in a single curve can be referred to as a hierarchical matching process.

  FIG. 15 shows an example of hierarchical matching.

  In the animation, a curve S having pixels A and B exists in the first digital picture shown in FIG. In the second digital picture shown in FIG. 15B, curves S ′, S1 and S2 exist. By hierarchical matching, the curve group F can be determined to comprise curves S, S ', S1 and S2, all of which are used to indicate the shape of the left eye. Furthermore, for curve group F, a curve subgroup F1 for frame 1 containing curve S can be determined, and another curve subgroup F2 for frame 2 containing curves S ′, S1 and S2 is determined. be able to. In a further step, a single curve correspondence can be determined. For example, the curve S in the curve subgroup F1 can be determined to correspond to the curve S ′ in the curve subgroup F2, and the curves S and S ′ are arranged at the same position in the left eye and have similar geometric characteristics. Share

  Curve support can be used for a wide range of applications, including smooth curve interpolation for in-vitroening, various style artistic stroke rendering for stylized animation, and manipulation of animation in curve space. it can. Finding curve correspondences using curve grouping provides perceptual cues in new advanced algorithms under development for various applications.

  FIG. 16 illustrates a computer 1600 according to one embodiment.

  In one embodiment, computer 1600 can include a processor 1601. In one embodiment, the computer 1600 can further include a memory 1602. In one embodiment, the computer 1600 can further comprise an input 1603 for receiving a three-dimensional representation for each of the plurality of curves, each curve at least partially representing the shape of the three-dimensional object. In one embodiment, computer 1600 can further comprise user input 1604. In one embodiment, the computer 1600 can further comprise a display 1605. In one embodiment, the computer 1600 may further comprise a code reading unit 1606 for reading codes from another computer readable medium. For example, all components of computer 1600 are connected to each other via computer bus 1607.

  In one embodiment, the memory 1602 stores a program adapted to cause the processor 1601 to perform a method for generating a digital picture, the memory 1602 relating to each of the plurality of curves. Code of a program that executes reception of a three-dimensional representation, wherein each curve at least partially represents the shape of a three-dimensional object, and uses the three-dimensional information given to the processor 1601 by the three-dimensional representation And a program code for performing grouping of a plurality of curves into at least one curve group based on a three-dimensional representation of the curve, and causing the processor 1601 to execute each curve group and picture based on the curve of the curve group. Code of a program for executing association with an element, and processor 1 01, and a code of a program for executing the formation of digital picture using the picture element.

  In one embodiment, processor 1601 reads a program on memory 1602 to execute a program that generates a digital picture.

  In one embodiment, the processor 1601 obtains a 3D representation of the curve via the curve representation input 1603.

  In one embodiment, processor 1601 obtains input from a user via user input 1604.

  In one embodiment, the program code may be recorded on another computer readable medium (not shown). In this case, the processor 1601 may execute the method for reading the code from the other computer readable medium via the code reading unit 1606 and generating the digital picture described herein.

  In one embodiment, the generated digital picture is shown on display 1605.

  While the invention has been particularly shown and described with respect to particular embodiments, those skilled in the art will recognize that various forms and details can be used without departing from the scope of the invention as defined in the appended claims. It should be understood that changes can be made. Accordingly, the scope of the present invention is intended to be defined by the appended claims, and therefore, all modifications that come within the meaning and range of equivalents of each claim are intended to be encompassed.

In this specification, the following documents are referred to.
1. Appel, “The notion of quantitative invisibility and the machine rendering of solids” (Proceedings of the 22nd National Conference of 1967)
2. Gooch et al., “Interactive technical illustration” (Proceedings of 1999 Symposium on Interactive 3D graphics)
3. Hertzmann and Zorin, “Illustrating smooth surfaces” (minutes from Sigmagraph 2000)
4). Kalnins et al., “WYSIWYG NPR: Drawing strokes direct on 3d models” (Minutes of Sigmagraph 2002)
5. Interrant et al., “Enhancing transparencing skin surfaces with ridge and valley lines” (IEEE Visualization 1995).
6). Wilson and Ma, “Representing complexity in computer-generated pen-and-ink illustrations” (minutes of NPAR 2004)
7). Ni et al., “Proceedings of Multi-scale line drawings from 3D meshes 2006 Symposium on Interactive 3D graphics”
8). DeCarlo et al., “Suggestive controls for conveying shape” (minutes of Sigma 2003).

1 region 204 of ... Frame 2 ... Frame 200 surface 202 ... 3D object _{··· D w || ∇f ||> 0} ··· D w || ∇f || <0 region 900 ... Device 901 ... Reception unit 902 ... Grouping unit 903 ... Association unit 904 ... Generation unit 1001 to 1004, 1201 to 1204 ... End point 1005 to 1012, 1205 to 1220 Sampling point 1600... Computer 1601... Processor 1602... Memory 1603... Input of curve expression 1604... User input 1605... Display 1606.

Claims (16)

In a method for generating a digital picture,
A step receiving unit of the apparatus, the method comprising: receiving a three-dimensional representation of each of a plurality of curves, each curve, at least partially representative of the shape of the three-dimensional object, receiving,
Grouping the plurality of curves into at least one curve group based on the three-dimensional representation of the curve using the three-dimensional information provided by the three-dimensional representation by the grouping unit of the device When,
A grouping unit of the apparatus associating each curve group with a picture element based on the curve of the curve group;
Generation unit of the apparatus, using said picture elements, look-containing and forming the digital picture,
A measure of continuity between two of the plurality of curves;
A measure of the parallelism of the two curves;
A measure of the proximity of the two curves; and
A measure of the possibility of the two curves to be connected to a closed curve
Determining whether the two curves are grouped into the same curve group based on at least one of
Method. The method of claim 1, wherein the digital picture is a digital picture in a video sequence of a plurality of digital pictures. The method of claim 2, wherein at least two curves of the plurality of curves are associated with different digital pictures of the video sequence of digital pictures. A measure of continuity between the two curves,
A measure of the parallelism of the two curves;
Based on a comparison of at least one of the measure of proximity of the two curves and the measure of the likelihood of the two curves to lead to a closed curve with a predetermined threshold, the two curves are 4. A method according to any one of the preceding claims, wherein it is determined whether to group into the same curve group. A measure of continuity between the two curves,
A measure of the parallelism of the two curves;
Based on a comparison of a measure of the proximity of the two curves and a measure of the likelihood of the two curves to lead to a closed curve, and a comparison with a predetermined threshold, two curve whether grouped into the same curve group is determined, the method according to any one of claims 1 to 4. The combination is
A measure of continuity between the two curves,
A measure of the parallelism of the two curves;
6. The method of claim 5 , wherein the weighted sum of at least two of the proximity measurements of the two curves and the likelihood of the two curves to lead to a closed curve. It said three-dimensional representation of the curve is a set of vertices in the 3-dimensional space, the method according to any one of claims 1 to 6. 8. The vertex according to claim 7 , wherein if the three-dimensional object is visible from a predetermined viewpoint and illuminated by at least one predetermined light source, the vertex identifies a point in a three-dimensional space where the illumination of the three-dimensional object changes. The method described. The three-dimensional information, which is information on three-dimensional path of the curve in three-dimensional space, the method according to any one of claims 1 to 8. 4. The method of claim 3, wherein each curve group associated with one picture element is further divided into a plurality of curve subgroups, and each curve subgroup is associated with the same picture element for a plurality of different digital pictures. . Wherein for each picture element to be presented in a digital picture, style look further including the step of setting a re stick pattern, the stylistic pattern includes the characteristics of the non-photorealistic application, The method of claim 10. For the same picture element, the same stylistic pattern is applied to a plurality of different curves subgroups across different digital pictures, the method of claim 1 1. Further comprising the step of setting a stylistic pattern for at least one curve within the curve subgroup A method according to claim 1 0. The same stylistic pattern is applied to the at least one curve in the curve subgroup and to at least one corresponding curve in all other curve subgroups in the same curve group, and the at least one corresponding curve but geometrical properties located or similar similar for the three-dimensional object refers to a curve to be shared with the at least one curve, the method of claim 1 3. A receiving unit configured to receive a three-dimensional representation for each of a plurality of curves, each curve at least partially representing a shape of a three-dimensional object;
A grouping unit configured to group the plurality of curves into at least one curve group based on the three-dimensional representation of the curve using the three-dimensional information provided by the three-dimensional representation; ,
An association unit configured to associate each curve group with a picture element based on the curves of the curve group;
Using said picture elements, and <br/> Bei example configured generation unit to form a digital picture,
A measure of continuity between two of the plurality of curves;
A measure of the parallelism of the two curves;
A measure of the proximity of the two curves; and
A measure of the possibility of the two curves to be connected to a closed curve
Determining whether the two curves are grouped into the same curve group based on at least one of
,device. A computer readable recording medium having recorded thereon a program configured to cause a computer processor to execute a method for generating a digital picture, the computer readable recording medium comprising:
Code of the program causing the processor to receive a three-dimensional representation of each of a plurality of curves, wherein each curve at least partially represents a shape of a three-dimensional object;
Causing the processor to perform grouping of the plurality of curves into at least one curve group based on the 3D representation of the curve using 3D information provided by the 3D representation; Program code and
Code of the program causing the processor to perform association of each curve group with a picture element based on the curve of the curve group;
Recording the program code for causing the processor to perform the formation of the digital picture using the picture element ;
A measure of continuity between two of the plurality of curves;
A measure of the parallelism of the two curves;
A measure of the proximity of the two curves; and
A measure of the possibility of the two curves to be connected to a closed curve
Determining whether the two curves are grouped into the same curve group based on at least one of
Computer-readable recording medium.