CN102509357A - Pencil sketch simulating and drawing system based on brush stroke - Google Patents

Abstract

The invention discloses a pencil sketch simulation and drawing system based on strokes. The system takes a 3D geometric model as input, and generates pencil sketch pictures by means of automatic generation combined with interactive methods. The system divides the strokes in pencil sketch drawing into characteristic strokes and hatched strokes, and simulates and draws them respectively. The system mainly includes five modules: user interaction module, feature path automatic generation module, hatch path automatic generation module, stroke generation module from path, and drawing output module. The invention has the characteristics of conforming to the actual creation mechanism of human pencil sketches, realistic drawing effects, and easy editing and modification, and has important application value in computer-aided design, art design, special effect production, digital entertainment, etc.

Description

Stroke-based Pencil Sketch Simulation and Drawing System

technical field

The invention belongs to the interdisciplinary technical field of combining computer graphics, computer-aided design and art design, and relates to a method for automatically or interactively generating pencil sketches based on strokes according to an input three-dimensional geometric model.

Background technique

The simulation of various art media has been widely used in the fields of digital art design, digital entertainment, film animation production, and digital image processing, and pencil sketch drawing, as the basic content of painting, has attracted extensive attention. The existing pencil sketch simulation drawing methods can be roughly divided into four categories: methods based on volume rendering, methods based on texture mapping, methods based on images and methods based on strokes.

The earliest work in the pencil sketch simulation rendering work is the volume rendering model established by Takagi et al. Volume rendering methods usually require a high amount of storage and calculation, and can only be used for the simulation of pencil drawing materials.

The method based on texture mapping usually generates some pencil sketch textures in advance, and pastes the generated textures on the geometry surface according to the curvature direction. Since current graphics hardware devices have good support for texture mapping, this method can realize real-time drawing of pencil sketches on 3D geometry. The disadvantage of the texture mapping method is that because this method only describes the object in the three-dimensional object space and has no two-dimensional image space information, the result is too regular and lacks the hand-painted effect.

Image-based pencil sketch simulations all take a two-dimensional digital image as input, first generate a noise image and a two-dimensional vector field based on the input image, and then use linear integral convolution (LIC) to generate the final pencil sketch image. Contrary to the texture mapping method, this kind of method only describes the drawing object in the two-dimensional image space, and cannot utilize the geometric information of the object's three-dimensional space. Due to the lack of structural information, data representation is generally loose, which is not conducive to editing.

Both the texture-mapping-based method and the image-based method treat the pencil sketch lines as a collection (texture or image), lacking individual pencil line information, and thus cannot reflect human's drawing process and drawing habits. Pencil sketch simulation drawing method based on stroke then overcomes above-mentioned shortcoming. Strokes have their own geometric description, and each stroke corresponds to a stroke in the painting. Various attributes can also be stored in strokes, such as the pressure used when drawing, color, etc., which provides convenience for applications involving simulated drawing and interaction. Users can also change strokes through operations such as picking, editing, and deleting. A picture composed of brush strokes preserves almost all the information of painting. In addition, due to the geometric structure of strokes, the drawing results can be output with arbitrary precision. These advantages make the stroke-based method highly practical and widely used in various art design software.

Contents of the invention

The purpose of the present invention is to provide a stroke-based tool that can input three-dimensional A system for automatically or interactively generating pencil sketch drawings from geometric models. The pencil sketch generated by the system is composed of strokes, which can reflect the creation mechanism and process of a real pencil sketch, and is easy to be edited and modified by users, and can be used for virtual creation of various pencil sketches.

The present invention proposes a pencil sketch simulation and drawing system based on strokes, the system includes five modules:

The user interaction module is used to provide an interface for the user to select a 3D geometric model and output the selected 3D geometric model to the feature path automatic generation module and the hatch path automatic generation module;

A feature path automatic generation module is used to automatically generate a feature path according to the input 3D geometric model;

The hatch path automatic generation module is used for automatically generating the hatch path according to the input 3D geometric model;

Generate a stroke module from the path, for generating the geometry and attributes of the feature stroke and the hatch stroke according to the feature path generated by the feature path automatic generation module and the hatch path automatically generated by the hatch path automatic generation module;

The drawing output module is used for drawing pencil strokes by using the pencil physical model according to the geometry and attributes of the strokes.

The beneficial effect of the present invention is to propose a stroke-based pencil sketch simulation and drawing system, which generates pencil sketch drawing by using a fully automatic generation algorithm or an interactive tool according to an input three-dimensional geometric model. The difference between the present invention and previous methods is mainly reflected in that more types of characteristic strokes can be generated, and a new method for fully automatic generation of hatched strokes by using sampling algorithm is proposed, and a new method of using hatching frame ( hatching carrier) interactive tool for stroke design. Users can also use the interactive tools provided by the system to edit, modify or delete the automatically generated feature strokes and hatch strokes, and can also add new feature strokes and hatch strokes. It can be seen from the results of the final pencil sketch that the system can quickly generate realistic pencil sketch drawings with hand-painted effects. Because the system has an auxiliary design function, it reduces the user's drawing skill requirements and accelerates the creation speed of pencil sketch works. In addition, compared with the traditional creation using paper and pencil, digital virtual design has advantages in transmission, reproduction and preservation, so the system has high application value in digital art design, film animation special effects, digital entertainment and other fields.

Description of drawings

Fig. 1 is a modular flow chart of the system of the present invention.

Fig. 2 is a schematic diagram of paths extracted from triangular patches.

Fig. 3 is a flow chart of automatically generating pencil hatch strokes.

Fig. 4 is a flowchart of a sampling algorithm based on cumulative probability density.

Figure 5 is a flowchart of the sampling algorithm based on Metropolis-Hastings.

Fig. 6 is a schematic diagram of two sampling point correction strategies.

Figure 7 is a diagram of the sampling point correction effect.

Figure 8 is the rendering of the teapot pencil sketch automatically generated by the system.

Fig. 9 is an effect diagram of a pencil sketch of a plant automatically generated by the system.

Fig. 10 is a schematic diagram of constructing strokes from paths.

Fig. 11 is a schematic diagram of different tip shapes for constructing paths.

Fig. 12 is a schematic diagram of the interactive stroke design tool provided by the system.

Figure 13 is an example of the trajectory of the control particles in the interactive stroke design tool.

Figure 14 is an example of drawing hatched strokes on a reference image using the interactive stroke design tool.

Figure 15 is an example of various strokes drawn by the interactive stroke design tool.

Fig. 16 is an effect diagram of each step of interactive pencil sketch design.

Figure 17 is an interactively generated pencil sketch rendering of a plant.

Figure 18 is the rendering of the colored pencil sketch.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

Figure 1 shows the modular flow chart of the whole system. The whole system mainly includes five modules: user interaction module, feature path automatic generation module, hatch path automatic generation module, stroke generation module from path, and drawing output module. Among them, the feature path automatic generation module, the hatch path automatic generation module, the stroke generation module from the path, and the drawing output module are automatically controlled by the system, and the user interaction module mainly provides various interactive tools for users to use.

The system first generates feature paths and hatch paths by the feature path automatic generation module and hatch path automatic generation module, and then the output feature path and hatch path are input into the stroke generation module from path. The brush stroke generation module from path generates stroke geometry and attributes from the input feature path and hatch path according to the stroke construction rules, and outputs them to the drawing output module. Finally, the drawing output module generates the final pencil sketch image according to the input stroke geometry and attributes, as well as the physical model drawn by the pencil. The user interaction module provides interactive tools for the entire generation process, enabling users to supplement, delete and edit the automatically generated results.

In the description of the present invention, a path (path) refers to a trajectory that a stroke passes through. This trajectory forms the backbone of the stroke and corresponds to the trajectory that the center point of the brush tip travels on the paper in actual manual painting. Stroke (stroke) refers to a graphic representation with geometric shapes and attributes in computer graphics, corresponding to a certain line in actual manual painting. Feature path (feature path) refers to the line path used to describe the feature, usually used to reflect the geometric characteristics of the object (such as the boundary of the object), the boundary of color, the boundary of light and shade, etc. The hatching path is mainly used to express the light and shadow, tone, color, material, etc. in the scene, and generally appears in the form of a group set composed of a large number of paths. The strokes generated by feature paths and hatch paths are called feature strokes and hatching strokes, respectively.

Reference image (reference image) is a technique in graphics that uses two-dimensional data structures to record and reflect three-dimensional information. It was first proposed by Kowalski et al. in 1999. Since the data stored in the early frame buffer needs to be read back to the memory for processing, and is usually stored in an image format, it is called a reference image. With the development of programmable graphics pipeline (programmable graphics pipeline), data can be processed directly in video memory at present, so it can also be called reference buffer (reference buffer).

Three kinds of reference images are used in the present invention, which are ID reference image, color reference image and illumination reference image. Among them, the identity reference image uses different color values to distinguish the drawn objects, which is used for the hidden line algorithm of the feature segment; the color reference image and the illumination reference image are used to assist in the design of the shadow line.

The user interaction module, the feature path automatic generation module, the hatch path automatic generation module, the stroke generation module from the path, and the drawing output module will be described one by one in conjunction with Figure 1 below.

1. User interaction module

The user interaction module provides the following tools for the user to perform parameter selection and interactive design:

(1) Select the desired 3D geometric model and output it to the feature path automatic generation module and hatch path automatic generation module, and on this basis, select and analyze the model coordinates, camera coordinates, light source position and type, light model, etc. set up. Users can also select the automatically generated feature path type according to their needs;

(2) After the system draws the color reference image according to the selected geometric model and illumination model, the user can edit the color reference image as required;

(3) Users can use the path design tool to design paths based on the reference image according to their needs. For details, refer to the description in the interactive tool of path design;

(4) The system provides different types of pencils, which are divided into 9B, 8B, ... 2B, B, F, HB, H, 2H, ... 6H, etc. Different types of pencils contain graphite, soil, The percentage of paraffin is also different;

(5) The system uses Perlin noise to generate the height field of various papers, and can be used for the physical model of pencil drawing;

(6) After the system automatically generates feature strokes and hatch strokes, users can also use interactive tools to pick up, edit, modify, and delete generated strokes, and can also use interactive tools to add new strokes.

2. Feature path automatic generation module

The function of the feature path automatic generation module is to generate the required feature path from the input 3D geometric model, which mainly includes the following four steps:

(1) Extract feature segments from the triangular faces stored in the input 3D geometric model;

(2) Constructing an identity reference image according to the input 3D geometric model and the feature fragments extracted in step (1), for the subsequent hidden line algorithm;

(3) Utilize the identity reference image, use the hidden line algorithm to hide the feature segment;

(4) Link the hidden feature segments into a feature path.

The feature paths used in the system include silhouettes or occluding contours, suggestive contours, creases, borders boundary edges, ridges, valleys , apparent ridges, suggestive highlights, principal highlights, etc. These feature paths can be automatically generated from 3D geometric models by existing methods. There is no distinction between various feature paths. Each feature path has its own unique characteristics, and each feature path also has applicable and inapplicable occasions. There are overlapping and similar components between some feature paths, such as ridges and obvious ridges, obvious ridges and hint contours. Therefore, in actual use, the types of feature paths are not the better, too many are easy It appears disorganized, and the feature paths are too few to reflect the required features, reducing the expressive content of the final pencil sketch result. Therefore, the present invention not only provides a module for automatically generating feature paths, but also provides tools in the user interaction module to allow users to add paths by themselves. In actual use, it is more convenient and effective for the user to let the system automatically generate feature paths and hatch paths according to needs, and then select, edit, and delete operations based on the automatically generated results, or add new feature paths and hatch paths by themselves. s solution.

Existing feature path extraction methods that take 3D geometric models as input all obtain feature paths from the calculation of triangular patches. The feature path extracted from a single triangular patch is called a feature segment, which consists of two endpoints and the connecting line between the two endpoints. Multiple feature line segments connected end to end are linked to form a feature path. Here, the contour line is taken as an example for illustration, other types of feature path extraction algorithms can refer to the corresponding literature.

For smooth surfaces, the contour line can be defined mathematically as:

n _i ·(cv _i )=0,

Among them, v _i is the space coordinate of vertex i, _ni is the vertex normal vector of v _i , and c is the space coordinate of the camera. The specific algorithm in practical application is as follows: First, calculate the normalized dot product value for each vertex in the triangular patch:

${\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; ,</span>$

Vertex symbols are then labeled according to the following formula:

${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{c}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; :</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; 0</span>}\\ <span\; class="notranslate">\; -</span><span\; class="notranslate">\; :</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.<span\; class="notranslate">\; ,</span>$

If the _si values of the three vertices in the triangle facet have different signs (2 are + sign and 1 is - sign, or 2 are - sign and 1 is + sign), it means that there is Feature fragments, as shown in Figure 2(a). The two endpoints (s ₁ , s ₂ ) of the feature segment can be obtained by using the following linear interpolation formula:

${\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="d"\; filenames="HDA0000096640330000021.png"\; state="\{\{state\}\}">d</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="d"\; filenames="HDA0000096640330000021.png"\; state="\{\{state\}\}">d</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="d"\; filenames="HDA0000096640330000021.png"\; state="\{\{state\}\}">d</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span>}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="v"\; filenames="HDA0000096640330000021.png"\; state="\{\{state\}\}">v</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>$

${\mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="2"\; label="s"\; filenames="HDA0000096640330000021.png"\; state="\{\{state\}\}">s</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; |</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; |</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; ,</span>$

Since the surface is continuous and smooth, the feature segments extracted from each triangular patch can be naturally connected end to end to form a feature path, as shown in Figure 2(b).

In the present invention, after the feature segments are extracted, the representation of the current feature path is a collection of a series of feature segments. Due to deep occlusion, many of the fragments are invisible, but the information of the original 3D geometric model surface no longer exists after the feature fragments are extracted. When drawing these feature fragments, the z-buffer in the graphics processor will only be on the fragment pixels For depth detection, the drawn feature segments cannot reflect the depth information of the original 3D geometry, so it is necessary to use the hidden line algorithm to hide the feature segments. The algorithm used here is to use the identity reference image for concealment. This algorithm was proposed by Northrup et al. in 2000, which is a hybrid concealment algorithm of geometry and image. The algorithm first constructs an ID reference image: mark feature fragments and 3D geometric models in different colors and draw them to the frame buffer, and then read the drawing results back from the frame buffer to the memory to construct an ID reference image . When performing feature segment concealment, each feature segment is detected and matched with some specified pixels in the identity reference image to achieve concealment.

After concealing the extracted feature fragment set, the next work is to connect these concealed fragments end to end to form a feature path, so as to facilitate the subsequent generation of stroke geometry for drawing. If some feature segments are too short to be smaller than one pixel on the screen, these segments can be merged with adjacent segments.

If the generated path is too long and does not conform to the actual painting style, the long path can be divided into several shorter paths according to the maximum path length, or according to the included angle between adjacent segments in the path, in the path with a relatively large curvature Where to split the path.

3. Hatch path automatic generation module

The function of the hatch path automatic generation module is to generate the required hatch path from the input 3D geometric model, which mainly includes the following three steps:

(1) Draw the corresponding color reference image and illumination reference image according to the input 3D geometric model, and which reference image to generate can be determined by the illumination and coloring model;

(2) Users can use color editing tools to edit the color reference image as needed;

(3) Generate the hatch path from the color reference image and lighting reference image. Afterwards, users can also use the interactive tools provided by the user interaction module to edit and modify the automatically generated results.

The hatching in pencil sketch is a collection of a series of strokes arranged according to certain rules. Hatch strokes can simultaneously express light, color, material, shape and other information through tone. Strokes have two properties: 1. Each stroke can express tone and texture at the same time, that is, the attributes of a single stroke. 2. The collection of multiple strokes can also reflect the tone and texture, that is, the group attributes of the strokes.

In our system, the shadow lines can be designed according to the color reference image and the lighting reference image respectively. They correspond to the intrinsic property (intrinsic property) and extrinsic property (extrinsic property) of the object respectively. Distinguishing intrinsic properties from extrinsic properties enables these two parts to be edited independently, such as using color editing software or lighting editing software independently. Another more important reason is that mixing intrinsic and extrinsic attributes often causes visual errors, which is difficult to grasp even for professional painters, because users can estimate the original color of the object very well, but It is difficult to correctly estimate the color of light reflected off an object. Treating the color system and the lighting system separately, and using the color reference image and the lighting reference image to locate the hue, will avoid this visual error. The user specifies the original color of the object, and the existing lighting rendering system in graphics can quantify the light intensity, so that the division of labor between the user and the computer can be realized. In fact, the design method of distinguishing inherent attributes from external attributes is also described in professional art teaching techniques. Due to different professional backgrounds, computer workers and art workers have different terms and descriptions, but the concept and essence are the same. For example, Dodson described in his classic painting tutorial "Keys to Drawing" how to understand the painting creation process from the perspectives of natural tone, light and shadow pattern, and lightness pattern: the painter combines the light and shadow pattern with the natural tone to get the final basis. Lightness pattern for photo painting. The distinction between color reference images and lighting reference images in the system is also based on this concept of painting creation.

The main attributes of a single hatch include position, direction, length, pressure distribution, etc. In addition, the set of hatches also has group attributes such as density. In the actual painting process, the painter arranges the brush strokes on the drawing paper one by one on the basis of constantly observing the object, and finally uses all the brush strokes on the drawing paper to approach the painting object. This process can be abstracted as a sampling problem. We represent the drawing object with a reference image, so the generation and distribution of hatching can be regarded as a sampling problem of a two-dimensional reference image.

During specific implementation, various attributes of the hatching can be determined step by step. Refer to FIG. 3 for the entire hatching stroke generation process. The following are the steps for generating hatch strokes, wherein steps (1)-(4) are steps for generating hatch paths:

(1) First draw a reference image (Fig. 3(b)) according to the input 3D geometric model (Fig. 3(a));

(2) Then according to the reference image, some sampling method is used to obtain the position and distribution of the shadow line (Fig. 3(c)), and the specific sampling method will be described below;

(3) Due to the size and shape of the stroke, considering that the coincidence of sampling points will affect the drawing effect, the sampling points need to be corrected (Figure 3(d));

(4) After the sampling point is corrected, the direction of the hatch path is determined according to the gradient information of the reference image, and then the length of the hatch path is determined according to the gray value of the reference image, and then the shape of the hatch path is determined by Bezier curve interpolation, and finally determined The hatched pressure distribution (Fig. 3(e)), so far the hatched path has been generated;

(5) The final generated hatch path is generated by the path generating stroke module, and the final result is drawn by the drawing output module (Fig. 3(f)).

Assuming that the two-dimensional probability density function of the reference image is f(x, y), the hatching design is equivalent to obtaining a number of sampling points from f(x, y) to generate strokes, and these generated strokes can be used to reflect the original reference The distribution characteristics of the image. Here we discuss two sampling methods: sampling based on cumulative probability density and sampling based on Metropolis-Hastings algorithm.

Sampling based on cumulative probability density is to redistribute uniformly distributed random numbers using the slope of the cumulative probability density function, and its one-dimensional algorithm is shown in Figure 4. The algorithm first calculates the cumulative probability density function according to the probability density function, then generates a uniformly distributed random vector, and uses the inverse function of the cumulative probability density function to transform the uniformly distributed random vector, then the transformed random vector conforms to the original probability density function distribution. For the two-dimensional probability density function f(x, y), you can first obtain the marginal probability density function for y, then use the above-mentioned one-dimensional cumulative probability density algorithm, and then obtain the distribution of x according to the conditional probability formula and use the above-mentioned one-dimensional cumulative probability density algorithm. The method is actually to convert two-dimensional sampling into sampling y first and then sampling x through marginal probability density and conditional probability density.

当前的采样点为z_n，生成候选采样点z^*的提议分布为Q(z^*|z_n)，则生成N个采样点的Metropolis-Hastings算法如图5所示。算法开始时将生成的采样点个数n初始化为0。当生成的采样点个数小于N时，算法循环执行，否则算法结束。每个循环内算法根据提议分布生成一个候选采样点，计算其接受概率，并和一个随机数比较，从而决定该生成的候选采样点是否应该被接受。Metropolis-Hastings算法每次根据当前采样点z_n生成下一个采样点，其采样点序列(z₁，z₂，...，z_n)形成一条马尔可夫链，且序列路径为二维平面的随机游走。算法确定接受A(z_n→z^*)≥1的采样点，随机接受A(z_n→z^*)＜1的采样点。随着采样点个数N的增加，采样点序列的分布可以趋近于概率密度函数f(x，y)。The Metropolis-Hastings algorithm is an algorithm that can sample arbitrarily complex distributions. The algorithm only needs to know the estimated probability of the points to be sampled or the ratio of probability estimates between the points to be sampled, and it can sample without knowing the probability distribution function, which has good universality. For two-dimensional digital images, this algorithm can directly use the gray value of the image as the estimated value for two-dimensional sampling, avoiding the estimation of probability density function, marginal probability density function and conditional probability density function in the cumulative probability density sampling algorithm. Assuming that we need N sampling points, the two-dimensional sampling points on the image are represented by z=(x, y), and the estimated probability is

The current sampling point is z _n , and the proposed distribution for generating candidate sampling point z ^* is Q(z ^* |z _n ), then the Metropolis-Hastings algorithm for generating N sampling points is shown in Figure 5. At the beginning of the algorithm, the number n of sampling points generated is initialized to 0. When the number of sampling points generated is less than N, the algorithm is executed cyclically, otherwise the algorithm ends. The algorithm in each loop generates a candidate sampling point according to the proposed distribution, calculates its acceptance probability, and compares it with a random number to determine whether the generated candidate sampling point should be accepted. The Metropolis-Hastings algorithm generates the next sampling point each time according to the current sampling point z _n , and its sampling point sequence (z ₁ , z ₂ ,..., z _n ) forms a Markov chain, and the sequence path is a two-dimensional plane random walk. The algorithm determines to accept sampling points with A(z _n →z ^* )≥1, and randomly accepts sampling points with A(z _n →z ^* )<1. As the number of sampling points N increases, the distribution of the sampling point sequence can approach the probability density function f(x, y).

In the pencil sketch simulation application of the present invention, the sampling points are used to reflect the stroke distribution, and the strokes have an area. Since the sampling is carried out in a limited reference image area, there must be some sampling points that are close to each other. even overlap. From the visual effect of the final drawing, the sampling points with similar positions appear as a series of overlapping strokes, thereby weakening the visual effect of the drawing. This is more noticeable when the stroke transparency is relatively low, the stroke lacks an overlay effect, etc. The more average pixels of strokes in the final generated image, the greater the influence caused by the overlap of sampling point positions. Although the distribution of sampling points does approach the required probability distribution from the statistical histogram, the actual drawing effect does not meet the requirements.

In view of the above problems, the present invention corrects the obtained sampling points in a small range. The specific sampling point overlap correction algorithm is as follows:

(1) For each sampling point z _n obtained, find out whether there are other sampling points in its neighborhood, where the radius of the neighborhood is the tolerable minimum sampling point spacing, which can be set to half of the average stroke width;

(2) If there are other sampling points in the neighborhood, then calculate the position center of other sampling points, and then let the investigated sampling point z _n move a small distance δ along the direction opposite to the center of other sampling points, as shown in Figure 6( a) as shown. In order not to affect the probability distribution of the obtained sampling points, we only let the sampling points move to the target position with the same gray value as its original position.

(3) Check whether the target position meets the requirements. If the target position does not meet the requirements, let the sampling point _zn move a small distance δ along the opposite direction from its nearest sampling point, as shown in Figure 6(b);

(4) If the gray value of the target position still does not meet the requirements, the correction of the sampling point z _n is abandoned, and the effect of avoiding overlapping is achieved by the correction of other sampling points;

(5) The above correction process can be repeated several times until most of the sampling points meet the requirements.

Using the sampling point overlap correction algorithm can use as few sampling points as possible to achieve a better drawing effect, and maximize the use of each sampling point to generate a brushstroke drawing effect.

Figure 7(c) and Figure 7(d) respectively show the effect diagrams before and after the overlap correction of sampling points. Its corresponding input 3D geometric model and reference image are shown in Fig. 7(a) and Fig. 7(b) respectively. It can be seen that the sampling point overlap correction algorithm proposed by the present invention can avoid overlapping of sampling points as much as possible, and improve the visual effect of strokes generated by a single sampling point.

After obtaining the hatching position and distribution information, the hatching direction can be determined according to the gradient information of the reference image. In order to maintain the characteristic stroke information in the generated image, the system uses the direction perpendicular to the image gradient as the hatching direction, which can also minimize the influence of stroke size and shape on sampling. If the gradient of the reference image is less than a certain threshold, the "lower left-upper right" direction is used as the default direction of hatching. This direction is the most frequent direction for human hand-painted hatching, and it is also most in line with human drawing habits and ergonomic characteristics. Since painters often use the "blind painting" painting technique (the eyes keep on drawing the object while the hand does not stop painting) during the painting process, and the brush tip often jumps, this default direction has a certain degree of science in both theory and experiment in accordance with.

After determining the position, distribution and direction of the hatch, the length and path of the hatch can be obtained by the following algorithm:

1. Generate control particles: Generate a pair of control particles [p ₊ , p _- ] at each generated sampling point. The control particles have attributes such as speed (v _x , v _y ) and initial color c ₀ . The velocity direction of each pair of control particles is consistent with the direction of the shadow line and the opposite direction of the shadow line;

2. Moving control particles: Calculate the target position of each control particle, that is, the position where the control particle moves a step along the direction of velocity (v _x , v _y ). The step size can be set according to needs, generally 1 to 5 pixels value; if the reference image gray value c _p of the target position of the control particle satisfies |c ₀ -c _p |<δ _c , let the control particle move to the target position; if it is not satisfied or the control particle reaches the maximum stroke length, let the control particle The particles stop; where δ _c is a threshold value, the larger the δ _c , the more obvious the soft edge effect of the hatched stroke; the smaller the δ _c , the more obvious the hard edge effect of the hatched stroke;

3. According to the final position of each pair of control particles, use the Bezier curve interpolation method to obtain the hatch path. The hatching path also records the pressure distribution of the corresponding hatching strokes according to the gray value of the reference image, so as to be used for subsequent drawing of pencil strokes.

It can be seen from the sampling point overlap correction algorithm and the shadow path generation algorithm that we always make the area covered by each shadow line have an approximate gray scale, that is, distribute the shadow lines according to the color blocks in the image, which is in line with the pencil sketch professional The concept of drawing according to the shape decomposition in the tutorial can make the generated pencil sketch picture have a better hand-painted effect.

Figure 8(c) shows the automatic pencil sketch generation results of the teapot geometry, in which both feature strokes and hatch strokes are automatically generated by the algorithm. There are 38 characteristic brushstrokes and 8000 hatched strokes in the picture. Figure 8(a) and Figure 8(b) respectively show the input 3D geometric model and the generated lighting reference image.

Figure 9(c) shows another automatically generated pencil sketch picture, which contains 266 automatically generated feature strokes and 8000 hatch strokes, and the corresponding 3D geometric model input and lighting reference images are shown in Figure 9(a ) and Figure 9(b).

4. Generating stroke modules from paths

The function of this module is to generate feature strokes or hatch strokes from feature paths or hatch paths. Referring to Figure 10, the path is represented by square vertices and dotted lines, representing the trajectory of the pen tip; the stroke is represented by round vertices and solid lines, representing the shape and texture of the lines left by the pen tip on the paper. As mentioned in the previous part, strokes have two attributes: 1. Each stroke can express tone and texture (separate attributes) at the same time. 2. A collection of multiple strokes can also reflect tone and texture (attributes of the group). This part mainly involves the characteristics of a single stroke, and the group attribute of strokes has been described in the hatch path automatic generation module.

The characteristic strokes and hatched strokes in the system are jointly generated by the characteristic path and hatched path obtained by the above two modules, and the tip shape of the pencil, as shown in Figure 10. Among them, the square vertex represents the vertex forming the path, and it forms the geometric representation of the path together with the path line segment; the round vertex and the solid line segment form the geometry of the stroke. The nib shape positions the geometry of the stroke at each path vertex, and together with the path, forms the geometry of the stroke. The nib shape is composed of a series of predefined shapes, such as typical shape, broad shape, chisel shape, etc., as shown in FIG. 11 . There are also a range of nib sizes for users to choose from. For feature paths, we can also automatically adjust the size of the pen tip with the details of the input geometry, because users usually use thinner pen tips at the details to maintain the details. The tip of the pen in painting is also affected by pressure. The greater the pressure, the larger the size of the tip formed by the pencil on the paper. Therefore, the final nib size S can be expressed by the following formula:

S=s _t ×f _g ×f _p ,

Among them, s _t is the global tip size selected by the user, f _p is the pressure factor, and f _g is the geometric detail scaling factor, which can be defined by the following formula:

是控制因子。Among them, A _v is the Voronoi area (one-ring Voronoiarea) of the ring triangle of the vertex of the input geometric model,

is the control factor.

5. Draw the output module

The function of this module is to draw the final pencil effect picture from strokes. The pencil physical model used for drawing adopts the Sousa pencil model, and the corresponding algorithm is implemented in the graphics processing unit (GPU). The GPU is the computing unit of modern graphics cards. Unlike the CPU, the GPU has a very powerful parallel processing capability and is suitable for processing massive graphics data.

The Sousa pencil model involves multiple physical variables when the pencil interacts with the paper, and uses the grain in the paper geometry as the basic unit for simulation. Each grain is composed of pyramids surrounded by four adjacent heights in the paper height field. The system stores the height of the paper and the composition of the pencil lead (including graphite, earth, and paraffin) into the R, G, B, and A channels of the texture, and packs them into a double buffer. When drawing, iteratively swap two textures as drawing targets and update each component in the texture according to the pixel shader program. Finally, the final pencil effect image is generated according to the graphite, dirt and paraffin content stored in the texels (each pixel corresponds to a grain). For example, if the color of graphite is C _G , the color of soil is C _C , the color of paraffin is C _V , and the contents of graphite, soil and paraffin in a grain are V _G , V _C and V _V respectively, then the final The color value C displayed by this grain is:

C＝C _G ×V _G +C _C ×V _C +C _V ×V _V ,

The color values displayed by all the grains on the paper reflect the color values of the final pencil sketch image together, forming the drawing effect of the pencil sketch. Users can also edit and modify the final strokes, and then repaint. Since the drawing speed of the graphics processing unit is very fast, it can basically meet the requirements of real-time interaction.

6. Interactive tools for hatch design

The system provides several tools for designing hatching in batches on reference images, they are parallel curve hatching tool, wave hatching tool, cross hatching tool and Free hatching tool (free hatching tool), as shown in Figure 12. Among them, the Parallel Curve Hatch tool has parameters to control its curvature and symmetry. The Wavy Hatch tool provides parameters to control how smooth its waves are. Through parameter adjustment, parallel curved hatching can degenerate into parallel line hatching, and wavy hatching can degenerate into zigzag hatching. These types of hatching are common hatching in pencil sketch drawings. The system also provides a free hatching tool for users to design their own hatching styles. Since the free hatch tool can draw strokes of any style, it can be used not only to design the path of hatch strokes, but also to design the path of feature strokes. The designed feature strokes and hatch strokes are generated through the path-generated stroke module to obtain the required feature strokes and hatch strokes. We use the concept of a hatching carrier as the carrier of the stroke path design tool, as shown in the longest path from left to right in Figure 12, the hatching carrier is a single line drawn by the user Path, the hatch path is automatically generated in batches according to the path of the hatch frame. During the painting process, users only need to use interactive input devices, such as mouse, pressure pen, etc. to draw a stroke frame on the reference image, and the hatching tool will automatically draw a series of hatching lines that can reflect the local tone according to the shape of the area. Curves drawn by hand are simple and flexible, and the hatching design of an area can be completed according to a hatching frame, so that the purpose of quickly designing hatches can be achieved, similar to gesture recognition in a browser.

The specific algorithm for generating hatches in batches with a hatch frame is as follows:

(1) When the user draws the shadow frame, record the input coordinate point sequence and the passed reference image pixel value, and calculate the median pixel value c _m of the passed reference image pixel;

其中δ_d为密度控制参数，以像素为单位。这里假设所有颜色值c_m，c₀，c_i0，c_j0等均已经规范化到[0，1]；(2) Initialize some control particles p ₁ , p ₂ , ... p _i , p _j , ... p _n on the shadow frame, these control particles have speed (v _x , v _y ) and initial color c ₀ and other attributes, where the initial color c ₀ is determined by the color value of the reference image that controls the position of the particle, and the initial speed is set according to the type of hatching. In the following, when specifying the initial color value of a certain control particle, the serial number of the control particle and 0 are used to represent it together. For example, the initial color value of the control particle P _i is c _i0 , and c ₀ is used to represent the color value of a control particle in general. Control the spacing between particles p _i and p _j as

where δ _d is the density control parameter in pixels. It is assumed here that all color values c _m , c ₀ , c _i0 , c _j0 , etc. have been normalized to [0, 1];

(3) The system allows the control particles to move to both sides of the shadow frame according to the type of the shadow line, as shown by the white arrows in Figure 13, and the thin white lines in the figure indicate the movement path of the control particles. Here, only the control particle is allowed to survive in the region satisfying |c ₀ -c _m |<δ _c , and the control particle will die if it leaves this region (the meaning of death here is consistent with the meaning discussed in the particle system, that is, the end of particle function and release of storage space). where _δc is the color difference control parameter within the region, which can be adjusted by the user. The larger the δ _c is, the more obvious the soft edge effect is; the smaller the δ _c is, the more obvious the hard edge effect is:

(4) According to the death position of the control particle, the shadow path is obtained by Bezier curve interpolation or Catmull-Rom interpolation.

It can be seen that the hatching design tool has the following characteristics: the hatching type is specified by the user, and the hatching direction is indirectly specified by the hatching frame (controlled by the user). Tint and hatch density are automatically determined (computer controlled) from the color reference image and lighting reference image. The hatch design tool is aware of shapes in reference images and can automatically distinguish between soft and hard edges (computer controlled). Figure 14 shows the interface diagram of using the hatch design interactive tool to design hatches based on reference images.

Figure 15(b) shows a simple pencil sketch, and its input 3D geometric model is shown in Figure 15(a). The results mainly showcase the automatically generated feature strokes and various hatch strokes (including parallel curved hatch, wavy hatch, and crosshatch) drawn by interactive tools. The picture contains 291 characteristic strokes and 777 hatched strokes. It can be seen from the figure that the pencil lines drawn by the system are relatively real. The thickness and pressure of the lines are varied, similar to the effect of human hand-painting.

Both Figure 1 and Figure 16(a)-(d) show the drawing process of the pencil sketch of the cow. Among them, Figure 16(a) shows the automatically generated feature strokes; Figure 16(b) shows the hatched strokes (inherent attributes, representing the original tone) interactively generated according to the edited color reference image; Figure 16(c) shows Hatch strokes (extrinsic properties, representing light and shadow patterns) generated interactively from a lighting reference image. All strokes in Figure 16(a), Figure 16(b) and Figure 16(c) are drawn together to form the final drawing effect, as shown in Figure 16(d). It contains a total of 211 characteristic strokes and 2747 hatched strokes. As mentioned earlier, this drawing method is in line with the cognition and habit of human drawing.

Figure 17(c) shows a plant pencil sketch generated using the interactive tools provided by the system, and the corresponding input 3D geometric model and reference image are the same as those shown in Figure 9(a) and Figure 9(b), respectively as Figure 17(a) and Figure 17(b). Figure 17(c) contains a total of 266 feature strokes and 7231 hatch strokes. From the comparison of Figure 17(c) and Figure 9(c), it can be seen that both the automatic generation method and the interactive design method can generate real pencil sketches, but the results drawn using interactive tools are compared with the automatically generated results. The details are handled better and have a more hand-painted effect.

Figure 18 shows a colored pencil sketch generated using the interactive tool, which contains a total of 766 feature strokes and 2631 hatch strokes. Among them, the feature strokes are automatically generated by the system, and the hatch strokes are generated by interactive tools. The colors in Figure 18 are from the color reference image. It can be seen from the results that the pencil sketch pictures generated by the system are more realistic and have the effect of hand-drawing.

Since the system in the present invention can automatically generate characteristic strokes and hatched strokes, and allows users to edit and modify the generated strokes, and can also use interactive tools to add strokes by themselves, the interactive tools provided by the system can also be based on reference images. The hatching is automatically arranged according to the color tone, thus reducing the professional painting skills required by the user. In addition, according to statistics, users without drawing foundation use interactive tools to draw hatching at a speed of 5-11 strokes/second, which is much faster than traditional manual hatching. The advantages of using interactive tools to design hatches are more prominent when drawing hatches in large areas, so this system is more suitable for drawing large-scale works. In addition, the system currently uses a normal mouse as input. If you use tools such as a professional stylus and pressure tablet, it is estimated that the efficiency will be further improved.

As can be seen from the above analysis, the system of the present invention can simulate realistic pencil sketch works. Even users without drawing foundation can quickly generate pencil sketches with hand-painted effects with the assistance of the system. Because the system is based on strokes, compared with other pencil sketch generation methods such as image, volume rendering, and texture mapping, it has the advantages of conforming to the actual painting creation process and being easy to edit and modify, so it has higher practical value.

The above experimental results and the brush stroke-based pencil sketch simulation and drawing method can be used in fields such as computer graphics, computer-aided design, movie animation special effects, art design, etc. It has the advantages of simple operation, fast generation speed, realistic effect, easy modification and editing, The characteristics of wide application prospects.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (11)

1. the pencil sketch based on style of writing is simulated and drawing system, it is characterized in that this system comprises:

User interactive module, being used to the user provides the interface to carry out that 3-D geometric model is selected and export the 3-D geometric model of selecting to characteristic path automatically-generating module and hachure path automatically-generating module;

Characteristic path automatically-generating module is used for the automatic generating feature of the 3-D geometric model path according to input;

Hachure path automatically-generating module is used for generating the hachure path automatically according to the 3-D geometric model of input;

Generate the style of writing module by the path, be used for according to the characteristic path of characteristic path automatically-generating module generation and the hachure path of hachure path automatically-generating module generation, the geometry and the attribute thereof of generating feature style of writing and hachure style of writing;

Draw output module, be used for geometry and attribute, adopt the pencil physical model to draw out the pencil style of writing according to style of writing.

2. the system of claim 1; It is characterized in that said user interactive module also provides interactive tools to supply the user to carry out following operation: color reference picture editting, hachure path design, pencil attribute are chosen, the sketch paper selection of parameter, add the deletion style of writing and edit-modify has generated style of writing.

3. system as claimed in claim 2; It is characterized in that; Said hachure path design is meant that system's common hachure type in to the pencil sketch concludes on the basis of summary; It is divided into four types; And use parallel curves hachure instrument, wave hachure instrument, cross hatch instrument and free hachure instrument respectively, with hachure frame (hatching carrier) as carrier carry out interactive mode, the hachure path generates in batches, wherein free hachure instrument also can be used for the path of design feature style of writing.

4. system as claimed in claim 3 is characterized in that, said hachure frame is the independent paths that the user drew, and the hachure path is carried out in batches, generated automatically according to the path of hachure frame.

5. system as claimed in claim 3 is characterized in that, said with the hachure frame as carrier carry out interactive mode, in batches the step that generates of hachure comprises:

Step 1: when the user drew hachure frame path, system log (SYSLOG) descended the coordinate point sequence of input and the reference picture pixel value of process, and calculated the median pixel value c of the reference picture pixel of process _m

Step 2: system is some control particle p of initialization on the hachure frame ₁, p ₂... p _i, p _j... p _n, these control particles have speed v _x, v _yWith priming color c ₀, and control particle p _iAnd p _jBetween spacing do

δ wherein _dBeing the density controlled variable, is unit with the pixel, c _I0Be control particle p _iThe priming color value, c _J0Be control particle p _jThe priming color value;

Step 3: system let the control particle according to the type of hachure to two lateral movements of hachure frame, and only let the control particle satisfy | c ₀-c _m|＜δ _cRegion memory live δ wherein _cIt is the color distortion controlled variable in the zone;

Step 4: according to the dead position of control particle, with Bezier (B é zier curve) interpolation or Ka Temaer-obtain the hachure path like (Catmull-Rom) interpolation not.

6. the system of claim 1 is characterized in that, the step in said automatic generating feature path comprises:

(1) tri patch of from the 3-D geometric model of input, storing extracts characteristic fragment;

(2) construct the identity reference picture according to the 3-D geometric model and the middle characteristic fragment that extracts of step (1) of input;

(3) utilize the identity reference picture, use the blanking line algorithm that characteristic fragment is carried out blanking;

(4) characteristic fragment after the blanking is linked into the characteristic path.

7. the system of claim 1 is characterized in that, the step in said automatic generation hachure path comprises:

(1) draws out corresponding color reference image and illumination reference picture according to the 3-D geometric model of input;

The position and the distribution of (2) color reference image and illumination reference picture being sampled and obtaining hachure;

(3) sampled point is revised, to weaken the overlapping influence of style of writing to final drafting effect;

(4) sampled point is revised after, confirm the hachure path direction according to the gradient information of reference picture, confirm the hachure path according to gray-scale value, confirm the pressure distribution on the hachure path then, thereby generate the hachure path.

8. system as claimed in claim 7; It is characterized in that; Said color reference image and illumination reference picture correspond respectively to build-in attribute and the external attribute of drawing object, and its implication corresponds respectively to this color harmony shadow pattern of drawing object in the study course of specialty drawing.

9. system as claimed in claim 7 is characterized in that, the employed algorithm of said sampling have based on the sampling of cumulative probability density and based on the ultra Pohle of rice this-sampling of Haas spit of fland (Metropolis-Hastings) algorithm.

10. system as claimed in claim 7 is characterized in that, the said step that sampled point is revised comprises:

(1) each sampled point z to obtaining _n, search whether there is other sampled point in its neighborhood, wherein the minimum sampled point spacing of the radius of neighborhood for tolerating can be set at the half the of average style of writing width;

(2) if having other sampled point in the neighborhood, then calculate the place-centric of other sampled point, let the sampled point z that is investigated then _nMove a bit of along the direction opposite apart from δ with other sampling point position center;

(3) whether the inspection target location meets the demands, if the target location does not meet the demands, then lets sampled point z _nMove a bit of apart from δ along reverse direction from its nearest sampled point;

(4) if the gray-scale value of target location does not still meet the demands, then abandon revising sampled point z _n, avoid overlapping by the correction of other sampled point;

Iterate above-mentioned makeover process for several times, till most sampled points all meet the demands.

11. system as claimed in claim 10 is characterized in that, said target location can use following two kinds of methods to differentiate:

In order not influence the probability distribution of the sampled point that obtains, the requirement of target location is: the sampled point target location has same gray-scale value with its origin-location;

Perhaps in order to reduce the influence of correction algorithm to the sampled point probability distribution as far as possible, the requirement of target location is: the gray-scale value of sampled point target location differs certain threshold value with the gray-scale value of its origin-location.

Images