KR100896762B1 - Apparatus and method for painterly animation - Google Patents

Abstract

Disclosed animation apparatus and method are disclosed. The motion vector calculator is configured to generate a second image frame based on a position shift degree of pixels whose positions on a second image frame subsequent to the first image frame are moved among pixels constituting the first image frame that is temporally advanced among the plurality of image frames. Calculate the motion vector corresponding to. The edge image generator generates an edge image by detecting an edge in the first image frame. The first motion map generator generates a first motion map by connecting the first motion map generator from each of the first pixels constituting the edge image to a point obtained by adding a motion vector to the coordinates of the first pixel. The second motion map generation unit connects the second motion map generator from the second pixels whose positions except for the first pixel are moved among the pixels constituting the first image frame to a point obtained by adding a motion vector to the coordinates of the second pixel. Create a motion map. The rendering image generator generates a second rendering image corresponding to the second image frame by performing an overlay on the first rendering image corresponding to the first image frame based on the first motion map and the second movement map. According to the present invention, it is possible to maintain the temporal and structural consistency of brush strokes between image frames, to reduce the time required for rendering, and to reduce the effects and the flickering of brush strokes between frames.

Description

Apparatus and method for painterly animation

The present invention relates to an apparatus and method for pictorial animation, and more particularly, to an apparatus and method for expressing a continuous input image with a pictorial feeling as if drawn by hand.

The study of computer graphics, which began in the 1960s, aimed to produce photorealistic images. As represented by the term photo-realism over the last three decades, the ultimate interest of traditional computer graphics has focused on how realistic pictures can be produced. These traditional computer graphics produced very realistic images by mimicking the physical and mechanical characteristics of things, such as radiosity and ray-tracing.

Since the 1990s, the interest of computer graphics has shifted from non-photorealistic images to photorealistic images. He no longer clung to producing only realistic images, but began to study ways to produce non-realistic images. Non-Photorealistic Rendering has become a field of traditional computer graphics as a way of producing these non-realistic images. Non-realistic rendering is the opposite of traditional realistic rendering. While the conventional realistic rendering was intended to express the object more realistically, the non-realistic rendering aimed at expressing various characteristics of the image well and conveying the meaning well rather than being bound by the realistic representation in the object. To be. Therefore, the evaluation of non-realistic rendering is more influenced by human subjectivity than objective. Table 1 lists the differences between realistic and non-realistic renderings.

Photo-Realistic Rendering Non-Photorealistic Rendering Approach simulation Stylization Characteristic Objective Subjective effect Physical process Artistic process accuracy Clarity Approximate Perfection Complete results Incomplete results

Non-realistic rendering is a rendering method that expresses an effect, such as an image produced by an artist, of a specific object (2D image or 3D data). Non-realistic rendering techniques include Painterly Rendering, Mosaic Rendering, Cartoon Rendering, Technical Illustration Rendering, and Pen and Ink Rendering.

Painterly rendering is mainly done through brush strokes. Brush strokes are painted with a myriad of brushes when the artist draws a work. One movement is called a brush stroke. In pictorial rendering, we studied methods for determining various parameters of brush strokes in general in order to produce hand-drawn result images such as oil paintings. Parameters for brush strokes include position, color, size, direction, shape, texture, and whether to paint. Such a painterly rendering is a method of automatically expressing a painterly feeling that is drawn by hand using a brush stroke, and is highly dependent on the characteristics of the brush stroke. The characteristics of brush strokes are color, position, direction, shape, size and texture. Previous researchers tried to express various forms of pictorial effects by adjusting the parameters of these characteristics.

Haeberli proposed a method for determining the color, shape, size, and direction of brush strokes through user input on an input image. This method saves the brush strokes to be used in the database and expresses the abstract effect using the stored brush, but user input is essential.

Litwinowicz created brush strokes with lines and textures. The direction of the brush stroke used in this method is based on the gradient-based thin-plate spline interpolation method of the input image, and various brush strokes are created by adding random noise values to the parameters of each brush stroke. In addition, the brush stroke clipping method that considers the edge of the image is introduced. However, there is a disadvantage in that the size of the brush is small and monotonous effect is obtained using only a straight stroke.

Hertzmann proposed a method for automatically expressing pictorial rendering effects on input images. This method creates brush strokes along the edges of the spline curve and the slope of the image. The image is divided into several layers and each layer is created by rendering brush strokes of the same size. In addition, in order to improve the quality of the final result image, each layer was applied in order from the widest brush to the smallest. However, this method did not express the texture of the brush strokes shown in the actual painterly works, and the brushed images with different gradients were also expressed at the edges, resulting in the blurring effect of the resulting image.

On the other hand, pictorial animation is a method of expressing a pictorial feeling that is drawn by hand using a continuous input image. It is very important to maintain structural and temporal consistency of brush strokes between frames. To maintain temporal coherence between frames, Collomosse created stroke surfaces through analysis of video sequences such as spatiotemporal volumes. This method generates pictorial animation images based on the regions created using the segmentation algorithm, not the brush strokes. In order to maintain the temporal consistency of each region between frames, the user divides the region directly and There is also a problem with establishing relationships. In addition, Meier used the method of moving from the position of the brush stroke overlaid on the previous frame to the current frame using the 3D geometry information and the camera position information. This method has the advantage of accurately finding the position of the brush stroke between frames because it uses three-dimensional geometric information, but has the disadvantage of modeling all the scenes to be animated.

Traditional painterly animation researchers have established the relationship of brushes between frames based on brush strokes. The relationship between brush strokes between frames is highly dependent on the elements of the brush strokes such as position, color and direction. In particular, existing researchers use the motion vector data calculated by the Optical Flow method to move all the brush strokes existing on the canvas of the previous frame to the canvas of the current frame, The canvas is completed by filling the hole. Since these methods focus on the position of the elements of the brush strokes, there is a problem that flickering between frames does not occur or brush strokes are overlaid, but the color and shape are not changed, but the position is shifted. Done. This problem is a very artificial phenomenon that can not be expressed in the works created by animators.

The most important factor to be considered in pictorial animation with video input is to maintain the structural temporal consistency of brush strokes between frames and to paint the part where objects (foreground and background) are moving. To express.

To maintain the temporal consistency of the brush strokes between frames, Meier modeled polygons, such as 3D particle sets, that would render 2D paint brush strokes on screen space as artists paint many brush strokes on the canvas. This method maintains temporal consistency by converting 3D particles to 2D using the normal direction of the 3D polygon. In addition, the surface geometry and lighting effects were used to control the brushing effect.

Litwinowicz relocated by adding a new brush to the current frame using the optical flow method and Delaunay triangulation to maintain the temporal consistency of brush strokes. This method moves all brush strokes existing on the previous frame to the current frame by adding the motion vector calculated using the optical flow method. Set the center of the brush to one seed point and apply the Delany triangle. The spacing between the brushes was adjusted by rearranging the seed points appropriately by comparing the triangle area formed between the seed points with the maximum area specified by the user.

Hertzmann used the Paint-over method and the Light Flow method to maintain the temporal consistency of brush strokes between planes. The paint over method is similar to the paint on glass method, in which the resultant image of the previous frame is set as the initial canvas of the current frame. Although the paint over method improves the temporal consistency, flickering occurs when the static area of the video frame differs from the corresponding area of the previous frame over which the brushes are overlaid. This is caused by video noise, but Hertzmann reduced the flickering by using difference masking. It also reduces flickering by warping the brushes that are moved to the current frame by motion vectors.

Hayes maintains temporal coherence between frames by moving all the brushes from the previous frame canvas to the current frame canvas using estimated motion vectors between frames. It is said to be impossible. This method adds and redefines new brushes on the canvas to fill these holes. In addition, flickering occurs due to inconsistencies between the newly added brushes and existing brushes. Haze solved this problem by reducing the opacity of the brush by 10%. However, this method is so dependent on opacity that it gives the feeling of simple movement, not the brush between the frames. This phenomenon is not seen in the actual paint on glass animation and is very artificial.

The technical problem to be achieved by the present invention, by using a continuous input image to express the painterly feeling like a hand-drawn while maintaining the structural and temporal consistency of the brush stroke between the frames, the effect and the flicker that the brush strokes between the frames are overlaid To provide a pictorial animation device and method that can reduce the ringing phenomenon.

Another technical problem to be achieved by the present invention is to use a continuous input image to express the painterly feeling as if drawn by hand while maintaining the structural and temporal consistency of the brush strokes between the frames, and the effect that the brush strokes between the frames are overlaid with the effect. The present invention provides a computer-readable recording medium that records a program for executing a pictorial animation method that can reduce flickering.

In accordance with an aspect of the present invention, a pictorial animation device includes a plurality of image frames on a second image frame subsequent to the first image frame among pixels constituting a first image frame that is temporally advanced. A motion vector calculator configured to calculate a motion vector corresponding to the second image frame based on the degree of positional movement of pixels whose positions have been moved; An edge image generator for detecting an edge in the first image frame and generating an edge image; A first motion map generator configured to connect a first motion map from each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel to generate a first motion map; A second motion map by connecting from the second pixels whose positions except for the first pixel among the pixels constituting the first image frame to the point obtained by adding the motion vector to the coordinates of the second pixel; A second motion map generator for generating a signal; And a rendering image of generating a second rendering image corresponding to the second image frame by performing a painting operation on the first rendering image corresponding to the first image frame based on the first movement map and the second movement map. It comprises a generation unit.

In accordance with another aspect of the present invention, there is provided a pictorial animation method comprising: on a second image frame subsequent to the first image frame among pixels constituting a first image frame that is temporally advanced among a plurality of image frames. A motion vector calculating step of calculating a motion vector corresponding to the second image frame based on the positional movement degree of the pixels to which the position of the pixel is moved; An edge image generation step of generating an edge image by detecting an edge in the first image frame; A first motion map generation step of generating a first motion map by connecting each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel; A second motion map by connecting from the second pixels whose positions except for the first pixel among the pixels constituting the first image frame to the point obtained by adding the motion vector to the coordinates of the second pixel; Generating a second motion map; And generating a second rendered image corresponding to the second image frame by performing a painting operation on the first rendered image corresponding to the first image frame based on the first motion map and the second motion map. Generating an image.

According to the pictorial animation apparatus and method according to the present invention, a motion map including motion information of edges identified by motion vectors obtained from image frames subsequent to a pixel corresponding to an edge of a previous image frame in which a rendered image is generated. By determining the area of the brush corresponding to the current image frame by using, it is possible to maintain the temporal and structural consistency of the brush strokes between the image frames. In addition, based on the motion information between the frames, only a partial region of the rendered image corresponding to the previous image frame is overwritten, thereby reducing the time required for rendering. Can be reduced.

Hereinafter, exemplary embodiments of the pictorial animation apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a preferred embodiment of the pictorial animation apparatus according to the present invention.

Referring to FIG. 1, the pictorial animation apparatus according to the present invention includes a motion vector calculator 110, an edge image generator 120, a first motion map generator 130, and a second motion map generator 140. And a rendering image generator 150.

The motion vector calculator 110 is configured to determine the current image frame based on the positional shift degree of pixels whose positions on the current image frame subsequent to the previous image frame among the pixels constituting the previous image frame that advance in time are among the plurality of image frames. Calculate the motion vector corresponding to. In this case, the previous image frame and the current image frame are image frames constituting a moving image, and are selected to be adjacent to each other or one or a plurality of different image frames are positioned therebetween.

There are two methods for evaluating motion between image frames: optical flow method and block matching method. The optical flow method is a method of grasping the apparent velocity distribution of motion in an image caused by slowly changing the brightness type. The block matching method divides an image into blocks of a certain size and expresses all the pixels in the block as a motion vector. The block matching method moves the blocks of the previous frame by one pixel to find the motion vector. For the matching criteria, mean absulute error (MAE) and mean square error (MSE) methods are used. The following description refers to an embodiment in which a block matching method is applied to reduce an error of motion vector data. However, other methods including an optical flow method may be used to evaluate motion between image frames.

The edge image generator 120 detects an edge in a previous image frame and generates an edge image. The edge image for the previous image frame may be obtained by using an edge filter, using a directional edge, or the like. When using an edge filter, the edge image generator 120 generates an edge image by applying an edge mask to the source image. In this case, the edge image generator 120 first removes the noise from the source image using a Gaussian mask and then generates an edge image by performing an edge detection mask such as a Sobel mask or a Canny mask. In the first of these two processes, noise is removed, and in the second process, edges are detected in the image from which the noise is removed. In addition to the Sobel edge image, the canny edge image is well represented on the outside of the objects but the inner edge is not well represented. 2A to 2C show a source image, a Sobel edge image, and a canny edge image, respectively. The Sobel edge image is obtained by setting the threshold value to 100, and the Canny edge image is obtained by setting the Low Threshold to 1 and the High Threshold to 128.

The directional edge refers to a line formed along the grain of contrast in the image. The grain of contrast is different from the edges that appear using conventional edge filters such as Sobel edge filters or Canny edge filters. This is because the contrast of the contrast means the boundary between two contrast values, whereas the general edge means a pixel whose difference in brightness or color between neighboring pixels is larger than a user-specified threshold. In other words, it can be said that the general edge image is included in the resolution of the contrast meaning the directional edge. In addition, since it represents the boundary between two contrast values, the internal edges as well as the external edges of objects can be represented well. This directional edge becomes the reference edge for determining the direction of the brush strokes to be overlaid on the canvas. In order to generate the directional edge image, the edge image generator 120 quantizes the consecutive contrast values of the source image by specifying among N discrete contrast levels designated by the user (for example, 4 and 8). Next, the edge image generator 120 generates the directional edge image by setting the boundary of the contrast value as the edge using the quantized image. 3 illustrates a process of generating the directional edge image by the edge image generator 120.

The use of directional edges in generating an edge image has several advantages over the conventional edge mask. First, when using Canny Edge in a video, it is very difficult for the user to set an appropriate threshold for every frame. Also, if the same threshold is applied to all frames, the edges that were represented in the previous frame may not appear in the current frame. In addition, since the directional edge uses the boundary of the intensity value of each frame, the edge between frames is maintained even if the intensity value changes. In addition, the edges can be expressed in the region where the gradation phenomenon occurs because the boundary between the two contrast values is used.

Maintaining the temporal consistency of brush strokes between frames is important for converting video into pictorial renderings. According to the present invention, a strong motion map (hereinafter referred to as 'first motion map') and a weak motion map (hereinafter referred to as 'second') are obtained by using edge vector of the previous image frame and motion vector data calculated between the previous frame and the current frame. Create two types of motion maps.

The first motion map refers to pixels in which pixels corresponding to the edge of the previous frame are moved to the current frame by the motion vector. The pixel at the edge of the current frame may be calculated by adding the motion vector to the pixel corresponding to the edge of the previous frame using the calculated motion vector. The first motion map refers to an area from pixel to pixel. In other words, the first motion map refers to an area where the foreground between frames changes to the background or the background turns to the foreground. In addition, the second motion map refers to an area other than the edge although the inter-frame motion occurs. Unlike the first motion map, the map does not include pixels corresponding to the edges in each frame, and thus the color difference between neighboring pixels is small. In other words, the second motion map refers to an area where the foreground between frames changes to the foreground or the background turns to the background. In the present invention, by using the two motion maps to accurately grasp the position of the brush stroke to be overlaid in the next frame, it is possible to maintain the temporal consistency of the brush stroke between the frames and to express the painterly feeling as if drawn by hand.

The first motion map generator 130 generates the first motion map Motion_Map _str by connecting the first motion map motion to the point obtained by adding the motion vector to the coordinates of the first pixel from each of the first pixels constituting the edge image of the previous image frame. do. This process is expressed as a mathematical expression as follows. In this case, it is preferable to use only a vector having a value of (1.0, 1.0) or more.

In Equations 1 and 2, A (x, y) is a pixel in which motion occurs among pixels constituting an edge image of a previous image frame, and MV _{(x, y)} (Δu, Δv) is a previous image frame and a current image. Inter-frame motion vector, DL (A (x, y), B (x, y)) is a function that draws a line between two points A (x, y) and B (x, y).

4A to 4F respectively show an image obtained by extracting only an edge where a motion occurs from an edge image, an edge image of a previous image frame, an edge image, an image obtained by adding a motion vector to an edge where a motion occurs in an edge image, and The first motion map is shown.

The second motion map generator 140 adds a motion vector to the coordinates of the second pixel from each second pixel whose position other than the first pixel constituting the edge image among the pixels constituting the first image frame is moved. The second motion map is generated by connecting up to the obtained points. This process is expressed as a mathematical expression as follows. In this case, since the second motion map is a part of which color difference between neighboring pixels is small, it is preferable to use only a vector having a value of (2.0, 2.0) or more, unlike the first motion map.

In Equation 3, C (x, y) is a pixel excluding edges among pixels constituting the previous image frame, and MV _{(x, y)} (Δu, Δv) is a movement between the previous image frame and the current image frame. Vector, DL (C (x, y), D (x, y)) is a function that draws a line between two points C (x, y) and D (x, y).

5A to 5F respectively show a previous image frame, a current image frame, an image composed of pixels other than the edge at which the movement occurred in the previous image frame, an image composed of pixels other than the edge at which the movement occurred at the previous image frame, and a second motion map, respectively. Is shown.

The first and second motion maps described above are somewhat different depending on the type of edge image used. First, since the canny edge image includes only the outer edges of the objects, the motion map created based on this may not be painted inside the objects when brush strokes are applied, and thus there is a possibility that flickering occurs between frames. 6A to 6E illustrate a previous image frame, a current image frame, a canny edge image of a previous image frame, a first motion map, and a second motion map, respectively. The directional edge image then contains both the outer and inner edges of the objects. Therefore, a more accurate motion map can be generated, and even when brush strokes are applied, a more accurate motion map can be applied than a motion map created using the Canny edge image. 7A to 7E illustrate a previous image frame, a current image frame, a direction edge image of the previous image frame, and a first motion map and a second motion map, respectively.

The rendering image generation unit 150 generates a second rendering image corresponding to the current image frame by performing a repainting on the first rendering image corresponding to the previous image frame based on the first motion map and the second movement map. The rendered image generator 150 may have various configurations according to the type of brush stroke, a color determination method, a rendering method, and the like.

8A illustrates a configuration of the first embodiment of the rendered image generator 150. 8A illustrates an example of the rendered image generator 150 when the brush stroke pattern is a curved brush texture.

Referring to FIG. 8A, the rendered image generating unit 150 includes an overcoat direction determiner 810, an overcoat point determiner 820, a stroke pattern selector 830, and a renderer 840.

The overlay direction determiner 810 determines the overlay direction for each pixel constituting the previous image frame based on the gradient value of each pixel constituting the edge image. For this purpose, the overcoat determination unit 810 includes a gradient calculator 812, a strong edge selector 814, and a determiner 816. 8B shows the detailed configuration of the re-coating direction determiner 810. The gradient calculator 812 calculates a gradient value of pixels on the edge detected in the previous image frame by the following equation.

The strong edge selector 814 sequentially selects a reference pixel from pixels having a large gradient value of pixels on the edge, and selects a predetermined first area (eg, adjacent to the reference pixel) around the selected reference pixel. The strong edge is selected by removing other pixels on the edge existing in the area including the pixel). The determination unit 816 may include a gradient of respective pixels constituting the previous image frame calculated based on the sum of the weights and gradients of the strong edges existing in the second predetermined area (eg, a circle having a radius of 2 pixels). The oversize direction of each pixel constituting the previous image frame is determined by the size. 9A to 9E illustrate a previous image frame, a strong edge image using a canny edge image, a strong edge image using a directional edge image, a gradient interpolation image by a strong edge image using a canny edge image, and a strong edge using a directional edge image, respectively. Gradient interpolation image by image is shown.

The overlay point determiner 820 selects third pixels located in an overlay region corresponding to the first motion map and the second motion map from the pixels constituting the first rendered image corresponding to the previous image frame, and then selects the third pixel. Among the fourth pixels to be overwritten is selected. In this case, the overlay point determining unit 820 randomly sets the position of the brush stroke, sets the position using the moment of the image, divides the entire canvas into grids, applies one brush per grid, or uses segmentation. Set one divided region as one brush. In the present invention, in setting the position of the brush stroke to be painted, it is preferable to use a method of setting randomly or in grid units.

When randomly generating seed values and brush strokes, the seed values are set to the center positions of the brush strokes and should be spread evenly in accordance with the characteristics of the objects throughout the canvas. However, since randomly generated brush strokes are values that are generated without considering the characteristics of objects, there is a problem in that seed values must be generated very much in order to express the pictorial feeling in detail. 9A and 9B are diagrams illustrating images obtained by randomly generating 3507 seed values and arranging them on a canvas and brushing the brush corresponding to the 3507 seed values, respectively.

Alternatively, a method of dividing the canvas into grids and applying one brush stroke to each grid may be applied. This method divides the canvas into four or six layers, and each layer is divided into grids based on the width of the brush. The divided grid is covered with a brush by comparing color values between the canvas and the source image. Also, since the width of the brush is from the largest to the smallest, the grid size is also divided from the largest to the smallest. 10 shows the steps in which brush strokes are overlaid on the canvas. The image shown in FIG. 10 is an image in which 87 brush strokes are overlaid on a canvas by attempting to paint a total of 100 times. The reason why only 87 paints are overwritten in 100 paint attempts is to paint only when the area error value AreaError is larger than the threshold T set by the user.

Here, D _{i , j} is a function for calculating the color difference between the source image and the rendered image in which (i, j) is overlaid, M is a grid area centered on (x, y), and grid is a grid Is the area.

The stroke pattern selector 830 selects a stroke pattern corresponding to the color of the pixel corresponding to the fourth pixel to be overwritten among the pixels constituting the current image frame. For this purpose, a plurality of stroke patterns having different shapes and textures are defined in advance. First, the shape of the brush stroke pattern is divided into a straight brush stroke and a curved brush stroke. In the case of the straight brush stroke, the start position (x ₀ , y ₀ ) is determined, and the end point (x ₁ , y ₁ ) of the brush stroke is determined according to the value of the length and width of the brush specified by the user. 11A illustrates a method for generating a straight brush stroke. To set the end point of the brush stroke, select a point away from the vertical direction of the gradient of the starting position (x ₀ , y ₀ ). Curved brush strokes are defined by spline curves. The curved brush stroke starts at the first control point (x ₀ , y ₀ ) and is determined by D ₀ , the vertical direction of the gradient G ₀ . Thus, from the second control point (x ₁ , y ₁ ), select D _1, which has the smallest curvature of the brush stroke, from the two vertical directions (θ ₁ + π / 2, θ ₁ -π / 2). Curved brush strokes are created. 11B illustrates a method for generating curved brush strokes.

Next, the texture of the brush stroke pattern is divided into a solid brush stroke and a brush texture. Solid brush strokes are brushes that have one color per brush. Solid brushes have only one color, so when you paint on the canvas, the mix of colors can be adjusted simply by using opacity. In addition, too much ratio of opacity may have the effect of ink painting. Brush strokes that incorporate these features have the drawback of not being able to express the feeling of painting due to mixing between neighboring brushes when painting with multiple solid brush strokes having the same color. It is also difficult to distinguish between brush strokes that are overlaid. In contrast, a brush texture is an image that is scanned and stored after drawing with a real brush. Brush strokes with this feature can express the feeling of overwriting even when several brush textures are crossed, and the color of the brush can be expressed by the color of the underlying brush between the brush strokes. 12A and 12B show examples of solid brush strokes and brush textures, respectively.

The stroke pattern selector 830 randomly selects one brush from among brush stroke patterns having various shapes and textures which are predefined and stored, and gives a color. In this case, the stroke pattern selection unit 830 may use a method other than the method of applying a color corresponding to the color of the pixel corresponding to the fourth pixel to be repainted among the pixels constituting the current image frame to the selected brush stroke pattern. Color can be given to a brush stroke pattern by this. For example, the stroke pattern selector 830 corresponds to a color value applied to a brush stroke pattern selected from a plurality of brush stroke patterns having different shapes and textures, and a fourth pixel to be overlaid among pixels constituting the current image frame. The difference in the color value of the corresponding pixel corresponds to the color value of the pixel corresponding to the fourth pixel to be overwritten among the pixels constituting the first rendered image and the fourth pixel to be overwritten among the pixels constituting the current image frame. Choose a stroke pattern that is smaller than the difference in the color values of the pixels.

The rendering unit 840 generates a second rendered image by arranging a selected stroke pattern in a direction of the overwriting determined for each of the fourth pixels to be selected for the first rendered image. In this case, the rendering unit 840 places a stroke pattern selected for each of the fourth pixels determined in correspondence with the first motion map, and selects each of the fourth pixels determined in correspondence with the second motion map. The stroke pattern is disposed on the first rendered image to generate a second rendered image. In the rendering of the fourth pixels determined in correspondence with the first motion map, holes can occur when a brush texture is used to paint on the canvas, because there is a brush on one brush. You will need to reapply the brush texture several times. Therefore, when using the brush texture, the brush stroke is applied according to the number of pixels corresponding to the first motion map, and the actual brush texture is overlaid using the color difference value between the current image frame and the first rendered image.

The process of applying the brush texture to the first rendered image corresponding to the first motion map is largely composed of six steps. First, the first rendered image corresponding to the previous image frame is set as the initial image of the second rendered image corresponding to the current image frame. Second, one brush texture is randomly selected from a plurality of brush textures. Third, the pixels corresponding to the first motion map are databased, and one of the databased pixels is selected to set a position at which the brush texture is to be overlaid. Fourth, an arbitrary region of the position where the brush texture is to be painted is put in a memory buffer and a color difference value (Error 1) between the current image frame and the buffer region is calculated. Fifth, the brush texture is overlaid and the color difference value (Error 2) between the painted area and the current image frame is calculated. Finally, by comparing Error 1 and Error 2, if the Error 1 value is greater than the Error 2 value, set the position to be overwritten. If the Error 1 value is less than the Error 2 value, the contents of the memory buffer stored before the painting are rewritten. Copies to the initial image of the rendered image. 13A to 13D illustrate a previous image frame, a current image frame, a first motion map, and a result image obtained by painting a brush texture through the above-described process.

The process of applying a brush texture to the first rendered image in which the primary rendering is performed based on the first motion map corresponding to the second motion map is also largely performed in six steps. First, the resultant image of which the brush texture is overlaid using the first motion map is set as the initial image of the second rendered image corresponding to the current image frame. Second, one brush texture is randomly selected from a plurality of brush textures. Third, pixels corresponding to the second motion map are databased. At this time, the position to be randomly painted is set, and it is checked whether the set position exists in the database corresponding to the first motion map. If the location exists in the database, it attempts to overwrite the brush texture at that location, otherwise it selects the next location. At this time, the number of brush painting attempts used is set to about 1000 times. This is because, unlike the first motion map, the second motion map is a portion where color difference between neighboring pixels is small, and thus, the second motion map does not need to be painted for each image frame. 1000 is a user-specified number and is determined experimentally. Fourth, an arbitrary region of the position where the brush texture is to be painted is put in the memory buffer, and a color difference value (Error 1) between the current image frame and the buffer region is calculated. Fifth, the brush texture is overlaid and the color difference value (Error 2) between the painted area and the current image frame is calculated. Finally, compare Error 1 and Error 2, and if the Error 1 value is greater than the Error 2 value, set the next place to overwrite and if the Error 1 value is less than the Error 2 value, remove the contents of the memory buffer stored before painting. 2 Set as the initial image of the rendered image. 14A to 14D illustrate a previous image frame, a current image frame, a second motion map, and a resultant image obtained by painting a brush texture through the above-described process.

8C illustrates the configuration of the second embodiment of the rendered image generator 150. 8C illustrates an example of the rendered image generator 150 corresponding to the case where the brush stroke pattern is a curved solid brush stroke.

Referring to FIG. 8C, the rendering image generator 150 includes an over-direction determining unit 860, an over-adding determining unit 870, and a rendering unit 880. The second embodiment of the rendered image generating unit 150 shown in FIG. 8C is a configuration of the overlaid direction determining unit 860 as compared with the first embodiment of the rendered image generating unit 150 shown in FIG. 8A. Since the functions are the same, only the configurations and functions of the over-the-head determining unit 870 and the rendering unit 880 will be described below.

The overlay determining unit 870 sets a grid having a predetermined size on the first rendered image, the first motion map, and the second motion map, and applies the grid to the first rendered image by a decision rule defined by the following equation. It is determined whether the grids corresponding to the grids set on the first motion map and the second motion map are overlaid. In this case, the overwrite determination unit 870 increases the first threshold value TH ₁ when the A / C value is greater than the second threshold value TH ₂ .

Where N _all is the number of pixels present in any grating, N _{st _ motion} and N _{we _ motion} are the number of pixels present in the corresponding grating of the first motion map and the second motion map, respectively, and A is the first The number of pixels present in the motion map, C is the number of pixels obtained as a result of performing an exclusive OR between an image obtained by applying a stroke pattern selected from a plurality of stroke patterns to the first rendered image and the first motion map, and a Diff Map (i) is a color difference between the second image frame and the first rendered image, N is the number of pixels constituting a region of a predetermined size including a lattice, and TH ₁ to TH ₄ are thresholds set for each .

The first motion map includes an edge portion where the color change between frames is apparent. For this reason, the first rendered image is set as the initial image of the second rendered image, and the first motion map portion in which the color change is clearly displayed is overlaid on it. Depending on the type of brush stroke used (divided by the width of the brush stroke), the canvas to be rendered is divided into N layers, and the brush stroke is applied to the minimum area corresponding to the first motion map for each layer. The overwrite determination unit 870 determines whether or not to overwrite the brush stroke on the canvas to be overlaid using Equation (6). At this time, one layer is divided into several grids according to the width of the brush strokes, and one brush stroke is applied in one grid. The first threshold value TH ₁ is inversely proportional to the brush stroke frequency.

The rendering unit 880 generates a second rendered image by arranging a selected stroke pattern among a plurality of stroke patterns in the overwrite direction determined with respect to the lattice in which the repainting is determined among the gratings set for the first render image. In this case, the rendering unit 880 arranges a stroke pattern selected for each of the fourth pixels determined in correspondence with the first motion map, and then selects each of the fourth pixels determined in correspondence with the second motion map. The stroke pattern is disposed on the first rendered image to generate a second rendered image.

First, the rendering unit 880 determines whether to overwrite by Equation 6, the result of applying a brush stroke on the temporary canvas, the OverlapRate value (ie, C / The first threshold value TH ₁ is adjusted until the value of A) is smaller than the second threshold value TH ₂ . If the OverlapRate value is larger than the second threshold value, it means that the brush stroke is overwritten more than the first motion map area, and the OverlapRate value can be reduced by increasing the first threshold value TH ₁ of Equation 6. 15A to 15E illustrate a previous image frame, a current image frame, a first motion map, a first result image obtained by applying a solid brush stroke through the above-described process (first threshold value TH ₁ : 0.5), and the above-mentioned image. A first resultant image (first threshold value TH ₁ : 0.85) obtained by applying a solid brush stroke through the process is shown.

Next, the rendering unit 880 minimizes the change from the previous canvas by painting only the smallest brush stroke using the second motion map on the canvas on which the first motion map is painted. That is, the rendering unit 880 does not paint the brush stroke on the canvas if the color difference between the neighboring pixels is small despite the movement of any pixel or area portion. This is because when a solid brush stroke is overlaid on a canvas where a small color difference between neighboring pixels is applied to the canvas, the color difference between neighboring pixels may be increased to cause flickering of the brush stroke between frames. In the present invention, to reduce this phenomenon, the brush stroke is overlaid on the canvas only when two conditions set by Equation 6 are satisfied. As a result, the rendering unit 880 performs the repainting in consideration of the ratio of the second motion map area to the entire grid area. 16A to 16E illustrate a previous image frame, a current image frame, a second motion map, a first result image obtained by applying a solid brush through the above-described process (third threshold value TH ₃ : 0.82), and the above-described process. The first resultant image (third threshold TH ₃ : 0.85) obtained by overlaying the solid brush stroke is shown.

17 is a flowchart illustrating a process of performing a preferred embodiment of the pictorial animation method according to the present invention.

Referring to FIG. 17, the motion vector calculating unit 110 moves a position of pixels shifted in position on a current image frame following a previous image frame among pixels constituting a previous image frame that is temporally advanced among a plurality of image frames. A motion vector corresponding to the current video frame is calculated based on the step S1700. Next, the edge image generator 120 detects an edge in the previous image frame and generates an edge image (S1710). Next, the first motion map generator 130 generates a first motion map by connecting the first motion map to the point obtained by adding the motion vector to the coordinates of the first pixel from each of the first pixels constituting the edge image (S1720). . Next, the second motion map generator 140 moves from the second pixel whose position except the first pixel is moved among the pixels constituting the previous image frame to a point obtained by adding a motion vector to the coordinates of the second pixel. Connect to generate a second motion map (S1730). Finally, the rendered image generator 150 performs a repainting on the first rendered image corresponding to the previous image frame based on the first motion map and the second motion map to generate a second rendered image corresponding to the current image frame. (S1740).

FIG. 18 is a diagram illustrating a process of determining, by a rendering image generator 150, a painting direction, a painting point, and a stroke pattern when generating a second rendering image using a brush texture.

Referring to FIG. 18, the overlay direction determining unit 810 calculates gradient values of pixels on the edge detected in the previous image frame by Equation 4 (S1800). Next, the overlay direction determiner 810 sequentially selects the reference pixel from the pixels having the large gradient values of the pixels on the edge, and selects a predetermined first area (for example, the reference pixel) based on the selected reference pixel. A strong edge is selected by removing the other pixels on the edge existing in the region including the pixel adjacent to (S1810). Next, the overlay direction determining unit 810 configures each of the previous image frames calculated based on the sum of the weights and gradients of the strong edges existing in the second predetermined area (eg, a circle having a radius of 2 pixels). The overwriting direction of each pixel constituting the previous image frame is determined by the gradient size of the pixels (S1820).

The overlay point determiner 820 selects third pixels located in an overlay region corresponding to the first motion map and the second motion map from the pixels constituting the first rendered image corresponding to the previous image frame, and then selects the third pixel. Fourth pixels to be repainted are selected from among them (S1830). In this case, the overlay point determining unit 820 randomly sets the position of the brush stroke, sets the position using the moment of the image, divides the entire canvas into grids, applies one brush per grid, or uses segmentation. Set one divided region as one brush. Next, the stroke pattern selection unit 830 selects a stroke pattern corresponding to the color of the pixel corresponding to the fourth pixel to be overwritten among the pixels constituting the current image frame (S1840). In this case, the stroke pattern selection unit 830 randomly selects one brush from among brush stroke patterns having various shapes and textures that are predefined and stored, and then gives a color.

19A and 19B illustrate an example of a process in which the rendered image generator 150 generates a second rendered image by using a brush texture.

19A and 19B, the rendering unit 840 first sets a first rendering image corresponding to a previous image frame as an initial image of a second rendering image corresponding to a current image frame (S1900). Next, the rendering unit 840 databases the pixels corresponding to the first motion map, and selects one of the databased pixels to set a position at which the brush texture selected from the plurality of brush textures is randomly overlaid (S1905). ). Next, the rendering unit 840 inserts an arbitrary region of the position where the brush texture is to be painted into the memory buffer and calculates a color difference value (Error 1) between the current image frame and the buffer region (S1910). Next, the rendering unit 840 paints the brush texture and calculates a color difference value (Error 2) between the painted area and the current image frame (S1915). Subsequently, the rendering unit 840 compares Error 1 and Error 2 (S1920), sets the position to be next overlaid when the Error 1 value is larger than the Error 2 value (S1925), and if the Error 1 value is smaller than the Error 2 value, The contents of the memory buffer stored before are copied back to the initial image of the second rendered image (S1930).

After the rendering by the first motion map is completed by the above process, the rendering unit 840 uses the first motion map as an initial image of the second rendered image corresponding to the current image frame using the brush image. It is set (S1935). Next, the rendering unit 840 databases the pixels corresponding to the second motion map, sets a position to be randomly painted, and checks whether the set position exists in the database corresponding to the first motion map (S1940). . If the position exists in the database (S1945), the brush texture selected from the plurality of brush textures is randomly overlaid on the position (S1950), otherwise, the next position is selected (S1955). Next, the rendering unit 840 inserts an arbitrary region of the position where the brush texture is to be painted into the memory buffer, and calculates a color difference value (Error 1) between the current image frame and the buffer region (S1960). Next, the rendering unit 840 paints the brush texture and calculates a color difference value (Error 2) between the painted area and the current image frame (S1965). Finally, the rendering unit 840 compares Error 1 and Error 2 (S1970). If the Error 1 value is greater than the Error 2 value, the rendering unit 840 sets the next position to be painted (S1975). If the Error 1 value is less than the Error 2 value, the rendering unit 840 stores the memory buffer before painting. The content of the image is set again as the initial image of the second rendered image (S1980).

20 is a diagram illustrating an example of a process in which the rendered image generator 150 generates a second rendered image by using a solid brush stroke.

Referring to FIG. 20, the overlay determining unit 870 sets a grid having a predetermined size on the first rendered image, the first motion map, and the second motion map, and initializes the first threshold value of Equation 6 below. A value (for example, 0.5) is set (S2000). Next, when the ratio of the number of all pixels in the grid to be repainted and the number of pixels corresponding to the first motion map in the grid is greater than or equal to the first threshold value (S2005). The overwrite determining unit 870 applies the solid brush stroke selected by the method as described with reference to FIG. 18 to the grid on the temporary canvas (S2010). Next, the rendering unit 880 performs an overlapping overlap operation based on the image obtained by performing an exclusive OR operation between the resultant image obtained by applying the solid brush stroke selected on the temporary canvas and the first motion map image. That is, C / A of Equation 6 is calculated (S2015). Next, the rendering unit 880 compares the OverlapRate value with the second threshold value (S2020). If the OverlapRate value is greater than the second threshold value, the rendering unit 880 increases the first threshold value (S2025). If the OverlapRate value is less than or equal to the second threshold value, the rendering unit 880 is the first threshold value. After determining this, the selected solid brush stroke is applied to the grid area on the first rendered image (S2030).

When the rendering by the first motion map is completed by the above process, the rendering unit 880 uses the first motion map to display the initial image of the second rendering image corresponding to the current image frame, as a result of the solid brush stroke being overlaid. It is set to (S2035). Next, when the determination condition for the second motion map of Equation 6 is satisfied, the rendering unit 880 applies the solid brush stroke selected by the method as described with reference to FIG. 18 to the grid (S2040).

Meanwhile, the first rendered image corresponding to the previous image frame is generated by the rendered image generator 150 described with reference to FIGS. 8A through 8C. In this case, it is not necessary to generate an edge image and a motion map for the first rendered image, and determine a painting direction and a painting point for each pixel of a previous image frame, and then select a stroke pattern to generate a pictorial rendering image.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific preferred embodiments described above, and the present invention belongs to the present invention without departing from the gist of the present invention as claimed in the claims. Various modifications can be made by those skilled in the art, and such changes are within the scope of the claims.

1 is a block diagram showing the configuration of a preferred embodiment of the pictorial animation device according to the present invention;

2A to 2C are views illustrating a source image, a sobel edge image, and a canny edge image, respectively;

3 is a view illustrating a process of generating a directional edge image by the edge image generator 120;

4A to 4F illustrate an image obtained by extracting only edges in which motion occurred in an edge image, an edge image of a previous image frame, a current image frame, a previous image frame, an image obtained by adding a motion vector to an edge in which motion occurs in an edge image, and FIG. A drawing showing a first motion map,

5A through 5F illustrate an image consisting of pixels other than the edge where a movement occurs in a previous image frame, a current image frame, a previous image frame, an image consisting of pixels other than an edge in which a movement occurs in a previous image frame, and a second motion map, respectively. Figure,

6A to 6E illustrate a previous image frame, a current image frame, a canny edge image of a previous image frame, a first motion map, and a second motion map, respectively;

7A to 7E illustrate a previous image frame, a current image frame, a direction edge image of the previous image frame, a first motion map, and a second motion map, respectively;

8A is a diagram illustrating a configuration of a first embodiment of a rendered image generator 150;

8B is a view showing a detailed configuration of the re-coating direction determining unit 810,

8C is a diagram illustrating a configuration of a second embodiment of the rendered image generator 150;

9A and 9B are views illustrating an image generated by randomly generating 3507 seed values and placing them on a canvas, and a result image obtained by overlaying a brush corresponding to 3507 seed values.

10 shows steps in which brush strokes are overlaid on a canvas;

11A and 11B illustrate a method for generating straight and curved brush strokes, respectively.

12A and 12B illustrate examples of solid brush strokes and brush textures, respectively;

13A to 13D illustrate a result image obtained by painting a previous image frame, a current image frame, a first motion map, and a brush texture, respectively;

14A to 14D illustrate a result image obtained by overwriting a previous image frame, a current image frame, a second motion map, and a brush texture, respectively;

15A to 15E illustrate a first resultant image (first threshold value TH ₁ : 0.5) obtained by overlaying a previous image frame, a current image frame, a first motion map, a solid brush stroke, and a solid brush stroke, respectively. A diagram showing the obtained first resultant image (first threshold value TH ₁ : 0.85),

16A to 16E are respectively obtained by overlaying a previous image frame, a current image frame, a second motion map, a first result image obtained by painting a solid brush (third threshold value TH ₃ : 0.82), and a solid brush stroke, respectively. A diagram showing a first resultant image (third threshold TH ₃ : 0.85),

17 is a flowchart illustrating a process of performing a preferred embodiment of the pictorial animation method according to the present invention;

FIG. 18 is a diagram illustrating a process of determining a painting direction, a painting point, and a stroke pattern when the rendering image generating unit 150 generates a second rendering image by using a brush texture; FIG.

19A and 19B illustrate an example of a process in which the rendered image generator 150 generates a second rendered image by using a brush texture.

20 is a diagram illustrating an example of a process in which the rendered image generator 150 generates a second rendered image by using a solid brush stroke.

Claims (19)

The second image frame corresponds to the second image frame based on a position shift degree of pixels whose positions on the second image frame subsequent to the first image frame are moved among pixels constituting the first image frame that is temporally advanced among the plurality of image frames. A motion vector calculator for calculating a motion vector to be performed; An edge image generator for detecting an edge in the first image frame and generating an edge image; A first motion map generator configured to connect a first motion map from each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel to generate a first motion map; A second motion map by connecting from the second pixels whose positions except for the first pixel among the pixels constituting the first image frame to the point obtained by adding the motion vector to the coordinates of the second pixel; A second motion map generator for generating a signal; And Generates a rendered image by performing an overwriting on a first rendered image corresponding to the first image frame based on the first motion map and the second motion map to generate a second rendered image corresponding to the second image frame. Painter animation device comprising a. The method of claim 1, The rendering image generation unit, An overlaying direction determining unit which determines an overlaying direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image; Among the pixels constituting the first rendered image, third pixels positioned in an overwrite area corresponding to the first motion map and the second motion map are selected, and among the third pixels, fourth pixels to be repainted are selected. An overcoat point selector for selecting; A stroke pattern selector for selecting a stroke pattern corresponding to a color of a pixel corresponding to the fourth pixel among pixels constituting the second image frame; And And a rendering unit arranged to generate the second rendered image by arranging the selected stroke pattern in the overwrite direction determined for each of the fourth pixels selected for the first rendered image. The method of claim 1, The rendering image generation unit, An overlaying direction determining unit which determines an overlaying direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image; A grid having a predetermined size is set on the first rendered image, the first motion map, and the second motion map, and the first motion is selected from among the grids set for the first rendered image according to a following decision rule. An overwrite determining unit configured to determine whether or not to overlay gratings corresponding to the grating set on the map and the second motion map; And And a rendering unit generating the second rendered image by arranging a selected stroke pattern among a plurality of stroke patterns in the determined overlaid direction with respect to the lattice in which the overlaid is determined among the lattices set for the first rendered image. Painterly animation device: [Decision Rule]

Where N _all is the number of pixels present in any grating, N _{st _ motion} and N _{we _ motion} are the number of pixels present in the corresponding grating of the first motion map and the second motion map, respectively, and A is the first The number of pixels present in the motion map, C is the number of pixels obtained as a result of performing an exclusive OR between an image obtained by applying a stroke pattern selected from a plurality of stroke patterns to the first rendered image and the first motion map, and a Diff Map (i) is a color difference between the second image frame and the first rendered image, N is the number of pixels constituting a region of a predetermined size including a lattice, and TH ₁ to TH ₄ are thresholds set for each . The method of claim 3, wherein The paint-over animation determining apparatus, characterized in that for increasing the TH ₁ if the A / C value is greater than the TH ₂ . The method of claim 1, The rendering image generation unit, An overlaying direction determining unit which determines an overlaying direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image; Among the pixels constituting the first rendered image, third pixels positioned in an overwrite area corresponding to the first motion map and the second motion map are selected, and among the third pixels, fourth pixels to be repainted are selected. An overcoat point selector for selecting; The difference between the color value assigned to the selected stroke pattern among the plurality of stroke patterns having different shapes and textures and the color value of the pixel corresponding to the fourth pixel among the pixels constituting the second image frame is different from the first rendered image. Selecting a stroke pattern that selects a stroke pattern smaller than a difference between a color value of a pixel corresponding to the fourth pixel and a color value of a pixel corresponding to the fourth pixel among pixels constituting the second image frame; part; And And a rendering unit arranged to generate the second rendered image by placing the selected stroke pattern in the first rendering image in the over-direction determined for each of the fourth pixels. The method according to claim 3 or 5, The rendering unit arranges the selected stroke pattern for each of the fourth pixels determined in correspondence with the first motion map to the first rendering image, and then, for each of the fourth pixels determined in correspondence with the second motion map. The pictorial animation device, wherein the second rendered image is generated by arranging a selected stroke pattern on the first rendered image. The method according to any one of claims 1 to 5, And the edge image generator generates an edge image corresponding to the first image frame by a Sobel edge filter or a Canny edge filter. The method according to any one of claims 1 to 5, The edge image generation unit generates an edge image corresponding to the first image frame by quantizing the first image frame by a predetermined quantization coefficient, and then setting a boundary between contrast values of the quantized image as an edge. Painterly animation device. The method according to any one of claims 2 to 5, The coating direction determining unit, A gradient calculator configured to calculate a gradient value of pixels on an edge detected in the first image frame; Strong edge that selects a strong edge by sequentially selecting a reference pixel from the pixels having a large gradient value of the pixels on the edge, and removing other pixels on the edge existing in the first area around the selected reference pixel Selection unit; And The gradient of each pixel constituting the first image frame is calculated by the gradient size of each pixel constituting the first image frame calculated based on the sum of the weights and gradients of the strong edges existing in the second predetermined area. Determination unit for determining the coating direction; Painter's animation device comprising a. The second image frame corresponds to the second image frame based on a position shift degree of pixels whose positions on the second image frame subsequent to the first image frame are moved among pixels constituting the first image frame that is temporally advanced among the plurality of image frames. A motion vector calculating step of calculating a motion vector to be performed; An edge image generation step of generating an edge image by detecting an edge in the first image frame; A first motion map generation step of generating a first motion map by connecting each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel; A second motion map by connecting from the second pixels whose positions except for the first pixel among the pixels constituting the first image frame to the point obtained by adding the motion vector to the coordinates of the second pixel; Generating a second motion map; And Generates a rendered image by performing an overwriting on a first rendered image corresponding to the first image frame based on the first motion map and the second motion map to generate a second rendered image corresponding to the second image frame. Pictorial animation method comprising a. The method of claim 10, The rendering image generating step, A painting direction determining step of determining a painting direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image; Among the pixels constituting the first rendered image, third pixels positioned in an overwrite area corresponding to the first motion map and the second motion map are selected, and among the third pixels, fourth pixels to be repainted are selected. Selecting a pixel; A stroke pattern selection step of selecting a stroke pattern corresponding to a color of a pixel corresponding to the fourth pixel among pixels constituting the second image frame; And And a rendering step of generating the second rendered image by arranging the selected stroke pattern in the overwrite direction determined for each of the fourth pixels selected with respect to the first rendered image. The method of claim 10, The rendering image generating step, A painting direction determining step of determining a painting direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image; A grid having a predetermined size is set on the first rendered image, the first motion map, and the second motion map, and the first motion is selected from among the grids set for the first rendered image according to a following decision rule. An overwrite determining step of determining whether or not to overlay gratings corresponding to the grating set on the map and the second motion map; And And a rendering step of generating a second rendered image by arranging a stroke pattern selected from a plurality of stroke patterns in the determined overlaid direction with respect to the lattice overlaid among the lattices set for the first rendered image. Painterly animation methods: [Decision Rule]

Where N _all is the number of pixels present in any grating, N _{st _ motion} and N _{we _ motion} are the number of pixels present in the corresponding grating of the first motion map and the second motion map, respectively, and A is the first The number of pixels present in the motion map, C is the number of pixels obtained as a result of performing an exclusive OR between an image obtained by applying a stroke pattern selected from a plurality of stroke patterns to the first rendered image and the first motion map, and a Diff Map (i) is a color difference between the second image frame and the first rendered image, N is the number of pixels constituting a region of a predetermined size including a lattice, and TH ₁ to TH ₄ are thresholds set for each . The method of claim 12, In the reversal direction determining step, if the A / C value is greater than the TH ₂ , the TH ₁ to increase the pictorial animation method. The method of claim 10, The rendering image generating step, A painting direction determining step of determining a painting direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image; Among the pixels constituting the first rendered image, third pixels positioned in an overwrite area corresponding to the first motion map and the second motion map are selected, and among the third pixels, fourth pixels to be repainted are selected. Selecting a pixel; The difference between the color value applied to the selected stroke pattern among the plurality of stroke patterns having different shapes and textures and the color value of the pixel corresponding to the fourth pixel among the pixels constituting the second image frame may cause the first rendered image. A stroke pattern selection step of selecting a stroke pattern smaller than a difference between a color value of a pixel corresponding to the fourth pixel and a color value of a pixel corresponding to the fourth pixel among pixels constituting the second image frame. ; And And a rendering step of generating the second rendered image by arranging the selected stroke pattern on the first rendered image in the overwrite direction determined for each of the fourth pixels. The method according to claim 12 or 14, The rendering step, A stroke pattern arrangement step of disposing the selected stroke pattern on the first rendered image for each of the fourth pixels determined corresponding to the first motion map; And And an image generating step of generating the second rendered image by placing the selected stroke pattern on the first rendered image for each of the fourth pixels determined corresponding to the second motion map. Animation method. The method according to any one of claims 10 to 14, In the edge image generation step, an edge image corresponding to the first image frame is generated by a Sobel edge filter or a Canny edge filter. The method according to any one of claims 10 to 14, In the edge image generating step, the first image frame is quantized by a predetermined quantization coefficient, and then an edge image corresponding to the first image frame is generated by setting an edge of a contrast value of the quantized image as an edge. Painterly animation method. The method according to any one of claims 11 to 14, The coating direction determining step, Calculating a gradient value of pixels on the edge detected in the first image frame; Strong edge that selects a strong edge by sequentially selecting a reference pixel from the pixels having a large gradient value of the pixels on the edge, and removing other pixels on the edge existing in the first area around the selected reference pixel Selection step; And The gradient of each pixel constituting the first image frame is calculated by the gradient size of each pixel constituting the first image frame calculated based on the sum of the weights and gradients of the strong edges existing in the second predetermined area. Determination step of determining the painting direction; Painter's animation method comprising a. A computer-readable recording medium having recorded thereon a program for executing the pictorial animation method according to any one of claims 10 to 14.

Images