KR20210077204A - Apparatus for painterly animation - Google Patents

Abstract

Disclosed are an apparatus and method for providing pictorial representation to an animation. The apparatus of the present invention comprises: a movement vector calculation unit for calculating a movement vector corresponding to a second video frame among a plurality of video frames, based on location shifts of pixels whose location are shifted on a second video frame following a first video frame among pixels constituting the first video frame ahead of other video frames in terms of a time; an edge video generation unit for detecting an edge in the first video frame to generate an edge video; a first movement map generation unit for generating a first movement map by connecting from each of first pixels constituting the edge video to a point obtained by adding a movement vector to coordinates of the first pixels; a second movement map generation unit for generating a second movement map by connecting from each of the second pixels whose locations are shifted except for first pixels among pixels constituting the first video frame to a point obtained by adding a movement vector to coordinates of the second pixels; and a rendered video generation unit for performing overcoating on a first rendered video corresponding to a first video frame based on the first movement map and the second movement map to generate a second rendered video corresponding to the second video frame. In accordance with the present invention, brush strokes between video frames can keep temporal and structural consistency, a time required for rendering can be shortened, and an effect of overcoating of brush strokes between frames and a flickering phenomenon can be reduced.

Description

An apparatus for pictorial expression of animation {Apparatus for painterly animation}

The present invention relates to an apparatus for pictorial expression of animation, and more particularly, to an apparatus for expressing a continuous input image with a pictorial feeling as if drawn by hand.

Research on computer graphics, which began in the 1960s, aimed to create photorealistic images. As represented by the term photo-realism for the past 30 years, the ultimate interest in existing computer graphics has focused on how realistic images can be created. These traditional computer graphics created very realistic images by mimicking the physical or mechanical characteristics of objects, such as radiosity and ray-tracing.

Since the 1990s, interest in computer graphics has changed from photorealistic images to non-photorealistic images. Instead of focusing on creating only realistic images, I started researching how to create non-realistic images. As a method of creating such non-realistic images, the field of Non-Photorealistic Rendering has been established as a field of existing computer graphics. Non-photorealistic rendering is a concept opposite to conventional realistic rendering. While the existing realistic rendering tried to express the object more realistically, the non-realistic rendering aims to express the various characteristics of the image well and convey its meaning well rather than being bound by realistic expression in expressing the object. is taken as Therefore, the evaluation of non-realistic rendering is more influenced by human subjectivity than objective. Table 1 lists the differences between photorealistic and non-photorealistic rendering.

photorealistic rendering
(Photo-Realistic Rendering) unrealistic rendering
(Non-Photorealistic Rendering) Approach simulation styling Characteristic objective subjective effect physical process artistic process accuracy clarity Approximate Perfection complete result incomplete results

Non-realistic rendering is a rendering method that expresses an effect such as an image created by an artist on a specific object (2D image or 3D data). Non-photorealistic rendering techniques include Painterly Rendering, Mosaic Rendering, Cartoon Rendering, Technical Illustration Rendering, and Pen and Ink Rendering technology.

Pictorial rendering is mainly done through brush strokes. In the case of brush strokes, when an artist draws a work, he paints countless times with a brush. At this time, one movement is called a brush stroke. In pictorial rendering, methods for determining various parameters of brush strokes have been studied in order to create images that look like hand-drawn images, such as oil paintings. The parameters of a brush stroke include position, color, size, direction, shape, texture, and whether to overcoat. This pictorial rendering is a method that automatically expresses a pictorial feeling as if drawn by hand using brush strokes, and is highly dependent on the characteristics of brush strokes. The characteristics of brush strokes include color, position, direction, shape, size, and texture. Existing researchers tried to express various types of pictorial effects by adjusting the parameters of these characteristics.

Haeberli proposed a method of determining the color, shape, size, and direction of brush strokes through user input to an input image. In this method, the brush strokes to be used are stored in the database in advance and used to express abstract effects using the saved brushes, but user input is essential.

Litwinowicz created brush strokes with lines and textures. The brush stroke direction used in this method used the gradient-based thin-plate spline interpolation method of the input image, and various brush strokes were created by adding random noise values to the parameters of each brush stroke. Also, a brush stroke clipping method that considers the edges of the image was introduced. However, there is a disadvantage that the size of the brush is small and a monotonous effect is obtained by using only straight strokes.

Hertzmann proposed a method of automatically expressing the pictorial rendering effect on the input image. In this method, brush strokes are made along the spline curve and the edge with the image slope. One image is divided into several layers, and each layer is rendered by creating brush strokes of the same size. In addition, in order to improve the quality of the final result image, the brushes were applied to each layer in the order from the largest to the smallest. However, this method did not express the texture of brush strokes shown in actual pictorial works, and multiple brush strokes with different gradients are expressed even at the edges, showing a blurring effect in the resulting image.

On the other hand, pictorial animation is a method of expressing a pictorial feeling as if drawn by hand using continuous input images, and it is very important to maintain the structural and temporal consistency of brush strokes between frames. In order to maintain temporal coherence between frames, Colomosse created Stroke Surfaces through analysis of video sequences such as spatiotemporal volumes. In this method, a pictorial animation image is generated based on each region created using the region division algorithm, not brush strokes. In order to maintain temporal consistency of each region between frames, the user directly divides the region and separates the regions between frames. There is also the problem of establishing a relationship. Also, Meier used a method of moving the brushstrokes overlaid on the previous frame to the current frame using 3D geometric information and camera position information. Since this method uses 3D geometric information, it has the advantage of accurately finding the position of brush strokes between frames, but has the disadvantage of requiring modeling for all scenes to be animated.

Existing pictorial animation researchers have established the relationship of brushes between frames based on brush strokes. Setting the relationship of brush strokes between frames is very dependent on elements of brush strokes such as position, color, and direction. In particular, existing researchers use motion vector data calculated by the optical flow method to move all brush strokes existing on the canvas of the previous frame to the canvas of the current frame, and then move the brush strokes that occur between the moved brushes to the canvas of the current frame. The canvas is completed by filling in the hole. Because these methods focus on the position among the elements of the brush stroke, there is a problem that only the position is moved without changing the color and shape, not the flickering phenomenon between frames or the feeling that the brush strokes between the frames are overpainted. will do This problem is a very artificial phenomenon that cannot be expressed in the works made by actual animators.

For pictorial animation with video input, the most important factors to consider are the maintenance of structural and temporal consistency of brush strokes between frames and the pictorial feeling of hand-drawn parts in which the movement of objects (foreground and background) occurs. is to express

To maintain temporal consistency of brushstrokes from frame to frame, Meier modeled polygons as a set of 3D particles that would render 2D paint brushstrokes in screen space, just as artists paint over many brushstrokes on a canvas. This method maintains temporal consistency by converting 3D particles to 2D using the normal direction of 3D polygons. In addition, the geometric properties of the surface and the lighting effect were used to control the brush stroke's overpainting effect.

Litwinowicz added a new brush to the current frame and rearranged it using the optical flow method and the Delaunay triangulation to maintain temporal consistency of brush strokes. In this method, all brush strokes existing on the previous frame are calculated using the optical flow method as a motion vector.

move to the current frame by adding Set the center of the displaced brushes as one seed point and apply the Delaney triangle. By comparing the area of the triangle formed between the seed points with the maximum area specified by the user, the spacing between the brushes was adjusted by properly relocating the seed points.

Hertzmann used the paint-over method and optical flow method to maintain the temporal consistency of brush strokes between planes. The paint-over method is a method similar to the paint-on-glass method, and sets the result image of the previous frame as the initial canvas of the current frame. Although the paint-over method improves temporal consistency, flickering occurs when the static area of a video frame is different from the corresponding area of the previous frame overlaid with brushes. This phenomenon is caused by video noise, and Herzmann reduced the flickering phenomenon by using difference masking. Also, the flickering phenomenon is reduced by warping the brushes that are moved to the current frame by the motion vector.

Hays uses motion vectors estimated between frames to maintain temporal consistency between frames by moving all brushes from the previous frame canvas to the current frame canvas. It is mentioned that it cannot. This method adds and redefines new brushes on the canvas to fill these holes. In addition, the flickering phenomenon occurs due to the inconsistency of characteristics between newly added brushes and existing brushes. Haze solved this problem by reducing the opacity of the brush color by 10%. However, since this method relies too much on opacity, it gives the impression that the brushes between frames are simply moved, not painted over. This phenomenon cannot be seen in actual paint-on-glass animation, and it can be said that it is very artificial.

The technical problem to be achieved by the present invention is that it is possible to express a pictorial feeling as if hand-drawn while maintaining the structural and temporal consistency of brush strokes between frames using continuous input images, and the effect of overlapping brush strokes between frames and flicker It is to provide a pictorial expression device for animation that can reduce the ring phenomenon.

Another technical object of the present invention is to be able to express a hand-drawn pictorial feeling while maintaining the structural and temporal consistency of brush strokes between frames using continuous input images, An object of the present invention is to provide a computer-readable recording medium in which a program for executing a pictorial animation method capable of reducing the flickering phenomenon is recorded on a computer.

In order to achieve the above technical object, an apparatus for pictorial expression of animation according to the present invention provides a second image frame following the first image frame among pixels constituting a temporally preceding first image frame among a plurality of image frames. a motion vector calculator for calculating a motion vector corresponding to the second image frame based on the degree of positional movement of pixels whose positions have been moved on the image; an edge image generator configured to generate an edge image by detecting an edge in the first image frame; a first motion map generator configured to generate a first motion map by connecting each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel; A second motion map by connecting each second pixel whose position is moved except for the first pixel among pixels constituting the first image frame to a point obtained by adding the motion vector to the coordinates of the second pixel. a second motion map generator to generate and performing overpainting on the first rendered image corresponding to the first image frame based on the first motion map and the second motion map to generate a second rendered image corresponding to the second image frame. generating unit;

In order to achieve the above other technical object, the pictorial animation method according to the present invention provides a second image frame following the first image frame among pixels constituting a temporally preceding first image frame among a plurality of image frames. a motion vector calculating step of calculating a motion vector corresponding to the second image frame based on the degree of position movement of pixels whose positions have been moved; an edge image generating step of generating an edge image by detecting an edge in the first image frame; a first motion map generating step of generating a first motion map by connecting each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel; A second motion map by connecting each second pixel whose position is moved except for the first pixel among pixels constituting the first image frame to a point obtained by adding the motion vector to the coordinates of the second pixel. a second motion map generation step of generating and performing overpainting on the first rendered image corresponding to the first image frame based on the first motion map and the second motion map to generate a second rendered image corresponding to the second image frame. generation step;

According to the pictorial expression apparatus for animation according to the present invention, a motion map including movement information of edges identified by motion vectors obtained from image frames successive to pixels corresponding to edges of a previous image frame in which a rendered image is generated The temporal and structural consistency of brush strokes between image frames can be maintained by determining the brush stroke area corresponding to the current image frame using . In addition, the time required for rendering can be shortened by overpainting only a portion of the rendered image corresponding to the previous image frame based on the motion information between frames, and the effect of overlapping brush strokes between frames and the flickering phenomenon can be reduced. can be reduced

1 is a block diagram showing the configuration of a preferred embodiment of an apparatus for pictorial expression of animation according to the present invention;
2A to 2C are views showing a source image, a Sobel edge image, and a Canny edge image, respectively;
3 is a diagram illustrating a process of generating a directional edge image by the edge image generating unit 120;
4A to 4F are respectively a previous image frame, a current image frame, an edge image of a previous image frame, an image obtained by extracting only an edge in which motion occurs in an edge image, an image obtained by adding a motion vector to an edge in which motion occurs in the edge image, and A drawing showing the first motion map,
5A to 5F are a previous image frame, a current image frame, an image composed of pixels other than the edge where motion occurred in the previous image frame, an image composed of pixels other than the edge where the motion occurred in the previous image frame, and a second motion map, respectively. the drawing shown,
6A to 6E are views showing a previous image frame, a current image frame, a Canny edge image of a previous image frame, a first motion map, and a second motion map, respectively;
7A to 7E are views showing a previous image frame, a current image frame, a directional edge image of a previous image frame, a first motion map, and a second motion map, respectively;
8A is a diagram showing the configuration of the first embodiment for the rendering image generating unit 150;
8B is a view showing a detailed configuration of the overcoat direction determining unit 810;
8C is a diagram showing the configuration of the second embodiment for the rendering image generating unit 150;
9A and 9B are diagrams showing images obtained by randomly generating 3507 seed values, respectively, arranged on a canvas, and a resultant image obtained by overlaying brushes corresponding to 3507 seed values;
Fig. 10 shows the steps in which brush strokes are painted over a canvas;
11A and 11B are diagrams showing a method of generating straight and curved brush strokes, respectively;
12A and 12B are diagrams showing examples of solid brush strokes and brush textures, respectively;
13A to 13D are views showing result images obtained by overpainting a previous image frame, a current image frame, a first motion map, and a brush texture, respectively;
14A to 14D are diagrams illustrating a result image obtained by overpainting a previous image frame, a current image frame, a second motion map, and a brush texture, respectively;
15A to 15E are respectively a previous image frame, a current image frame, a first motion map, a first result image obtained by overlaying a solid brush stroke (first threshold value TH1: 05), and a solid brush stroke obtained by overlaying the solid brush stroke, respectively. A drawing showing the first result image (first threshold TH1: 085);
16A to 16E are respectively a previous image frame, a current image frame, a second motion map, a first result image obtained by overpainting a solid brush (third threshold TH3: 082), and a third image obtained by overpainting a solid brush stroke, respectively. 1 A drawing showing the result image (third threshold TH3: 085);
17 is a flowchart showing a process of performing a preferred embodiment of the pictorial animation method according to the present invention;
18 is a diagram illustrating a process in which the rendering image generating unit 150 determines an overcoat direction, an overpainting point, and a stroke pattern when generating a second rendered image using a brush texture;
19A and 19B are diagrams illustrating an example of a process in which the rendering image generating unit 150 generates a second rendered image by using a brush texture;
20 is a diagram illustrating an example of a process in which the rendering image generator 150 generates a second rendered image by using a solid brush stroke.

Hereinafter, a preferred embodiment of the apparatus for pictorial expression of animation according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a preferred embodiment of an apparatus for pictorial expression of animation according to the present invention.

Referring to FIG. 1 , the apparatus for pictorial expression of animation according to the present invention includes a motion vector calculation unit 110 , an edge image generation unit 120 , a first motion map generation unit 130 , and a second motion map generation unit ( 140 ) and a rendered image generating unit 150 .

The motion vector calculator 110 is configured to calculate the current image frame based on the degree of position movement of pixels whose positions on the current image frame following the previous image frame are moved among the pixels constituting the temporally preceding image frame among the plurality of image frames. A motion vector corresponding to is calculated. In this case, the previous image frame and the current image frame are image frames constituting a moving picture, and are selected to be adjacent to each other, or one or a plurality of other image frames are positioned between them.

Methods for evaluating motion between image frames include an optical flow method and a block matching method. The optical flow method is a method to grasp the velocity distribution of the apparent motion in the image that is generated by gradually changing the brightness type. The block matching method divides the image into blocks of a certain size and expresses all pixels within the block as a single motion vector. In order to find the motion vector, the blocks of the previous frame are moved pixel by pixel to find the block most similar to the current block , and the mean absolute error (MAE) and mean square error (MSE) methods are used as matching criteria. In the following description, an embodiment in which a block matching method capable of reducing an error of motion vector data is applied will be described, but other methods for evaluating motion between image frames including an optical flow method may also be used.

The edge image generator 120 generates an edge image by detecting an edge in a previous image frame. The edge image for the previous image frame may be obtained by a method using an edge filter, a method using a directional edge, and the like. When an edge filter is used, the edge image generator 120 generates an edge image by applying an edge mask to the source image. In this case, the edge image generator 120 first removes noise from the source image using a Gaussian mask, and then performs an edge detection mask such as a Sobel mask or a Canny mask to generate an edge image. Among these two processes, in the first process, noise is removed, and in the second process, an edge is detected in the image from which the noise is removed. In the Canny Edge image as well as the Sobel edge image, only the outer edges of objects are well expressed, but the inner edges are not well expressed. 2A to 2C show a source image, a Sobel edge image, and a Canny edge image, respectively. The Sobel edge image is obtained by setting the threshold value to 100, and the Canny edge image is obtained by setting the Low Threshold to 1 and the High Threshold to 128.

The direction edge refers to a line formed along the texture of light and dark in an image. The contrast texture is different from the edges appearing using a typical edge filter such as a Sobel edge filter or a Canny edge filter. This is because the contrast texture means the boundary between two contrast values, whereas a general edge means a pixel whose difference in brightness or color between neighboring pixels is greater than a user-specified threshold. In other words, it can be said that the general edge image is included in the resolution of the contrast, which means the directional edge. In addition, since the boundary between two contrast values is expressed, not only the outer edges of objects but also the inner edges can be expressed well. This directional edge becomes the reference edge for determining the direction of the brush stroke to be overpainted on the canvas. In order to generate the directional edge image, the edge image generator 120 quantizes the continuous intensity values of the source image by designating them from among N (eg, 4, 8, etc.) discrete intensity values designated by the user. Next, the edge image generator 120 generates a directional edge image by setting the boundary surface of the contrast value as an edge using the quantized image. 3 illustrates a process of generating a directional edge image by the edge image generator 120 .

A method using a directional edge in generating an edge image has several advantages compared to a method using a general edge mask. First, it is very difficult for a user to set an appropriate threshold for every frame when using Canny Edge in a video. Also, when the same threshold is applied to all frames, the edge expressed in the previous frame may not appear in the current frame. Also, since the directional edge uses the boundary between the contrast values of each frame, the edge between frames is maintained even if the contrast value is changed. In addition, an edge can be expressed even in an area where a gradation phenomenon occurs because the boundary between two contrast values is used.

It is important to maintain temporal consistency of brush strokes between frames in order to transform a video into a pictorial rendering. In the present invention, a strong motion map (hereinafter referred to as a 'first motion map') and a weak motion map (hereinafter referred to as a 'second motion map') using the edge image of the previous image frame and motion vector data calculated between the previous frame and the current frame Two types of motion maps are generated: a 'motion map').

The first motion map refers to pixels in which pixels corresponding to edges of a previous frame are moved to a current frame by a motion vector. The pixel of the edge of the current frame may be calculated by adding the motion vector to the pixel corresponding to the edge of the previous frame using the calculated motion vector. The first motion map refers to an area from pixel to pixel. In other words, the first motion map refers to a region in which the foreground between frames is changed to the background or the background is changed to the foreground. In addition, the second motion map refers to an area other than the edge although motion between frames has occurred. Unlike the first motion map, since this map does not include pixels corresponding to edges in each frame, a color difference between neighboring pixels is also small.

In other words, the second motion map refers to a region in which the foreground is changed to the foreground or the background is changed to the background between frames. In the present invention, by accurately grasping the location of the brush stroke to be painted in the next frame using these two motion maps, the temporal consistency of the brush strokes between frames can be maintained and a hand-drawn pictorial feeling can be expressed.

The first motion map generator 130 generates a first motion map Motion_Mapstr by connecting each first pixel constituting the edge image of the previous image frame to a point obtained by adding a motion vector to the coordinates of the first pixel. . This process can be expressed in mathematical terms as follows. In this case, it is preferable to use only vectors with a value of (10, 10) or more of the motion vector.

Equation 1

Equation 2

In Equations 1 and 2, A(x,y) is a pixel in which motion occurs among pixels constituting the edge image of the previous image frame, and MV(x,y)(Δu,Δv) is the previous image frame and the current image frame It is an inter-motion vector, and DL(A(x,y), B(x,y)) is a function that draws a line between two points A(x,y) and B(x,y).

4A to 4F, the previous image frame, the current image frame, the edge image of the previous image frame, the image obtained by extracting only the edge where motion occurs in the edge image, the image obtained by adding a motion vector to the edge where the motion occurs in the edge image, and A first motion map is shown.

The second motion map generator 140 adds a motion vector to the coordinates of the second pixel from each of the second pixels whose positions are moved except for the first pixel constituting the edge image among pixels constituting the first image frame. The second motion map is created by connecting the points obtained by This process can be expressed in mathematical terms as follows. In this case, since the second motion map is portions having a small color difference between neighboring pixels, it is preferable to use only vectors having a value of (20, 20) or more of the motion vector, unlike the first motion map.

Equation 3

In Equation 3, C(x,y) is a pixel in which motion except for an edge has occurred among pixels constituting the previous image frame, and MV(x,y)(Δu,Δv) is a motion between the previous image frame and the current image frame. As a vector, DL(C(x,y), D(x,y)) is a function that draws a line between two points C(x,y) and D(x,y).

5A to 5F, the previous image frame, the current image frame, the image composed of pixels other than the edge where the motion occurred in the previous image frame, the image composed of pixels other than the edge where the motion occurred in the previous image frame, and the second motion map are respectively shown. is shown.

The first and second motion maps as described above are slightly different depending on the type of edge image used. First of all, since the Canny edge image includes only the outer edges of objects, the motion map created based on this does not perform over-painting on the inside of objects when brush strokes are overlaid, so there is a possibility of flickering between frames. 6A to 6E show a previous image frame, a current image frame, a Canny edge image of the previous image frame, a first motion map, and a second motion map, respectively. Next, the directional edge image includes both the outer and inner edges of the objects.

Therefore, a more accurate motion map can be created, and when brush strokes are overlaid, they can be painted more accurately than a motion map created using canny edge images. 7A to 7E, a previous image frame, a current image frame, a directional edge image of the previous image frame, a first motion map, and a second motion map are respectively shown.

The rendering image generator 150 generates a second rendered image corresponding to the current image frame by performing overpainting on the first rendered image corresponding to the previous image frame based on the first motion map and the second motion map. The rendering image generator 150 may have various configurations according to a type of brush stroke, a color determination method, a rendering method, and the like.

FIG. 8A shows the configuration of the rendering image generator 150 according to the first embodiment. The embodiment illustrated in FIG. 8A is an example of the rendering image generating unit 150 corresponding to the case in which the brush stroke pattern is a curved brush texture.

Referring to FIG. 8A , the rendering image generator 150 includes an overfill direction determiner 810 , an overfill point determiner 820 , a stroke pattern selector 830 , and a renderer 840 .

The overfill direction determiner 810 determines the overfill direction for each of the pixels constituting the previous image frame based on the gradient values of each of the pixels constituting the edge image. To this end, the overcoat direction determining unit 810 includes a gradient calculating unit 812 , a strong edge selecting unit 814 , and a determining unit 816 . 8B shows a detailed configuration of the overcoat direction determining unit 810 . The gradient calculator 812 calculates gradient values of pixels on edges detected in the previous image frame by the following equation.

Equation 4

The strong edge selector 814 sequentially selects a reference pixel from pixels having large gradient values of pixels on the edge, and a predetermined first area (eg, adjacent to the reference pixel) with respect to the selected reference pixel. A strong edge is selected by removing other pixels on the edge that exist within the region including the pixel. The determiner 816 determines the gradient of each pixel constituting the previous image frame calculated based on the sum of the gradient and the weight of the strong edges existing in the predetermined second region (eg, a circle having a radius of 2 pixels). The overlapping direction of each pixel constituting the previous image frame is determined by the size. 9A to 9E, a gradient interpolation image using a previous image frame, a strong edge image using a Canny edge image, a strong edge image using a directional edge image, a strong edge image using a Canny edge image, and a strong edge using a directional edge image, respectively A gradient interpolation image by image is shown.

The overpainting point determiner 820 selects third pixels located in the overpainting area corresponding to the first motion map and the second motion map from among pixels constituting the first rendered image corresponding to the previous image frame, and the third pixel Among them, fourth pixels to be overpainted are selected. In this case, the overpainting point determining unit 820 randomly sets the position of the brush stroke, sets the position using the moment of the image, divides the entire canvas into grids and applies one brush per grid, or uses region division to Set one divided area as one brush. In the present invention, in setting the position of the brush stroke to be overpainted, it is preferable to use a random or grid unit setting method.

In case of overpainting brush strokes by generating a seed value at random, the seed value is set as the center position of the brush stroke and should be evenly spread across the canvas according to the characteristics of the objects. However, since the randomly generated brush stroke position is a value generated without considering the characteristics of the objects, there is a problem in that it is necessary to generate a very large number of seed values to express a detailed pictorial feeling. 9A and 9B are diagrams illustrating images obtained by randomly generating 3507 seed values and arranged on a canvas, and a resultant image obtained by overpainting with brushes corresponding to 3507 seed values.

Alternatively, a method of dividing the canvas into grids and repainting each grid with one brush stroke may be applied. In this method, the canvas is divided into 4 to 6 layers, and each layer is divided into a grid according to the width of the brush. The grid divided in this way is painted with a single brush through color value comparison between the canvas and the source image. Also, since the width of the brush is in the order from largest to smallest, the grid size is also classified from the largest to the smallest. 10 shows the steps in which brush strokes are painted over the canvas. The image shown in FIG. 10 is an image in which 87 brush strokes are overpainted on the canvas by over-painting a total of 100 times. The reason why only 87 overlapping attempts were made in 100 overlapping attempts is that in the following equation, the area error value AreaError is greater than the threshold T set by the user.

Equation 5

Here, Di,j is a function that calculates the color difference between the source image and the rendered image on which the overlay is performed at (i,j), M is the grid area centered on (x,y), and grid is the is the area

The stroke pattern selector 830 selects a stroke pattern corresponding to a color of a pixel corresponding to a fourth pixel to be overpainted from among pixels constituting the current image frame. For this purpose, a plurality of stroke patterns having different shapes and textures are defined in advance. First, the shape of the brush stroke pattern is divided into a straight brush stroke and a curved brush stroke. For a straight brush stroke, when the start position (x0, y0) is determined, the end point (x1, y1) of the brush stroke is determined according to the user-specified brush length and width values. 11A shows a method of generating a straight brush stroke. To determine the end point of the brush stroke, select a point separated by a length in the vertical direction of the gradient from the start position (x0, y0). A curved brush stroke is defined by a spline curve. A curved brush stroke starts at the first control point (x0, y0) and is determined by the gradient G0, which is the vertical direction D0. Thus, from the second control point (x1, y1), select D1 with a small brush stroke curvature among the two vertical directions (θ1+π/2, θ1-π/2), and repeat this process to create a curved brushstroke do. 11B shows a method of generating a curved brush stroke.

Next, the texture of the brush stroke pattern is divided into a solid brush stroke and a brush texture. A solid brush stroke is a brush that has one color per brush. Since a solid brush has only one color, when painting over the canvas, color mixing can be controlled simply by using Opacity. Also, if the opacity ratio is too high, an effect like ink painting may appear. Brush strokes with these features have a disadvantage in that when multiple solid brush strokes having the same color are intersected and painted over, the feeling of overpainting cannot be expressed due to mixing between surrounding brushes. Also, it is difficult to distinguish between brush strokes that are overpainted. Contrary to this, the brush texture refers to a brush in which the texture of the brush appears as an image that is scanned and saved after drawing with an actual brush. Brush strokes with these characteristics can express the feeling of over-painting even when multiple brush textures are intersected, and color mixing can also be expressed as the color of the underlying brush is buried between the brush textures. 12A and 12B show examples of solid brush strokes and brush textures, respectively.

The stroke pattern selector 830 randomly selects one brush from among the previously defined and stored brush stroke patterns having various shapes and textures, and then assigns a color to the brush. In this case, as described above, the stroke pattern selector 830 applies a color corresponding to the color of the pixel corresponding to the fourth pixel to be overpainted among the pixels constituting the current image frame to the selected brush stroke pattern, other than the method described above. color can be given to the brush stroke pattern. For example, the stroke pattern selector 830 corresponds to a color value given to a brush stroke pattern selected from a plurality of brush stroke patterns having different shapes and textures and a fourth pixel to be overpainted among pixels constituting the current image frame. The difference between the color values of pixels to be overpainted corresponds to a color value of a pixel corresponding to a fourth pixel to be overpainted among pixels constituting the first rendered image and a fourth pixel to be overpainted among pixels constituting the current image frame. Select a stroke pattern that is smaller than the difference in color values of pixels.

The rendering unit 840 generates the second rendered image by arranging the selected stroke pattern in the determined overpainting direction for each of the fourth pixels to be overpainted on the first rendered image. At this time, the rendering unit 840 arranges the stroke pattern selected for each of the fourth pixels determined in response to the first motion map in the first rendered image, and then selects the stroke pattern selected for each of the fourth pixels determined in response to the second motion map. A second rendered image is generated by arranging the stroke pattern on the first rendered image. In the rendering of the fourth pixels determined in response to the first motion map, when painting over the canvas using a brush texture, a hole may occur because a brush texture exists in one brush, and the same color is required to fill the hole. You need to paint over the brush texture with . Therefore, when using a brush texture, overpainting of brush strokes is attempted according to the number of pixels corresponding to the first motion map, and the actual brush texture is overpainted using the color difference value between the current image frame and the first rendered image.

The process of overpainting the brush texture on the first rendered image in response to the first motion map is largely comprised of six steps. First, a first rendered image corresponding to a previous image frame is set as an initial image of a second rendered image corresponding to a current image frame. Second, one brush texture is randomly selected from among a plurality of brush textures. Third, the pixels corresponding to the first motion map are databased, and a position where the brush texture is to be applied is set by selecting one of the databased pixels. Fourth, an arbitrary area where the brush texture is to be painted is put into the memory buffer, and a color difference value (Error 1) between the current image frame and the buffer area is calculated. Fifth, paint over the brush texture and calculate the color difference value (Error 2) between the painted area and the current image frame. Finally, by comparing Error 1 and Error 2, if the Error 1 value is greater than the Error 2 value, the next overcoating position is set. If the Error 1 value is less than the Error 2 value, the contents of the memory buffer saved before painting are re-written to the second second. Copies to the initial image of the rendered image. 13A to 13D show a previous image frame, a current image frame, a first motion map, and a result image obtained by overpainting a brush texture through the above-described process.

The process of overpainting the brush texture on the first rendered image on which the primary rendering is performed based on the first motion map in response to the second motion map is also largely comprised of six steps. First, the result image overlaid with the brush texture using the first motion map is set as the initial image of the second rendered image corresponding to the current image frame. Second, one brush texture is randomly selected from among a plurality of brush textures. Third, the pixels corresponding to the second motion map are databased. At this time, a location to be randomly applied is set, and it is checked whether the set location exists in the database corresponding to the first motion map. If the location exists in the database, it tries to paint over the brush texture at that location, otherwise selects the next location. In this case, the number of attempts to repaint the brush texture used is set to about 1000 times. This is because, unlike the first motion map, the second motion map is a part in which a color difference between neighboring pixels is small, and therefore, it is not necessary to repaint for each image frame. 1000 is a number randomly assigned by the user and is determined experimentally. Fourth, an arbitrary area where the brush texture is to be painted is put into the memory buffer, and a color difference value (Error 1) between the current image frame and the buffer area is calculated. Fifth, paint over the brush texture and calculate the color difference value (Error 2) between the painted area and the current image frame. Finally, by comparing Error 1 and Error 2, if the Error 1 value is greater than the Error 2 value, the next overcoating position is set. If the Error 1 value is less than the Error 2 value, the contents of the memory buffer saved before painting are re-created. 2Set as the initial image of the rendering image. 14A to 14D show a previous image frame, a current image frame, a second motion map, and a result image obtained by overpainting the brush texture through the above-described process.

8C shows the configuration of the second embodiment of the rendering image generator 150 . The embodiment illustrated in FIG. 8C is an example of the rendering image generating unit 150 corresponding to the case in which the brush stroke pattern is a curved solid brush stroke.

Referring to FIG. 8C , the rendering image generating unit 150 includes an overcoat direction determining unit 860 , an overpainting determining unit 870 , and a rendering unit 880 . The second embodiment of the rendering image generating unit 150 shown in FIG. 8C has a configuration of the overlaying direction determining unit 860 compared with the first embodiment of the rendering image generating unit 150 shown in FIG. 8A and Since the functions are the same, only the configuration and functions of the overpainting determination unit 870 and the rendering unit 880 will be described below.

The overpainting determination unit 870 sets a grid having a predetermined size on the first rendered image, the first motion map, and the second motion map, and is applied to the first rendered image by a decision rule defined by the following equation. It is determined whether to overpaint the grids corresponding to the grids set on the first motion map and the second motion map from among the grids set for each other. At this time, the overcoat decision unit 870 increases the first threshold value TH1 when the A/C value is greater than the second threshold value TH2.

Equation 6

Here, Nall is the number of pixels present in an arbitrary grid, Nst_motion and Nwe_motion are the number of pixels present in the corresponding grids of the first and second motion maps, respectively, and A is the number of pixels present in the first motion map. Number, C is the number of pixels obtained as a result of performing an exclusive OR between an image obtained by applying a stroke pattern selected from among a plurality of stroke patterns to the first rendered image and the first motion map, and Diff Map(i) is the second image A color difference between the frame and the first rendered image, N is the number of pixels constituting an area of a predetermined size including a grid, and TH1 to TH4 are threshold values set for each.

The first motion map includes an edge portion in which a color change between frames is clearly displayed. For this reason, the first rendered image is set as the initial image of the second rendered image, and the first motion map portion on which the color change is clearly displayed is overlaid. The canvas on which the rendering is performed is divided into N layers according to the type of brush stroke used (divided based on the width of the brush stroke), and the brush stroke is applied to the minimum area corresponding to the first motion map for each layer. The overpainting decision unit 870 determines whether to overpaint the brush stroke on the canvas to be overpainted by using Equation (6). At this time, one layer is divided into several grids according to the width of the brush stroke, and one brush stroke is applied within one grid. Also, the first threshold value TH1 and the frequency of brush strokes are inversely proportional to each other.

The rendering unit 880 generates a second rendered image by arranging a stroke pattern selected from a plurality of stroke patterns in an overpainting direction determined with respect to a grid for which an overcoat is determined from among the grids set for the first rendered image. In this case, the rendering unit 880 arranges the stroke pattern selected for each of the fourth pixels determined in response to the first motion map in the first rendered image, and then selects the stroke pattern selected for each of the fourth pixels determined in response to the second motion map. A second rendered image is generated by arranging the stroke pattern on the first rendered image.

First, the rendering unit 880 determines whether to overpaint according to Equation 6, and an OverlapRate value (ie, C/ The first threshold value TH1 is adjusted until the value of A) is less than the second threshold value TH2. When the OverlapRate value is greater than the second threshold value, it means that more brush strokes are overpainted than in the first motion map area, and the OverlapRate value can be decreased by increasing the first threshold value TH1 of Equation (6). 15A to 15E show the previous image frame, the current image frame, the first motion map, and the first result image (first threshold value TH1: 05) obtained by overpainting the solid brush stroke through the above-described process, and the process described above. A first result image (first threshold value TH1: 085) obtained by overpainting a solid brush stroke through .

Next, the rendering unit 880 paints only the smallest brush strokes using the second motion map on the canvas on which the first motion map has been overpainted, thereby minimizing the change from the previous canvas. That is, the rendering unit 880 does not overpaint the brush strokes on the canvas if the color difference between neighboring pixels is small even though a movement occurs in an arbitrary pixel or region. This is because, when a solid brush stroke is used to paint over the canvas with a small color difference between neighboring pixels, the color difference between neighboring pixels is rather aggravated, leading to flickering of the brush strokes between frames. In the present invention, in order to reduce this phenomenon, brush strokes are overpainted on the canvas only when two conditions set by Equation 6 are satisfied. As a result, the rendering unit 880 performs overpainting in consideration of the ratio of the second motion map area to the entire area of one grid. 16A to 16E show the previous image frame, the current image frame, the second motion map, the first result image obtained by overpainting the solid brush through the above-described process (third threshold value TH3: 082), and the above-described process. A first result image (third threshold value TH3: 085) obtained by overpainting a solid brush stroke through a brush stroke is shown.

17 is a flowchart illustrating a process of performing a preferred embodiment of the pictorial animation method according to the present invention.

Referring to FIG. 17 , the motion vector calculator 110 calculates the degree of position movement of pixels whose positions on the current image frame following the previous image frame are moved among pixels constituting the temporally preceding previous image frame among the plurality of image frames. A motion vector corresponding to the current image frame is calculated based on ( S1700 ). Next, the edge image generator 120 generates an edge image by detecting an edge in the previous image frame (S1710). Next, the first motion map generating unit 130 generates a first motion map by connecting from each first pixel constituting the edge image to a point obtained by adding a motion vector to the coordinates of the first pixel (S1720) . Next, the second motion map generation unit 140 extends from each second pixel whose position is moved except for the first pixel among pixels constituting the previous image frame to a point obtained by adding a motion vector to the coordinates of the second pixel. to generate a second motion map (S1730). Finally, the rendered image generating unit 150 performs overpainting on the first rendered image corresponding to the previous image frame based on the first and second motion map. to generate a second rendered image corresponding to the current image frame (S1740).

18 is a diagram illustrating a process in which the rendering image generating unit 150 determines an overcoating direction, an overpainting point, and a stroke pattern when generating a second rendered image by using a brush texture.

Referring to FIG. 18 , the overfill direction determiner 810 calculates gradient values of pixels on the edge detected in the previous image frame by Equation 4 ( S1800 ). Next, the overfill direction determining unit 810 sequentially selects a reference pixel from pixels having large gradient values of pixels on the edge, and a predetermined first area (eg, a first area centered on the selected reference pixel).

For example, a strong edge is selected by removing other pixels on the edge existing in the area including the pixel adjacent to the reference pixel (S1810). Next, the overcoating direction determining unit 810 may configure each of the previous image frames calculated based on the sum of the weights and gradients of strong edges existing in a predetermined second area (eg, a circle having a radius of 2 pixels). The overlapping direction of each pixel constituting the previous image frame is determined by the gradient size of the pixels in S1820.

The overpainting point determiner 820 selects third pixels located in the overpainting area corresponding to the first motion map and the second motion map from among pixels constituting the first rendered image corresponding to the previous image frame, and the third pixel Among them, fourth pixels to be overpainted are selected ( S1830 ). In this case, the overpainting point determining unit 820 randomly sets the position of the brush stroke, sets the position using the moment of the image, divides the entire canvas into grids and applies one brush per grid, or uses region division to Set one divided area as one brush. Next, the stroke pattern selector 830 selects a stroke pattern corresponding to the color of the pixel corresponding to the fourth pixel to be overpainted from among the pixels constituting the current image frame (S1840). In this case, the stroke pattern selector 830 randomly selects one brush from among the previously defined and stored brush stroke patterns having various shapes and textures, and then assigns a color to the brush.

19A and 19B are diagrams illustrating an example of a process in which the rendering image generator 150 generates a second rendered image by using a brush texture.

19A and 19B , first, the rendering unit 840 sets the first rendered image corresponding to the previous image frame as the initial image of the second rendered image corresponding to the current image frame ( S1900 ). Next, the rendering unit 840 databaseizes the pixels corresponding to the first motion map, selects one of the databased pixels, and sets a location where the brush texture selected from among the plurality of brush textures is to be painted over at random (S1905). ). Next, the rendering unit 840 places an arbitrary area at a position where the brush texture is to be painted into the memory buffer and calculates a color difference value (Error 1) between the current image frame and the buffer area ( S1910 ). Next, the rendering unit 840 overpaints the brush texture and calculates a color difference value (Error 2) between the painted area and the current image frame (S1915). Subsequently, the rendering unit 840 compares Error 1 and Error 2 (S1920). ), if the Error 1 value is greater than the Error 2 value, the next overcoating position is set (S1925), and if the Error 1 value is less than the Error 2 value, the contents of the memory buffer saved before painting are rewritten as the initial image of the second rendering image. is copied to (S1930).

When the rendering by the first motion map is completed through the above process, the rendering unit 840 converts the resulting image overlaid with the brush texture using the first motion map as an initial image of the second rendering image corresponding to the current image frame. set (S1935).

Next, the rendering unit 840 databaseizes the pixels corresponding to the second motion map, sets a location to be overpainted at random, and checks whether the set location exists in the database corresponding to the first motion map (S1940) If the location exists in the database (S1945), a brush texture selected from among a plurality of brush textures is randomly applied to the location (S1950), otherwise the next location is selected (S1955). Next, the rendering unit 840 puts an arbitrary area where the brush texture is to be painted into the memory buffer, and calculates a color difference value (Error 1) between the current image frame and the buffer area (S1960). Next, the rendering unit 840 ) overpaints the brush texture and calculates a color difference value (Error 2) between the painted area and the current image frame (S1965). Finally, the rendering unit 840 compares Error 1 and Error 2 (S1970). If the Error 1 value is greater than the Error 2 value, the rendering unit 840 sets a position to be next painted (S1975),

If the Error 1 value is less than the Error 2 value, the rendering unit 840 sets the contents of the memory buffer stored before painting as the initial image of the second rendered image again (S1980).

20 is a diagram illustrating an example of a process in which the rendering image generator 150 generates a second rendered image by using a solid brush stroke.

Referring to FIG. 20 , the overcoat determination unit 870 sets a grid having a predetermined size on the first rendered image, the first motion map, and the second motion map, and sets the initial value of the first threshold of Equation (6). A value (eg, 05) is set (S2000). Next, when the ratio of the number of pixels in the grid to be overpainted to the number of pixels corresponding to the first motion map in the grid is greater than or equal to the first threshold value (S2005), the overpainting decision unit 870 determines whether the grid on the temporary canvas The solid brushstroke selected by the method as described with reference to FIG. 18 is applied to the image (S2010). Next, the rendering unit 880 applies the selected solid brushstroke to the temporary canvas and the resultant image and the first motion map image. An OverlapRate value (ie, C/A in Equation 6) indicating the degree of overlapping is calculated based on the image obtained by performing an exclusive OR operation between the two (S2015). Next, the rendering unit 880 compares the OverlapRate value and the second threshold value (S2020). If the OverlapRate value is greater than the second threshold value, the rendering unit 880 increases the first threshold value (S2025), and if the OverlapRate value is less than or equal to the second threshold value, the rendering unit 880 increases the first threshold value After determining , the selected solid brush stroke is applied to the corresponding grid area on the first rendered image (S2030).

When the rendering by the first motion map is completed through the above process, the rendering unit 880 converts the result image overlaid with solid brush strokes using the first motion map as an initial image of the second rendering image corresponding to the current image frame. set to (S2035). Next, when the determination condition for the second motion map of Equation 6 is satisfied, the rendering unit 880 applies the solid brush stroke selected by the method described with reference to FIG. 18 to the corresponding grid (S2040).

Meanwhile, the first rendered image corresponding to the previous image frame is generated by the rendered image generator 150 described with reference to FIGS. 8A to 8C . In this case, it is not necessary to generate an edge image and a motion map for the first rendered image, and a pictorial rendering image is generated by determining an overlay direction and an overlay point for each pixel of the previous image frame, and then selecting a stroke pattern.

The present invention can also be implemented as computer-readable codes on a computer-readable recording medium.

The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and may also be implemented in the form of a carrier wave (eg, transmission through the Internet). include In addition, the computer-readable recording medium is distributed in a network-connected computer system so that the computer-readable code can be stored and executed in a distributed manner.

Although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific preferred embodiments described above, and in the technical field to which the present invention belongs, without departing from the gist of the present invention as claimed in the claims Anyone with ordinary skill in the art can implement various modifications, of course, such changes are within the scope of the claims.

Claims (19)

Among the plurality of image frames, among the pixels constituting the temporally preceding first image frame, corresponding to the second image frame based on the degree of movement of the pixels whose positions on the second image frame following the first image frame have been moved. a motion vector calculator that calculates a motion vector of
an edge image generator configured to generate an edge image by detecting an edge in the first image frame;
a first motion map generator configured to generate a first motion map by connecting each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel;
A second motion map by connecting each second pixel whose position is moved except for the first pixel among pixels constituting the first image frame to a point obtained by adding the motion vector to the coordinates of the second pixel. a second motion map generator to generate and
Rendering image generation for generating a second rendered image corresponding to the second image frame by performing overpainting on the first rendered image corresponding to the first image frame based on the first motion map and the second motion map A pictorial animation chapter*, comprising: The method of claim 1,
The rendering image generating unit,
an overfill direction determiner configured to determine an overfill direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image;
Selecting third pixels located in an overcoat area corresponding to the first motion map and the second motion map from among pixels constituting the first rendered image, and selecting fourth pixels to be overpainted from among the third pixels Overpainting point determining unit to select;
a stroke pattern selection unit for selecting a stroke pattern corresponding to a color of a pixel corresponding to the fourth pixel from among pixels constituting the second image frame; and
A pictorial expression apparatus for animation, comprising: a rendering unit configured to generate the second rendered image by arranging the selected stroke pattern in the overpainting direction determined for each of the fourth pixels selected for the first rendered image . The method of claim 1,
The rendering image generating unit,
an overfill direction determiner configured to determine an overfill direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image;
A grid having a predetermined size is set on the first rendered image, the first motion map, and the second motion map, and the first motion among the grids set for the first rendered image according to the following decision rule an overpainting determination unit that determines whether to overpaint the grids corresponding to the grids set on the map and the second motion map; and
A rendering unit configured to generate the second rendered image by arranging a stroke pattern selected from a plurality of stroke patterns in the determined overpainting direction with respect to a grid for which an overcoat is determined from among the grids set for the first rendered image. A pictorial expression device for animation:
[Decision Rule]

Here, Nall is the number of pixels present in an arbitrary grid, Nst_motion and Nwe_motion are the number of pixels present in the corresponding grids of the first and second motion maps, respectively, and A is the number of pixels present in the first motion map. Number, C is the number of pixels obtained as a result of performing an exclusive OR between an image obtained by applying a stroke pattern selected from among a plurality of stroke patterns to the first rendered image and the first motion map, and Diff Map(i) is the second image A color difference between the frame and the first rendered image, N is the number of pixels constituting an area of a predetermined size including a grid, and TH1 to TH4 are threshold values set for each. 4. The method of claim 3,
The pictorial expression apparatus of animation, characterized in that the overcoat determining unit increases the TH1 when the A/C value is greater than the TH2. The method of claim 1,
The rendering image generating unit,
an overfill direction determiner configured to determine an overfill direction for each pixel constituting the first image frame based on a gradient value of each pixel constituting the edge image;
Selecting third pixels located in an overcoat area corresponding to the first motion map and the second motion map from among pixels constituting the first rendered image, and selecting fourth pixels to be overpainted from among the third pixels Overpainting point determining unit to select;
A difference between a color value given to a stroke pattern selected from a plurality of stroke patterns having different shapes and textures and a color value of a pixel corresponding to the fourth pixel among pixels constituting the second image frame is determined in the first rendered image Stroke pattern selection for selecting a stroke pattern that is smaller than a difference between a color value of a pixel corresponding to the fourth pixel among pixels constituting a pixel and a color value of a pixel corresponding to the fourth pixel among pixels constituting the second image frame part; and
and a rendering unit configured to generate the second rendered image by arranging the selected stroke pattern on the first rendered image in the overlapping direction determined for each of the fourth pixels. 6. The method according to claim 3 or 5,
The rendering unit arranges the selected stroke pattern for each of the fourth pixels determined in response to the first motion map in the first rendered image, and then applies the selected stroke pattern to each of the fourth pixels determined to correspond to the second motion map. A pictorial animation chapter*, characterized in that the second rendered image is generated by arranging the selected stroke pattern on the first rendered image. 6. The method according to any one of claims 1 to 5,
The edge image generator generates an edge image corresponding to the first image frame by a Sobel edge filter or a Canny edge filter. 6. The method according to any one of claims 1 to 5,
The edge image generator generates an edge image corresponding to the first image frame by quantizing the first image frame by a predetermined quantization coefficient and setting a boundary between the contrast values of the quantized image as an edge. A pictorial expression device for animation. 6. The method according to any one of claims 2 to 5,
The coating direction determining unit,
a gradient calculator for calculating gradient values of pixels on edges detected in the first image frame;
A strong edge selecting a strong edge by sequentially selecting a reference pixel from pixels having a large gradient value of the pixels on the edge, and removing other pixels on the edge existing in a predetermined first area around the selected reference pixel. selection department; and
Each pixel constituting the first image frame is determined by the gradient size of each pixel constituting the first image frame calculated based on the sum of the gradient and the weight of the strong edges existing in the second predetermined region. A pictorial expression device for animation, comprising: a determining unit that determines the overcoating direction. Among the plurality of image frames, among the pixels constituting the temporally preceding first image frame, corresponding to the second image frame based on the degree of movement of the pixels whose positions on the second image frame following the first image frame have been moved. a motion vector calculation step of calculating a motion vector of
an edge image generating step of generating an edge image by detecting an edge in the first image frame;
a first motion map generating step of generating a first motion map by connecting each first pixel constituting the edge image to a point obtained by adding the motion vector to the coordinates of the first pixel;
A second motion map by connecting each second pixel whose position is moved except for the first pixel among pixels constituting the first image frame to a point obtained by adding the motion vector to the coordinates of the second pixel. a second motion map generation step of generating and
Rendering image generation for generating a second rendered image corresponding to the second image frame by performing overpainting on the first rendered image corresponding to the first image frame based on the first motion map and the second motion map Step; Conversational animation method comprising the. 11. The method of claim 10,
The rendering image generation step includes:
an overlay direction determining step of determining an overlay direction for each pixel constituting the first image frame based on the gradient values of each pixel constituting the edge image;
Selecting third pixels located in an overcoat area corresponding to the first motion map and the second motion map from among pixels constituting the first rendered image, and selecting fourth pixels to be overpainted from among the third pixels a pixel selection step to select;
a stroke pattern selection step of selecting a stroke pattern corresponding to a color of a pixel corresponding to the fourth pixel from among pixels constituting the second image frame; and
and a rendering step of generating the second rendered image by arranging the selected stroke pattern in the overlapping direction determined for each of the fourth pixels selected for the first rendered image. 11. The method of claim 10,
The rendering image generation step includes:
an overlay direction determining step of determining an overlay direction for each pixel constituting the first image frame based on the gradient values of each pixel constituting the edge image;
A grid having a predetermined size is set on the first rendered image, the first motion map, and the second motion map, and the first motion among the grids set for the first rendered image according to the following decision rule an overcoat determination step of determining whether to overpaint the grids corresponding to the grids set on the map and the second motion map; and
A rendering step of generating the second rendered image by arranging a stroke pattern selected from a plurality of stroke patterns in the determined overpainting direction with respect to a grid for which an overcoat is determined from among the grids set for the first rendered image. How to do pictorial animation:
[Decision Rule]

Here, Nall is the number of pixels present in an arbitrary grid, Nst_motion and Nwe_motion are the number of pixels present in the corresponding grids of the first and second motion maps, respectively, and A is the number of pixels present in the first motion map. Number, C is the number of pixels obtained as a result of performing an exclusive OR between an image obtained by applying a stroke pattern selected from among a plurality of stroke patterns to the first rendered image and the first motion map, and Diff Map(i) is the second image A color difference between the frame and the first rendered image, N is the number of pixels constituting an area of a predetermined size including a grid, and TH1 to TH4 are threshold values set for each. 13. The method of claim 12,
In the step of determining the overcoat direction, if the A/C value is greater than the TH2, the TH1 is increased. 11. The method of claim 10,
The rendering image generation step includes:
an overlay direction determining step of determining an overlay direction for each pixel constituting the first image frame based on the gradient values of each pixel constituting the edge image;
Selecting third pixels located in an overcoat area corresponding to the first motion map and the second motion map from among pixels constituting the first rendered image, and selecting fourth pixels to be overpainted from among the third pixels a pixel selection step to select;
The difference between a color value given to a stroke pattern selected from a plurality of stroke patterns having different shapes and textures and a color value of a pixel corresponding to the fourth pixel among pixels constituting the second image frame is the first rendered image. A stroke pattern selection step of selecting a stroke pattern smaller than a difference between a color value of a pixel corresponding to the fourth pixel from among the pixels and a color value of a pixel corresponding to the fourth pixel from among pixels constituting the second image frame ; and
and a rendering step of generating the second rendered image by arranging the selected stroke pattern on the first rendered image in the overlapping direction determined for each of the fourth pixels. 15. The method of claim 12 or 14,
The rendering step is
a stroke pattern arrangement step of arranging the selected stroke pattern for each of the fourth pixels determined to correspond to the first motion map in the first rendered image; and
and an image generating step of generating the second rendered image by arranging the selected stroke pattern for each of the fourth pixels determined in response to the second motion map on the first rendered image. animation method. 15. The method according to any one of claims 10 to 14,
In the edge image generating step, a pictorial animation method, characterized in that generating an edge image corresponding to the first image frame by a Sobel edge filter or a Canny edge filter. 15. The method according to any one of claims 10 to 14,
In the edge image generating step, after quantizing the first image frame by a predetermined quantization coefficient, an edge image corresponding to the first image frame is generated by setting a boundary of a contrast value of the quantized image to an edge. pictorial animation method. 15. The method according to any one of claims 11 to 14,
The step of determining the overcoating direction is
a gradient calculation step of calculating gradient values of pixels on edges detected in the first image frame;
A strong edge selecting a strong edge by sequentially selecting a reference pixel from pixels having a large gradient value of the pixels on the edge, and removing other pixels on the edge existing in a predetermined first area around the selected reference pixel. selection stage; and
Each pixel constituting the first image frame is determined by the gradient size of each pixel constituting the first image frame calculated based on the sum of the gradient and the weight of the strong edges existing in the second predetermined region. A pictorial animation method comprising; a determining step of determining an overcoat direction. A computer-readable recording medium recording a program for executing the pictorial animation method according to any one of claims 10 to 14 on a computer.

Images