WO2000063843A1

WO2000063843A1 - A method and apparatus for hierarchically combining regions

Info

Publication number: WO2000063843A1
Application number: PCT/US2000/009960
Authority: WO
Inventors: Joseph Scott Howlett
Original assignee: Avid Technology, Inc.
Priority date: 1999-04-16
Filing date: 2000-04-13
Publication date: 2000-10-26
Also published as: AU4458500A

Abstract

A method and apparatus for combining regions in a hierarchical manner includes combining two regions to produce an intermediate region. This process may be repeated at successive points along a path to form multiple intermediate regions. The intermediate regions are, in turn, combined to create a combined region which represents a resultant region. Each region may represent a brush outline in a brush stroke.

Description

A METHOD AND APPARATUS FOR HIERARCHICALLY COMBINING REGIONS

BACKGROUND Manually operable graphics tools are available in video editing systems which may include graphics paint applications. These paint applications allow a user to create and apply paint effects to image data. For example, a user may place a brush stroke onto a computer screen by using a mouse. A user may also select paint options such as the shape of a brush and a color. The paint application may be configured to store information, such as the color and shape of the brush stroke selected by the user. However, most paint applications store the brush stroke as a raster image or bit map which limits the ability of a user to scale an image or otherwise manipulate the brush stroke.

SUMMARY Combining regions in a hierarchical manner includes combining two regions to produce an intermediate region. This process may be repeated at successive points along a path to form multiple intermediate regions. The intermediate regions are, in turn, combined to create a combined region which represents a resultant region. Each region may represent a brush outline in a brush stroke.

One embodiment is a method of combining regions along a path that includes the steps of, for each point along the path, selecting a first brush region at a first point along the path, combining a second brush region at a second point along the path with the first brush region to create a first intermediate region, selecting a third brush region at a third point along the path, combining a fourth brush region at a fourth point along the path with the third brush region to create a second intermediate region, and combining the first and second intermediate regions in a hierarchical manner.

Another embodiment is a system for combining regions including a combiner receiving a first brush region and a second brush region, wherein the combiner creates a first combined region from the first brush region and the second brush region. A storage location receives the first combined region, and the combiner receives a third brush region and a fourth brush region to create a second combined region. The storage location grows dynamically to receive combined regions, and stores the first and second combined regions. Another embodiment is a computer product for combining regions along a path in a hierarchical manner including a computer readable medium having a program stored thereon. The program, when executed by a processor, performs the steps of, for each point along the path, combining brush regions at adjacent points to create an intermediate region, and combining pairs of intermediate regions.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings,

Figure la is a pictorial representation of an example path and brush region; Figure lb is a pictorial representation of an example brush stroke region; Figure lc is a pictorial representation of an example brush stroke region; Figure 2a is a data flow diagram illustrating one embodiment of combining two regions;

Figure 2b is a data flow diagram illustrating one embodiment of combining two regions;

Figure 3 is a pictorial representation of a hierarchical process of combining regions; Figure 4 is a flowchart describing in general how regions are combined in a hierarchical process;

Figure 5 is a pictorial representation of a stack structure including combined regions; and

Figure 6 is a flow chart illustrating a process of combining regions in one embodiment

DETAILED DESCRIPTION The following detailed description should be read in conjunction with the attached drawing in which similar reference numbers indicate similar structures. In some paint applications, characteristics of a brush stroke are stored and the paint applications have the capability of re-rendering the brush stroke. In these applications, the brush stroke is resolution independent and therefore can be enlarged or reduced in size without causing significant artifacts in the resulting image. One example paint application is the Paint Effect a component of Avid® Symphony™ available from Avid Technology, Inc., of Massachusetts. USA. In this type ofpaint appiication, the brush stroke may be defined as a path and a brush outline. The brush path may be represented as a curve along which a user drags a brush. The brush outline may be represented as a closed outline which may be

RECTIFIED SHEET (RULE 91) dragged by the user along the path of the curve. Any suitable data structure may be used to represent each of the path and the brush outline, and thus the brush stroke. Example representations for each of the path and the brush outline include a Bezier curve, a cubic spline and a polygon. Since the brush stroke may be stored as a path and a brush outline, the brush stroke may be manipulated by a user (e.g., reshaped, enlarged, or reduced), rendered and then used to apply a variety of effects (e.g., scaling, lightening, blurring, etc.) to image data. However, in conventional paint applications the re-rendering of a brush stroke may take a significant amount of time, particularly if the user has created a long brush stroke. The delay in the re-rendering process affects operations in the paint application that rely on, or occur after, the rendered brush stroke.

The rendered brush stroke specifies an arbitrary two-dimensional area. This area may be of any shape, may be composed of multiple disjoint pieces, and may have holes in it. The area may be represented by a data structure such as a region. A region data structure is supported by operating systems such as MacOS and Win32. Both operating systems provide routines for manipulating regions, for example, for creating a region which is a union, intersection, or difference of two other regions.

As the brush stroke region becomes larger and more complex, the re-rendering process is slowed down dramatically and performance degradation occurs because of the number of region primitives that are combined. The performance degradation occurs in part because an increasingly large brush stroke region is involved in each operation involved in producing the complete region.

The time for combining the regions is dependent in part on the characteristics of the region combining operations supported by the operating system. However, the time may be estimated according to the number of rectangles that make up a region (e.g., rectangles 104, 105 shown in Figure lc). For estimating purposes, it is assumed that the complexity of the brush stroke region grows by a complexity i at each point on the path. Combining two regions, for example, on a conventional system may have the complexity of an order O of O(i+j) where i and j are measures of the complexity of each of the two regions to be combined. This complexity is roughly equivalent to the number of non-overlapping rectangles, (e.g., 104, 105), used to fully describe the region. In this example, a brush region of complexity i may be placed onto a path N times such that the complexity of the entire rendering operation by combining a series of regions may be approximately represented as O(N^"). However, in the process shown in Figures 4 and 6 the time for combining regions may be reduced as described below.

As shown in Figures la and lb, the brush stroke region 102 may be represented by a brush region 106 which is dragged along a path 108. A user may create a brush stroke which is displayed in real time, for example, on a computer screen. An operating system calculates the regions representing each point along the path of brush stroke according to the process shown in Figures 3 and 6.

Many simple brush region shapes 106 may be stored by the operating system which controls the application rendering the brush stroke, and may be used to reproduce the brush stroke 102 using the path 108. The brush stroke is re-rendered by repeatedly positioning the brush region 106 at successive points along path 108. Information about the brush region 106 is accumulated at each point to form a brush stroke region 102 as shown in Figure lb. As a user is painting a brush stroke, the paint application may display a simplified representation of the brush stroke on the screen. However, after the user has completed a brush stroke, the paint application may render the brush stroke, use the rendered brush stroke to apply a currently chosen effect (e.g., coloring, lightening, blurring, etc.) to the image data, and display the image data. Figure 1 c illustrates an example brush stroke region 102 that may be created by the user using a paint application. This brush stroke region 102 may be represented by a series of rectangles 104 which may be used to approximate the shape of the brush stroke region 102.

Referring now to Figures 2a and 2b, data flow diagrams for combining two regions in an accelerated fashion are shown and will be described further in connection with Figures 5 and 6. As shown in Figure 2a, a region 202 may be combined with region 204 by using a hierarchical region combiner 206. The result is a combined region 208 which is, for example, the union of regions 202 and 204. Similarly, as shown in Figure 2b a region 202 may be combined with a combined region 208 which may be provided to hierarchical region combiner 206 in order to produce a combined region 210. The combined region may be the result of combining two or more regions through, for example, unioning, intersecting, or subtracting regions. The process used by hierarchical region combiner 206 to combine the brush regions changes the conventional linear process into a hierarchical process. This hierarchical process

RECTIFIED SHEET (RULE 91) combines regions in an optimal binary-tree fashion. The complexity ofthe hierarchical process is derived as shown in the following equations:

= i * NLog₂(n) (Eq. 2)

→ 0(NLogN) (Eq. 3)

where N is the number of brush regions which are defined using a tree such that the brush regions appear at the "leaves" ofthe tree and are combined hierarchically using a tree- type process to create the brush stroke region at the "root" ofthe tree, and where i is the complexity ofthe brush region.

Figure 3 illustrates a pictorial representation of a hierarchical process of combining regions according to one embodiment Regions (e.g., 302, 304, 306, 308, 310, 312, 314, 316) representing a brush outline at points along a path ofthe brush stroke are combined to form intermediate regions (e.g., 318, 320, 324). The intermediate regions are, in turn, combined to form additional intermediate regions (e.g., 326, 328) until all regions are combined to form a final brush stroke region 330. This process will be further described in connection with Figure 4. Figure 4 is a flowchart illustrating the process used to combine regions in a hierarchical manner, as shown in the embodiment of Figure 3. Prior to performing this process, any scaling or other manipulation ofthe brush stroke may been performed. In step 402, a first brush region 302 at a first point on a path of a brush stroke is selected. A second brush region 304 is selected at a second point on the path in step 404. Each point along the path may be selected by defining a step in pixels along the path ofthe brush stroke and defining a region according to the brush outline defined at the point. The step size may be one pixel. A first intermediate region 318 is created in step 406 by combining the first 302 and second 304 brush regions.

This process continues in step 408 to select a third brush region 306 at a third point on the path. In step 410, a fourth brush region 308 is selected at a fourth point on the path. In step 412, the third 306 and fourth 308 brush regions are combined to form a second intermediate region 320. The first 318 and second 320 intermediate regions are combined in

RECTIFIED SHEET (RULE 91) step 414. The combination of intermediate regions may be performed each time two intermediate regions are available. Alternatively, all primary regions may be processed first, with stored intermediate regions being combined together in subsequent steps.

The process of Figure 4 may continue for each point along a brush stroke curve until a complete brush stroke is rendered. As shown in Figure 3, the combination ofthe first 318 and second 320, third 322 and fourth 324 intermediate regions results in additional intermediate regions 326 and 328. This additional intermediate region 326 is combined with an intermediate region 328 to form a complete brush stroke region 330. The tree-type structure of Figure 3 may be expanded to include each point along a brush stroke, with the final combination resulting in a complete brush stroke region at the "root" ofthe tree. This process will be described in greater detail below.

The process of combining regions using the hierarchical region combiner 206 of Figure 2 will now be described in connection with Figures 5 and 6.

A stack structure 500 used to store combined brush regions, as shown in Figure 5, may be created for example, in the memory of a computer. Each location in the stack may include a flag and a reference to stored region data. The operation ofthe stack 500 will now be described in connection with Figure 6.

Figure 6 is a flow chart which illustrates the process which accumulates regions at each point along a path of a brush stroke and continues until a complete brush stroke can be rendered. In step 602, a brush region such as brush region 106 (Figure la), is formed at a point along a path 308. In step 604, the stack 500 is reviewed to determine if a stack location, i.e., location 502, is full. If the location has a region stored, a flag indicating the full state of the location is set.

In Figure 6 each ofthe stack locations are represented in multiple stages, for example, as 501 , 507, 511, 517, 529, 531 , 533, 535, and 537 to more clearly show the effect of combining the regions in a hierarchical fashion. For example, in Figure 5, stack location 502 is represented in two stages as an empty location 502, and as a full location with region 506 stored, although contents ofthe location may be the result of one combining operation discussed below. Further, Figure 5 illustrates the stack at several points in time. The use of the stack combines regions in a tree-like hierarchical manner.

A brush region (i.e., brush region 106) to be combined is shown at 504. In step 606 of Figure 6, a region is stored in an empty stack location. A location in a stack may be logically

RECTIFIED SHEET (RULE 91) empty because the flag for the location is not set, even though some data may be stored in the location. By indicating a location is empty, additional or new data may be stored in the location. In the example shown in Figure 5, the region 504 is stored in the empty stack location 502. The stored region is represented as stored region 506 in Figure 5. After a region is stored in the stack 504, it is determined in step 620 whether all ofthe regions in the brush stroke have been combined to form the brush stroke.

If all ofthe brush regions have been accumulated, then the combined region is provided as a result, as shown in step 622. If all ofthe regions in the brush stroke have not been accumulated in step 620, then the brush region is moved to the next point along path 108 (Figure la) so that the next region is accumulated into the combined regions.

Returning now to step 604, if the stack location is full, then as shown in step 608 of Figure 6, a brush region is unioned with the region in the full stack location. As shown in stage 507 of Figure 5, if stack location 502 is full, then the brush region 508 is combined with the region 506 in full stack location 502 to create a combined intermediate region. In one embodiment, the regions are combined through a union operation. The flag 509 for stack location 502 is changed to indicate that the location is empty. As discussed above, the stack location 502 may still include data for region 506. However, since the data has been used and accumulated, the location may be overwritten by subsequent operations.

In step 610, it is determined whether or not a next stack location exists in the stack to store the combined brush region 506 and brush region 508. If no next location exists, then a next location is created. In the example of Figure 5, location 510 is created. The result ofthe combined region is stored in the next stack location in step 614. The combined region is represented in Figure 5 as region 512 and is formed by region 508 and region 506. The result is stored in stack location 510. Next, in step 616, the previously full location in the stack is emptied and a flag is set to indicate that the location is empty. A flag is also set to indicate that the new location is now full. For example, location 502 is empty and location 510 is full.

As shown in 620, it is determined whether or not all ofthe brush regions have been accumulated along the path 108. If all ofthe regions have not been accumulated, then the process continues at step 602 to move a brush region to a point along the path ofthe brush stroke curve. Referring to Figure 5, if the stack location 502 is not full, then region 514 is stored in the empty stack location. However, if the stack location is full in step 604, as shown in stage 517 which illustrates a full stack location 502 and a full stack location 512, then additional combining operations are performed. Thus, as shown in Figure 5, if there is a next location 510 (step 610) and that location

510 is full (step 618) then the brush region 518 is combined with the region 514 in the full stack location 502. If a next location 510 exists (step 610) and the next location 510 is full (step 618), then the combined region 520 is combined with region 512, which is the region in the next full location 510. If there is a next location, i.e., location 524, then the combined region may be stored in location 524 as region 526 (step 614). The full locations 502 and 510 in the stack are then emptied and the flags for these locations indicate the locations are empty, step 616.

Figure 5 represents only a portion ofthe numerous combining operations performed in rendering a brush stroke. This combining process continues until all ofthe regions representing the brush stroke have been accumulated to create one brush stroke region which is the combination ofthe separate brush regions.

In step 620, if all ofthe regions have been accumulated, then the accumulated region which represents the combined regions for a complete brush stroke may be provided, step 622. For example, after all ofthe input regions have been processed, the full locations are combined to produce a final combined region. In the embodiment of Figure 5, regions 542 and 544 are combined. In one embodiment, the final combined region is produced by starting at a lowest location to combine a full region with a full location at a next highest location, and continuing up through the stack until all regions are combined. The final region then may reside at the top ofthe stack and may be taken from the top to be rendered. Figure 5 is provided as an example only and may be continued to include many locations to represent the brush stroke.

By using the process in Figure 6, the regions which define a brush stroke may be combined in a hierarchical manner to allow efficiencies in the combining process which are approximately ten times more efficient than conventional processes for combining regions. The hierarchical combining process results in a two to three times faster process for the operations performed in the paint application. A computer system for implementing the system of Figures 2a and 2b as a computer program typically includes a main unit connected to both an output device which displays information to a user and an input device which receives input from a user. The main unit generally includes a processor connected to a memory system via an interconnection mechanism. The input device and output device also are connected to the processor and memory system via the interconnection mechanism.

It should be understood that one or more output devices may be connected to the computer system. Example output devices include a cathode ray tube (CRT) display, liquid crystal displays (LCD), printers, communication devices such as a modem, and audio output. It should also be understood that one or more input devices may be connected to the computer system. Example input devices include a keyboard, keypad, track ball, mouse, pen and tablet, communication device, and data input devices such as sensors. It should be understood the invention is not limited to the particular input or output devices used in combination with the computer system or to those described herein. The computer system may be a general purpose computer system which is programmable using a computer programming language, such as "C++," JAVA or other language, such as a scripting language or even assembly language. The computer system may also be specially programmed, special purpose hardware, such as an application specific integrated circuit (ASIC). In a general purpose computer system, the processor is typically a commercially available processor, of which the series x86 and Pentium processors, available from Intel, and similar devices from AMD and Cyrix, the 680X0 series microprocessors available from Motorola, the PowerPC microprocessor from IBM and the Alpha-series processors from Digital Equipment Corporation, are examples. Many other processors are available. Such a microprocessor executes a program called an operating system, of which WindowsNT, UNIX, DOS, VMS and OS8 are examples, which controls the execution of other computer programs and provides scheduling, debugging, input/output control, accounting, compilation, storage assignment, data management and memory management, and communication control and related services. The processor and operating system define a computer platform for which application programs in high-level programming languages are written.

A memory system typically includes a computer readable and writeable nonvolatile recording medium, of which a magnetic disk, a flash memory and tape are examples. The disk may be removable, known as a floppy disk, or permanent, known as a hard drive. A disk has a number of tracks in which signals are stored, typically in binary form, i.e., a form interpreted as a sequence of one and zeros. Such signals may define an application program to be executed by the microprocessor, or information stored on the disk to be processed by the application program. Typically, in operation, the processor causes data to be read from the nonvolatile recording medium into an integrated circuit memory element, which is typically a volatile, random access memory such as a dynamic random access memory (DRAM) or static memory (SRAM). The integrated circuit memory element allows for faster access to the information by the processor than does the disk. The processor generally manipulates the data within the integrated circuit memory and then copies the data to the disk after processing is completed. A variety of mechanisms are known for managing data movement between the disk and the integrated circuit memory element, and the invention is not limited thereto. It should also be understood that the invention is not limited to a particular memory system. Such a system may be implemented in software or hardware or firmware, or a combination ofthe three. The various elements ofthe system, either individually or in combination may be implemented as a computer program product tangibly embodied in a machine-readable storage device for execution by a computer processor. Various steps ofthe process may be performed by a computer processor executing a program tangibly embodied on a computer-readable medium to perform functions by operating on input and generating output. Computer programming languages suitable for implementing such a system include procedural programming languages, object-oriented programming languages, and combinations ofthe two.

It should be understood the invention is not limited to a particular computer platform, particular processor, or particular high-level programming language. Additionally, the computer system may be a multiprocessor computer system or may include multiple computers connected over a computer network.

Having now described a few embodiments, it should be apparent to those skilled in the art that the foregoing is merely illustrative and not limiting, having been presented by way of example only. Numerous modifications and other embodiments are within the scope of one of ordinary skill in the art and are contemplated as falling within the scope ofthe invention.

Claims

1. A method of combining regions along a path, wherein the path includes at least four points, the method of combining comprising the steps of: for each point along the path, combining brush regions at adjacent points to create an intermediate region; and combining pairs of intermediate regions.

2. The method of claim 1 , wherein the step of combining brush regions comprises: selecting a first brush region at a first point along the path; combining a second brush region at a second point along the path with the first brush region to create a first intermediate region; selecting a third brush region at a third point along the path; and combining a fourth brush region at a fourth point along the path with the third brush region to create a second intermediate region.

3. The method of claim 1 , wherein the step of combining pairs of intermediate regions comprises combining the first and second intermediate regions in a hierarchical manner.

4. A system for combining regions comprising: a combiner receiving a first brush region and a second brush region, wherein the combiner creates a first combined region from the first brush region and the second brush region; and a storage location for receiving the first combined region; wherein the combiner receives a third brush region and a fourth brush region and creates a second combined region, and wherein the storage location grows dynamically to receive combined regions, and combines the first and second combined regions.

5. The system of claim 4, wherein the brush region is a Bezier form.

6. The system of claim 4, wherein the combiner performs a union operation.

7. The system of claim 4, wherein the storage location stores regions in a hierarchical manner.

8. The system of claim 4, wherein, the first brush region is selected at a first point along the path, wherein the second brush region is selected at a second point along the path, wherein the third brush region is selected at a third point along the path, and wherein the fourth brush region is selected at a fourth point along the path.

9. A computer program product for combining regions along a path in a hierarchical manner comprising: a computer readable medium having a program stored thereon, wherein the program, when executed by a processor, performs the steps of: for each point along the path, combining brush regions at adjacent points to create an intermediate region; and combining pairs of intermediate regions.

10. The computer program product of claim 9, wherein the step of combining brush regions comprises: selecting a first brush region at a first point along the path; combining a second brush region at a second point along the path with the first brush region to create a first intermediate region; selecting a third brush region at a third point along the path; combining a fourth brush region at a fourth point along the path with the third brush region to create a second intermediate region; and combining the first and second intermediate regions in a hierarchical manner.

11. The computer program product of claim 9, wherein the step of combining pairs of intermediate regions comprises combining the first and second intermediate regions in a hierarchical manner.