JP2009521062A - Modeling the 3D shape of an object by shading 2D images - Google Patents

Abstract

A computer graphics method generated by shading comprising: Receiving shading information provided by an operator representing a change in lightness level of at least a portion of the image in relation to the image of the object; Generating an updated geometric model of the object in response to the shading information used to determine at least one geometric feature of the updated geometric model; Displaying an image of the object as defined by the updated geometric model; D. The method can operate with digital input of any hierarchical subdivision surface, polygon mesh, or NURBS surface.
[Selection] Figure 13

Description

  The present invention relates to the fields of computer graphics, computer-aided geometric design, and more particularly to an improved system and technique for modeling the three-dimensional shape of an object by shading two-dimensional images.

  In computer graphics, computer-aided geometric design, etc., artists, drafters, etc. (herein collectively referred to as “operators”) use a three-dimensional model of an object maintained by a computer as a two-dimensional object. Attempts to generate from the line that defines the view. Conventionally, computer graphics devices generate a three-dimensional model from various two-dimensional line drawings including, for example, the contour and / or cross-section of an object, and perform a number of operations on such lines to produce a three-dimensional model in a three-dimensional space. Generate a two-dimensional curved surface. Then, such curved surface parameters and control points are corrected to correct or correct the shape of the model as a result of the object. Once the 3D model for the object has been generated, it can be viewed or displayed from any of a number of directions.

  In the field of artificial intelligence, commonly referred to as robot vision or machine vision (collectively referred to here as “machine vision”), it is recorded by the camera using a method called “formation from shading”. A three-dimensional model of the actual object is generated from one or more two-dimensional images of the object. In general, in machine vision, the type of object recorded on the image is not initially known by the machine, and the model of the object that is generated is, for example, by the machine or other device. Generally used to facilitate identification of the type of object shown above.

上での画像照射関数Ｉ（ｘ，ｙ）により表現され、一方、対象物の形状は、定義域

上の高さ関数ｚ（ｘ，ｙ）により与えられる。 In the formation method from shading, an object to be modeled is illuminated by a light source, an image is recorded by a camera such as a camera or a video camera, and the object is modeled from the image. It is assumed that the direction of the light source, the position of the camera, and the image plane for the object are known. Furthermore, it is assumed that the reflection characteristics of the surface of the object are also known. It is further assumed that the surface of the object is projected onto the image plane using orthographic projection techniques. That is, it is assumed that an unspecified camera that records an image on the image plane has an infinite focal length. The image plane represents the x, y coordinate axes (ie, any point on the image plane can be specified by coordinates (x, y)), and the z axis is orthogonal to the image plane. As a result, any point on the curved surface of the object that can be projected on the image plane can be expressed by coordinates (x, y, z). The image of the object projected on the image plane is a two-dimensional domain.

Represented by the image illumination function I (x, y) above, while the shape of the object is defined by the domain

Given by the height function z (x, y) above.

  The image irradiation function I (x, y) represents the brightness of the object at each point (x, y) in the image. In the formation method from shading, if I (x, y) is given to all points (x, y) in the domain, the shape of the object is determined by z (x, y). Is done.

  It would be desirable to provide an improved method and system for generating a three-dimensional model of an object by shading applied to a two-dimensional image of the object.

  The present invention provides an improved method and system for generating a three-dimensional model of an object by shading.

  One form of the present invention improves the shading formation (SBS) system and method described in commonly owned US Pat. No. 6,724,383.

  Another aspect of the present invention relates to special shaping techniques, methods and algorithms that can be implemented in a Shaded Formation (SBS) modeler according to the present invention, and more particularly to methods and algorithms that advantageously utilize trust region methods. .

Cross-reference to related applications This patent application is a co-pending US application Ser. No. 10 / 795,704 filed Mar. 5, 2004 (Attorney Docket No. MENT- 003D1) is a partial continuation application. The above application is a divisional application of US Application No. 09 / 027,175 (Attorney Docket No. MENT-003) filed on February 20, 1998, and filed on February 21, 1997. And claims the benefit of US Provisional Application No. 60 / 038,888 (Attorney Docket No. MENT-003-PR). The above three applications are incorporated herein by reference as references.
The present invention also relates to US Provisional Application No. 60 / 752,230 (attorney docket number MNTL-106-PR) filed on December 20, 2005, and dated August 24, 2006. Claims the benefit of US Provisional Application No. 60 / 823,464 (Attorney Docket No. MENT-089-B-PR). These two applications are also incorporated herein by reference as references.
In addition, all of which are hereby incorporated by reference, filed May 9, 2001, U.S. Application No. 09 / 852,906 (Ment-060), and February 1, 2002. US Application No. 10 / 062,192 (MENT-062), filed on the appendix, is also shared.
These and other aspects, embodiments, implementations, implementations, and examples of the invention are described in the detailed description below, which is divided into the following sections / sections.
I. Digital processing environment in which the invention can be implemented II. SBS modeler III. SBS Modeler Improvements 3.1 Introduction 3.2 SBS Shading and Shaping Processing 3.3 Surface Treatment Improvements 3.4 Additional Shaping Algorithms 3.5 SBS C ++ API
3.6 Prototype: SBS plug-in for 3ds max 3.7 SBS extension IV. Shaping methods and algorithms implemented within the SBS modeler 4.1 Introduction 4.2 Rasterization 4.3 Minimized function reduction 4.4 Trust region Newton-CG method 4.5 Calculation of trust region model 4.6 4. Minimization of trust region model 4.7 Convergence of trust region Newton-CG method Generalized method flow chart

I. Digital Processing Environment in Which the Invention can be Implemented Before describing specific examples and embodiments of the invention, the following is a description of the digital processing structure and environment on which the invention can be implemented and implemented, in conjunction with FIGS. And give a review to read.

  One skilled in the art will appreciate that the present invention can be used for image generation and composition, such as for display in moving images or other dynamic displays. The techniques described herein can be implemented as part of a computer graphics system where pixel values are generated for pixels in the image. Pixel values represent points in the scene that are recorded on the image plane of the simulated camera. The underlying computer graphics system can be configured to generate pixel values for the image using some selected method, such as the method of the present invention.

  The detailed description below provides examples of methods, structures, systems, and computer software products according to these techniques. Those skilled in the art will recognize that the methods and systems described are in software, hardware, or a combination of software and hardware, according to a conventional operating system such as Microsoft Windows, Linux, or Unix, over a stand-alone configuration or over a network. It will be appreciated that the implementation can be performed using a conventional computer device, such as a personal computer (PC) or equivalent device, that operates (or by emulation). Accordingly, the various processing forms and means described herein can be implemented in software and / or hardware components of a suitably configured digital processing device or network of devices. Processing can be performed sequentially or in parallel and can be implemented using special purpose or reconfigurable hardware.

  By way of example, FIG. 1 attached here shows an exemplary computer system 10 capable of performing such computer graphics processing. Referring to FIG. 1, a computer system 10 in one embodiment includes a processor module 11 and operator interface elements (or generally operator input elements, including operator input components such as a keyboard 12A and / or a mouse 12B). A digitizing tablet or other similar element identified as 12) and an operator output element such as a video display 13. The example computer system 10 may be a conventional stored program computer architecture. The processor module 11 may include, for example, a mass storage device such as one or more processors, memory, and disk and / or tape storage elements (not separately shown). Perform processing and storage operations associated with the digital data provided. An operator input element 12 can be provided to allow the operator to enter information for processing. The video display device 13 can provide output information generated by the processor module 11 on the screen 14 to be displayed to the operator. The output information includes data that can be input by the operator for processing and processing by the operator. Information that can be entered for control is included with the information generated during processing. The processor module 11 uses a so-called “graphical user interface” (“GUI”) in which information for various application programs is displayed using various “windows” for display by the video display device 13. Information can be generated.

  The computer system 10 is shown including special components such as a keyboard 12A and mouse 12B for receiving input information from an operator, and a video display device 13 for displaying output information to the operator. It will be appreciated that the system 10 can include a variety of components in addition to or in place of those shown in FIG.

  In addition, the processor module 11 may include one or more network ports, generally identified by reference numeral 14, that are connected to a communication link that connects to a computer system 10 in the computer network. Yes. The network port allows the computer system 10 to send information to and receive information from other computer systems and other devices in the network. For example, in a typical network configured according to a client-server paradigm, one computer system in the network is designated as a server and data and programs (collectively “information”) for processing by other client computer systems. So that client computer systems can conveniently share information. Client computer systems that need access to information maintained by a special server allow the server to download information there over the network. After processing the data, the client computer system also returns the processed data to the server for storage. In addition to computer systems (including the servers and clients described above), the network is also shared among various computer systems connected within the network, such as printers, facsimile machines, digital audio or video storage and distribution devices, for example. A possible device may be included. Communication links interconnecting computer systems within a network may comprise any convenient information carrying medium, including wires, optical fibers, or other media for carrying signals between computer systems, as is conventional. Good. Computer systems transfer information over a network by means of messages transferred over a communication link, each message including information and an identifier that identifies the device that receives the message.

  In addition to the computer system 10 shown in the figure, a method, apparatus, or software product according to the present invention is a conventional PC 102, laptop, such as shown by way of example in FIG. 2 (eg, network system 100). 104, including handheld or portable computer 106, whether stand alone, networked, portable, fixed, or via the Internet or other network 108, and As a result, it can operate in any of a wide range of conventional computer devices and systems that can include server 110 and storage device 120.

  In synchronization with the execution of conventional computer software and hardware, a software application constructed in accordance with the present invention can run in, for example, a PC 102 as shown in FIGS. 2 and 3, and program instructions can be stored in ROM or It can be read from CD ROM 116 (FIG. 3), magnetic disk, or other storage device 120 and loaded into RAM 114 for execution by CPU 118. Data can be entered into the system via any known device or means including a conventional keyboard, scanner, mouse, digitizing tablet, or other element 103.

  Those skilled in the art will recognize that the method forms of the present invention described herein are specially constructed to perform the processes described herein using ASIC construction techniques known to ASIC manufacturers. It will be appreciated that it can be implemented on a hardware element such as an application specific integrated circuit (ASIC). While various forms of ASIC are available from a number of manufacturers, currently available ASICs do not provide the functionality described in this patent application. Such manufacturers include Intel Corporation and NVIDIA Corporation, both located in Santa Clara, California. The actual semiconductor elements and equivalent integrated circuits of conventional ASICs are not part of the present invention and will not be discussed in detail here.

  Those skilled in the art will also be able to examine the methods of the present invention described below in more detail, using the teachings of the present invention described herein in more detail, where an ASIC or other conventional integrated circuit or semiconductor element is used. It will also be appreciated that it can be implemented and practiced in a manner that allows the shading shaping module 150 to be practiced within the processing system 102, as shown in FIG. According to the systems and techniques described below, the shading shaping module 150 may include one or more of the following sub-modules, a shading information input module 150a, a model generator module 150b, and a display output module 150c. it can. The shading shaping module 150 may also include other components described herein, generally indicated as a “tool / API / plug-in” in the square 150d. As further shown in FIG. 4, the output of the shading shaping module 150 can be provided in a number of different forms, such as displayable images, digitally updated geometric models, subdivision surfaces, and the like.

  Those skilled in the art will also understand that the method forms of the present invention operate under the commands of a workstation or PC operating system, such as a workstation and a personal computer (PC), and a computer program product configured in accordance with the present invention. It will also be appreciated that it can be performed within a digital processing system available on the market. The term “computer program product” can include any collection of computer readable program instructions encoded on a computer readable medium. A computer readable medium may be, but is not limited to, a computer hard disk, a computer floppy disk, a computer readable flash drive, a computer readable RAM or ROM element, or any that encodes, stores or provides digital information. Including any form of computer readable element, including other known means whether or not located at a workstation, PC, or other digital processing device or system it can. Since various forms of computer readable elements and media are well known in the computing arts, the choice is left to the practitioner. In any case, the present invention is operable to allow a computer system to calculate a pixel value, which is a hardware element within the computer system that may be a conventional element such as a graphics card or display controller. Can be used to generate a display control electronic output. Conventional graphics cards and display controllers are well known in computing technology and are not necessarily part of the present invention, so the choice is left to the practitioner.

II. SBS Modeler FIG. 5 is constructed in accordance with the present invention to generate a three-dimensional model of an object by shading applied to a two-dimensional image of the object in a given state of its creation at any time, such as by an operator. Shows a computer graphics system. Referring to FIG. 5, the computer graphics system includes a processor module 201, one or more operator input devices 202, and one or more display devices 203. Display device 203 typically includes things such as frame buffers, video display terminals, etc., which display information to the operator in the form of text and / or graphics on a display screen. The operator input device 202 of the computer graphics system 200 typically includes a pen 204 and a trackball or mouse device 206 that are typically used in conjunction with a digitizing tablet 205. In general, the pen 204 and digitizing tablet are used by the operator in several modes. In one mode, pen 204 and a digitizing tablet are used to provide updated shading information to a computer graphics system, although particularly useful in the context of the present invention. In other modes, the pen and digitizing tablet may input conventional computer graphics information, such as line drawings for surface trimming, and other information into the computer graphics system 200, thereby causing the system 200 to Is used to allow conventional computer graphics operations. The trackball or mouse device 206 can be used to move a cursor or pointer on the screen to a special point in the screen where the operator can provide input with a pen and digitizing tablet. The computer graphics system 200 can also include a keyboard (not shown) that can be used by an operator to provide text input to the system 200.

  The processor module 201 generally includes a processor and main memory, which may be in the form of one or more microprocessors, and typically includes a mass storage subsystem that includes one or more disk storage devices. including. In general, the memory and the disk storage device store data and a program (collectively “information”) processed by the processor, and store processing data generated by the processor. The processor module includes a connection to the operator input device 202 and the display device 203, receives information input by the operator via the operator input device 202, processes the input information, and stores the processed information in memory and / or memory. Store in the capacity storage subsystem. In addition, the processor module can provide video display information, which can be part of the information obtained from the processing data generated by the memory and disk storage device to the display device for display to the operator. Can be formed. The processor module 201 also connects the system 200 to a hardcopy output device such as a printer that facilitates the generation of hardcopy output, to a public telephone system, and / or in a computer network to facilitate the transmission of information. A connection to a modem and / or network interface (not shown) or the like (also not shown) may be included.

  The computer graphics system 200 generates information defining the initial and subsequent shape of a three-dimensional object from input provided by an operator via a pen, a digitizing tablet, and a mouse, It can be used to generate a two-dimensional image of the corresponding object for display to the operator, thereby generating a model of the object. The image displayed by the computer graphics system 200 represents an image of an object that is illuminated from the illumination direction and projected onto the image plane. The object is a spatial position and rotation with respect to the illumination direction and the image plane. With the direction and scaling and / or zoom settings selected by the operator. The initial model used in the model generation process may be one of a plurality of default models provided by the computer graphics system itself, such as a model that defines a hemisphere or an elliptical shape. Alternatively, the initial model may be provided by the operator by using pen 204 and digitizing tablet 205 to provide initial shading of at least one pixel in the image plane. When the initial model is provided by the operator, one of the pixels on the image plane is selected to provide a “reference” portion of the initial surface fragment for the object, and the reference initial surface fragment portion is selected with respect to the image plane. The computer graphics system has the rest of the surface fragments (if any) associated with the shading (if any) applied to other pixels on the image plane. Determine the initial model for. In one embodiment, the reference initial surface fragment portion is selected as the portion of the surface fragment for the first pixel on the image plane on which the operator shades. Further, in this embodiment, the reference initial surface fragment portion is determined to be parallel to the image plane so that the vector perpendicular to the reference initial surface fragment portion is orthogonal to the image plane and the reference initial surface fragment portion is Has a height value selected by the operator. In any case, the computer graphics system displays an image of the initial model, which is illuminated from a special illumination direction and defines the object shading associated with the initial model projected onto the image plane. .

  The operator uses a mouse, pen and digitizing tablet to provide updated shading of the image of the initial object and / or extend the object by shading adjacent areas on the image plane, and computer graphics The system 200 generates an update model that represents the shape of the object based on the update shading provided by the operator. When updating the shading, the operator can increase or decrease the amount of shading applied to a particular point on the image plane. In addition, the operator can use a mouse or trackball and pen and a digitizing tablet to perform conventional computer graphics operations associated with the image, such as cropping the surface representation of the object defined by the model. it can. The computer graphics system 200 uses the updated shading provided by the operator and other computer graphics information to generate an updated model that defines the shape of the object, and from the updated model to the operator. Can be generated from the respective spatial position, rotation direction, and scaling and / or zoom settings selected by the operator. When the operator determines that the shape of the object represented by the updated model is satisfactory, the operator can cause the computer graphics system 200 to store an updated model that defines the shape of the final object. On the other hand, if the operator determines that the shape of the object represented by the updated model is not satisfactory, the operator cooperates with the computer graphics system 200 to perform three-dimensional rotation, translation, and In a process that uses scaling or zooming, shading and other computer graphics information can be further updated. As the shading and other computer graphics information is updated, the computer graphics system 200 updates the model information, which includes the rotation direction, translation or spatial positioning selected by the operator, and scale and / or From the zoom setting, it is again used to provide a two-dimensional image of the object. These operations can continue until the operator determines that the shape of the object is satisfactory, at which point the computer graphics system 200 assumes that the updated model represents the final object. Store.

かつ

であって、Ｎが列の最大数（添え字ｉは画像平面内の列の範囲）であり、Ｍが列の最大数（添え字ｊは画像平面内の行の範囲）である

により表現される画素、または「ピクセル」に対応している。図６に示されている例としての画像平面２２０においては、列数Ｎは８であり、行数Ｍは９である。画像平面２２０をオペレータに示すために使用される表示装置２０３がラスター走査装置の場合は、行は、画像を表示するために装置により使用される走査線に対応する。各ピクセル

は座標系の特別な点（ｘ_ｉ，ｙ_ｊ）に対応し、Ｍ×Ｎは画像の解像度を特定する。更に、コンピュータグラフィックスシステム１０は、対象物が方向

、ここで、

はベクトルである方向を有する光源により照明され、対象物の表面はランベルト面であることを仮定している。明示されていないカメラは、その画像平面は画像平面２２０により表現されているが、「ＣＡＭＥＲＡ」と記されている矢印により表現されているように、画像平面２２０に直交する方向から画像平面２２０を見ると仮定されている。 Detailed operations performed by the computer graphics system 200 when determining the shape of the object will be described in conjunction with FIGS. Referring to FIG. 6, in the operation of the computer graphics system 200, an image of an object is displayed on a two-dimensional image plane 220 formed on a mosaic pixel 221 (i, j) having a predetermined number of rows and columns. It is assumed that it is projected. Image plane 220 defines an x, y Cartesian plane with rows extending in the x direction and columns extending in the y direction. The projection of the surface of the object to be formed is identified by reference numeral 222 in FIG. 6, but is an orthographic projection, and the direction of the “eye” of the camera is the z direction orthogonal to the x and y image planes. . Each point on the image plane is

And

Where N is the maximum number of columns (subscript i is the range of columns in the image plane) and M is the maximum number of columns (subscript j is the range of rows in the image plane).

Corresponds to a pixel or “pixel”. In the example image plane 220 shown in FIG. 6, the number of columns N is 8 and the number of rows M is 9. If the display device 203 used to show the image plane 220 to the operator is a raster scanning device, the rows correspond to the scan lines used by the device to display the image. Each pixel

Corresponds to a special point (x _i , y _j ) in the coordinate system, and M × N specifies the resolution of the image. In addition, the computer graphics system 10 allows the object to be oriented.

,here,

Is illuminated by a light source having a direction that is a vector, and the surface of the object is assumed to be a Lambertian plane. An unspecified camera has its image plane represented by the image plane 220, but the image plane 220 is viewed from a direction orthogonal to the image plane 220 as represented by an arrow marked “CAMERA”. It is assumed to see.

に対して、ピクセル上に投影される対象物の部分の高さを定義する高さ値ｚ（ｘ，ｙ）は既知であり、下記の

ここで、

は、「定義域

におけるすべての点（ｘ，ｙ）」を意味し、定義域

は画像平面２２０である式のように高さフィールドＨ（ｘ，ｙ）として定義される。更に各ピクセル

に対して、その上に投影される基本初期対象物の表面の部分の法線ｎ（ｘ，ｙ）もまた既知であり、下記の

のような法線フィールドＮ（ｘ，ｙ）として定義される。図６において、画像平面２２０の１つのピクセル上に投影される対象物の表面２２２に関連する法線は、「ｎ」と記されている矢印により表現される。 As described above, the computer graphics system 200 initializes at least a very small portion of an object to be modeled as an initial model. Each pixel

On the other hand, the height value z (x, y) defining the height of the part of the object projected onto the pixel is known and

here,

Is the domain

Means all points (x, y) in the domain

Is defined as a height field H (x, y) as in the equation for image plane 220. Each pixel

On the other hand, the normal n (x, y) of the surface portion of the basic initial object projected onto it is also known,

Is defined as a normal field N (x, y). In FIG. 6, the normal associated with the surface 222 of the object projected onto one pixel of the image plane 220 is represented by an arrow labeled “n”.

は、ピクセル

における画像の相対明度を表現し、逆に、ピクセルの相対シェーディングを表現する関連強度値Ｉ（ｘ，ｙ）（ここでは、「ピクセル値」とも称される）を有する。各ピクセル

に対する初期ピクセル値が、画像平面２２０上の位置（ｘ，ｙ）におけるそれぞれのピクセル

の画像強度値または明度を表現するＩ_０（ｘ，ｙ）により与えられ、シェーディング後のピクセル値がＩ_１（ｘ，ｙ）により表現されると、オペレータは、好ましくは、各ピクセル値に対して、

ここで、∫_Ｉ（∫_Ｉ＞０）は選択された所定の境界値である式が成り立ち、それにより、式（２．０３）が各ピクセルに対して満たされれば、対象物の形状は、オペレータにより提供されるシェーディングに基づいて更新できるように、画像に対するシェーディングを更新する。 After the computer graphics system 200 displays an image representing the object defined by the initial model that is displayed to the operator on the display device 203 as an image on the image plane 220, the operator Using the digitizing tablet 205 (FIG. 5), image correction can be started by updating the shading of the image. The initial model image displayed by the computer graphics system is itself defined by the initial model, illuminated from a predetermined illumination direction, and shaded to represent the shape of the object projected onto the image plane. That will be understood. Each pixel on the image plane

Is a pixel

Has an associated intensity value I (x, y) (also referred to herein as a “pixel value”) that represents the relative brightness of the image at, and vice versa. Each pixel

For each pixel at position (x, y) on the image plane 220.

Given by I ₀ (x, y) representing the image intensity value or brightness of the pixel and the pixel value after shading is represented by I ₁ (x, y), the operator preferably And

Here, ∫ _I (∫ _I > 0) is an expression that is a selected predetermined boundary value, and if the expression (2.03) is satisfied for each pixel, the shape of the object is The shading for the image is updated so that it can be updated based on the shading provided by the operator.

に対して、それぞれの新しい法線ベクトルｎ_１（ｘ，ｙ）を決定し、
（ｉｉ）更新法線ベクトルｎ_１（ｘ，ｙ）を生成後、新しい高さ値ｚ（ｘ，ｙ）を決定する。 After the operator updates the shading for the pixel, the computer graphics system 10 performs two general operations in generating an updated shape for the object. In particular, the computer graphics system 200 includes:
(I) First, each pixel whose shading has been updated

For each new normal vector n ₁ (x, y),
(Ii) After generating the updated normal vector n ₁ (x, y), a new height value z (x, y) is determined.

に対して実行し、それにより、新しい法線ベクトルフィールドＮ（ｘ，ｙ）と高さフィールドＨ（ｘ，ｙ）を提供する。ピクセル

に対する法線ベクトルｎ_１（上記の（ｉ）の説明）のシェーディングに関連して、コンピュータグラフィックスシステム２００により実行される操作は、図７と図８と関連して説明し、各ピクセル

に対する高さ値ｚ（ｘ，ｙ）（上記の（ｉｉ）の項目）の更新に関連して実行される操作は、図９と図１０と関連して説明する。 When the shading is updated, the computer graphics system 10 performs the operations (i) and (ii) for each pixel whose shading is updated.

To provide a new normal vector field N (x, y) and a height field H (x, y). pixel

The operations performed by the computer graphics system 200 in connection with the shading of the normal vector n ₁ (description of (i) above) with respect to are described in conjunction with FIGS.

The operations performed in connection with the update of the height value z (x, y) (item (ii) above) for will be described in conjunction with FIGS.

により定義される円錐２３１の表面上に存在、つまり、照明ベクトルとの点乗積に対するベクトルの集合は、オペレータにより提供されるシェーディングの更新後のピクセルに対するピクセル値「Ｉ」に対応する。更に、法線ベクトルｎ_１は、すべての法線ベクトルの場合と同様に、好ましくは値「１」の所定の大きさの値を有するように正規化されているので、更新法線ベクトルは下記の

ここで、「

」は更新法線ベクトルｎ_１の大きさを示す式に対応する大きさを有する。 Referring first to FIG. 7, a portion of the object identified by reference numeral 230 after the shading is updated by the operator is shown. In the following, it is assumed that the update normal vector specified by the arrow specified by the symbol “n ₁ ” is determined for the point value z (x, y) on the surface of the object 230. is doing. The normal vector specified by the symbol “n ₀ ” represents the normal to the surface before update. The illumination direction is represented by a line extending from the vector corresponding to the arrow identified by the symbol “L”. Specifically, “L” is based on the illumination direction from the light source that illuminates the object, and the magnitude represents the magnitude of illumination on the object provided by the light source. The lighting vector is expressed. In that case, based on the update, the set of possible new normal vectors is

The set of vectors on the surface of the cone 231 defined by, i.e., the dot product with the illumination vector, corresponds to the pixel value “I” for the updated pixel of the shading provided by the operator. Further, since the normal vector n ₁ is normalized so as to have a predetermined magnitude value of the value “1”, as in the case of all normal vectors, the updated normal vector is of

here,"

"Has the size corresponding to formula indicating the size of the updated normal vector n _1.

Equations (2.04) and (2.05) define a set of vectors and the magnitude of each vector, one of which is an update to the updated object at point z (x, y) It is a normal vector. The computer graphics system 200 selects a vector from the set of vectors as the appropriate update normal vector n ₁ as follows. As described above, the update normal vector exists on the surface of the cone 231. Obviously, if the original normal vector n ₀ and the illumination vector L are not parallel, they (ie, the original normal vector n ₀ and the illumination vector L) define a plane. This is because the point z (x, y) where the illumination vector L hits the object 230 and the origin of the normal vector n ₀ on the object 230 are the same point, and the tail of the illumination vector and the original normal vector The n ₀ head provides two additional points because, together with the point z (x, y), it is sufficient to define a plane. As a result, when the plane specified by reference numeral 232 on which both the illumination vector L and the preceding normal vector n ₀ exist is constructed, the plane 232 is represented by the line 33 in FIG. Intersects the cone along two lines. One of the lines 233 exists on the surface of the cone 231 that is on the side of the illumination vector L toward the original normal vector n ₀ , and the other line 233 is on the side of the illumination vector L away from the normal vector n _0. The exact updated normal vector n ₁ is defined by the line on the cone 231 that is on the side of the illumination vector L towards the previous normal vector n ₀ .

となる。 Based on these observations, the direction of the updated normal vector can be determined from equation (2.04) and: Since the previous normal vector n ₀ and the illumination vector L form a plane 232, their cross product, “n ₀ × L”, defines a vector that is orthogonal to the plane 232. As a result, since the updated normal vector n ₁ is also present in the plane 232, the updated normal vector n ₁ and the vector defined by the cross product between the previous normal vector n ₀ and the illumination vector L The dot product of has the value zero, that is,

It becomes.

として書き直せる。 Furthermore, the difference between the pixel values I ₀ and I ₁ provided by the previous shading and the update shading is bounded by ∫ _I (Equation (2.03) above) and the previous normal vector n ₀ And the updated normal vector n ₁ is also bounded by some maximum positive value ∫ _δ . As a result, equation (2.06) becomes

Can be rewritten as

This is illustrated in FIG. FIG. 8 is constrained by the presence of an update normal vector n ₁ , identified by a portion of the cone 232 shown in FIG. 7, the update normal vector n _1, and the reference number 234. that indicates a region representing the maximum angle ∫ _[delta] from the previous normal vector.

に対する更新法線ベクトルｎ_１を生成し、それにより、更新ベクトルフィールドＮ（ｘ，ｙ）を生成する。コンピュータグラフィックスシステム２００が、ピクセルに対して更新法線ベクトルを生成した後、そのピクセルに対して新しい高さ値ｚ（ｘ，ｙ）を生成でき、それにより、更新シェーディングに基づいて高さフィールドＨ（ｘ，ｙ）を更新する。高さ値ｚ（ｘ，ｙ）の更新と関連して、コンピュータグラフィックスシステム２００により実行される操作は、図９と図１０と関連して説明する。図９は、図６に示されている画像平面２２０に対する例としての更新シェーディングを示している。図９に示されている画像平面２２０に対して、ピクセル

には座標が与えられ、行は、１から８のその両端の数を含む範囲の数により特定され、列は、ＡからＩのその両端の文字を含む範囲の文字により特定される。図９に示されているように、更新シェーディングにおいては、

から

、

から

、および

から

のピクセルはすべて修正されており、コンピュータグラフィックスシステム２００は、更新高さフィールドＨ（ｘ，ｙ）におけるピクセルに対する更新高さ値として、それらに対して更新高さ値ｈ（ｘ，ｙ）を生成することになる。これを達成するために、コンピュータグラフィックスシステム２００は、下記に説明するいくつかの操作を実行して、垂直方向、水平方向、および２つの対角線方向に沿ってシェーディングが修正された各ピクセル

に対して高さ値を生成し、４つの高さ値（つまり、垂直、水平、および２つの対角線方向に沿う高さ値）の平均として、ピクセルに対する最終高さ値を生成する。 The computer graphics system 200 (FIG. 5) uses each pixel in the image plane 220 based on shading provided by the operator.

An update normal vector n ₁ is generated for, thereby generating an update vector field N (x, y). After the computer graphics system 200 generates an updated normal vector for a pixel, it can generate a new height value z (x, y) for that pixel, thereby creating a height field based on the updated shading. Update H (x, y). The operations performed by the computer graphics system 200 in connection with updating the height value z (x, y) will be described in conjunction with FIGS. FIG. 9 shows an example update shading for the image plane 220 shown in FIG. For the image plane 220 shown in FIG.

Is given a coordinate, a row is identified by a number of ranges including the numbers at its ends from 1 to 8, and a column is identified by a range of characters including the characters at its ends from A to I. As shown in FIG. 9, in update shading,

From

,

From

,and

From

And the computer graphics system 200 uses the updated height value h (x, y) as the updated height value for the pixel in the updated height field H (x, y). Will be generated. To accomplish this, the computer graphics system 200 performs several operations, described below, for each pixel whose shading has been modified along the vertical, horizontal, and two diagonal directions.

And a final height value for the pixel as the average of the four height values (ie, vertical, horizontal, and height values along two diagonal directions).

との関連において説明する。他の方向との関連において実行される操作と、シェーディングが更新された他のピクセルは、当業者には明白であろう。更新高さ値の生成において、コンピュータグラフィックスシステム２００はBezier-Bernstein内挿を利用し、Bezier-Bernstein内挿は、次元「ｎ」の曲線Ｐ（ｔ）を下記の

として定義する。ここで、ｔは０と１のその両端を含む区間上の数値パラメータであり、ベクトルＢ_ｉ（成分（ｂ_ｉｘ，ｂ_ｉｙ，ｂ_ｉｚ）により定義される）は、曲線Ｐ（ｔ）に対する「ｎ＋１」個の制御点を定義し、制御点Ｂ_０とＢ_ｎは曲線の端点を含む。端点における曲線Ｐ（ｔ）の接線は、ベクトルＢ_０Ｂ_１とＢ_ｎ−１Ｂ_ｎに対応する。１つの実施形態において、コンピュータグラフィックスシステム２００は、立体Bezier-Bernstein内挿

を使用して、更新高さ値を生成する。点Ｂ_０、Ｂ_１、Ｂ_２、およびＢ_３は立体曲線Ｐ_ｎ＝３（ｔ）に対する制御点である。 In generating the updated height value, the operation performed by the computer graphics system 200 is one of the modified pixels in the image plane 220, ie, one of the directions, ie, the pixels along the horizontal direction.

Will be described in relation to The operations performed in relation to other directions and other pixels with updated shading will be apparent to those skilled in the art. In generating the updated height value, the computer graphics system 200 uses Bezier-Bernstein interpolation, which uses the curve P (t) of dimension “n” as follows:

Define as Here, t is a numerical parameter on the interval including both ends of 0 and 1, and the vector B _i (defined by the components (b _ix , b _iy , b _iz )) is “ n + 1 "control points are defined, and control points B ₀ and B _n include the end points of the curve. The tangent line of the curve P (t) at the end points corresponds to the vectors B ₀ B ₁ and B _n-1 B _n . In one embodiment, the computer graphics system 200 performs stereoscopic Bezier-Bernstein interpolation.

To generate an updated height value. Points B ₀ , B ₁ , B ₂ , and B ₃ are control points for the solid curve P _{n = 3} (t).

に対する更新高さ値ｈ_１の決定に適用される式（２．０９）は、

に対応する。 pixel

Equation (2.09) applied to determine the updated height value h ₁ for

Corresponding to

に対する更新高さ値ｈ_１は、ピクセル

に対する高さ値であるｈ_ａに対応し、ｔ＝１に対しては、ピクセル

に対する更新高さ値ｈ_１は、ピクセル

に対する高さ値であるｈ_ｂに対応する。一方、０または１以外の値を有するｔに対しては、更新高さ値ｈ_１は、ピクセル

と

の高さ値ｈ_ａとｈ_ｂと、制御点Ｂ_１とＢ_２に対する高さ値の関数である。 From equation (2.10), for t = 0 the pixel

The updated height value h ₁ for the pixel

Corresponds to _a height value h _a , and for t = 1 the pixel

The updated height value h ₁ for the pixel

Corresponding to h _b is the height value for. On the other hand, for t having a value other than 0 or 1, the updated height value h ₁ is equal to pixel

When

Is _a function of height values h _a and h _b and height values for control points B ₁ and B ₂ .

において法線ベクトルｎ_ａに直交し、ベクトルＢ_２Ｂ_３は、ピクセル

において法線ベクトルｎ_ｂに直交する。その結果、

となり、

となる。 As described above, for the n-th order curve P (t), the tangent lines at the end points B ₀ and B ₁ correspond to the vectors B ₀ B ₁ and B _n−1 B _n . As a result, with respect to the curve _{P n =} 3 shown in FIG. 10 (t), the vector _B 1 _{B 0,} which is defined by the control points _{B 1} and the adjacent end point _{B 0} is the curve P at the end point _{B 0} A vector B ₂ B ₃ defined by a control point B ₂ that is adjacent to the end point B ₃ and touches _{n = 3} (t) is a tangent line that touches the curve at the end point B ₃ . Thus, the vector B ₁ B ₀ is a pixel

Perpendicular to the normal vector _{n a} in, the vector _B 2 _{B 3,} the pixel

Perpendicular to the normal vector _{n b} in. as a result,

And

It becomes.

および

ここで、数式（２．１３）と数式（２．１４）とのｘとｚの下付き文字は、数式（２．１０）におけるそれぞれのベクトルに対する、それぞれｘとｚとの成分を示している式を誘導する。式（２．１３）と（２．１４）とに対しては、高さ値のｚ成分の値ｈ_１ｚのみが分かっておらず、「ｘ」成分の値ｈ_１ｘは、この場合はピクセル

である、その高さ値が決定されるピクセルの位置の関数となることは理解されよう。更に、式（２．１２）は、下記の２つの

と

ここで、式（２．１５）と（２．１６）とにおける下付き文字ｘ、ｙ、およびｚは、式（２．１２）におけるそれぞれのベクトルに対する、それぞれｘ、ｙ、およびｚ成分を示している式を誘導する。 For the determination of the updated height value h _{1 in} the horizontal direction (see FIG. 9), equation (2.10), which is a vector format, has dimensions “x” and “z” (“z” dimension is For each of the

and

Here, the subscripts of x and z in Equation (2.13) and Equation (2.14) indicate the components of x and z, respectively, for the respective vectors in Equation (2.10). Deriving the formula. For equations (2.13) and (2.14), only the z component value h _1z of the height value is not known, and the “x” component value h _1x is the pixel in this case.

It will be appreciated that the height value is a function of the position of the pixel being determined. Further, the formula (2.12) is expressed by the following two

When

Here, the subscripts x, y, and z in the equations (2.15) and (2.16) indicate the x, y, and z components for the respective vectors in the equation (2.12), respectively. Deriving the formula that is.

に対応する点において、曲線に対して垂直とならなければならないという制約がある。図１０におけるベクトルＢ_０１２Ｂ_１２３がピクセル

に対応する点において曲線に接するならば、そのｚ成分は更新高さ値に対応する点ｈ_１もまたベクトルＢ_０１２Ｂ_１２３上に存在する。その結果、

と

となる。図１０に示されている凸型組合せに基づくと、

と

になり、これは

と

という結果になる。式（２．１７）と式（２．１９）と式（２．２１）とを組み合わせると、

になり、この結果、それぞれのベクトルのｘとｚ成分に対しては、

となる。同様に、式（２．１８）と式（２．２０）と式（２．２２）とに対しては、それぞれのベクトルのｘとｚ成分に対して、

となる。 Furthermore, as mentioned above, there are further constraints on the curve P _{n = 3} (t), in particular the update normal n ₁ is the pixel

There is a restriction that it must be perpendicular to the curve at the point corresponding to. The vector B ₀₁₂ B ₁₂₃ in FIG. 10 is a pixel.

If the tangent to the curve at a point corresponding to, then the z component of the point h ₁ corresponding to the updated height value is also present on the vector B ₀₁₂ B ₁₂₃ . as a result,

When

It becomes. Based on the convex combination shown in FIG.

When

This is

When

Result. Combining equation (2.17), equation (2.19) and equation (2.21),

As a result, for the x and z components of each vector,

It becomes. Similarly, for Equation (2.18), Equation (2.20), and Equation (2.22), for the x and z components of the respective vectors,

It becomes.

に対する点Ｐ_ｎ＝３（ｔ）へのベクトルｈ_１のｚ成分（つまり、ｈ_１ｚの値）が未知である。（２．１３）から（２．１６）、（２．２４）および（２．２５）の８個の式は、当業者には明白な方法により、未知の値を決定できる連立方程式に対しては十分である。 It will be understood that the eight equations (2.13) through (2.16), (2.24), and (2.25) are all one-dimensional in their respective x and z components. . For equations (2.13) to (2.16), (2.24) and (2.25), there are six unknown values, ie the value of parameter t, vector B ₁ x and z component values (ie, b _1x and b _1z values), vector B ₂ x and z component values (ie, b _2x and b _2z values), and pixels

The z component of the vector h ₁ to the point P _{n = 3} (t) with respect to (ie the value of h _1z ) is unknown. The eight equations (2.13) through (2.16), (2.24) and (2.25) are for simultaneous equations that can determine unknown values in a manner apparent to those skilled in the art. Is enough.

に対する更新高さベクトルｈ_１を決定する。４方向すべてに対して更新高さベクトルを決定した後、コンピュータグラフィックスシステム２００は、それらを平均する。更新高さベクトルの平均のｚ成分は、対象物に対する更新モデルの高さ値に対応する。 The computer graphics system 200 performs the above operations in relation to the horizontal direction (corresponding to the “x” coordinate axis) and corresponding operations similar to the above operations for the vertical and two diagonal directions, respectively. Run the pixel

An update height vector h ₁ is determined for. After determining the updated height vectors for all four directions, the computer graphics system 200 averages them. The average z component of the update height vector corresponds to the height value of the update model for the object.

  Operations performed by the computer graphics system 200 will be described in conjunction with the flowcharts of FIGS. 11A and 11B. In general, it is expected that the operator will have an image of the object modeled by the computer graphics system in his head. Referring to FIGS. 11A and 11B, an initial model for the object is determined (step 250), and the computer graphics system determines the display direction based on a predetermined illumination direction, the image plane (shown in FIG. 6). The two-dimensional image is displayed to the operator so as to correspond to the reference image plane 20) (step 251). As described above, the initial model can define a default shape such as a hemisphere or ellipse provided by a computer graphics system, or a shape provided by an operator. In either case, the shape defines an initial normal vector field N (x, y) and a height field H (x, y) to define a normal vector and height value for each pixel in the image. After the computer graphics system 200 displays the initial model, the operator can select one of a plurality of operating modes including a shading mode associated with the present invention, one of a plurality of conventional computer graphics modes such as erasing and trimming. Both can be selected (step 252). If the operator selects the shading mode, the operator updates the shading of the two-dimensional image, for example, with the pen and digitizing tablet of the system (step 253). While the operator is applying shading to the image at step 253, the computer graphics system 200 can display the shading to the operator. The shading applied by the operator is preferably a representation of the shading of the final object that is illuminated from a predetermined illumination direction and projected onto the image plane displayed by the computer graphics system 200.

  When the operator updates the shading for the pixel at step 253, the computer graphics system 200 generates an update to the object model. In generating the update model, the computer graphics system 200 first determines an update normal vector for each pixel in the image, as described above with respect to FIGS. 7 and 8, thereby updating the object. A normal vector field is provided (step 254). Thereafter, the computer graphics system 200 determines an updated height value for each pixel in the image, as described above in connection with FIGS. 9 and 10, thereby updating the height for the object. A field is provided (step 255).

  After generating an update normal vector field and an update height field to provide an update model of the object, the computer graphics system 200 is selected by the operator (step 257), image rotation, translation, and In the scaling and / or zooming process, as selected by the operator (step 256), an image of the updated model is displayed to the operator in one or more directions and zooms. If the operator determines that the updated model is satisfactory (step 258), this is the case, for example, when the updated model matches the image of the object to be modeled that the operator has drawn in mind, The operator can save the updated model in the computer graphics system 200 as the final model of the object (step 259). On the other hand, if the operator determines in step 257 that the updated model is not satisfactory, the operator can return the computer graphics system 200 to step 251.

  Returning to step 252, if the operator selects another operating mode, such as a conventional operating mode such as an erase mode or a trimming mode, at that step, the computer graphics system will cause the computer graphics system 200 to be A series of steps up to step 260 is performed to update the model based on the erasure information or trimming and other conventional computer graphic information provided. The computer graphics system performs a series of steps up to step 257 to display an image of the object based on the updated model. If the operator determines that the updated model is satisfactory (step 108), the operator can cause the computer graphics system 200 to save the updated model as the final model of the object (step 259). On the other hand, if the operator determines in step 257 that the updated model is not satisfactory, the operator can return the computer graphics system 200 to step 251.

  When the operator updates the shading of the image of the object (step 253) or provides other computer graphic information (step 260), the operator performs steps 251, 253 to 257, and 260 on the computer graphics system 200. The computer graphics system 200 can generate an updated normal vector field and an updated height field in steps 254 and 255, or a conventional computer graphic component in step 260, so that the object's Define the update model. If the operator determines in step 258 that the update model corresponds to an image of the object that the operator has drawn on the head, or if it determines that the update model is satisfactory, the operator may send an update normal to the computer graphics system 200. The vector field and the updated height field can be stored to define the final model of the object (step 259).

  The present invention provides a number of advantages. In particular, the present invention provides an interactive computer graphics system whereby an operator, such as an artist, can imagine the desired shape of an object, and the shading for the object is illuminated from a special illumination direction, You can imagine how it will appear in the object when viewed from a special visual direction (direction defined by the position of the image plane). After the operator provides certain shading input corresponding to the desired shape, the computer graphics system displays the object model to the operator as updated based on the shading. The operator can accept the model as the final object or further update the shading, so that the computer graphics system further updates the object model. A computer graphics system constructed in accordance with the present invention avoids the need to solve partial differential equations that were necessary in prior art systems operating according to the shaping method from shading.

  A further advantage of the present invention is that it facilitates the use of a hierarchical representation for the model of the object being generated. As a result, for example, if the operator causes the computer graphics system 200 to scale the object or zoom in on the object, thereby providing a higher resolution, the pixels of the image will be at a lower resolution. It will be appreciated that the portion of the image that was associated with a single pixel will be displayed. In this case, when the operator updates the shading of the higher resolution image, the computer graphics system generates a normal vector and height value for each higher resolution pixel whose shading is updated as described above, Thereby generating and / or updating parts of the model related to update shading at an increased resolution. The updated portion of the higher resolution model is associated with a special portion of the model previously defined at the lower resolution, thereby providing a hierarchical representation that can be stored. As a result, the object defined by the model inherits the level of detail corresponding to the higher resolution in the underlying surface representation. Corresponding operations can be performed if the operator can reduce the scale of the object or zoom out the object to the computer graphics system 200, thereby providing a lower resolution.

  It will be appreciated that numerous variations and modifications may be made to the computer graphics system 200 as described above in connection with FIGS. For example, the computer graphics system 200 can maintain object model information, ie, normal vector field information and height field information, for multiple updates of shading provided by an operator (ie, system 200). ) Can be used to display a model of the object for each update. This allows the operator to see images of each model, for example, to be able to see the progress of the object through each update. In addition, this allows the operator to return to the model from the previous update as the base to be updated. This allows the operator to generate, for example, a tree of objects based on different shading in a special model.

  Further, although the computer graphics system 10 has been described as determining an updated height field h (x, y) using Bezier-Bernstein interpolation, such as, for example, Taylor polynomials and B-splines It will be appreciated that other forms of interpolation can also be used. In addition, multiple forms of surface representation can be used with the present invention. In fact, the model generation method used by computer graphics system 200 is a general application, so that all free-form surface representations are piecewise composed of, for example, triangles, squares, and / or pentagons. Can be used with linear surfaces.

  Further, although the computer graphics system 200 has been described as utilizing an orthographic projection and a single light source, it will be appreciated that other types of projections and multiple light sources may be used, including perspective projection.

  Furthermore, although the computer graphics system 200 has been described as providing the shape of an object by shading the image of the object, it can also be trimmed and erased via the appropriate operating mode of the pen 204 and the digitizing tablet. It will be appreciated that such computer graphics operations can also be provided.

  Furthermore, although the computer graphics system has been described as generating a model of an object assuming that the surface of the object is a Lambertian surface, when rendering an image of the object, It will be appreciated that other surface treatments can be used.

  It will be appreciated that the system according to the present invention can be constructed in whole or in part from special purpose hardware or general purpose computer systems, or any combination thereof, where any part can be controlled by a suitable program. Any program, in whole or in part, can include a portion of the system in a conventional manner, or be stored on the system, or a mechanism for communicating information over a network or other conventional method The system can be provided in whole or in part. In addition, the system is provided by an operator using an operator input element (not shown) that can be connected directly to the system or via a network or via a mechanism that can convey information in a conventional manner. It will be understood that the information can be manipulated and / or controlled.

III. Improving the SBS modeler 3.1 Introduction The above systems and technologies have made great progress. Section 3.2 provides a brief summary of the SBS shading and shaping process. Sections 3.3 to 3.7 describe specific extensions and other improvements to SBS shading and shaping processes.

3.2 SBS Shading and Shaping Process FIG. 12 shows a flow diagram illustrating SBS shading and shaping cycle 300.

  Step 301: Hierarchical subdivision surfaces, polygon meshes, and non-uniform rational B-spline (NURBS) surfaces are supported as input surfaces for shading formation (SBS). The SBS internal algorithm uses the properties of hierarchical subdivision surfaces, so each of the latter two types is converted to subdivision surfaces before shading begins.

  Step 302: Once the subdivision surface is in place and displayed to the user, it contains information about the grid corners, grid width and height, pixel size, and camera-to-object conversion. Aligned with 2D model view.

  Step 303-Step 305: Using the 2D model view, the user sets the lighting direction, adjusts the input parameters, and performs shading, ie, modifies the intensity of the selected pixel or has been previously shaded Load a set of pixels. This information is passed to the shaping algorithm 306.

  Steps 306-309: The shaping algorithm 306 determines the exact geometric changes to make to the surface. If necessary, more surface features are added via subdivision in the area of shading to ensure that sufficient details are present (step 307). A height field is obtained that reflects the changes requested in 2D settings in 3D (step 308), and the subdivision surface is modified so that it reflects these heights (step 309).

  The result is a shaped hierarchical subdivision surface that can be further transformed (steps 302-309), stored (step 310), or converted to the desired output surface type.

3.3 Improved Surface Treatment The SBS system and technique described in Section II above is designed for the treatment of NURBS surfaces. The systems and techniques described herein extend SBS to accept arbitrary hierarchical subdivision surfaces, polygon meshes, or NURBS surfaces. The incident mesh is converted to a hierarchical subdivision surface if it is not yet a hierarchical subdivision surface, and the resulting subdivision surface undergoes SBS shading and shaping cycles. Use adaptive subdivision to add details to the surface, and use analysis and synthesis to communicate the change to all levels of the surface, allowing for correction at the specified level of details. The method images® hierarchical subdivision surface (HSDS) library provides all necessary subdivision support. The features of the HSDS library are described in patents owned by the owners of the present patent application. The subdivision surface model resulting from the SBS process can be converted to another surface type if desired. In this way, SBS allows flexibility in the selection of both incoming and outgoing mesh types and utilizes hierarchical subdivision characteristics in its algorithm.

ここで、Ｎ_Ｈは表面に対する離散法線であり、λは、無限に遠くにある光源の方向を指し示す単位ベクトルである式により定義される。 3.4 Additional shaping algorithms The surface of interest that is assumed to be continuous is orthographically projected onto the viewing plane. This projection has an associated height field, the intensity of which is determined by one light source with a Lambertian reflection map, so that a point (u,) in the model view of a given surface with a height field H The discrete intensity I in v) is

Here, _NH is a discrete normal to the surface, and λ is defined by an expression that is a unit vector indicating the direction of a light source that is infinitely far away.

  The intensity of the selected pixel on the projection surface is changed by shading or loading a predefined set of pixels. The SBS shaping algorithm obtains a shape determined by shading. One solution includes Bezier-Bernstein polynomials. A new technique for interpreting 2D shading in the model view as a 3D shape on the surface is now practiced in SBS and described in the rest of this section.

ここで、Ｃ_Ｈは関連する高さフィールドＨを有する曲面の曲率を示し、λは平滑係数と称される定数で、合計は、投影曲面の内部と交差するモデルビュー内のピクセルに渡り実行される式を最小化する、モデルビュー全体にわたる高さ増分Γの集合を生成する。 The techniques described here are:

Where C _H is the curvature of the curved surface with the associated height field H, λ is a constant called the smoothing factor, and the sum is performed over the pixels in the model view that intersect the interior of the projected curved surface. Generate a set of height increments Γ over the entire model view that minimizes

とまとめられるピクセルに縮小できる。更に、高さ増分Γの集合を、各ピクセルに対して１つの入力を含むベクトルｘ、または対応する高さ値がアルゴリズムにより変更できる接続領域に縮小できる。Γのサイズは、投影曲面の高さフィールドのサイズに整合する。ベクトルｘは、潜在的にはるかに小さい集合である、Γにおける固有非ゼロ高さ増分の数に基づいてサイズを縮小する。そして、関数（３．０２）は、

の非制約最小化に縮小できる。 Let P be the set of pixels in the model view with modified intensity. This set is referred to as a set of modified pixels. Through a series of simplifications, the function (3.02) is

Can be reduced to pixels that can be grouped together. Furthermore, the set of height increments Γ can be reduced to a vector x containing one input for each pixel, or a connected region whose corresponding height value can be changed by the algorithm. The size of Γ matches the size of the height field of the projection surface. The vector x reduces in size based on the number of intrinsic non-zero height increments in Γ, which is potentially a much smaller set. And the function (3.02) is

Can be reduced to unconstrained minimization.

  The reduced function is less computationally expensive to minimize, but this means that instead of all the pixels in the model view, it is only necessary to find the sum in the vicinity of the modified pixels, The scale of the problem is due to both reasons that the size of Γ is reduced to the length of x.

ここで、

はｆの勾配ベクトルであり、

はｆのヘッシアン行列である二次関数によりモデル化される。そして、モデルＦは、ある

に対して、

である領域において最小化される。最小化を行うために使用される方法は、特殊疎行列乗算を有するCG-Steihaug法である。テスト値は、結果としての最小点ｘ_１から構築され、

である。 The method used to minimize the function (3.03) is the trust region method. First, the function

here,

Is the gradient vector of f,

Is modeled by a quadratic function that is a Hessian matrix of f. And model F is

Against

Is minimized in an area. The method used to perform the minimization is the CG-Steihaug method with special sparse matrix multiplication. Test value is constructed from the minimum point x ₁ as a result,

It is.

が十分に小さい、つまり、ｆの極小値が達成されたかどうかということである。この方法は収束することが証明できる。 If ρ is close to 1, F is considered to be a good model for f in the confidence region. The center of the confidence region is moved to x _1, the radius of the confidence region is increased. If ρ is away from 1, the radius of the trust region decreases. This process is repeated, that is, a minimum value is determined for F in the new confidence region. The criteria for stopping processing is at the center of the current trust region

Is sufficiently small, that is, whether a minimum value of f has been achieved. This method can be proved to converge.

3.5 SBS C ++ API
The C ++ application programming interface (API) has recently been developed for SBS. The C ++ API functions on the above mental matter (registered trademark) library and requests libmentalmatter.lib at the time of connection. The mental matter library starts and ends inside the SBS library.

Three major SBS classes
miSbs_surface
miSbs_ogl_view
miSbs_solver
There is. Access to these classes and how to create them is provided by miSbs_module.

  The initialization of miSbs_module is unconditional and is performed when the class is first accessed. An instance of miSbs_module is returned by its static method get. The terminate method must be called when unloading a library. SBS uses objects from the miCapi_cl_subsurf class of the mental matter library that are wrapped in the miSbs_surface object via the create_surface method. The plug-in writer provides the SBS API with an instance of miCapi_cl_subsurf or a mosaic mesh in the form of miGeoBox. Another possibility is not to provide a surface, in which case the library creates a wrapper that holds an empty subdivision surface. Other methods in the miSbs_module class include creat_viewer and get_solver.

  The miSbs_surface class is used for access to the SBS shaping algorithm in calls to miSbs_solver, and for display and interaction in miSbs_ogl_view. The miSbs_surface implementation is instantiated using the create_surface method of miSbs_module and destroyed by the destroy method. Other methods of the miSbs_surface class include get_subsurf, get_depth, and convert (converts a mesh index to or from a surf index). Other related classes are provided to speed up the integration. For example, miSbsMax_mesh hides the technical details of transforming a 3ds max® mesh into and out of a miSbs_surface object.

  The miSbs_ogl_view class is used to display an instance of miSbs_surface. The miSbs_ogl_view implementation is instantiated using the create_viewer method of miSbs_module and destroyed by the destroy method. The miSbs_ogl_view entity does not maintain a reference to a given miSbs_surface instance. It only holds data related to its mesh representation (possibly simplified) and auxiliary data such as OpenGL® contexts and triangle strip buffers. The methods of the miSbs_ogl_view class include set_settings and update along with a set of methods around 2D projection of mesh vertices and faces. These include get_face_in_pixel, get_vtxs_in_face, get_vtx_pixels, get_2d_distance_from_vtx, get_face_color, set_face_color, reset_face_colors, set_pixel_buffer, get_pixels_no and copy_pixel_buffer. An additional helper class miSbs-wind32_viewport is provided to be hooked up on an existing window. It provides basic refresh and message handling functions.

  A third class called miSbs_solver is presented in the API to perform SBS operations. It is implemented as a static, stateless instance and is accessed via the miSbs_module get_solver method. The methods include get_default_settings, udpdate (the main algorithm that determines 3D shape from 2D shading), and cancel.

3.6 Prototype: SBS Plug-in for 3ds max (R) SBS was recently implemented as a modifier for 3ds max (R) from Autodesk (R) / discreet (R). The modifier allows artists to perform SBS modeling by shading in 2D directly on the 3ds max viewpoint with a simple brush. All functions of the SBS modeling library are available in the plug-in. FIG. 13 shows a screen shot 350 of the SBS Modifier at 3ds max.

Plug-in function
A shading tool with the ability to load and save a basic 2D paint package and shading;
Light control,
Updating the surface shape based on shading information, light direction, and input parameters;
Undo / redo inside the modifier,
A tool to select the area to be updated (masking),
Includes a selection tool with standard subdivision surface manipulation functions.

3.7 SBS Extension The above systems and technologies could be augmented in a number of ways. For example, these systems and techniques can be modified to include:

  Mesh-Based SBS: A function to perform SBS processing on a polygon mesh without first converting to the characteristics of subdivision surfaces and / or using the characteristics of subdivision surfaces.

  Real-Time Mesh Display: A function that displays large and complex meshes in a conversational format. Other modifications can include view dependent simplification and custom mesh manipulation.

  Trimming: A function that arbitrarily trims the entire surface.

  Creasing: A function that arbitrarily wrinkles the entire surface.

  Shape from Contour Lines: A function for creating an initial 3D shape by drawing an outline as a tool for completing the SBS modeling process.

  NURBS-Based SBS: A function that starts SBS processing on non-uniform rational B-splines (NURBS).

は、表面に与えられた照明条件λ表わし、Ｃ_Ｈは、曲面の曲率を表わすとする。ＳＢＳシェーディングアルゴリズムは、高さ値Ｈに加算され、下記の

ここで、λは平滑化係数と称される正の定数で、積分は、ビューイングウィンドウ内の投影表面の領域全体にわたり行われる関数を最小化する増分Γを求めようとする。理想的な結果は、強度がシェーディングの強度と整合する、λにより決定された平滑性を有する表面高さの新しい集合である。２Ｄシェーディングは、３Ｄ修正という結果になる。 IV. Shaping methods and algorithms implemented within the SBS modeler 4.1 Introduction The goal of Shading Shaping (SBS) processing is to interpret 2D (2D) shading as 3D (3D) shapes. A number of techniques and systems for accomplishing this are described herein. For purposes of this discussion, grayscale shading has already been performed on the projection of the surface onto the viewing plane, the underlying surface is continuous, and the gray of the projection surface. It is assumed that the scale intensity can be described mathematically. H represents the height value in camera space associated with the part of the surface that can be seen in the viewing window;

Represents the illumination condition λ given to the surface, and C _H represents the curvature of the curved surface. The SBS shading algorithm is added to the height value H and

Where λ is a positive constant called the smoothing factor, and the integration seeks to find an increment Γ that minimizes the function performed over the entire area of the projection surface in the viewing window. The ideal result is a new set of surface heights with smoothness determined by λ whose intensity matches the intensity of the shading. 2D shading results in 3D modification.

  The SBS process must be performed in a way that is efficient and does not disturb the continuity of the underlying surface. In order to move from theoretical settings to computational settings, the discretize discussed in the next section is required.

4.2 Rasterization In order to find a solution for SBS, the problem must be explained accordingly and rasterized for discrete computation.

4.2.1 Model View The viewing window is rasterized as an orthogonal grid M of pixels having integer value coordinates (u, v) and corresponding camera space coordinates (x, y). This rasterized image is called a “model view”.

は、ピクセルそれ自身を含み、｜ｕ_０−ｕ｜≦１と｜ｖ_０−ｖ｜≦１となるようなすべてのピクセル

として定義される。ピクセルの集合Ａの近傍は、nbhd（Ａ）として表わされる。ピクセルは、ピクセルがＡのメンバーであれば集合Ａの境界上にあると定義されるが、その隣の１つまたは２つ以上はＡ内にない。Ａの境界は、∂Ａと表わされる。Ａの内部は、

と定義され、符号「＼」は、集合の減算を表わす。モデルビューの内部のピクセルは、９個のピクセルを含む近傍を有している。 Raster pixel neighborhood

All pixels that contain the pixel itself, such that | u ₀ −u | ≦ 1 and | v ₀ −v | ≦ 1

Is defined as The neighborhood of the pixel set A is denoted as nbhd (A). A pixel is defined to be on the boundary of set A if the pixel is a member of A, but one or more of its neighbors are not in A. The boundary of A is denoted ∂A. The inside of A is

The symbol “\” represents set subtraction. The pixels inside the model view have a neighborhood that contains nine pixels.

4.2.2 Surface projection The SBS algorithm assumes an orthographic projection of the surface onto the viewing plane. S represents the set of pixels in the model view that intersect the projection surface. The surface height field, represented by H, is defined on pixels in S and includes the height of the surface in camera space. H (u, v) is the floating point height of the portion of the surface that can be seen at pixel (u, v) in the model view.

4.2.3 First derivative The raster points in the model view are arranged so that their vertical and horizontal distances are equal, that is, the model view pixels are square and the same size. It is assumed. However, it should be noted that the systems and techniques described herein can also be implemented for non-square pixels. Non-square pixels can be used, for example, in SBS mesh-based implementations. In that case, the projected basic apex angle can be used to define the pixel.

と定義される。 Let ∫ be the floating point width (height) between neighboring raster points in camera space. The discrete first derivative in the u direction of the surface with height field H is based on a simple gradient formula over two pixels across the point of interest:

Is defined.

と定義される。 Similarly, the discrete first derivative in the v direction is

Is defined.

と定義され、表面接ベクトル＜１、０、Ｄ_１Ｈ（ｕ，ｖ）＞と＜０、１、Ｄ_２Ｈ（ｕ，ｖ）＞とのクロス積である。 4.2.4 Surface normal The discrete normal for a surface with a height field H is

And is the cross product of the surface contact vector <1, 0, D ₁ H (u, v)> and <0, 1, D ₂ H (u, v)>.

で定義される。 4.2.5 Illumination condition and intensity In SBS, the illumination condition is assumed to be a Lambertian surface by one light source. However, it is possible to use different illumination models and / or to use multiple illumination sources. The light source is described by a unit vector λ indicating the direction of light and is infinitely far away. The discrete intensity I and light vector λ for a given surface with height field H is

Defined by

  This formula produces a scalar value between 0.0 (black) and 1.0 (white), implying that the surface area facing the light source is brighter than the unfaced area. is doing.

と、

を、ｕおよびｖ方向それぞれにおける離散第１導関数に対して使用することに基づいている。ｕ方向における第２導関数は、

と定義される。同様に、

と、

であり、ここで、

である。 4.2.6 Second Derivative The discrete second derivative of the surface with height field H is an alternative formula

When,

Is used for discrete first derivatives in the u and v directions, respectively. The second derivative in the u direction is

Is defined. Similarly,

When,

And where

It is.

として定義される。 4.2.7 Curvature The discrete curvature of a curved surface with a height field H is a non-negative scalar.

Is defined as

ここで、λは平滑係数と称される正の定数であり、Γは、投影表面Ｓ上で定義される高さ増分の集合である式として定義される。

に対して、Γ（ｕ，ｖ）＝０．０の条件が課せられ、それにより、モデルビュー上へのその投影の境界における、またはモデルビューのエッジにおける表面上のアーティファクトが回避される。 4.2.8 Discrete function to be minimized A truncated version of the function (4.01) can be created using the definition of discrete strength and curvature. Since these equations depend on neighboring pixel values, only pixels that are both inside the model view and inside the projection surface are considered to be included in the region of interest. The discrete version of the function (4.01) is

Here, λ is a positive constant called a smoothing coefficient, and Γ is defined as an expression that is a set of height increments defined on the projection surface S.

Is subject to the condition Γ (u, v) = 0.0, thereby avoiding artifacts on the surface of the projection on the model view or at the edges of the model view.

におけるすべてのピクセルではなく、ユーザーにより強度が修正されたピクセルの近傍におけるピクセルにわたる合計を含むことのみが必要であることが示される。更に、関数の規模は、提案された高さ増分を、よりコンパクトな方法で表現することにより縮小されることが示される。 4.3 Reduction of function to be minimized In this section, function (4.10) is

It is shown that it is only necessary to include the sum over the pixels in the vicinity of the pixels whose intensity has been modified by the user, rather than all the pixels in. Furthermore, it is shown that the function scale is reduced by expressing the proposed height increment in a more compact way.

を、ユーザーが強度を修正したモデルビューにおけるピクセルの集合とする。Ｐは「修正ピクセル」の集合と称され、近傍情報が強度を計算するために必要なので、

が仮定されている。 4.3.1 Corrected pixels

Let be the set of pixels in the model view whose intensity has been modified by the user. P is referred to as a set of “corrected pixels”, and neighborhood information is needed to calculate the intensity, so

Is assumed.

として定義され、それにより、

となる。集合Ａは、

ならば集合Ｂの接続成分であり、各

に対して、Ａに完全に含まれる（ｕ_０，ｖ_０）から（ｕ_１，ｖ_１）のパスがある。ビューコードは、高さ増分ΓはＳ＼Ｐの接続成分上では定数であることを仮定している。表面アーティファクトを回避するために、この定数は、接続成分が、

と交差するならば０．０と仮定されている。 4.3.2 View codes View codes support the evolution of reduction in computation. The view code sorts the pixels by the proposed height increment. The notation of the set connected to the path is necessary to construct the view code. The path from pixel (u ₀ , v ₀ ) to pixel (u _j , v _j ) is an arbitrary sequence of pixels

Is defined as

It becomes. Set A is

Is the connected component of set B, and

On the other hand, there are paths (u ₀ , v ₀ ) to (u ₁ , v ₁ ) that are completely included in A. The view code assumes that the height increment Γ is a constant on the connected component of S \ P. In order to avoid surface artifacts, this constant is

Is assumed to cross 0.0.

と交差するすべての接続成分の和集合とし、０≦ｉ＜ｎに対するＴ_ｉを、Ｓ＼Ｐの接続成分で、それを適切に含むものがないという意味で最大である接続成分とする。そうすると、増分の集合Γは、多くとも（ｍ＋ｎ）の固有非ゼロ値を、修正ピクセルに対して１つ、各Ｔ_ｉに対して１つ有し、それによりｉ≠−１である。 _Let T = (T ₋₁ , T ₀ , T ₁ ,..., T _n−1 ) be a division of S \ P, and T ₋₁

Let T _i for 0 ≦ i <n be the largest connected component in the sense that none of the connected components of S \ P is appropriate. Then, the incremental set Γ has at most (m + n) unique non-zero values, one for the modified pixel and one for each T _i , so that i ≠ -1.

と定義される。これは、例えば、０≦Ｖ（ｕ_０，ｖ_０）＜ｍであれば、（ｕ_０，ｖ_０）は修正ピクセルであることが分かる。また、Ｖ（ｕ_０，ｖ_０）＝−１であれば、ピクセルＶ（ｕ_０，ｖ_０）は２Ｄシェーディング情報の結果として変更されない。これは、ビューコードが割り当てられていないピクセル、つまり、モデルビュー上への表面の投影上にないピクセルについても当てはまる。 "View code"

Is defined. For example, if 0 ≦ V (u ₀ , v ₀ ) <m, it can be seen that (u ₀ , v ₀ ) is a modified pixel. If V (u ₀ , v ₀ ) = − 1, the pixel V (u ₀ , v ₀ ) is not changed as a result of the 2D shading information. This is also true for pixels that have not been assigned a view code, that is, pixels that are not on the projection of the surface onto the model view.

が成り立つ可能なピクセルは、

に含まれる。下記の

ここで、ｑ_ｉはモデルビューにおけるピクセル（ｕ_ｉ，ｖ_ｉ）を表わす式を仮定する。Ｑは「関数ピクセル」の集合と称され、関数（４．１０）は、ここでは、

と縮小され、ここでＫは、集合

上の

の合計であり、定数である。 4.3.3 First Reduction in Function Pixel and Calculation Even if the height of the cross section of the surface is moved by a certain amount, the intensity or curvature inside the cross section does not change. Since the pixels at each T _i are moved by a fixed amount, and T is a split of S \ P,

The possible pixels for which

include. below

Here, q _i is assumed to be an expression representing a pixel (u _i , v _i ) in the model view. Q is referred to as the set of “function pixels” and the function (4.10) is here:

Where K is the set

upper

Is a constant.

ここで、ｘ_ｉはＶ（ｕ，ｖ）＝ｉとなるような、すべてのピクセルに対するΓの増分値であるとする。ｘは「増分ベクトル」と称される。

に対して、

とする。すると、関数（４．１０）は、

の最小化に縮小される。 4.3.4 Increment vector and second reduction in computation Since Γ has a maximum (m + n) intrinsic non-zero height increments, the function to be minimized is determined by the view code as It only needs to be a function of the increment. below

Here, x _i is an increment value of Γ for all pixels such that V (u, v) = i. x is referred to as an “incremental vector”.

Against

And Then the function (4.10) is

Reduced to minimization.

ここで最小化はまた、非制約であってもよい。関数（４．１７）は、整形アルゴリズムにより最小化される離散関数の最終バージョンである。この関数は「目的関数」と称され、その大きさは（ｍ＋ｎ）である。 4.3.5 Objective Function The function (4.13) and the function (4.16) can be combined to form the following function, and when the function is minimized, the function (4.10) Is equivalent to minimizing.

Here the minimization may also be unconstrained. Function (4.17) is the final version of the discrete function that is minimized by the shaping algorithm. This function is called an “objective function”, and its size is (m + n).

4.4 Trust Region Newton-CG Method 4.4.1 Overview SBS uses the trust region Newton-CG method to minimize the objective function (function (4.17)). The main idea of the trust region method is to construct a quadratic model for the function, minimize the model function within the “trust region”, adjust the trust region according to a certain criterion, and model function within the new trust region , Adjust the area again, minimize it again, and so on. Given certain restrictions, such a method is guaranteed to converge to a point corresponding to the local minimum of the original function.

は、

であり、ここで、

は勾配ベクトル

であり、

はヘッシアン行列

である。信頼領域ニュートン−ＣＧという名称における「ニュートン」という名前は、ヘッシアン行列がモデルで使用されるという事実から来ている。他の、普通は正の定符号行列を代わりに使用でき、その場合は、ニュートンは名前から落ちる。ＳＢＳの場合、ヘッシアン行列の使用は便利であり、より弱い収束条件の使用が可能になる。 Trust region model of objective function centralized in incremental vectors

Is

And where

Is the gradient vector

And

Is a Hessian matrix

It is. The name “Newton” in the name of trust region Newton-CG comes from the fact that the Hessian matrix is used in the model. Other, normally positive constant code matrices can be used instead, in which case Newton falls from the name. For SBS, the use of the Hessian matrix is convenient and allows the use of weaker convergence conditions.

が十分に小さい場合、つまり、ある収束閾値よりも小さい場合は、極小値オフ（off）がｘ^０において起こり、処理は終了したと結論される。そうでない場合は、モデルＦ_０は、ある正数Δ_０に対する円形信頼領域

において、下記に説明するCG-Steihaug法を介して最小化される。ｐ^０を、信頼領域においてモデルの最小化が起きた結果としての増分ベクトルとする。

が、収束閾値よりも小さい場合は、ｆの極小値がｘ^０＋ｐ^０で起こり、処理は終了したと結論される。それ以外では、テスト値

を計算する。ρ_０が閾値を超えると、実際の縮小と予想される縮小は、お互いに若干近くなり、信頼領域の中心は、ｘ^１＝ｘ^０＋ｐ^０に移動される。それ以外では、信頼領域の中心はｘ^１＝ｘ^０に留まる。ρ_０が１に近ければ、Ｆ_０は信頼領域内においてｆに対する良好なモデルと考えられ、半径は増大する。ρ_０が１から離れれば、信頼領域の半径は減少する。新しい領域の半径はΔ_１と表示され、ｘ^１に集中化された新しいモデルＦ_１が構築され、最小化処理が繰り返される。

は０との近さをテストされる。十分に近くない場合は、処理は、

が収束閾値以下になるまで再び繰り返される。ｆの極小値は、ｘ^ｉ＋ｐ^ｉにおいて求められる。モデルをどのように計算するか（特に、ｆ、

、

をどのように求めるか）についての詳細は５節で、CG-Steihaug法に必要な計算は６節で、そして、

の収束についての検討は７節で行う。

If sufficiently small, that is, if less than a certain convergence threshold, minimum value off (off) occurs at x ^0, the process is concluded to be completed. Otherwise, the model F ₀ is a circular confidence region for some positive number Δ ₀

Is minimized via the CG-Steihaug method described below. Let p ⁰ be the incremental vector as a result of model minimization in the confidence region.

However, if it is smaller than the convergence threshold, the minimum value of f occurs at x ⁰ + p ⁰ , and it is concluded that the process is finished. Otherwise, test value

Calculate When ρ ₀ exceeds the threshold, the actual reduction and the expected reduction are slightly closer to each other, and the center of the trust region is moved to x ¹ = x ⁰ + p ⁰ . Otherwise, the center of the trust region remains at x ¹ = x ⁰ . If ρ ₀ is close to 1, F ₀ is considered a good model for f in the confidence region and the radius increases. If ρ ₀ is away from 1, the radius of the trust region decreases. The radius of the new region appears as delta _1, a new model F ₁ which is centralized in x ¹ is built, minimization processing is repeated.

Is tested for proximity to zero. If not close enough, the process

Is repeated again until becomes below the convergence threshold. minimum value of f ^is determined in x ⁱ + ^p i. How to calculate the model (especially f,

,

For details on how to find the CG-Steihaug method in Section 6, and

A discussion of convergence is given in Section 7.

4.4.2 Pseudocode FIG. 14 shows a list of pseudocodes for the described technique.

、および

に対する式を求めるために残差が使用される。 4.5 Computation of the trust region model The goal of this section is to build a tool that supports the computation of the function (4.18), a quadratic model used for the trust region method. In particular, f (x),

,and

The residual is used to find the expression for.

であることを思い起こして、関数（４．１８）の残差を０≦ｉ＜ｔに対して、

のように定義する。関数（４．１８）の残差ベクトルを、次のように第１、第２、第３、および第４種の残差すべての結合と定義する。

ここで、残差ベクトルのノルムの平方は、強度と曲率に分解される。

その結果、

となる。実際、残差は、上記の式が真であるように選択された。ｆの勾配とヘッシアンに対する式もまた残差から導出できる。

とすると、

となる。その結果、

である。同様な導出が、ヘッシアンに対する下記の式という結果になり、

ここで、

は、以前に定義したように残差勾配ベクトルであり、

は残差ヘッシアン、

である。簡単な式が下記の

と、

のそれぞれに対して、残差式を分解し、その導関数を取ることにより導出される。ビューコードは計算を簡略化するために使用される。結果としての式は、これらの２つの値が、既知の値の小さな集合上を除いて０でなければならないことを示している。この情報は、

と、

とを求め、その結果、ｆ（ｘ）、

、および、

を求めるために使用できる。その最終的な結果は、信頼領域モデル関数（関数（４．１８））を計算する簡単な方法である。 4.5.1 Using residuals to obtain functions, gradients, and Hessians

Recall that the residual of the function (4.18) is 0 ≦ i <t,

Define as follows. The residual vector of function (4.18) is defined as the combination of all residuals of the first, second, third and fourth types as follows:

Here, the square of the norm of the residual vector is decomposed into intensity and curvature.

as a result,

It becomes. In fact, the residual was chosen so that the above equation is true. Equations for the slope of f and Hessian can also be derived from the residual.

Then,

It becomes. as a result,

It is. A similar derivation results in the following equation for Hessian:

here,

Is the residual gradient vector as defined previously,

Is the residual Hessian,

It is. The simple formula is

When,

Is derived by decomposing the residual equation and taking its derivative. View codes are used to simplify calculations. The resulting equation shows that these two values must be zero except on a small set of known values. This information

When,

And as a result, f (x),

,and,

Can be used to The final result is a simple method of calculating the trust region model function (function (4.18)).

と書くことで使用できる。偏微分をするために商法則を使用すると、

と、

という結果になる。式（４．３１）と式（４．３２）との主要部分は、より処理し易い形式に分解できる。

の定義に鎖法則を使用することができ、

が得られ、

の偏微分を計算することにより、

という結果になる。式（４．３３）と式（４．３４）とは、Ｄ_１Ｈ_ｘ（ｑ_ｉ）の偏微分とＤ_１Ｈ_ｘ（ｑ_ｉ）に対する式を求めることにより更に分解できる。下記の

ここで、Ｖはビューコードである式を呼び出す。ｕ方向における離散第１導関数の定義により、下記の

ここで、δ_１はラスター点（１，０）である式が与えられ、これは、

を意味する。同様に、下記の

ここで、δ_２はラスター点（０，１）である式が与えられる。ビューコードにより、

が与えられる。ここで、第１種の残差に対する第１および第２偏微分の式が可能な限り分解されたので、今度は、その微小部分を使用して、鎖を遡って計算する。式（４．３８）を式（４．３６）と式（４．３７）とに代入して、

と、

とを計算する。そしてこれらを、式（４．３３）と式（４．３４）とに代入して、

と、

とを計算する。最後に、結果を式（４．３１）と式（４．３２）とに代入して所望するように、

と、

とを計算する。特に、

および、

であることに留意されたい。 4.5.2 Type 1 residual The equation for intensity is the type 1 residual for 0 ≦ i <t.

It can be used by writing Using the commercial law to do partial differentiation,

When,

Result. The main parts of equations (4.31) and (4.32) can be broken down into a more manageable form.

The chain law can be used to define

Is obtained,

By calculating the partial derivative of

Result. And formula (4.34) Equation _(4.33), can be further decomposed by obtaining the expression for D ₁ H x _{(q i)} of the partial differentiation and _{D 1 H x (q i)} . below

Here, V calls an expression that is a view code. By the definition of the discrete first derivative in the u direction,

Where δ ₁ is given the equation that is the raster point (1,0),

Means. Similarly,

Here, an expression is given in which δ ₂ is a raster point (0, 1). View code

Is given. Now that the first and second partial differential equations for the first type of residual have been resolved as much as possible, this small part is used to calculate the chain retrospectively. Substituting Equation (4.38) into Equation (4.36) and Equation (4.37),

When,

And calculate. Substituting these into the equations (4.33) and (4.34),

When,

And calculate. Finally, as desired by substituting the results into equations (4.31) and (4.32),

When,

And calculate. In particular,

and,

Please note that.

であり、それにより、

が与えられる。上記の計算に対して式（４．３８）を使用して、

および、

であることに留意されたい。第３種の残差は、０≦ｉ＜ｔに対して、

であり、

が与えられる。上記の計算に対して式（４．３８）を使用して、

および、

であることに留意されたい。第４種の残差は、０≦ｉ＜ｔに対して、

であり、ここで、

であり、

が与えられる。上記の計算に対して式（４．３８）を使用して、

および、

であることに留意されたい。第１、第２、第３、および第４の残差に対して、つまり、０≦ｉ＜４ｔに対して、

および、

であることに留意されたい。 4.5.3 Second, Third, and Fourth Type Residues The second type residuals are for 0 ≦ i <t,

And thereby

Is given. Using equation (4.38) for the above calculation,

and,

Please note that. The third type residual is 0 ≦ i <t,

And

Is given. Using equation (4.38) for the above calculation,

and,

Please note that. The fourth type of residual is 0≤i <t,

And where

And

Is given. Using equation (4.38) for the above calculation,

and,

Please note that. For the first, second, third and fourth residuals, ie for 0 ≦ i <4t,

and,

Please note that.

これにより問題は、φ（ｘ）の最小化という問題になる。ステップｉにおける、ｆに対する信頼領域モデルＦ_ｉを、

として呼び出す。φ（ｘ）、Ａ、およびｂは、信頼領域法の立場から、下記の

および、

と表現できる。Ａは実際には対称であることに留意されたい。上記の観点から、CG-Steihaug法により最小化される関数は、

であり、ＣＧ法は、共役勾配（ＣＧ）の表記を使用して、φ（ｘ）の最小値に収束するベクトル列を構築する。所与のｉに対して、ベクトル集合

は、すべてのｋ≠１に対して、

ならば、行列Ａに関して共役（ＣＧにおける「Ｃ」）と称される。信頼領域法のステップｉにおいてそのような集合Ｄが与えられると、ｐ^ｉ，ｊ＝０とｊ≧０に対して下記の

ここで、ａ_ｉ，ｊは、

に沿う、φ（ｘ）の一次元最小化因子である数列ｐ^ｉ，ｕを定義する。数列ｐ^ｉ，ｊは、最大（ｍ＋ｎ）（以前のように、（ｍ＋ｎ）が目的関数の大きさの場合）ステップで、所望の最小値ｐ^ｉに収束する。ここでの目標は、どのようにして集合Ｄを構築し、各ｊ≧０に対してａ_ｉ，ｊをどのように求めるかである。 4.6 Minimization of the trust region model: CG-Steihaug method 4.6.1 Overview At each step in the trust region Newton-CG method, the SBS uses the CG-Steihaug method to minimize the model function in the trust region. Find the value. The conjugate gradient method tries to solve a linear simultaneous equation Ax = −b, where A is a symmetric matrix. The problem can be reformulated as follows:

This causes a problem of minimizing φ (x). The trust region model F _i for f in step i is

Call as. φ (x), A, and b are the following from the standpoint of the trust region method:

and,

Can be expressed as Note that A is actually symmetric. From the above viewpoint, the function minimized by the CG-Steihaug method is

The CG method uses a conjugate gradient (CG) notation to construct a vector sequence that converges to the minimum value of φ (x). Vector set for a given i

For all k ≠ 1

Then, it is called conjugate (“C” in CG) with respect to the matrix A. Given such a set D in step i of the trust region method, for p ^{i, j} = 0 and j ≧ 0,

Where a _{i, j} is

A number sequence p ^{i, u} that is a one-dimensional minimization factor of φ (x) is defined. The sequence p ^{i, j} converges to the desired minimum value p ⁱ in steps of maximum (m + n) (as before, where (m + n) is the magnitude of the objective function). The goal here is how to build the set D and how to find a _{i, j} for each j ≧ 0.

とする。言い換えれば、最小値を探す第１方向を、最も急峻な降下方向として選択する。これは、勾配（ＣＧにおける「Ｇ」）により決定される方向である。系の残差はｒ（ｘ）＝Ａｘ＋ｂとして定義され、次の探す方向を決定する各ステップにおいて使用される。ｐ^ｉ，０＝０なので、ステップ０における残差は、ｊ≧０に対してｒ^ｉ，０＝ｂであり、

である。ｊ≧０に対して、ｄ^{ｉ，ｊ＋１}を、

ここで、

であるように定義する。β_{ｉ，ｊ＋１}に対するそのような選択は、すべてのｋ＜ｊ＋１に対して、

となるようなｄ^{ｉ，ｊ＋１}の結果になり、それにより、集合Ｄが各ｊに対して共役であることが保証される。

And In other words, the first direction in which the minimum value is searched is selected as the steepest descent direction. This is the direction determined by the gradient (“G” in CG). The system residual is defined as r (x) = Ax + b and is used in each step of determining the next search direction. Since p ^{i, 0} = 0, the residual in step 0 is r ^{i, 0} = b for j ≧ 0,

It is. For j ≧ 0, let d ^{i, j + 1} be

here,

To be defined. Such a choice for β _{i, j + 1} is for all k <j + 1

Resulting in d ^{i, j + 1} such that the set D is conjugate to each j.

ここで、Ｋはαに関して定数である。次に、αについての導関数を求める。

上記の量を０に等しいと置いて、臨界点

を求める。臨界点が最小値に対応することを確実にするために、第２導関数を計算して、関数の凹性を求める。

上記の量が正かどうかは分からない。

の場合も後で扱うが、導関数は、

であることを仮定し続けている。α_ｉ，ｊに対する式は、各ｊ＞０に対して、

という事実を使用して、より良好な方法で書くことができ、ｋ＜ｊのすべてに対しては、帰納法によりある事実が証明される。ｊ＝１に対しては、

である。ここでｋ＜ｊ−１に対して、

を仮定する。すると、ｋ＜ｊ−１に対して、

であり、ここで、帰納法仮説により、

であり、Ｄの共役性により、

である。ここでｋ＝ｊ−１の場合を扱うと、

である。従って、各ｊ＞０に対して、

がｋ＜ｊのすべてに対して成り立つ。ここで、α_ｉ，ｊに対する新しい式は、

と表わされる。要約すると、CG-SteihaugＳ法は、信頼領域ニュートン−ＣＧ法においてステップｉにおいて使用され、信頼領域において、モデル関数の最小値が起こる点を求める。これを行うために、３つの数列が使用される。それは、ｄ^ｉ，ｊ、ｒ^ｉ，ｊ、およびｐ^ｉ，ｊである。ｄ^ｉ，ｊは、共役勾配方向ベクトルの数列であり、ｒ^ｉ，ｊは残差の数列であり、ｐ^ｉ，ｊは、所望の最小値ｐ^ｉに収束する。ｄ^ｉ，ｊの共役性は、最大（ｍ＋ｎ）ステップでの収束を保証する。 In order to calculate the minimizing factor α _{i, j} , terms are multiplied together and arranged according to the power of α to obtain an equation along φp ^{i, j} + α ^{di, j} .

Here, K is a constant with respect to α. Next, a derivative with respect to α is obtained.

Put the above quantity equal to 0, the critical point

Ask for. In order to ensure that the critical point corresponds to the minimum value, the second derivative is calculated to determine the concave nature of the function.

I don't know if the above amount is positive.

Will be dealt with later, but the derivative is

We continue to assume that. The formula for α _{i, j} is for each j> 0:

Can be written in a better way, and for all k <j, an induction proves a fact. For j = 1

It is. Where k <j−1

Assuming Then, for k <j-1,

Where, by induction hypothesis,

And due to the conjugation of D,

It is. Here, when the case of k = j−1 is handled,

It is. Therefore, for each j> 0

Holds for all k <j. Where the new expression for α _{i, j} is

It is expressed as In summary, the CG-SteihaugS method is used in step i in the trust region Newton-CG method to find the point where the minimum value of the model function occurs in the trust region. Three sequences are used to do this. It is d ^{i, j} , r ^{i, j} and p ^{i, j} . d ^{i, j} is a sequence of conjugate gradient direction vectors, r ^{i, j} is a sequence of residuals, and p ^{i, j} converges to a desired minimum value p ⁱ . The conjugacy of d ^{i, j} ensures convergence at maximum (m + n) steps.

と、

とを使用して与えられ、

および、

である。この時点までは、説明された方法は標準ＣＧ法である。数列ｐ^ｉ，ｊは、残差がある閾値以下になるまで生成される。ＣＧ法のSteihaug変形は、

の場合を考慮しており、これは計算されたα_ｉ，ｊが最小値に対応するという仮定に反することになり、モデルの最小値は、関心対象、つまり所与の信頼領域の外側で求まる。これらの場合を扱うために、２つの追加的停止基準が追加される。

または、

であるならば、信頼境界と方向ｄ^ｉ，ｊの交差点は、最終点ｐ^ｉとして割り当てられる。ｐ^ｉ，０＝０であるならば、各ｊ＞０に対して、

であり、信頼領域境界との交差点の解を返すことが、境界に到達したときに数列ができる最善のことであることを意味している。そして、信頼領域法は、ｐ^ｉを使用してテスト値を計算する。この結果に依存して、解は十分良好に見えるか、または信頼領域法は、信頼領域ｘ^ｉ＋１の中心が、ｘ^ｉと同じか、ｘ^ｉ＋ｐ^ｉに移動されて、反復ｉ＋１に入るかのいずれかである。 The starting quantities of the sequence are p ^{i, 0} = 0, r ^{i, 0} = − ▽ f (x ⁱ ) = b, and d ^{i, 0} = − ▽ f (x ⁱ ) = b. Subsequent terms in the sequence are helper expressions

When,

And given using

and,

It is. Up to this point, the described method is a standard CG method. The sequence p ^{i, j} is generated until the residual falls below a certain threshold. The Steihaug variant of the CG method is

This is contrary to the assumption that the calculated α _{i, j} corresponds to the minimum value, and the minimum value of the model is found outside the object of interest, ie the given confidence region. . To handle these cases, two additional stop criteria are added.

Or

The intersection of the confidence boundary and the direction d ^{i, j} is assigned as the final point p ⁱ . If p ^{i, 0} = 0, for each j> 0,

And returning the solution at the intersection with the trust region boundary means that the sequence is the best that can be done when the boundary is reached. The trust-region calculates a test value using p ^i. The result depends, or solutions look good enough, or trust region, the center of the trust region x ^{i + 1} is equal to or x ^i, is moved to x i ⁺ p ^i, if entering the iteration i + 1 Either.

4.6.2 Pseudocode In the following pseudocode, the sequence of points p ^{i, j} is represented as min_pt, the sequence of directions is represented by d ^{i, j} by the direction, and the sequence of residuals r ^{i, j} Is represented by the residual. “A dot b” is used to represent a ^T b.

ここで、各ｉに対して、

とする。（ａ_ｉ）_ｊ，ｋをＡ_ｉの（ｊ，ｋ）要素、ｅ^ｉを長さ（ｍ＋ｎ）のベクトルとし、

が成り立つとする。ｄは信頼領域法の固定ステップにおいて、CG-Steihaug法により構築された共役方向ベクトルの１つとする。すると、

となり、（ｊ，ｋ）の各対に対して１回、つまり、（ｍ＋ｎ）^２回の乗算を必要とする。しかし計算は、式（４．５３）と式（４．５４）とを使用することにより、より効率的に行うことができる。これらの式は、各ｉに対して、（ａ_ｉ）_ｊ，ｋは、ｊとｋが共に、

に含まれるときに限って非ゼロであることを示している。従って、それぞれのｉとｋに対して、

となり、これはＯ（１）乗算において計算できる。従って、全体の規模は、（ｍ＋ｎ）^２から（ｍ＋ｎ）に減少される。 4.6.3 Sparse matrix multiplication
In each iteration of the CG-Steihaug method, a much more laborious operation is multiplication with the A direction vector. Call the following expression for A in terms of the residual of the objective function:

Where for each i

And (A _i ) Let _{j, k be} a (j, k) element of A _i , e ^{i be} a vector of length (m + n),

Suppose that d is one of conjugate direction vectors constructed by the CG-Steihaug method in the fixed step of the trust region method. Then

Thus, one multiplication is required for each pair of (j, k), that is, (m + n) ² times. However, the calculation can be performed more efficiently by using the equations (4.53) and (4.54). These equations are as follows: for each i, (a _i ) _{j, k}

This indicates that it is non-zero only when it is included in. Therefore, for each i and k,

Which can be calculated in O (1) multiplication. Thus, the overall scale is reduced from (m + n) ² to (m + n).

を、ｉ→∞のとき、下記の条件が成立すれば満たす。
・信頼領域法の各ステップｉにおいて、最小化アルゴリズムにより構築されたｐ^ｉは、ある定数

に対して、

を満たす。
・最小化アルゴリズムにより構築された数列ｐ^ｉは、ある定数ｃ_２≧１に対して、

を満たす。
・信頼領域の中心を移動する閾値は、区間（０，１／４）に含まれる。
・ｆは、レベル集合

、つまり、ある定数

に対して、

が、ｆ（ｘ）≦ｆ（ｘ^０）となるようにすべてのｘに対して成り立つ集合に対して、下方から境界を区切られる。
・Ｌは、境界を区切られる、つまり、ある定数

に対して、

となる。
・すべてのｉ≧０に対して、

となるようなある定数

が存在する。
・ｆはＬ上でリプシッツ連続微分可能、つまり、任意の

に対して、

となるようなある定数

が存在する。上記に主張された条件のそれぞれを下記で証明する。 4.7 Convergence of Trust Region Newton-CG Method The sequence x ⁱ constructed by the trust region method is

If i → ∞, the following condition is satisfied.
In each step i of the trust region method, p ⁱ constructed by the minimization algorithm is a constant

Against

Meet.
The sequence p ⁱ constructed by the minimization algorithm is for some constant c ₂ ≧ 1

Meet.
The threshold value for moving the center of the trust region is included in the section (0, 1/4).
・ F is a level set

That is, a constant

Against

Is bounded from below for the set that holds for all x such that f (x) ≦ f (x ⁰ ).
L is bounded, ie a constant

Against

It becomes.
・ For all i ≧ 0

A constant such that

Exists.
F is Lipschitz continuously differentiable on L, that is, any

Against

A constant such that

Exists. Each of the above asserted conditions is proved below.

を求める。ステップｉにおいて、コーシー点（ｐ^ｃ）^ｉは、

として定義され、ここで、

が成り立つ。その結果、

となる。CG-Steihaug法により、

が与えられる。その結果、

となる。 4.7.1 Model estimation The SBS uses the CG-Steihaug method at each step i to correspond to the minimum value of the model F _i in the trust region.

Ask for. In step i, the Cauchy point (p ^c ) ⁱ is

Where, where

Holds. as a result,

It becomes. By CG-Steihaug method,

Is given. as a result,

It becomes.

である。 4.7.2 Model Minimum Boundary For any j, if pi ^{, j} computed by the CG-Steihaug algorithm is outside the trust region, it is replaced with an internal vector and the algorithm Returns a value,

It is.

に設定される。 4.7.3 Trust Region Center Movement Threshold Range To force the range for the threshold and move the center of the trust region is to set it appropriately. In the pseudo code, the movement threshold is

Set to

を呼び出す。定義により、各ｑ_ｉに対して、

であり、選択によりλ＞０である。従って、すべての

に対してｆ（ｘ）≧０であり、従って、ｘに対してもｆ（ｘ）≦ｆ（ｘ^０）となる。 4.7.4 Lower boundary of f in level set

Call. By definition, for each q _i

And λ> 0 by selection. Therefore, all

Therefore, f (x) ≧ 0, so that f (x) ≦ f (x ⁰ ) even for x.

および、

ここで、δ_１＝（１，０）とする。ｈを

で定義し、ｖを

で定義し、すべてのｘに対してｈ（ｘ）≦ｆ（ｘ）であり、従って、

となるようなｃ_４＞０が存在し、ｔ＞ｃ_４が、ｈ（ｔｘ）＞ｆ（ｘ^０）を暗示するので、

であり、ｔ＞ｃ_４がｆ（ｔｘ）＞ｆ（ｘ^０）を暗示し、ｆ（ｔｘ）≦ｆ（ｘ^０）が、

およびｔ≦ｃ_４、つまり、所望するように、

であることを暗示していることを意味する。 4.7.5 Level set boundaries

and,

Here, δ ₁ = (1, 0). h

And define v as

H (x) ≦ f (x) for all x, so

C ₄ > 0 such that and t> c ₄ implies h (tx)> f (x ⁰ )

T> c ₄ implies f (tx)> f (x ⁰ ), and f (tx) ≦ f (x ⁰ )

And t ≦ c ₄ , that is, as desired,

Means that it is.

ここで、ｋ_ｊはｓに関しての定数であり、それは、

ここで、ｋはｓに関しての定数である式を暗示する。任意のスカラーｔ＞０に対して、ｈ_ｘ（ｔ）＝ｈ（ｔｘ）とする。すると、

となり、ｈは平方の和であるからｈ≧０である。増分ベクトルｔｘが与えられれば、

がすべてに対して成り立つ場合のみｈ（ｔｘ）＝０となる。これは、増分ベクトルは、投影表面の境界上では０であることが要求されるので、ｔｘがゼロベクトルであることを暗示している。ｕ方向における投影表面を通る任意のピクセル線は、境界上では０でなければならず、

なので、０から変化できない。

およびｔ＞０なので、ｈｘ（ｔ）＞０であり、これは、

を暗示している。ここで、

および、

とする。すると、任意のｔ＞ｃ_４および

に対して、

であり、これは、

を暗示している。 For any scalar s> 0 and each j,

Where k _j is a constant with respect to s, which is

Here, k implies an expression that is a constant with respect to s. For any scalar t> 0, _let h _x (t) = h (tx). Then

Since h is the sum of squares, h ≧ 0. Given an increment vector tx,

H (tx) = 0 if only holds for all. This implies that tx is a zero vector because the incremental vector is required to be zero on the projection surface boundary. Any pixel line that passes through the projection surface in the u direction must be zero on the boundary,

So it cannot change from zero.

And t> 0, so hx (t)> 0,

Is implied. here,

and,

And Then any t> c ₄ and

Against

And this is

Is implied.

が信頼領域の中心を移動する閾値よりも大きい場合のみ容認される。閾値は（０，１／４）において選択され、従って、信頼領域の中心が移動されると、ρ_ｉ＞０となる。CG-Steihaug法は信頼領域におけるモデルを最小化するので、

が保証され、それはρ_ｉの分母が非負であることを暗示している。ρ_ｉ≧０ならば、

である。従って、すべてのｉ≧０に対して、

であり、それはＬがすべてのｘ^ｉを含むことを意味している。主張された条件がＬ上で証明されると、それはすべてのｘ^ｉに対しても証明される。ｆとその偏微分のすべては連続である。コンパクト集合［０，ｃ_４］（ｃ_４は、上記に計算されたＬ上の境界である）上では、ｆとその導関数のすべては境界が区切られている。各ｉとｊ≧０、および＜（ｍ＋ｎ）に対して、

がすべてのｘに対して、

となるように成り立ち、従って、

となるような定数ｋ_ｉ，ｊが存在する。 4.7.6 Hessian boundary In the trust region Newton-CG method, there are two possibilities for x ^{i + 1} . A ^{x i + 1} = x ^i, if its is ^{f (x i + 1) =} f (x i), or a ^{x i + 1 = x i +} p i, p i wherein the point resulting from the CG-Steihaug method It is. In the latter case, the model test value

Is allowed only if is greater than the threshold for moving through the center of the confidence region. The threshold is selected at (0, 1/4), so ρ _i > 0 when the center of the confidence region is moved. Since the CG-Steihaug method minimizes the model in the trust region,

Is guaranteed, which implies that the denominator of ρ _i is non-negative. If ρ _i ≧ 0,

It is. Therefore, for all i ≧ 0,

Which means that L contains all x ⁱ . If the asserted condition is proved on L, it is also proved for all x ⁱ . f and all of its partial derivatives are continuous. On the compact set [0, c ₄ ] (c ₄ is the boundary on L calculated above), f and all of its derivatives are bounded. For each i and j ≧ 0 and <(m + n),

For all x

Therefore, therefore,

There are constants k _{i, j} such that

とする。ｉを固定して、

および、

とする。すると、

となる。これは、

が各ｉに対してリプシッツ連続であることを示し、それは、

であることを示している。 4.7.7 Lipschitz continuous differentiability of f k _{i, j} as in the previous section,

And i is fixed,

and,

And Then

It becomes. this is,

Is Lipschitz continuous for each i, which is

It is shown that.

4.8 Notation FIGS. 16A and 16B are tables 500a and 500b showing a list of mathematical notations used to describe systems and techniques according to aspects of the present invention for convenience of reference. Is described.

V. Generalized Method Flowcharts FIGS. 17-22 represent a geometric model representing the geometry of at least a portion of the surface of a three-dimensional (3d) object, representing the object projected onto the image plane. A generalized method 600 and auxiliary methods 620, 640, 660, 680, generated by operator shading in the context of a two-dimensional (2d) image of the object, according to the above-discussed aspects of the invention, and auxiliary methods 620, 640, 660, 680, And a series of flow charts illustrating 700.

  The generalized method 600 shown in FIG. 17 includes the following steps.

  Step 601: Receive shading information provided by an operator representing a change in lightness level of at least a portion of an image in the context of an image of an object.

  Step 602: Generate an updated geometric model of the object in response to the shading information used to determine at least one geometric feature of the updated geometric model.

  Step 603: Display an image of the object as defined by the updated geometric model.

  As discussed above, the generalized method 600 can operate based on the digital input of any hierarchical subdivision surface, polygon mesh, or NURBS surface.

  The generalized method 600 includes the auxiliary method 620 shown in FIG. 18, and the auxiliary method 620 includes the following steps.

  Step 621: Once the subdivision surface is generated and displayed to the user, the subdivision surface is obtained with information about the corners of the grid, the width and height of the grid, the pixel size, and the camera-to-object conversion. Consistent with the containing 2d model view.

  Step 622: Utilizing the 2d model view, set the lighting direction, adjust input parameters and shades, thereby modifying the intensity of the selected pixels or loading a set of pre-shaded pixels .

  This information is used by the shaping algorithm. The parameters can include either an effect on how much subdivision occurs in the correction region (a) and an effect on how noticeable the geometric correction is (b).

  Step 623: Determine the exact geometric changes to the surface, add surface primitives if necessary, via subdivision, in the area of shading to ensure that there is enough detail 3D Result in a hierarchical subdivision surface shaped by determining a height field reflecting the requested change in 2D settings and changing the subdivision surface to reflect the determined height value This subdivision surface can be further modified, stored, or converted to the desired output surface type. The surface basic shape can include either a triangle, a quadrangle, or another polygon.

  The generalized method 600 includes the auxiliary method 640 shown in FIG. 19, and the auxiliary method 640 includes the following steps.

  Step 641: Create a subdivision curved surface as a base.

  Step 642: Display a 2D shade view.

  Step 643: Allow the user to set lighting and shading and adjust parameters.

  Step 644: (a) introducing details on the surface, (b) determining a new height parameter for the surface, and (c) performing a shaping process including shaping the subdivision surface, thereby Generate a 3D subdivision curved surface. The parameters can include either (a) the effect on how much subdivision occurs in the correction region and (b) the effect on how noticeable the geometric correction is.

  The generalized method 600 also includes the auxiliary method 660 shown in FIG. 20, which includes the following steps.

  Step 661: Receive input including either a mesh representation or a NURBS surface.

  Step 662: If not yet a hierarchical subdivision surface, convert the input to a hierarchical subdivision surface.

  Step 663: Perform shading and shaping on the hierarchical subdivision surface.

  Step 664: Use adaptive subdivision to add details to the surface, and use analysis and synthesis to propagate changes to all levels of the surface, thereby modifying the details at the selected level. enable.

  Step 665: Provide a hierarchical subdivision surface library.

  Step 666: If desired, convert the subdivision surface model as a result of the SBS process to another surface type.

  The generalized method 600 also includes the auxiliary method 680 shown in FIG. 21, which includes the following steps.

ここで、Ｉはピクセル（ｕ，ｖ）における離散強度であり、λは、無限に遠くに離れたシミュレートされた光源の方向を指し示す単位ベクトルであり、Ｈは、関連する高さフィールドであり、Ｃ_Ｈは、高さフィールドＨに関連する曲面の曲率を表わし、λは平滑係数であり、合計は、投影曲面の内部と交差するモデルビューにおけるピクセル上で行われる式により与えられる関数を最小化する、モデルビュー上の高さ増分集合Γを生成するように構成される選択された整形操作を適用する。 Step 681:

Where I is the discrete intensity at pixel (u, v), λ is a unit vector pointing to the direction of the simulated light source that is infinitely far away, and H is the associated height field , C _H represents the curvature of the curved surface associated with the height field H, λ is the smoothing factor, and the sum is the function given by the formula performed on the pixels in the model view that intersects the interior of the projected curved surface Apply a selected shaping operation that is configured to generate a height increment set Γ on the model view.

の非制約最小化に縮小し、ここで最小化を実行するために使用される方法は、信頼領域法である。 Step 682: The function

The method used to reduce to an unconstrained minimization, where the minimization is performed is the trust region method.

  Step 683: From the sum over all pixels in the model view that intersect the interior of the projection surface to the sum over only the reduced set called Q by intersecting it with the neighborhood of the modified pixel Reduction is performed so that calculations need not be performed across the projection surface, as seen in the model view.

  The generalized method 600 also includes the following auxiliary method 700 shown in FIG. 22, which includes the following steps.

ここで、

はｆの勾配ベクトルであり、

は、ｆのヘッシアン行列である式により関数をモデル化する。 Step 701: Quadratic function

here,

Is the gradient vector of f,

Models a function with an expression that is a Hessian matrix of f.

に対して、

である。 Step 702: Minimize model F in the selected region. In this example, the selected region is

Against

It is.

  Step 703: Minimize by using CG-Steihaug method with special sparse matrix multiplication.

であって、ここで、ρが１に近ければ、Ｆは信頼領域内で、ｆに対して良好なモデルと考えられ、信頼領域の中心はｘ_１へ移動され、信頼領域の半径は増大し、または、ρが１から遠く離れていれば、信頼領域の半径は減少する式で表わされるテスト値を構築する。 Step 704: the minimum point _{x 1} as a result,

Where ρ is close to 1, F is considered to be a good model for f in the trust region, the center of the trust region is moved to x ₁ and the radius of the trust region increases. Or, if ρ is far from 1, build a test value represented by a formula in which the radius of the confidence region decreases.

が、ｆの極小値が達成された現在の信頼領域の中心において、十分に小さいかどうかである。 Step 705: Repeat the process until the minimum value for f is determined based on established criteria in the trust region. In this example, the criterion for stopping the process is

Is sufficiently small at the center of the current confidence region where the local minimum of f is achieved.

  The generalized and auxiliary methods described above can interoperate with any computer-aided design (CAD) system, computer graphics system, or software application that can create, display, manipulate, or model geometry. Note that it can be implemented as a computer software plug-in product adapted to Plug-in product functions load and save 2d rendering functions, shading, light control, parameter adjustment, surface shape update based on shading information, light direction and input parameters, undo / redo functions inside the modifier A function shading tool, a tool for selecting an area to be updated using a masking technique, and a selection tool having a set of standard subdivision curved surface operations can be included. It can include either the effect on how much subdivision occurs in the region and (b) the effect on how noticeable geometric modifications are.

  Furthermore, as described above, the generalized method and the auxiliary method have the function of performing the SBS processing on the polygon mesh or the NURBS surface without first converting into the characteristics of the subdivision surface or using it. Can have. This method and the auxiliary method generate an initial 3d shape that can be used in conjunction with the ability to interactively display large, complex meshes, arbitrarily trim the entire curved surface, and the SBS modeling process. Therefore, a function of drawing a contour line can be further included.

  While the above description includes details that enable those skilled in the art to practice the invention, the description is by way of example only, and numerous modifications and variations have benefited from these teachings. It should be recognized that it will be apparent to those skilled in the art. Accordingly, the invention herein is defined solely by the claims appended hereto and the claims are intended to be construed as broadly as is acceptable by the prior art.

The invention is set forth with particularity by the appended claims. The above and further advantages of the present invention will be better understood with reference to the following description taken in conjunction with the accompanying drawings.
FIG. 6 is a series of diagrams illustrating components of an exemplary digital processing environment in which aspects of the invention can be developed. FIG. 6 is a series of diagrams illustrating components of an exemplary digital processing environment in which aspects of the invention can be developed. FIG. 6 is a series of diagrams illustrating components of an exemplary digital processing environment in which aspects of the invention can be developed. FIG. 6 is a series of diagrams illustrating components of an exemplary digital processing environment in which aspects of the invention can be developed. A computer graphics system constructed in accordance with the present invention that generates a three-dimensional model of an object by shading applied to a two-dimensional image of the object in a given state of its creation at any point in time, such as by an operator Show. Performed by the computer graphics system shown in FIG. 5 when determining an object model update by shading applied to a two-dimensional image of the object in a given state of its creation at any point in time FIG. 6 is a series of diagrams useful for understanding the operations performed. Performed by the computer graphics system shown in FIG. 5 when determining an object model update by shading applied to a two-dimensional image of the object in a given state of its creation at any point in time FIG. 6 is a series of diagrams useful for understanding the operations performed. Performed by the computer graphics system shown in FIG. 5 when determining an object model update by shading applied to a two-dimensional image of the object in a given state of its creation at any point in time FIG. 6 is a series of diagrams useful for understanding the operations performed. Performed by the computer graphics system shown in FIG. 5 when determining an object model update by shading applied to a two-dimensional image of the object in a given state of its creation at any point in time FIG. 6 is a series of diagrams useful for understanding the operations performed. Performed by the computer graphics system shown in FIG. 5 when determining an object model update by shading applied to a two-dimensional image of the object in a given state of its creation at any point in time FIG. 6 is a series of diagrams useful for understanding the operations performed. 6 is a flowchart illustrating operations performed by a computer graphics system and an operator related to the present invention. 6 is a flowchart illustrating operations performed by a computer graphics system and an operator related to the present invention. FIG. 2 shows a diagram illustrating the data flow in an SBS system according to the invention. FIG. 6 shows a screen shot 350 of an SBS modifier at 3ds max. FIG. 3 illustrates a pseudo code implementation of SBS technology according to an aspect of the present invention. FIG. 3 illustrates a pseudo code implementation of SBS technology according to an aspect of the present invention. FIG. 3 shows a table providing a list of mathematical notations used in the description of the invention. FIG. 3 shows a table providing a list of mathematical notations used in the description of the invention. 2 is a series of flow charts of generalized and auxiliary methods according to various aspects of the present invention. 2 is a series of flow charts of generalized and auxiliary methods according to various aspects of the present invention. 2 is a series of flow charts of generalized and auxiliary methods according to various aspects of the present invention. 2 is a series of flow charts of generalized and auxiliary methods according to various aspects of the present invention. 2 is a series of flow charts of generalized and auxiliary methods according to various aspects of the present invention. 2 is a series of flow charts of generalized and auxiliary methods according to various aspects of the present invention.

Claims (27)

A geometric model representing the geometry of at least a portion of the surface of a three-dimensional (3D) object, associated with a two-dimensional (2D) image of the object representing the object projected onto an image plane A drawing method by a computer generated by shading by an operator,
A. Receiving shading information provided by the operator representing a change in lightness level of at least a portion of the image in relation to the image of the object;
B. Generating the updated geometric model of the object in response to the shading information used to determine at least one geometric feature of the updated geometric model;
C. Displaying the image of the object as defined by the updated geometric model; and
D. The method can be operated by digital input of any hierarchical subdivision surface, polygon mesh, or NURBS surface,
A drawing method characterized by that. Once the subdivision surface is generated and displayed to the user, it is aligned with the 2D model view.
The method according to claim 1, wherein: The 2D model view includes information about grid corners, grid width and height, pixel size, and camera-to-object conversion.
The method according to claim 2, wherein: Using the 2D model view, the user sets the lighting direction, adjusts input parameters and shades, thereby modifying the intensity of selected pixels, or loads a pre-shaded set of pixels. This information is then used in the shaping algorithm,
The parameters can include either (a) an effect on how much subdivision occurs in the correction region and (b) an effect on how noticeable the geometric correction is.
The method of claim 3, wherein: The shaping algorithm determines an exact geometric change to the surface;
Additional surface primitives are added when necessary to ensure that sufficient details exist through subdivision in the shading area,
A height field is determined that reflects in 3D the change required in the 2D setting, and then the subdivision surface is modified to reflect the determined height value;
This results in a shaped, hierarchical subdivision surface that can be further modified, saved, or converted to the desired output surface type,
Surface basic shapes can include either triangles, squares, or other polygons,
The method of claim 4, wherein: Create a subdivision surface to be the foundation,
Display 2D shade view,
Allow users to set lighting, shading and adjust parameters,
Perform shaping, thereby generating a 3D subdivision surface,
The shaping process is
Introducing details on the surface,
Determine a new height parameter for the surface;
Shaping the subdivision surface, and
The parameters can include either (a) an effect on how much subdivision occurs in the correction region and (b) an effect on how noticeable the geometric correction is.
The method according to claim 1, wherein: Accepts an input containing either a mesh representation or a NURBS surface;
If the input is not yet a hierarchical subdivision surface, convert the input to a hierarchical subdivision surface;
Performing shading and shaping on the hierarchical subdivision surface;
The method according to claim 6, wherein: Use adaptive subdivision to add details to the surface, and use analysis and synthesis to propagate changes to all levels of the surface, thereby allowing modification at selected levels of details To
The method according to claim 7, wherein: Provides a hierarchical subdivision surface library,
The method according to claim 6, wherein: Converting the subdivision curved surface model resulting from the SBS processing into another surface type;
10. The method of claim 9, wherein:

Where I is the discrete intensity at the pixel (u, v), l is a unit vector pointing to the direction of the simulated light source that is infinitely separated, H is the associated height field, C _H represents the curvature of a curved surface with an associated height field H, λ is a smoothing factor, and the sum is a function given by the equation executed on the pixels in the model view that intersect the interior of the projected curved surface Applying a selected shaping operation configured to generate a set of height increments Γ on the model view according to the above equation,
The method according to claim 6, wherein: The set of height increments Γ has the function

The corresponding height value can be reduced by the shaping algorithm so that it can be reduced to a vector x containing one input for each pixel or connected region, and the minimization The method used to perform is the trust region method,
The method of claim 11, wherein: Correct it from the sum over all pixels in the model view that intersects the interior of the projection surface so that it does not have to be performed on the entire projection surface as seen in the model view Further including a reduction to the sum only on the reduced set called Q by intersecting with the neighborhood of the pixel;
The method according to claim 12, wherein: The trust region method has the following quadratic function:

here,

Is the gradient vector of f,

Is a first model of the function according to an expression that is a Hessian matrix of f,
next,

Is in the area

Minimizing the model F with respect to
The method according to claim 12, wherein: The method used to practice the minimization is the CG-Steihaug method with special sparse matrix multiplication,
The test value is constructed from the resulting minimum point x ₁ and is

And
the closer [rho is the 1, F is considered good model for f in the confidence region, the center of the confidence region is moved to x _1, the radius of the confidence region is increased, whereas, [rho 1 Away from the radius of the trust region is reduced,
The process is repeated until a minimum value for F is found in the confidence region,
The criterion for ending the process is:

Is sufficiently small at the center of the current confidence region where the local minimum of f has been achieved,
The method according to claim 14, wherein: The method is adapted to interoperate with either a computer aided design (CAD) system, a computer graphics system, or a software application that operates to create, display, manipulate, or model a geometry. Implement as a computer software plug-in product,
The method according to claim 15, wherein: The function of the plug-in product is as follows:
A shading tool having a 2D rendering function and a function for loading and saving shading, light control, parameter adjustment, surface shape update based on shading information, light direction, and input parameters, undo / redo functions inside the modifier,
A tool that uses masking technology to select areas to update,
A selection tool having a set of standard subdivision surface operations, and
The parameters can include either an effect on how much subdivision occurs in the correction region (a) and an effect on how noticeable the geometric correction is (b),
The method according to claim 16, wherein: Further including the ability to convert the SBS process to a subdivided surface property first or without using it on a polygon mesh or NURBS surface;
The method according to claim 10, wherein: Further includes the ability to display large, complex meshes with interactive capabilities,
The method according to claim 10, wherein: Further includes the ability to arbitrarily trim the entire surface;
The method according to claim 10, wherein: Further comprising the ability to delineate and generate an initial 3D shape that can be used in connection with the SBS modeling process;
The method according to claim 10, wherein: By an operator in the context of a two-dimensional image of the object, representing a geometric model representing the geometry of at least a portion of the surface of the three-dimensional object, as the object is projected onto an image plane A computer graphics system generated by shading,
A. An operator input device configured to receive shading information provided by the operator representing a change in lightness level of at least a portion of the image;
B. Configured to receive the shading information from the operator input device and generate an updated geometric model of the object in response, and using the shading information, at least one of the updated geometric models A model generator configured to determine one geometric feature;
C. An object display device configured to display an image of the object as defined by the updated geometric model;
D. The system can accept any hierarchical subdivision surface, polygon mesh, or NURBS surface;
A computer graphics system characterized by the above. A computer program product executable in a computer graphics system comprising a human-usable input device and a display device operable to produce a human-recognizable display comprising:
The computer program product includes computer software code instructions that can be executed by the computer graphics system and encoded on a computer-readable medium;
The computer program product is operable in the computer graphics system, and a geometric model representing a geometric shape of at least a part of a surface of a three-dimensional object is projected on the image plane. Can be generated by shading by an operator in the context of a two-dimensional image of the object expressed as:
A. First computer software code means operable to receive shading information provided by an operator using an input device representing a change in lightness level of at least a portion of the image;
B. It is operable to receive the shading information from the operator input device and generate an updated geometric model of the object in response, and using the shading information, at least one of the updated geometric models A model generator computer software code means operable to determine one geometric feature;
C. Object display computer software code means configured to display an image of the object on a display device as defined by the updated geometric model, the computer graphics system comprising:
D. The computer program product can operate to accept any hierarchical subdivision surface, polygon mesh, or NURBS surface.
A computer program product characterized by that. Further comprising minimizing by using a trust region Newton-CG method;
The method according to claim 14, wherein: Use the residual to obtain the function, its slope and its Hessian,
25. The method of claim 24, wherein: Calculations are integrated with or into the API,
The method according to claim 14, wherein: Calculations are integrated with or within computer software application plug-ins,
The method according to claim 14, wherein:

Images