KR20120118462A - Concave surface modeling in image-based visual hull - Google Patents

Abstract

The apparatus and methods disclosed herein comprise a collection of reference images obtained from a camera and a reference image obtained from a perspective for capturing the entire concave area of an object; A silhouette processing module for obtaining a silhouette image of the concave region of the object; And a virtual image synthesizing module connected to the silhouette processing module for synthesizing a virtual inverted image of the concave region from the calculated silhouette image and generating a visual hull of an object having the concave region.

Description

CONCAVE SURFACE MODELING IN IMAGE-BASED VISUAL HULL}

Image Based Rendering (IBR) is a method of creating a new scene from a set of reference images. The reference image is an actual picture of the object and the color can be calculated from the pixels in the reference image. Thus, IBR is a method for achieving photo realism. Image-based Visual Hull (IBVH) is a type of image-based rendering that can be used to reconstruct a three-dimensional (3D) model from a group of reference images of an object. Conventional IBVH techniques involve a 3D reconstruction method that generates a convex hull of a target object using a group of photos as input.

Conventional IBVH techniques use object silhouettes in a reference image to recover geometric information. However, a problem with this prior art is that the silhouette cannot show the information of a concave region (ie, called a concave object) on the surface of the object. As a result, concave objects are difficult to recover. In particular, conventional IBVH techniques result in raised defects, and as a result can reduce photo realism. An example of a reconstructed object with raised defects is shown in FIG. 2. The growth of raised defects is due in part to the fact that the concave area is always surrounded by the remainder of the object. As a result, conventional IBVH techniques have problems in obtaining the geometry of concave objects as reference images.

In the present disclosure, a computer readable medium and a method in which a computer executable program is stored are disclosed, the method comprising: obtaining a plurality of reference images of an object having a concave region or concave regions; And generating a virtual inside-out image of the concave region based on the reference image.

Alternatively, an apparatus is described in the present disclosure, the apparatus for storing a plurality of reference images obtained from a camera for an object having a concave region and a reference image obtained from a view point for capturing the concave region of the object. Storage devices; A silhouette processing module for obtaining a silhouette of each reference image; And a virtual image compositing module for obtaining a virtual inverted image of the concave region and composing a visual hull of the object.

Alternatively, an apparatus is described in the present disclosure, the apparatus comprising: a storage device for storing a plurality of reference images of an object obtained from a camera and a reference image obtained from a perspective of capturing an entire boundary of a concave region of the object; A silhouette processing module for obtaining a boundary of a silhouette and a concave region for each reference image; And a virtual image synthesizing module coupled to the silhouette processing module to obtain a virtual inverted image of the concave region and to synthesize a visual hull for an object having a concave region based on the virtual inverted image.

Alternatively, a method is described in this disclosure, the method comprising obtaining a plurality of reference images of an object having a concave area; Generating a virtual inverted image for the concave region based on the plurality of reference images; And a subtraction Boolean operation on a corresponding segment mapped from the virtual reference image, with the plurality of reference images, using the virtual inverted image generated as the virtual reference image. And obtaining a visual hull by performing an intersection Boolean operation on the corresponding segment mapped from the reference image.

The foregoing is a summary, and therefore, where necessary, simplifications, generalizations, and omissions of detail are included, and thus those skilled in the art will recognize that the summary is merely exemplary and is not intended to be limiting in any way. Other aspects, features, and advantages of the devices and / or processes and / or other objects described herein will become apparent in the teachings presented herein. The summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

1 is a block diagram illustrating an example of a configuration of a computing device arranged to reconstruct an image displayed in accordance with the method of the present disclosure, in accordance with an example embodiment.
2 illustrates the rendering result with the distortion of the raised shape of the concave object using IBVH.
3A illustrates an image before processing in accordance with the disclosed method.
3B illustrates a rendering result using a method of modeling a concave object, according to an exemplary embodiment.
4 is a block diagram illustrating a module executed in the computing device of FIG. 1, in accordance with an example embodiment.
5 is a flowchart illustrating a method of generating a visual silhouette cone for a concave region of an object, in accordance with an exemplary embodiment.
FIG. 6 is a flowchart illustrating steps for a method of determining 3D coordinates of FIG. 5.
7 is a conceptual diagram illustrating a visual hull obtained from a reference image.
8 illustrates the concept of a virtual camera for obtaining a virtual flipped image of a recessed area, according to an exemplary embodiment.
9 illustrates a projected flipped image of the concave region obtained by perspective projection from 3D to 2D.
10A is a reference image of an object obtained directly above, according to an exemplary embodiment.
10B illustrates intermediate results obtained using image segmentation, according to an example embodiment.
10C illustrates the synthesized virtual image, in accordance with an example embodiment.
11 is a flowchart illustrating a method of determining a visual hull using a projected flipped image of a concave region as a reference image.
12 illustrates a visual hull obtained using the projected flipped image of the concave region as a reference image.

In the following detailed description, reference is made to the accompanying drawings which form a part hereof. In the drawings, like reference numerals identify generally similar components, unless otherwise indicated in the context. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Without departing from the scope and spirit of the objects set forth herein, other embodiments may be utilized and other modifications may be made. It is readily understood that aspects of the present disclosure, as generally described herein and illustrated in the figures, may be arranged, substituted, combined, and designed in a variety of different configurations, all of which are expressly considered and part of the present disclosure. Will be.

The present disclosure, among other things, relates to a method, apparatus, computer program and system related to reconstructing an image of an object.

As can be seen for the example in FIGS. 3A and 3B, compared to the rendering result of the conventional processing method (FIG. 3A), the concave object modeling, which is the processing method of the present disclosure, is characterized by the shape and texture of the target concave object ( texture is accurately reconstructed and produces high quality photorealistic results (FIG. 3B).

According to an exemplary embodiment of the present disclosure, a virtual camera and an image are employed to provide a virtual silhouette cone characterizing the geometric information of the concave object, while the texture information is provided by an image obtained directly above the concave object.

1 is a block diagram illustrating an example of a configuration of a computing device arranged to reconstruct an image displayed in accordance with the methods of the present disclosure.

1 is a block diagram illustrating an exemplary computing device 100 for implementing a pre-processing module, a virtual image composition module, and a point relocation module. In the basic configuration 101, the computing device 100 generally includes one or more processors 110 and system memory 120. The memory bus 130 may be used to communicate between the processor 110 and the system memory 120.

Depending on the configuration desired, the processor 110 may be of any type including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. Do not. The processor 110 may include one or more levels of cache, such as a level 1 cache 111, a level 2 cache 112, a processor core 113, and a register 114. The processor core 113 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. Memory controller 115 may also be used with processor 110, or in some implementations, memory controller 115 may be an internal part of processor 110.

Depending on the configuration desired, system memory 120 may be of any type, including but not limited to volatile memory (such as RAM), nonvolatile memory (such as ROM, flash memory, etc.), or any combination thereof. . System memory 120 generally includes an operating system 121, one or more applications 122, and program data 124. The application 122 includes a concave object modeling processing application 123 arranged to acquire and render the object. Program data 124 includes reference image data 125, as will be described further below. In some embodiments, application 122 may be arranged to operate on program data 224 on operating system 121 such that stereoscopic image data may be generated. This described basic configuration is shown in FIG. 1 by its components in dashed line 201.

Computing device 100 may have additional features or functionality, and additional interfaces to facilitate communication between the basic configuration 101 and any desired devices and interfaces. For example, bus / interface controller 140 may be used to facilitate communication via storage interface bus 141 between basic configuration 101 and one or more data storage devices 150. The data storage device 150 may be a removable storage device 151, a non-removable storage device 152, or a combination thereof. Some examples of removable and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard disk drives (HDDs), compact disk (CD) drives, or digital general purpose disks (DVD). Optical disk drives such as drives, solid state drives (SSDs), and tape drives, and the like. Exemplary computer storage media includes volatile and nonvolatile removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data, Media.

System memory 120, removable storage 151, and non-removable storage 152 are all examples of computer storage media. Computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROMs, digital versatile disks (DVDs) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or Any other medium that can be used to store the required information and can be accessed by the computing device 100 includes, but is not limited to. Any such computer storage media may be part of device 100.

Computing device 100 also provides an interface bus for facilitating communication from various interface devices (eg, output interface, peripheral interface, and communication interface) to underlying configuration 101 via bus / interface controller 140. 142). Exemplary output device 160 includes a graphics processing unit 161 and an audio processing unit 162, which may be configured to communicate via various one or more A / V ports 163 to various external devices such as displays or speakers. Can be. Exemplary peripheral interface 170 includes a serial interface controller 171 or a parallel interface controller 172, which may be input devices (eg, keyboard, mouse, pen, voice input) through one or more I / O ports 173. Devices, touch input devices, etc.) or other peripheral devices (eg, printers, scanners, etc.). Exemplary communication device 180 includes a network controller 181, which may be arranged to facilitate communication with one or more computing devices 190 in network 102 communication via one or more communication ports 182. . Communication connections are one example of communication media. Communication media generally can be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier or other transport mechanism, and includes any information delivery media. do. A "modulated data signal" can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. For example, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared (IR), and other wireless media. This is not restrictive. The term computer readable media as used herein may include both storage media and communication media.

Computing device 100 may be a mobile phone, personal data assistant (PDA), personal media playback device, wireless web-watch device, personal headset device, special purpose device, or any of the above functions. It can be implemented as part of a small form factor portable (mobile) electronic device, such as a fusion device including that of. Computing device 100 may also be implemented as a personal computer that includes both a laptop computer and a non-laptop computer configuration.

As illustrated in FIG. 4, the modeling processing application 123 of the concave object may include two parts, a silhouette processing module 402 and a virtual image composition module 404. The silhouette processing module 402 extracts a silhouette of an object by capturing or approaching an image of the object. The virtual image synthesis module 404 sets up the virtual camera on the bottom of the concave object, maps the silhouette to the image space of the virtual camera, and synthesizes the virtual image to form a visual hull.

The modeling processing application 123 of the concave object involves IBVH technology based on silhouette-from-silhouette 3D reconstruction. Silhouette-based 3D reconstruction generally creates a geometrical entity called a silhouette cone. In silhouette based 3D reconstruction, the original image is thresholded to produce a binary image of the foreground / background referred to as the silhouette image. A foreground mask, known as a silhouette, is a 2D projection of the corresponding 3D foreground object. According to the camera viewing parameter, the silhouette defines a back-projected generalized cone surrounding the 3D object. This generalized cone is a silhouette cone.

An image based visual hull (IBVH) process creates a visual hull at the intersection of a generalized cone. In general, the IBVH process takes a group of photos as input and produces a convex hull of the object. Convex hull is a visual hull for an object. In the process, each reference image is divided into a foreground and a background. The foreground mask, or silhouette, defines a back projected cone in the 3D space containing the target object according to the camera's calibration information. The correction information of the camera includes a focal length and a camera position. The intersection of all silhouette cones forms the convex hull of the object. An example of a visual hull is shown in FIG. 7.

5 and 6 illustrate a method of removing the distortion of the raised shape from the visual hull for the object. The visual hull in FIG. 7 illustrates the elements described in the steps of the method. As shown in FIG. 5, in step 502, a visual hull 704 for an object having a concave area is obtained from a group of photos as a set of reference images 702, obtained from the outer side of the object. In step 504 of FIG. 5, the 3D coordinates of the boundary of the concave region are calculated on the visual hull. 10A shows an exemplary reference concave region image obtained from directly above the concave region.

6 illustrates performing step 504 of determining the 3D coordinates of the boundary of the concave region on the visual hull.

In step 602 of FIG. 6, the boundary 708 of the concave region on the 2D reference picture 706 is determined by manual interaction or image segmentation technique.

Image segmentation techniques generally refer to the process of dividing a digital image into a plurality of segments of a set of pixels (also known as superpixels). The purpose of image segmentation is to simplify and / or change the representation of an image to something more meaningful and easier to analyze. In particular, image segmentation involves assigning labels to all pixels in an image such that pixels with the same label share certain virtual features. Image segmentation is used to find objects and boundaries (lines, curves, etc.) in an image. In the case of step 602, image segmentation can be used to find the boundary for the concave area.

The result of image segmentation is a set of segments that selectively cover the entire image or a set of contours extracted from the image. Each pixel in the area is similar in some feature or computed properties such as color, intensity, or texture. Neighboring regions differ considerably in the same features.

10B shows an example of an intermediate result obtained using image segmentation, in which the boundary of the concave region is identified.

In step 604 of FIG. 6, the 3D coordinates for the boundary 710 of the concave region on the visual hull 704 are determined in the 2D reference picture 706 determined in step 602 by projecting the position on the 2D reference picture back into 3D space. Determined using the boundary 708 of the recessed area. Performing projection of the position on the 2D reference picture back into 3D space is an IBVH process step.

Continuing with the method illustrated in FIG. 5, at step 506, a location for the virtual camera is selected inside the recessed area. The position of the virtual camera is selected in accordance with observing the object from the inside of the concave area outward. In one embodiment, as shown by way of example in FIG. 8, a position where a virtual camera is disposed at the center of the bottom of the recessed area may be selected. Alternatively, other positions, such as at the corners of the bottom of the recessed area, can likewise be selected. The selected location allows the virtual flipped image to be created at that location to include the boundaries of the complete concave area. Thereafter, several locations can be selected where the virtual flipped image can entirely cover the boundaries of the complete concave service.

In step 508, camera parameters of the virtual camera are determined based on the selected position.

로 공통적으로 설정되어, 가상의 카메라의 방향은 위쪽으로 및 오브젝트의 저면에 수직으로 머물러, 오목한 오브젝트의 바로 아래의 저면으로부터 캡쳐하는 시뮬레이션을 가능하게 한다. 내부 파라미터에 관해서는, 초점 거리(f)만이 수정될 필요가 있으며, 기타 요소들은 참조 이미지를 찍는 실제 카메라의 대응하는 요인과 일치하여 남아있게 된다. 보통, 가상의 카메라의 초점 거리는 오목한 오브젝트의 깊이에 비례한다. 참조 이미지를 찍는 카메라의 알려진 파라미터에 기초하여, 오목 영역의 경계의 전체 영역이 가상의 카메라에 의해 찍힌 가상 이미지 내에 있도록 가상 이미지의 초점거리(f)의 값이 설정될 수 있다.When the center of the virtual camera is set, a transformation vector of external parameters of the camera is determined. For a rotation matrix of external parameters, three rotation angles

In common, the direction of the virtual camera remains upward and perpendicular to the bottom of the object, enabling simulation to capture from the bottom just below the concave object. Regarding internal parameters, only the focal length f needs to be corrected, and other factors remain consistent with the corresponding factors of the actual camera taking the reference image. Usually, the focal length of the virtual camera is proportional to the depth of the concave object. Based on the known parameters of the camera taking the reference image, the value of the focal length f of the virtual image may be set such that the entire area of the boundary of the concave area is in the virtual image taken by the virtual camera.

The camera parameter is selected such that the virtual flipped image obtained by the virtual camera based on the camera parameter entirely covers the boundary of the complete concave area.

In step 510, the image of the silhouette and the texture of the concave region are captured from directly above and outside the concave region, for example, as an image of the reference concave region that contains the boundaries of the complete concave region. Alternatively, several images may be obtained in which the boundaries of the complete concave area may be entirely covered by the various images.

In step 512, based on the 3D coordinates of the boundary of the concave area created in step 504, the camera parameters generated in step 508, and the reference concave area image obtained in step 510, the prior art relating to perspective projection from 3D to 2D is described. By using this, a virtual inverted image is generated.

Step 512 of perspective projection from 3D to 2D is shown in FIG. 9. Perspective projection from 3D to 2D in step 512 is how the reference concave region image 902 in the 2D plane is projected into the 3D space 904. The boundary 906 of the concave region in 3D space is projected into a virtual inverted image 908 in the 2D plane.

 One or several virtual flipped images (to the location of different virtual cameras) can be generated as long as such one or several virtual flipped images can cover the boundaries of the complete concave area.

10C shows an example of a virtual inverted image obtained by the virtual camera.

The process of perspective projection from 3D to 2D can be performed using a visual hull created using an image based visual hull (IBVH) process.

In addition, the modified visual hull of the concave object can be generated using the original reference image as well as one or several virtual inverted images as additional reference images. One or several virtual inverted images are used as reference images, and segments between intersections of the virtual inverted images are subtracted (via subtraction operations) from the view ray. In addition, when several virtual inverted images are used as reference images, the segments are totally subtracted.

In particular, in generating a modified visual hull for a concave object, a subtraction Boolean operation is performed on segments mapped from a virtual flipped image generated based on a virtual camera, and the cross Boolean operation is mapped from other reference photos. Is performed on the segment.

An example demonstrating calculating the modified visual hull for the concave region is illustrated in FIG. 12 and includes the following steps as shown in FIG. 11.

In step 1102 of FIG. 11, a new perspective taken by a new camera (“new” camera means an actual camera that takes an object from a new perspective different from the viewing direction of all other reference images 1204 in FIG. 12). For the pixel of the image of (1202 in FIG. 12), the line of sight is computed, which is emitted from the center of the new camera and passes through the pixel of the new perspective image. The projection of this line of sight in the reference image 1204, ie the projected line of sight, can be determined by using the position information of the new camera.

In a similar manner, the projected gaze is also determined for the virtual inverted image 1208 as the virtual reference image.

In step 1104 of FIG. 11, the point at which the projected gaze intersects the 2D silhouette on the reference image 1204 results in a set of 2D intervals on the reference image.

In step 1106 of FIG. 11, the 2D interval in the reference image is projected back into 3D space 1206 so that a corresponding 3D segment is obtained on the line of sight.

In step 1108 of FIG. 11, a 1D Boolean intersection operation is performed on all projected 3D line segments projected from all reference images 1204, so that the intersection of all 3D segments from all reference images 1204 is new. Represents a visual hull boundary for a pixel in a perspective image.

In step 1110 of FIG. 11, the 2D intervals in the one or more virtual inverted images 1208 are projected back into the 3D space 1206 to obtain a corresponding segment on the line of sight, after which the visual hull for the object with concave areas is obtained. A subtraction call operation is performed on the segment mapped from the gaze and virtual inverted images to obtain.

In accordance with the methods, computer readable storage media and apparatus disclosed herein, a concave region of a virtual inverted image or object of the concave region may be created. The virtual inverted image enables a view of the object which is not possible with conventional image-based visual hull techniques. In addition, a virtual inverted image can be used to create a refined visual hull of an object with concave regions, with the visual hull accurately depicting the object's geometry. In addition, refined visual hulls can be used to obtain photorealistic images of objects without raised defects.

The foregoing detailed description has described various embodiments and / or processes of the apparatus through block diagrams, flowcharts, and / or examples. As long as such block diagrams, flowcharts, and / or examples include one or more functions and / or operations, those skilled in the art will appreciate that each function and / or operation within such block diagrams, flowcharts, or examples is hardware, software, firmware, or virtual. It will be understood that they may be implemented individually and / or collectively by a wide range of any combination of these. In one embodiment, some portions of the subject matter described in this disclosure may be implemented in the form of an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), or other integrated form. However, those skilled in the art will appreciate that some aspects of the embodiments of the present disclosure may include one or more computer programs running on one or more computers (eg, one or more programs running on one or more computer systems), one running on one or more processors. It will be appreciated that the software and / or firmware may be implemented as a whole or in part, evenly, in whole or in part, with one or more programs (eg, one or more programs running on one or more microprocessors), firmware, or substantially a combination thereof. Writing code for and / or designing a circuit will be apparent to those skilled in the art in light of the present disclosure. Moreover, those skilled in the art will understand that the mechanisms of the subject matter of the present disclosure may be distributed in various forms of program products, and examples of the subject matter of the present disclosure are signal bearing media used to actually perform the distribution. It will be understood that this applies regardless of the particular type of medium). Examples of signal bearing media include readable type media such as floppy disks, hard disk drives, CD, DVD, digital tape, computer memory, etc., digital and / or analog communication media Wired communication link, wireless communication link, etc.), but is not limited thereto.

Those skilled in the art will describe the devices and / or processes in the format described herein, and then use engineering practice to describe such described devices (eg, transmitters, receivers, computing platforms, computing devices, etc.) and / or methods. It will be appreciated that integrating into a processing system is common in the art. That is, at least some of the devices and / or methods described herein may be incorporated into a data processing system through reasonable experimental quantities. As those skilled in the art, typical data processing systems typically include one or more system unit housings, video display devices, memory such as volatile and nonvolatile memory, processors such as microprocessors and digital signal processors, operating systems, drivers, graphical user interfaces and application programs. One or more interactive devices, such as computerized entities, touch pads or screens, and / or feedback loops and control motors (eg, feedback for sensing position and / or speed; components and / or quantities) It will be appreciated that it generally includes a control system comprising a control motor for moving and / or adjusting the motor. Typical data processing systems may be implemented using any suitable commercially available components as are typically found in data computing / communication and / or network computing / communication systems.

The subject matter described herein sometimes illustrates different components contained within or connected to different other components. It is to be understood that the illustrated architecture is merely exemplary and that many other architectures may be implemented that substantially achieve the same functionality. Conceptually, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Thus, any two components combined herein to achieve a particular function may be viewed as “associated with” each other such that the desired function is achieved, regardless of the architecture or intermediate components. Likewise, two associated components may also be considered to be "operatively connected" or "operatively connected" to each other to achieve the desired functionality, and any two components that may be associated as such It can be seen that they are "operably connectable" with one another to achieve a desired function. Certain examples of being operatively connectable are components that are physically compatible and / or physically interacting and / or wirelessly interacting and / or wirelessly interacting and / or logically. And / or logically interactable components with, but are not limited to.

With respect to the use of virtually any plural and / or singular term herein, one skilled in the art can interpret the plural to singular and / or the plural to plural, as appropriate to the context and / or application. Various singular / plural substitutions may be explicitly described herein for clarity.

Those skilled in the art will generally appreciate that the terms used in the present disclosure and specifically used in the appended claims (eg, appended claims) are generally "open" terms (eg, the term "comprising"). It is to be understood that the term "having" is intended to be interpreted as "including, but not limited to", "having" at least, and the term "comprising" should be interpreted as "including but not limited to" and the like. Moreover, those skilled in the art will also understand that, where a specific number of descriptions in the appended claims are intended, such intent will be expressly stated in the claims, and in the absence of such description, no such intention is intended. For example, to facilitate understanding, the following claims are intended to incorporate the claims, including the use of introduction phrases such as "at least one" and "one or more". It is to be understood, however, that the use of such phrases is not intended to limit the scope of the present invention to the use of an indefinite article "a" or "an" And should not be construed to limit the inclusion of a particular claim and should not be construed to imply that the same claim is not to be construed as an admission that it has been disclosed as an adverbial phrase such as "one or more" or "at least one" and " Quot; one "should be interpreted generally as meaning" at least one "or" at least one "). This also applies to the case of articles used to introduce claims. In addition, even if a particular number of introduced claim descriptions are expressly stated, it is generally known to one skilled in the art that such descriptions generally describe at least the number described (eg, “two descriptions” without other modifiers). Will be interpreted to mean at least two entries or two or more entries). In addition, where rules similar to “at least one of A, B and C, etc.” are used, it is generally intended that such interpretations will be understood by those skilled in the art (eg, “A, B and A system having at least one of C "has only A, has only B, has only C, has A and B together, has A and C together, has B and C together, or A Including, but not limited to, systems having B, and C together). If a rule similar to "at least one of A, B, or C" is used, then generally such interpretation is intended to assume that a person skilled in the art will understand the rule (e.g., A " and " B " together, A and C together, B and C together, or A, B, C together). It will also be understood by those skilled in the art that substantially any disjunctive word and / or phrase that represents more than one alternative term, whether in the detailed description, the claims or the drawings, , Or any combination of the terms < RTI ID = 0.0 > and / or < / RTI > For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B".

While various aspects and embodiments have been described in this disclosure, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments described in this disclosure are presented for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims (16)

Obtaining a plurality of reference images of an object having a concave region; And
Generating a virtual inside-out image for the concave region based on the plurality of reference images The method of claim 1,
Generating the virtual flipped image
Obtaining a visual hull for an outer boundary of the object based on the plurality of reference images;
Determining 3D coordinates of a boundary of the concave region on the visual hull;
Selecting a location inside the recessed area to calculate a virtual flipped image of the recessed area;
Determining image parameters for the selected location;
Capturing an image covering the complete boundary of the concave region as a reference concave region image; And
Generating the virtual inverted image for the concave region based on the determined 3D coordinates, the image parameters, and the captured reference concave region image by performing perspective projection from 3D to 2D.
≪ / RTI > The method of claim 2,
The determining of the 3D coordinates
Determining a boundary of the concave region in a 2D reference photograph by image segmentation; And
Determining a corresponding 3D coordinate of the boundary of the concave region on the visual hull. The method of claim 2,
Using the virtual flipped image generated as a virtual reference image,
Perform a subtraction Boolean operation on corresponding segments mapped from the virtual reference image with the plurality of reference images and cross Boolean on corresponding segments mapped from the reference image. Obtaining a modified visual hull by performing an intersection Boolean operation. The method of claim 4, wherein
Several virtual flipped images are generated for the object having the concave area and the multiple virtual flipped images are used as virtual reference images, and the segments connecting the intersecting points of the virtual flipped images are called A method that is totally subtracted in performing a subtraction operation. A computer readable storage medium storing program code executable on a computer, when executed by the computer,
Obtaining a plurality of reference images of an object having a concave region; And
Generating a virtual flipped image for the concave region based on the plurality of reference images
A computer readable storage medium storing computer executable program code for performing the step comprising a. The method of claim 6,
The generating step
Obtaining a visual hull of an outer boundary of the object based on the plurality of reference images;
Determining 3D coordinates of a boundary of the concave region on the visual hull;
Selecting a location inside the recessed area to calculate a virtual flipped image of the recessed area;
Determining image parameters for the selected location;
Capturing an image covering the complete boundary of the concave region as an image of a reference concave region; And
Performing 3D to 2D perspective projection generates the virtual inverted image for the concave region based on the determined 3D coordinates, the image parameters, and the image of the captured reference concave region. And a computer readable storage medium. The method of claim 7, wherein
Determine a boundary of the concave region in the 2D reference photograph by image segmentation;
Determine a corresponding 3D coordinate of the boundary of the concave region on the visual hull
Computer readable storage media The method of claim 7, wherein
Using the generated virtual inverted image as a virtual reference image, perform a subtraction boolean operation on a corresponding segment mapped from the virtual reference image, with the plurality of reference images, and correspondence mapped from the reference image. And obtaining a modified visual hull by performing a cross Boolean operation on the segment. 10. The method of claim 9,
Several virtual flipped images are generated for the object having the concave area and the multiple virtual flipped images are used as virtual reference images, and the segment connecting the intersection of the virtual flipped images is the boolean subtraction operation. Computer-readable storage media that is totally eliminated in performing the operation. As an apparatus,
A storage device for storing a plurality of reference images of an object obtained from a camera and reference images obtained from a perspective of capturing the entire boundary of the concave area of the object;
A silhouette processing module for obtaining a silhouette for the boundary of the concave region and each reference image; And
And a virtual image synthesizing module coupled to the silhouette processing module for obtaining a virtual flipped image of the concave region and synthesizing a visual hull for the object having the concave region based on the virtual flipped image. The method of claim 11,
And the silhouette processing module separates the concave region from the background and calculates the silhouette of the concave region. The method of claim 11,
The virtual image synthesizing module further comprises: the virtual camera being disposed inside the concave region of the object to determine a parameter for the virtual camera including a focal length and covering the entire boundary region of the concave region; And generate the virtual inverted image based on a reference image and the determined parameter. The method of claim 13,
And the virtual camera is disposed on a bottom surface of the concave area of the object. As a method,
Obtaining a plurality of reference images of an object having a concave region;
Generating a virtual inverted image for the concave region based on the plurality of reference images; And
Using the generated virtual inverted image as a virtual reference image, perform a subtraction boolean operation on the corresponding segment mapped from the virtual reference image, with the reference image and corresponding segment mapped from the reference image. Obtaining a visual hull by performing a cross Boolean operation on the. 16. The method of claim 15,
Several virtual inverted images are generated for the object having the concave region and the virtual inverted image is used as a virtual reference image such that the segments connecting the intersections of the virtual inverted images are called Boolean subtraction operations. How to be subtracted as a whole.

Images