KR20140081729A - Location correction of virtual objects - Google Patents

Abstract

A computer-implemented method is provided for use in location correction of virtual objects in a virtual model of a real-world scene. Location of an object consists of both position and orientation of the virtual object. The method includes generating the virtual model, including a virtual object, and acquiring at least one digital image of a real-world object within the real-world scene, wherein the real-world object corresponds to the virtual object. The method also includes calculating an image-based positional difference between at least one predefined point on the virtual object and at least one corresponding point on the real-world object, adjusting the position and/or the orientation of the virtual object based on this image positional difference, and adjusting the virtual model with respect to the corrected location of the virtual object.

Description

Location Correction of Virtual Objects {LOCATION CORRECTION OF VIRTUAL OBJECTS}

Embodiments described herein relate generally to graphical representations of real-world objects and / or scenes, and more particularly, to three-dimensional (3D) computer graphics modeling and simulation, and to virtual objects that represent real- Layout and location correction. The location of the object consists of both the position and orientation of the object.

Three dimensional computer graphics simulations of manufacturing work cells are now being used to simulate manufacturing processes. Thus, programs or simulation models are downloaded to computers on the job site to drive resources such as industrial robots. As part of this process, a virtual 3D model of the manufacturing workcell (including the machines, fixtures, and work pieces used in the manufacturing process) is simulated to match the actual position and orientation in the actual working cell on the job site The layout must be adjusted in the working cell. This enables the simulated manufacturing programs to verify processes for issues such as reachability and conflicts. This also enables correct operation of offline programs downloaded to manufacturing resources such as industrial robots at the factory.

The known methods of 3D location correction in the simulation model use expensive and complex measurement equipment such as coordinate measuring machines, theodolites, and / or laser range finders. However, such equipment must be set up at the factory. Measure data obtained from the factory is then imported into the 3D simulation and used to make adjustments to the model. Thus, these known methods require that separate devices be purchased and users trained to use them. Moreover, these known methods require multiple steps in different locations.

In one aspect, a computer-implemented method is provided for use in location correction of a virtual model of a real-world scene. The method includes generating a virtual model, including a virtual object, and obtaining at least one digital image of a real world object within a real world scene, wherein the real world object corresponds to a virtual object do. The method also includes calculating a difference on an image-based position between at least one predefined point on the virtual object and at least one corresponding point on the real world object, determining a position of the virtual object based on the difference in image position adjusting the position and / or orientation of the virtual object, and adjusting the virtual model for the corrected location of the virtual object.

In another aspect, a computer is provided for use in location correction of a virtual model of a real-world scene. The computer includes a virtual model and a memory area configured to store virtual objects in the virtual model. The computer also includes a processor coupled to the memory region. The processor is configured to generate a virtual model, including a virtual object, and to acquire at least one digital image of the real world object within the real world scene, wherein the real world object corresponds to the virtual object. The processor may also calculate a difference on an image-based position between at least one predefined point on the virtual object and at least one corresponding point on the real world object, and determine a position and / Or orientation, and to adjust the virtual model for the corrected location of the virtual object.

In another aspect, a computer program product comprises one or more non-temporal computer-readable storage media having computer-executable components for use in location correction of a virtual model of a real-world scene. The components include a generation component that causes the processor to generate a virtual model that includes a virtual object; And an acquisition component that causes the processor to acquire at least one digital image of a real-world object within the real-world scene, wherein the real-world object corresponds to a virtual object. The components can also be used by the processor to calculate a difference on an image-based position between at least one predefined point on the virtual object and at least one corresponding point on the real world object, And a correction component for adjusting the position and / or orientation of the object and for adjusting the virtual model for the corrected location of the virtual object.

Figure 1 is a schematic block diagram of an exemplary computer system for use in location correction of a virtual model of a real-world scene.
2 is a flow chart illustrating the operation of the computer system shown in Fig.
3 is a flow chart further illustrating the operation of the computer system shown in Fig.
4A-4G are examples of exemplary displays of client devices used in the computer system shown in FIG.
5 is a schematic block diagram of an exemplary computer architecture for use with the server system and / or client devices shown in FIG.

The description of exemplary embodiments of the present invention follows.

All patents, published applications and references cited herein are incorporated by reference in their entirety.

Exemplary embodiments of computer-implemented methods, computer devices and systems, and computer program products for use in location correction of a virtual model of a real-world scene are described herein. Embodiments described herein may be used for computing virtual objects in computer graphics models using, for example, a built-in digital camera, a portable tablet computer with 3D graphics simulation or virtual reality software, 3D location correction can be facilitated. The embodiments described herein utilize a combination of high resolution photographic and computational capabilities built into these portable tablet computers, along with mathematically rigorous rendering of 3D model images from software. This approach involves the form of "mediated reality" with the user superimposing on top of the 3D virtual image of the working cell model on the tablet computer to capture a small set of digital camera still images of the working cell do. It should be understood, however, that the embodiments described herein need not overlap images over a virtual model. For example, the embodiments described herein may also provide a computer device with a method for locating images and virtual models in a side-by-side manner, such as, for example, To be tracked. The computational approach described in detail below then adjusts the 3D locations of the virtual objects in the virtual model to solve the calibration problem.

Moreover, the embodiments described herein facilitate ease of use, speed of procedure, and a significant reduction in cost and setup time for measurement and installation. In addition, the embodiments described herein include an approach that eliminates the need for hardware-specific customization or length-standard type correction of the digital camera that is embedded in the tablet computer during 3D location adjustments. Moreover, the embodiments described herein are scalable from small dimensions of objects to large dimensions, as well as from coarse accuracy to coarse accuracy of adjustments, as described below.

While the embodiments described herein are described in connection with industrial and / or manufacturing resources, it should be understood that the embodiments are not limited to such uses. For example, embodiments may also be used during interior design to remodel the interior of a room, wherein a 3D graphic model of the layout of existing objects, such as furniture, is adjusted to accurately match the actual layout inside the building room prior to undertaking the redesign will be. Another example is an architecture where a 3D model of a set of buildings roughly ranked in a virtual model takes a tablet computer outdoors and captures digital images of buildings while walking around its grounds, Will be adjusted.

1 illustrates an exemplary computer system 100 for use in location correction of a virtual model of a real-world scene and / or for performing the processes described below and / or may be associated with those described below A schematic block diagram of additional processes. In an exemplary embodiment, the memory area 102 includes one or more storage devices 104 for use in storing data, such as data associated with simulations of real-world scenes. The storage devices 104 may be implemented as one or more databases and / or may be located in single or multiple geographic sites. In some embodiments, the memory area 102 is coupled to the server system 106. In an alternative embodiment, the memory area 102 is integrated with the server system 106. In an exemplary embodiment, server system 106 is coupled to one or more client devices 108 via network 110. The client devices 108 may include mobile client devices, including, but not limited to, laptop computers, tablet computers, and / or smart phones. Moreover, the client devices 108 may include non-mobile, non-mobile client devices such as desktop computers and the like.

As will be appreciated, the network 110 may be a public network, such as the Internet, or a private network, such as a LAN or WAN network, or any combination thereof, and may also include PSTN or ISDN sub-networks. The network 110 may also be wired, such as an Ethernet network, or wireless, such as a cellular network including EDGE, 3G, and 4G or LTE wireless cellular systems. The wireless network may also be WiFi, Bluetooth, or any other wireless type of communication known. Thus, the network 110 is merely illustrative and does not limit the scope of these enhancements in any way.

The server system 106 and / or client devices 108 may be any suitable computer architecture, such as those described below with reference to Fig. 5, or any other computing architecture known in the art. Moreover, it should be understood that the server system 106 is configured to perform the above-described processes and / or any additional processes that may be associated with those described above.

In some embodiments, the server system 106 may also provide data from the memory area 108 as needed by the client devices 108, such that the client devices 108 execute the processes described above. have. As such, FIG. 1 includes implementations of computer system 100 via cloud computing, distributed computing, and the like. Moreover, in some embodiments, the server system 106 stores computer-readable instructions for executing the processes described below and provides these instructions to the client devices 108 via the network 110. [ Such computer-readable instructions may be implemented as a computer program product having one or more non-transitory computer-readable storage media having computer-executable elements for use in location correction of a virtual model of a real-world scene. As will be described in further detail below, the components include a generating component that causes the processor to generate a virtual model that includes a virtual object, and a processor that causes the processor to generate at least one digital image of the real- , Wherein the real-world object corresponds to a virtual object. The components may also enable the processor to adjust the position and / or orientation of the virtual object based on a difference in image position between at least one predefined point on the virtual object and at least one corresponding point on the real world object, And a correction component that allows the virtual model to be adjusted for the corrected location of the virtual object.

FIG. 2 is a flowchart 200 illustrating the operation of the computer system 100 shown in FIG. 1 and 2, the server system 106 may provide a virtual model of a real-world scene or work cell, such as a 3D model for use in simulation or virtual display of real-world scenes, (202) to the client device (108) via the network (110). The client device 108 loads (204) the virtual model into a simulation or virtual reality software module and receives (206) a user selection of one or more virtual objects having spatial locations the user desires to adjust. The client device 108 also receives 208 a user selection of a reference object or reference point in the virtual model. The client device 108 then displays (210) a digital camera image that is nested or parallel to the virtual 3D view on the virtual 3D view. Based on the additional user input, the 3D views at the client device 108 may then be used to generate virtual rotations and transforms, a view of the zoom and orientation by display of the virtual model, To a desired observation point (212). A digital image with the corresponding 3D virtual view is then acquired 214 by the client device 108 and stored in the client device 108 and / or the server 106.

In an exemplary embodiment, the user then takes the client device 108 in a real world scene, such as a room layout, a work site, or a building site, and obtains a different viewpoint from the display of the digital camera image in accordance with the virtual model 210 (216) with the client device (108). In an exemplary embodiment, the user repeatedly adjusts the display of the virtual 3D view (212) and captures (214) the new digital image each time with its 3D view. The client device 108 and / or the server system 106 may then use the captured digital image (s) to create a virtual object (s), as described in detail below in the flowchart 300, And calls a location correction function that identifies the location and orientation (218). The client device 108 and / or the server system 106 complete the location correction function, which results in improved accuracy of the location and orientation of the virtual object in the virtual model for the real world object in the real world scene. The resulting manufacturing simulation or virtual display will be enhanced to such an extent that space and / or time errors are reduced. In some embodiments, the client device 108 sends virtual objects and / or properties of the virtual model to the server system 106, such as the new location and / or orientation of the virtual object for storage in the memory area 102 . In such embodiments, the stored data may be accessed later for subsequent simulations.

FIG. 3 is a flowchart 300 further illustrating the operation of the computer system 100 shown in FIG. In an exemplary embodiment, with reference to both FIGS. 1 and 3, the server system 106 generates 302 a virtual model representing a real-world scene. The server system 106 transmits the virtual model to the client device 108 via the network 110 (304). The client device 108 loads 306 the virtual model into a simulation or virtual reality software module. In an exemplary embodiment, the client device 108 receives (308) user inputs to identify virtual objects and reference objects in the virtual model. A virtual object is a virtual object that the user desires to perform the location correction steps described herein. The reference object is a reference object related to the completion of the location correction. For example, FIG. 4A is an illustration of an exemplary display 400 of a client device 108. The client device 108 displays the virtual object 402 and the reference object 404.

In some embodiments, the client device 108 may also receive 310 user input (s) and compare one or more predefined comparison properties on the virtual object 402 and one or more corresponding comparison properties on the reference object 404 . In alternative embodiments, vision processing and / or pattern recognition algorithms may be used to identify predefined characteristics on virtual object 402 and reference object 404. 4A, the 3D view of the virtual object 402 has three predefined properties at vertex points 406, and the type of reference object 404 is the shape of the vertex points 408 Lt; RTI ID = 0.0 > predefined < / RTI > 4B is another illustration of a display 400 of a client device 108 that represents a digital image from a camera embedded in a client device. 4B, the real-world object 410 has three points 412 corresponding to the user-selected points 406 on the virtual object 402. As shown in Fig. The real world reference object 414 has three points 416 corresponding to the user-selected points 408 on the virtual reference object 404. [ Although FIGS. 4A and 4B illustrate virtual objects 402, virtual reference objects 404, real world objects 410, and real world reference objects 414 as cube objects, the embodiments described herein refer to cube objects But are not limited to use with < / RTI > More precisely, the embodiments described herein may be used with any type of objects that enable selection of some identifiable sets of points. Furthermore, the predefined comparison characteristics 406, 408, 412, and / or 416 may be vertices, or may be lines, planes, or surfaces, as shown in Figs. 4A and 4B. In some embodiments, client device 108 sends data to server system 106 associated with identified points 406, 408, 412, and / or 416, such as coordinates of identified points .

After the predefined points 406 and / or 408 are identified, the client device 108 computes 312 the location transformation based on the identified points 406 and / or 408. In the virtual model, the position and orientation of each virtual object is defined by six parameters (X, Y, Z, W, P, and R). The first three parameters (X, Y, Z) are distances that specify the location of the virtual object in the x-, y-, and z-directions. The second three parameters (W, P, R) are the angles that specify the orientation of the virtual object, where W represents yaw, P represents pitch, and R represents roll. Collectively, these six parameters may be referred to as the location transformation of the virtual object. Furthermore, the location transformation of the virtual object 402 is computed, for example, for the reference coordinate system implied by the virtual reference object 404. Specifically, the location transformation of the virtual object 402 determines the location (X, Y, Z) of the identified points 406 on the virtual object 402 and the identified points 408 on the reference object 404, . As shown in FIG. 4A, points 406 on the virtual object 402 are those used to compute the location transform 312. In an exemplary embodiment, the reference object 404 provides both a reference coordinate system and a dimensional length standard for complete location correction. Therefore, the 3D model of the reference object 404 almost matches the real world reference object (not shown) in terms of the location of its identified points 408. That is, the respective distances between the identified points 408 of the fictitious reference object 404 and corresponding points of the real world reference object must be the same within an application-dependent predefined tolerance.

After the location transformation is calculated, the camera devices at the client device 108 acquire 314 digital images of real-world scenes with their 3D virtual views as described in 210-214 of FIG. 2, the client device 108 receives 212 user inputs in the form of manipulations of the fictitious reference object 404. For example, the user may use the touch screen to rotate the virtual object 402 and the reference object 404 such that the virtual object 402 and the reference object 404 are substantially aligned with the real world object 410 and the reference object 414, Using the zooming operations. 4C and 4D are additional illustrations of the display 400 of the client device 108. FIG. Specifically, FIG. 4C illustrates a virtual world 402, including its virtual world 402 and its corresponding real world object 410, as well as a real world world 402, including a reference object 404 and its corresponding real world reference object 414 Lt; / RTI > illustrates a client device 108 that overlay or simultaneously displays a scene. The situations shown in Figure 4c are the results after 212 the 3D virtual view is adjusted so that the user is almost aligned with the real world image. The client device 108 and / or the server 106 may then use the viewing rotation 404 to substantially match the predefined points 408 on the reference object 404 with the points 416 on the real world reference object 414. [ , And initiates the programmatic adjustment (316) of the 3D virtual view by fine-tuning of the transformation and / or zoom. This programmatic view adjustment is made for each of the images 314 obtained from different views of the real world scene. 4D illustrates the resulting situation after step 316 such that the fictitious reference object 404 coincides with the real world reference object 414. Specifically, the points 408 of the reference object 404 are substantially aligned with corresponding points 416 of the real world reference object 414. Furthermore, this is a programmatic adjustment 316 that eliminates the need for any hardware-specific camera correction and ensures that the length standard implied by the dimensions of the reference object 404 is automatically considered in the next location correction.

The client device 108 and / or the server system 106 are now in virtual object locations 406 when the reference object 404 is substantially aligned with the real world reference object 414 by programmatic adjustment 316. [ And the corresponding points 412 on the real world object 410. [ More specifically, the client device 108 and / or the server system 106 may provide an image-based (or object-based) interface between the points 406 on the virtual object 402 and corresponding points 412 on the real- (318). ≪ / RTI > In an exemplary embodiment, the difference is calculated in the pixels in each of the X and Y directions. Figure 4E is another illustration of the display 400 of the client device 108. [ More specifically, FIG. 4E shows the position of the virtual object 402 in each of the X-direction (DeltaX) and in the Y-direction (DeltaX) with respect to one of the corresponding points 412 and 406 of the real- (DeltaY). In an exemplary embodiment, however, the client device 108 and / or the server system 106 calculates (318) the difference for every pair of identified points 406 and 412. In an exemplary embodiment, the differences are stored in the memory area 102. The client device 108 and / or the server system 106 calculates the difference on this image location for each of the images 314 obtained from different views of the real world scene. 4F is another illustration of the display of the client device 108. FIG. 4F illustrates a difference calculation on the image location for another capture from an alternative view of the real world scene after the reference object 404 has been substantially reordered with the real world reference object 414. [

The client device 108 and / or the server system 106 may then perform a location transformation of the virtual object 402 to minimize the combination of differences in all image locations between the virtual object 402 and the real world object 410 (320). In one embodiment, the client device 108 and / or the server system 106 utilizes a standard non-linear regression that minimizes the cost function by adjusting a set of variable parameters. In an exemplary embodiment, this cost function is the sum of square root mean square (RMS) errors for all of the calculated DeltaX and DeltaY differences. The variable parameters may be the computed location transform of the (X, Y, Z, W, P, and R) values of the virtual object 402, Based on the results of this minimization step 320, the client device 108 and / or the server system 106 adjusts the position and / or orientation of the virtual object 402 (322) The virtual model is adjusted (324) for the corrected location of the virtual model. 4G is another illustration of the display of the client device 108. FIG. More specifically, FIG. 4G shows the end result regarding reorientation and / or repositioning of the virtual model and virtual object 402 as a whole.

Although the steps describe adjusting the virtual model based on adjusting only the position of a single virtual object, the embodiments described herein should be understood to consider adjusting the positions of multiple virtual objects more than once at a time do. In an exemplary embodiment, each virtual model adjustment uses the same reference object. However, in some embodiments, the reference object may be different for different virtual objects and / or may be reference points in space, rather than simply objects.

5 is a schematic block diagram of an exemplary computer architecture 500 for use with server system 106 and / or client devices 108 (each shown in FIG. 1).

In an exemplary embodiment, the computer architecture 500 includes one or more processors 502 (CPU) that perform the above-described processes and / or any additional processes that may be associated with those described above. The term "processor" generally refers to systems and microcontrollers, reduced instruction set circuits (RISC), application specific integrated circuits (ASIC), programmable logic circuits, and / or any of those capable of performing the functions Or any other circuitry or processor of the system. The examples are illustrative only and are not intended to limit the definition and / or meaning of the term "processor" in any way.

The steps described above and / or any additional processes that may be associated with those described above may be performed, for example, by a system bus 506 to enable the processor 502 to be operably and / May be stored in the memory area 504 as computer-executable instructions. "Memory region ", when used herein, is generally used to assist in location correction of a virtual model of a real-world scene, and / or to perform additional processes that may be associated with the processes described above and / Refers to any means for storing non-trivial program code and instructions executable by one or more processors. Memory region 504 may include one or more than one type of memory. For example, memory region 504 may include random-access memory (RAM) 508, which may include other forms of non-volatile RAM, magnetic RAM, ferroelectric RAM, and / . Memory region 504 may also include read only memory (ROM) 510 and / or flash memory and / or electrically-programmable read only memory (EEPROM). Any other suitable magnetic, optical, and / or semiconductor memory, such as a hard-disk drive (HDD) 512, may be included in the memory area 504, either alone or in combination with other types of memory. The HDD 512 may also be coupled to the disk controller 514 for use in sending and receiving messages to and from the processor 502. [ Moreover, the memory area 504 may also be or include removable or removable memory 516, such as a suitable cartridge disc, CD-ROM, DVD, or USB memory. The examples are illustrative only and are not intended to limit the definition and / or meaning of the term "memory area" in any way.

Computer architecture 500 also includes a display device 518 coupled to, e.g., operably coupled to, display controller 520. The display controller 520 receives data via the system bus 506 for display by the display device 518. The display device 518 may include, but is not limited to, a monitor, a television display, a plasma display, a liquid crystal display (LCD), a display based on light emitting diodes (LED), a display based on organic LEDs (OLED) A display based on surface-conducting electron emitters, a display comprising projected and / or reflected images, or any other suitable electronic device or display mechanism. Moreover, the display device 518 may include a touch screen 522 associated with the touch screen controller 524. The examples are illustrative only and are not intended to limit the definition and / or meaning of the term "display device " in any way.

In addition, the computer architecture 500 includes a network interface 526 for use in communicating with a network (not shown in FIG. 5). Moreover, the computer architecture 500 includes one or more input devices, such as a keyboard 528, and / or a pointing device 530, such as a roller ball, a mouse, a touchpad, The touch screen 522 and its controller 524 may also be regarded as an integrated keyboard 528 and / or a pointing device 530. The input devices are coupled to and controlled by an input / output (I / O) interface 532, which is further coupled to the system bus 506. The computer architecture 500 also includes at least one digital camera 534 with its associated digital camera controller 536. [

General characteristics of the display controller 520, the disk controller 514, the network interface 526, and the I / O interface 532 as well as the display device 518, the keyboard 528, the pointing device 530, The description of the function is omitted here for the sake of brevity since these characteristics are known.

2-5, processor 502 generates a virtual model representing a real-world scene and loads the virtual model into a simulation or virtual reality software module. In an exemplary embodiment, the processor 502 receives user inputs via the touch screen 522, for example, to identify virtual objects and reference objects in a virtual model. A virtual object is a virtual object that the user desires to perform the location correction steps described herein. And the reference object is completed to compensate for this.

In some embodiments, the processor 502 also receives user inputs to identify one or more predefined points on the virtual object and one or more corresponding points on the reference object. After the predefined points are identified, the processor 502 calculates the location transformation based on the identified points. Furthermore, the location transformation of the virtual object is calculated for the same reference point as the reference object. After the location transformation is calculated, the camera device 534 obtains a digital image of the real world object within the real world scene, where the real world object corresponds to the virtual object. Moreover, during image acquisition, the processor 502 receives user inputs in the form of operations 3D virtual views. For example, the user may reposition the reference object using the touch screen 522, e.g., using rotation, translation, and / or zooming operations so that the reference object is substantially aligned with the real world reference object. After this approximate alignment, the processor 502 may perform a programmatic adjustment of the 3D virtual view by fine alterations of the viewing rotation, transformation, and / or zoom to substantially match the reference object points with the real world reference object points. .

When the reference object is substantially aligned with the real world reference object, the processor 502 determines the mismatch between the virtual object points and corresponding points on the real world object. Specifically, the processor 502 calculates the difference in image-based position between points on the virtual object and corresponding points on the real world object. In an exemplary embodiment, the difference is calculated in the pixels in each of the X and Y directions. The processor 502 calculates the difference on this image-based location for each of the images obtained from different views of the real-world scene.

Processor 502 then repeats the location transformation of the virtual object to minimize the combination of differences on all image locations. Specifically, processor 502 utilizes a standard non-linear regression that minimizes the cost function by adjusting the set of variable parameters. Based on the minimization step, the processor 502 adjusts the position and / or orientation of the virtual object and adjusts the virtual model for the corrected location of the virtual object.

Exemplary embodiments of computer systems, computer devices, computer-implemented methods, and computer-program products for use in location correction of a virtual model of a real-world scene have been described in detail above. These embodiments are not limited to the specific embodiments described herein, and more precisely, that the operations of the methods and / or the components of the system and / or apparatus may be performed by other acts and / And may be used independently and separately. It should also be understood that the operations and / or components described may also be used in other systems, methods, and / or apparatuses, and / or in combination, And < / RTI > storage media.

A computer device, e.g., a computer device having a computer architecture as described herein, includes at least one processor or processing unit and system memory. Computer devices generally have at least some form of computer readable media. By way of example and not limitation, computer readable media include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data . Communication media generally embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. Those skilled in the art are familiar with the modulated data signal that has one or more of its set of characteristics or that is modified in such a way as to encode information in the signal. Combinations of any of the above are also encompassed within the scope of computer readable media.

Moreover, embodiments of the present invention may be implemented as a computer program product having one or more non-temporal computer-readable storage media having computer-executable components for use in location correction of a virtual model of a real-world scene. The components include an originating component that, when executed by a processor, causes the processor to generate a virtual model, including a virtual object. The generating component also causes the processor to receive a user selection of at least a portion of the plurality of identifiable points and to calculate a location transformation for the virtual object based on the plurality of identifiable points on the virtual object. Moreover, the generating component allows the processor to identify the virtual object and the corresponding reference object, and identify at least one predefined point on the virtual object and a corresponding point on the reference object.

The components also include an acquisition component that when executed by the processor causes the processor to acquire at least one digital image of a real-world object within a real-world scene, wherein the real-world object corresponds to a virtual object. Moreover, the components, when executed by a processor, may cause the processor to calculate a difference on an image-based position between at least one predefined point on the virtual object and at least one corresponding point on the real-world object, Adjust the position and / or orientation of the virtual object based on the difference in the virtual object, and adjust the virtual model for the corrected location of the virtual object. The correction component may also enable the processor to make programmatic adjustments of the 3D virtual view by fine changes in viewing rotation, transformation, and / or zoom to substantially match the reference object points with the real world reference object points . The correction component then causes the processor to calculate a difference on the image-based position between at least one predefined point on the reference object and at least one corresponding point on the real-world reference object. In some embodiments, the correction component causes the processor to repeat the location transformation of the virtual object to minimize the combination of differences on all image locations.

While the invention has been described in connection with an exemplary computer system environment, embodiments of the invention are interoperable with numerous other general purpose or special purpose computer system environments or configurations. The computer system environment is not intended to imply any limitation as to the scope of use or functionality of any aspect of the present invention. Moreover, the computer system environment should not be construed as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment. Examples of well-known computer systems, environments, and / or configurations that may be suitable for use with aspects of the present invention include, but are not limited to, personal computers, server computers, portable or laptop devices, , Distributed computing environments that include any of the above systems or devices, microprocessor-based systems, set top boxes, programmable consumer electronics, mobile telephones, network PCs, minicomputers, mainframe computers, And the like.

Embodiments of the invention may be described in the general context of computer-executable instructions that are executed by one or more computers or other devices, such as program components or modules. Aspects of the present invention may be implemented with any number and structure of components or modules. For example, aspects of the invention are not limited to the specific computer-executable instructions or specific components or modules illustrated in the drawings and described herein. Alternate embodiments of the present invention may include different computer-executable instructions or components having more or less functionality than those illustrated and described herein.

The order of execution or performance of operations in the embodiments of the present invention illustrated and described herein is not essential, unless otherwise specified. That is, operations may be performed in any order, unless otherwise specified, and embodiments of the present invention may include additional or fewer operations than those disclosed herein. For example, it is contemplated that performing or performing a particular operation before, concurrently with, or after another operation is within the scope of aspects of the present invention.

When introducing elements of aspects of the present invention or aspects of the embodiments thereof, the articles "a", "an", "the" and "said" It is intended to mean more than one. The terms " comprising, "" including, " and" having "are intended to be inclusive and may mean additional elements other than the listed elements.

This written description discloses the invention, including the best mode, using examples, and is provided to enable any person skilled in the art to make and use any devices or systems, including performing any of the contained methods, Thereby enabling the present invention to be practiced. The scope of the patent of the present invention is defined by the claims, and may include other examples that come to those skilled in the art. These other examples, if they include structural elements that do not differ from the literal language of the claims, or even structural elements that are not significantly different from the literal language of the claims, are intended to be within the scope of the claims, do.

While the invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention, which is encompassed by the appended claims. , ≪ / RTI >

Claims (20)

A computer-implemented method for use in location correction of a virtual model of a real-world scene,
Generating the virtual model including a virtual object;
Obtaining at least one digital image of a real-world object within the real-world scene, the real-world object corresponding to the virtual object;
Computing a difference on an image-based location between at least one predefined point on the virtual object and at least one corresponding point on the real-world object;
Adjusting at least one of a position and an orientation of the virtual object based on a difference on the image position; And
And adjusting the virtual model with respect to the corrected location of the virtual object. ≪ Desc / Clms Page number 19 > The method according to claim 1,
Wherein generating the virtual model comprises computing a location transformation for the virtual object based on a plurality of identifiable points on the virtual object. Computer-implemented method. 3. The method of claim 2,
Wherein computing the location transformation comprises receiving a user selection of at least a portion of the plurality of identifiable points. The method according to claim 1,
Wherein generating the virtual model comprises:
Identifying the virtual object and a corresponding reference object; And
Identifying the at least one predefined point on the virtual object and a corresponding point on the reference object for use in location correction of a virtual model of a real-world scene. 5. The method of claim 4,
Further comprising a programmatic adjustment of a virtual view for substantially matching the predefined points on the reference object with the points on a corresponding real world reference object, For a computer-implemented method. 5. The method of claim 4,
Programmatic adjustment of virtual views; And
Computing a difference on the image-based location between the at least one predefined point on the virtual object and the at least one corresponding point on the real-world object for each of the acquired digital images A computer-implemented method for use in a location correction of a virtual model of a real-world scene. The method according to claim 6,
Further comprising minimizing a difference on the image location by repeating the location transformation of the virtual object. ≪ Desc / Clms Page number 21 > A computer for use in location correction of a virtual model of a real-world scene,
A memory area configured to store virtual models of the virtual model and the virtual model; And
A processor coupled to the memory region,
The processor comprising:
Generate the virtual model including the virtual object;
Within the real-world scene, obtaining at least one digital image of a real-world object corresponding to the virtual object;
Calculating a difference on an image-based location between at least one predefined point on the virtual object and at least one corresponding point on the real-world object;
Adjust at least one of a position and an orientation of the virtual object based on a difference on the image position; And
And to adjust the virtual model for the corrected location of the virtual object. 9. The method of claim 8,
Wherein the processor is further configured to calculate a location transformation for the virtual object based on a plurality of identifiable points on the virtual object. 10. The method of claim 9,
Wherein the processor is further configured to receive a user selection of at least a portion of the plurality of identifiable points. 9. The method of claim 8,
The processor comprising:
Identify the virtual object and a corresponding reference object; And
Wherein the computer is further configured to identify the at least one predefined point on the virtual object and a corresponding point on the reference object. 12. The method of claim 11,
Wherein the processor is further configured to programmatically adjust the virtual view to substantially match the predefined points on the reference object with the points on the corresponding real world reference object, Computer for use in calibration. 12. The method of claim 11,
The processor comprising:
Programmatically adjusting a virtual view to substantially match the predefined points on the reference object with the points on a corresponding real-world reference object; And
For each of the acquired digital images, a difference on the image-based location between the at least one predefined point on the virtual object and the at least one corresponding point on the real world object A computer for use in location correction of a virtual model of a real world scene. 14. The method of claim 13,
Wherein the processor is further configured to minimize a difference on the image location by repeating the location transformation of the virtual object. As a computer program product,
Readable storage medium having computer-executable components for use in location correction of a virtual model of a real-world scene,
The components,
An execution component that, when executed by a processor, causes the processor to generate the virtual model, including a virtual object;
An acquisition component that, when executed by a processor, causes the processor to acquire at least one digital image of a real-world object within the real-world scene, the real-world object corresponding to the virtual object, ; And
Calculating a difference on an image-based position between at least one predefined point on the virtual object and at least one corresponding point on the real-world object when executed by the processor; Adjust at least one of a position and an orientation of the virtual object based on a difference on the image position; And a calibration component for causing the virtual model to adjust for the corrected location of the virtual object. 16. The method of claim 15,
The generating component may further cause the processor to:
Receiving a user selection of at least a portion of a plurality of identifiable points; And
And calculate a location transformation for the virtual object based on a plurality of identifiable points on the virtual object. 16. The method of claim 15,
The generating component may further cause the processor to:
Identify the virtual object and a corresponding reference object; And
Identify the at least one predefined point on the virtual object and a corresponding point on the reference object. 16. The method of claim 15,
Wherein the correction component further causes the processor to programmatically adjust the virtual view to substantially match the predefined points on the reference object with the points on the corresponding real world reference object, product. 16. The method of claim 15,
The correction component may further cause the processor to:
Adjust the virtual view programmatically; And
Based position between the at least one predefined point on the reference object and the at least one corresponding point on the real world reference object for each of the acquired digital images, Computer program products. 20. The method of claim 19,
Wherein the correction component further causes the processor to minimize a difference on the image location by repeating the location transformation of the virtual object.

Images