JP5635689B2 - Reference markers for augmented reality - Google Patents

Description

  Augmented reality often depicts a view or image of a real-world environment augmented with computer-generated content. The combination of real world images with computer generated content has proven useful in many different applications. Advertising, navigation, military, travel, education, sports, and entertainment are examples of areas where augmented reality can be used.

  The fusion of real-world environment images with computer-generated content results in an expanded image. However, the successful integration of real-world images with computer-generated content depends on how well the real-world images are recognized. For example, successful expansion of a monument image may depend on whether the monument is recognized by a device that displays the monument image. More specifically, the fusion of computer generated content such as monument name, location, creator, etc. may depend on whether the monuments in the image are recognized. If monuments are not recognized or known, providing computer-generated content is a very difficult proposition.

  One way to recognize objects in the image or real world objects is through the use of fiducial markers. Traditionally, fiducial markers are objects that appear in the resulting image that are used in the field of view of the imaging system. In other words, conventional markers are usually used as markers in the image, not as markers on real world objects.

  Often, the appearance of reference markers in the image serves as a reference for image scaling. For example, a reference marker at a known location in the image can be used to determine the relative scale of the image. Some cameras can create a rescross as a reference marker in the image. Reference markers can also be used to make image features more prominent. For example, a motion capture application can use a reference marker to track the movement of a marked object. A reference marker in the image may also allow independent images to be correlated.

  Unfortunately, the use of fiducial markers in augmented reality is very limited, and as mentioned earlier, fiducial markers are usually found in images rather than on real world objects.

  There may be some instances of real world objects with reference markers, but these markers are difficult to recognize. Recognition of medium and long distance reference markers is particularly problematic. Further, conventional fiducial markers cannot store a significant amount of data that can be converted or used to generate computer-generated content in augmented reality applications.

  Embodiments disclosed herein relate to fiducial markers. The fiducial marker stores data that can be used to augment the image with computer-generated content. In an exemplary embodiment, the fiducial marker can include a sheet of material and be attached to an object. The material is also configured to reflect electromagnetic radiation. The mask is applied to the material so as to cover the first part of the material and not the second part of the material. The first and second portions of material store data that is read by the reader.

  In an exemplary embodiment, the fiducial marker can include a material that can be attached to an object in the environment. The material includes a first portion having a retroreflector configured to reflect electromagnetic radiation received from the device back toward the device. The second part of the material is less reflective or non-reflective. The first portion and the second portion form a pattern and are arranged to store data within the reference marker. The electromagnetic radiation reflected by the material is modulated according to the pattern. The reflected electromagnetic radiation can be detected and read by a reader that emits the electromagnetic radiation.

  In another exemplary embodiment, a method of extending a real world image using a computer generated image includes emitting a light beam toward a reference marker disposed on the real world object. The reflected light beam contains data stored by the fiducial marker and can be read. From the fiducial marker, it is read by the device that emitted the light beam. Thereafter, computer generated content is generated using the data stored in the reflected light beam, and an expanded image containing the computer generated content is displayed on the display of the device.

  The foregoing summary is illustrative only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

It is a figure which shows the example of the environment containing the reference | standard marker arrange | positioned in space. It is a figure which shows the example of the apparatus which reads the reference | standard marker attached to the object. FIG. 6 illustrates an exemplary embodiment of a fiducial marker. FIG. 6 illustrates another exemplary embodiment of a fiducial marker. FIG. 6 illustrates an exemplary embodiment of a fiducial marker manufactured using a photonic crystal. FIG. 5 illustrates an exemplary embodiment of a fiducial marker that includes an array of retroreflectors. FIG. 6 illustrates an exemplary embodiment of a method for generating an expanded image using fiducial markers. FIG. 6 is a block diagram illustrating an example computing device arranged to read a fiducial marker and generate an augmented image using data read from the fiducial marker.

  In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized and other changes may be made without departing from the spirit or scope of the subject matter presented herein. The aspects of the present disclosure that are generally described herein and illustrated in the drawings can be arranged, replaced, combined, separated, and designed in a variety of different configurations, all of which are described herein. It will be readily understood that it is explicitly contemplated.

  Embodiments disclosed herein relate to fiducial markers that include fiducial markers for augmented reality applications. In augmented reality, fiducial markers can be implemented as tags that are placed in space or in the environment to assist in object recognition, object tracking, and / or object modeling. By way of example and not limitation, the reference marker embodiments disclosed herein support active interrogation, are recognizable at relatively longer distances, are less obvious to normal inspection, and Can store significantly more data or allow access to much more data.

  Some embodiments of the reference markers disclosed herein include a retroreflector. The retroreflector reflects light back toward the source of light. As a result, a plurality of devices can read the reference marker from a plurality of directions. Reference markers incorporating retroreflectors can be configured to store or encode data that can be read or interrogated by a properly configured device. By interrogating the reference marker, data stored or encoded in the reference marker can be read or retrieved.

  The retroreflector in the fiducial marker can be partially masked or otherwise obscured to encode or store data in the fiducial marker. The pattern of the mask formed in or on the reference marker can be one-dimensional or multi-dimensional. As a result, a reference marker that supports focused scanning read out allows an increase in storage factor. In other words, the amount of information that can be stored in the fiducial markers can be increased for a given size or area based on how the data is stored and / or how the data is read.

  A fiducial marker can be placed unobtrusively on an object in the environment. Thus, at least some embodiments of fiducial markers are attached to real world objects rather than being placed only in the resulting image. Devices that read these fiducial markers can generate augmented reality images using data stored within or made accessible by the fiducial markers. In addition, other components of the device (compass, global positioning sensor, etc.) can be used in combination with data read from fiducial markers to generate and display augmented reality images.

  A fiducial marker typically includes a sheet of material (eg, molded plastic, semiconductor material, printed paper, optical reflector, radio frequency reflector) that is attached to an object and configured to reflect electromagnetic radiation. The sheet of material can have a mask applied or secured thereon that obscures a portion of the reflective material. The mask can take the form of any material that blocks or masks reflected electromagnetic radiation, such as oils, metal shields, optically opaque paper, or dirt. The mask may be applied intentionally or inadvertently. The mask can be applied using mechanical attachments such as adhesives, stables, nails, rivets, glue, velcro, or other techniques.

  In some instances, the mask can be repositioned (eg, removed, given a new direction, and replaced with a new mask) so that the reference marker stores new information. For example, the sheet of material can include a reflective material. The masks can be separate so that the data stored changes as the pattern is changed by placing a new mask over the reflective material.

  Alternatively, some areas on the sheet of material may be free of reflective components. Thus, the sheet of material has a reflective portion and a portion that is less reflective or non-reflective. Together, the reflective and non-reflective portions can be arranged to store data that can be read by the reader. As previously mentioned, the mask may be formed by additional material obscuring the underlying reflective material, or by forming a sheet of material so that some portions do not contain or have reflective properties. it can. In another example, the non-reflective portion can be formed by damaging selected areas of the reflective portion. The part that is damaged is an example of a mask.

  In some examples, a sheet of material can be used to create multiple reference markers. The sheet of material is then diced to form individual fiducial markers, which are then packaged as needed or otherwise prepared for placement.

  In another example, the fiducial marker can be applied directly to the object, such as by painting the fiducial marker with retroreflective paint according to a pattern. A template having a pattern formed therein can be provided. A template can be applied to the object and a paint with retroreflective material can be applied to the template. When the template is removed, the paint on the object forms a reference marker that follows the pattern of the template.

  FIG. 1 shows an example of an environment including reference markers arranged in a space. In FIG. 1, the reference marker 122 is arranged on the object 120. The reference marker 122 can be conservatively placed and can be configured to be aesthetically compatible with the object 120. The size of the reference marker 122 may depend on the amount of data stored therein. This makes it possible to balance the environmental considerations (e.g. the influence of the reference marker on the environment) and the size of the reference marker 122.

  The fiducial marker 122 can be on a 1 inch square scale, but a better printing process can dramatically reduce the fiducial marker. Effective query distances may range from 1 inch to over one mile, for example and not limitation, depending on the sensitivity of the receiver, such as through a telescope imager. One skilled in the art can appreciate that the fiducial marker 122 can be a 1 inch square scale, or larger or smaller. The reading distance can depend on the sensitivity of the receiver.

  Also shown in FIG. 1 is a device 100 that includes a display 102. The device 100 can include an imaging system (eg, a camera or video camera) that allows the device 100 to display or show a real world image 104 of the object 120. Display 102 also presents computer-generated content 106 within real world image 104. Both real world image 104 and computer-generated content 106 are examples of augmented image 114.

  The computer-generated content 106 can be displayed in various ways and in various forms. The computer-generated content 106 (and thus the augmented image 114) can include text, images, videos, colors, highlights, or the like, or any combination thereof. The real world image 104 can be a real-time image of whatever is captured by the camera (or other input) of the device 100. As the device 100 moves and the view of the device 100 changes, the real world image 100 displayed by the device 100 changes accordingly. Where applicable, the positioning of the computer-generated content 106 in the augmented image 114 can be updated as the device 100 moves.

  Computer generated content 106 can be presented such that both real world image 104 and computer generated content 106 are visible simultaneously. Thus, in one example, at least one of the real world image 104 and the computer generated content 106 is partially transparent. In another example, computer generated content 106 may be placed on a particular portion of display 102 such that portions of real world image 104 are completely obscured by computer generated content. For example, text can be placed directly above the image 104 of the object 120 in the display 102 or at the bottom of the display 102 to minimally interfere with the real world image 104.

  In FIG. 1, the apparatus 100 includes a component 112 that is used to generate and display an expanded image 114. Component 112 includes, by way of example only, a camera, a compass, a global positioning sensor, or the like, or any combination thereof. The component 112 can be used by the device 100 to generate an expanded image 114. For example, the location of device 100 and the orientation of device 100 relative to the location of object 120 can be determined by a GPS sensor and a compass, respectively, and can be used in the generation of computer-generated content 106. In some instances, fiducial marker 122 can store location data. This location data may be combined with data from GPS sensors and / or compass to be used to locate the user relative to the object 120 and / or to generate computer-generated content 106. it can.

  In some instances, the device 100 may communicate with the server 108 via a network 110 such as a telephone network, the Internet, a local area network, or the like or any combination thereof. Information generated, detected, or otherwise determined by component 112 can be transmitted to server 108 and used to generate augmented image 114. The information determined by the component 112 can also be used directly by the device 100 and not transmitted to the server 108.

  For example, information read from the reference marker 122 can be transmitted to the server 108. Server 108 can respond to device 100 with at least some of the computer-generated content 106. In addition, a global positioning sensor can provide the location to the server 108. The location determined by the global positioning sensor and the data read from the reference marker 122 can be combined to generate computer generated content 106 and / or display the computer generated content 106. Alternatively, at least some of the data used to generate the augmented image 114 can be stored locally on the device 100. In addition, data represented in the computer-generated content 106 can be read from the reference marker 122 as a whole.

  The device 100 can be configured to read the fiducial marker 122. The data stored by the reference marker 122 or the data made accessible by the reference marker 122 can be rendered in the computer-generated content 106. For example, the reference marker 122 can store a description of the object 120. For example, if the object 120 is a monument, the reference marker 122 may store the creation date, artist name, monument reason, monument description, or the like. If the object 120 is a company, the reference marker 122 can store contact information such as a telephone number. One skilled in the art can appreciate that the data stored by or encoded within the reference marker 122 can vary significantly. Further, the format of the fiducial marker 122 can be standardized to allow any properly provisioned device to read and understand the data stored by the fiducial marker 122.

  In one embodiment, the fiducial marker 122 may store a link (eg, Uniform Resource Locator (URL)) that can be used by the device 100 to access data from the server 108. This example effectively allows the fiducial marker 122 to be updated over time and also increases the effective amount of data that can be stored or accessible by the fiducial marker 122. After the device 100 becomes accessible to the server 108, the corresponding data received from the server 108 includes the computer generated content 106. This allows the augmented image 114 to contain different types of data or different data at different times. The data that can be supplied by the server 108 can change over time, can be adapted to a particular device configuration, or the like can be done. For example, the server 108 may receive data describing the configuration of the device (eg, screen size, resolution, etc.). This allows the server 108 to deliver computer generated content specifically prepared for the requesting device.

  FIG. 2 shows an example of an apparatus 100 that reads the reference marker 122. The component 112 of the apparatus 100 can also include a light source 202 and a photodetector 204. The light source 202 is configured to emit a light beam. The light source 202 can include, for example, a laser that emits light at a predetermined frequency or within a predetermined frequency range. The detector 204 can be configured to detect the frequency of the light emitted by the light source 202. The detector 204 can be a photodetector such as a photodiode or the like.

  In FIG. 2, the light emitted by the light source 202 of the device 100 is directed to the object 100, and more specifically to the reference marker 122. The reference marker 122 reflects light back toward the device 100. The apparatus 100 uses the detector 204 to detect the reflected light and reads the data stored in the reference marker 122.

  In one example, the detector 204 detects the intensity of the reflected light and generates a waveform that is shaped according to the mask pattern in the reference marker 122. The generated waveform can then be decoded and used to generate computer generated content 106 included in the augmented image 114.

  The fiducial marker 122 can be or include a retroreflector that reflects light back toward its source. In this example, the fiducial marker 122 reflects the light emitted by the light source 202 back toward the detector 204. More generally, the fiducial marker 122 reflects the electromagnetic radiation back toward a vector that is parallel or substantially parallel to the source of the electromagnetic radiation. As described in more detail herein, the fiducial marker 122 can include a corner cube, a photonic crystal, a cat's eye retroreflector, or the like or any combination thereof.

  FIG. 3 illustrates an exemplary embodiment of the fiducial marker 300. The reference marker 300 is an example of the reference marker 122. In this example, the reference marker 300 is placed in a single dimension. The fiducial marker 122 includes a reflective area 304 and an area 302 that is less reflective or non-reflective. When the light source 202 reads the fiducial marker 122, the detector 204 can generate a waveform 306 corresponding to the pattern formed by the reflective areas 304 and the areas 302 that are less reflective or non-reflective. A simple passive reading of fiducial marker 300 (or of other fiducial markers disclosed herein) can be achieved by optically observing ambient reflected light. Instead, the beam of light can be actively scanned over the fiducial marker 300 by techniques such as raster scanning. Both approaches can prove effective at both close range and distances greater than 1 mile. However, longer distances may require both greater illumination energy and a larger size of fiducial marker 300.

  The reference marker 300 can be manufactured in different ways. The reference marker 300 can include a sheet of corner cube reflectors. However, a mask can be applied to the surface of the sheet to form areas 302 that are less reflective or non-reflective. Alternatively, areas 302 that are less reflective or non-reflective may not include a corner cube reflector.

  The fiducial marker 300 can be read using a raster scan technique. For example, the data in the reference marker 300 can be read by using a focused laser as the light source.

  FIG. 4 shows another exemplary embodiment of fiducial marker 400. The reference marker 400 is another example of the reference marker 122 that stores or codes data in multiple dimensions. The reference marker 400 includes a reflective area 402 and an area 404 that is less reflective or non-reflective. In this example, data stored or represented by fiducial marker 400 can be read using a scattered laser that can illuminate an area of fiducial marker or portions of fiducial marker 400. In this example, the multidimensional area of the reference marker 400 can be read in parallel using a suitable light source. Multidimensional coding of the reference marker 400 can encode much more information for a given size of the reference marker 400. Multi-dimensional coding allows a fixed size marker 400 to encode more information and / or use smaller reference markers. Even smaller multidimensional markers can store more data than larger single dimensional reference markers. This multidimensional approach further enables parallel reading. For example, two illumination sources can be used to simultaneously read data from the first two rows.

  In comparison, the fiducial marker 400 can store more data than the fiducial marker 300 and can therefore occupy less area. This can be useful, for example, when placing a fiducial marker on an object in the environment.

  FIG. 5 illustrates an exemplary embodiment of a fiducial marker 500 manufactured from a photonic crystal. The reference marker 500 is another example of the reference marker 122. In this example, fiducial marker 500 is formed from a plurality of layers 502 and 504. The layer 502 has a refractive index different from that of the layer 504. Layers 502 and 504 depict alternating layers of materials having a high and low refractive index or a high and low dielectric constant. Layers 504 and 504 can be formed or grown on the substrate. Examples of suitable materials include silicon, plastic, and some semi-rigid gels. The thickness of each of layers 502 and 504 can be configured for a particular frequency of light, for example. In some examples, the thickness of each of layers 502 and 504 is 1/4 of the desired wavelength.

  The correct choice of material for layers 502 and 504 results in a structure that reflects certain frequencies of light. As a result, the fiducial marker 500 can reflect light from the source back toward the source. The fiducial marker 500 can be constructed to provide unidirectional readout without having to align the illumination source to the fiducial marker 500 in any way.

  Encoding of data within the fiducial marker 500 can be accomplished by providing a mask 508. By masking a portion of the surface of the fiducial marker 500 according to the pattern, the data can be encoded and read by the device. In this example, mask 508 can include an additional layer of material that is selectively formed on the surface of fiducial marker 500. For example, the mask 508 can be a metal layer. Area 506 may be reflective and may not have a mask 508. In some examples, the fiducial marker 500 can have a protective layer formed on top of the fiducial marker 500 that does not interfere with the operation or function of the fiducial marker 500.

  FIG. 5 also shows that the mask can be formed by etching the reference marker 500. For example, the fiducial marker 500 can be etched and another material 510 can be deposited on the etched areas of the fiducial marker 500 to form a mask and create a pattern within the fiducial marker 500. Material 510 may not include alternating layers, or may be non-reflective for certain wavelengths of light such that the associated frequency or wavelength of light is not reflected for the portion of fiducial marker 500 occupied by material 510. It can be a material that is reflective.

  FIG. 6 illustrates an exemplary embodiment of a fiducial marker 600 that includes an array 602 of retroreflectors. The reference marker 600 is an example of the reference marker 122. In this example, the retroreflector array 602 includes a corner cube 604. Array 602 can include a plurality of rows and / or columns of corner cube 604. An array 602 of corner cubes 604 can be formed using molded plastic. Each of the corner cubes 604 typically includes three mutually perpendicular flat surfaces that reflect electromagnetic radiation, including light, back toward the source.

  The fiducial marker 600 can be read from different positions or locations after being placed in the environment. In other words, light that approaches the fiducial marker 600 from the front or at an angle is reflected back toward the source location of the light. As a result, the orientation of the reference marker 600 may not affect the ability of the reference marker 600 to be read by the device. The reference marker 600 can also include a mask to form a pattern for storing data. A mask can be formed by obscuring some of the corner cubes 604 or by fabricating the array 602 such that there are no corner cubes in an area.

  FIG. 7 illustrates an exemplary embodiment of a method 700 for generating an augmented image. At block 702 of method 700, the apparatus emits a light beam toward the reference marker. The light source can be configured to scan the fiducial marker, or the device can be moved so that the emitted light can interrogate the fiducial marker.

  In block 704, the device detects and reads the reflected light beam. The reflected light beam is typically modulated according to a reference marker mask or pattern. Thus, the reflected light beam contains data stored by the fiducial marker.

  At block 706, the data in the reflected light beam is used to generate computer generated content to be included in the expanded image. In one example, the computer generated content corresponds to data stored in a fiducial marker. In another example, data stored in a fiducial marker may allow the device to access computer generated content associated with the fiducial marker (eg, via the Internet). At block 708, the augmented image including the computer generated content and / or the image of the real world object is displayed on the device.

  Those skilled in the art will appreciate that the functions performed by the processes and methods can be performed in a different order for this process and method, as well as other processes and methods disclosed herein. Further, the outlined steps and actions are provided as examples only, and some steps and actions may be made optional, without departing from the essence of the disclosed embodiments, and with fewer steps and actions. It can be combined with actions or extended to additional steps and actions.

  This disclosure should not be limited to the specific embodiments described herein that are intended to be exemplary of various aspects. Many modifications and variations can be made without departing from its spirit and scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatus within the scope of the present disclosure will be apparent to those skilled in the art from the foregoing description, in addition to the methods and apparatus listed herein. Such modifications and variations are intended to be included within the scope of the appended claims. The present disclosure should be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It should be understood that the present disclosure is not limited to a particular method, reagent, compound synthesis, or biological system, and that these methods, reagents, compound synthesis, or biological system can of course vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

  In an exemplary embodiment, any of the operations, processes, etc. described herein can be implemented as computer readable instructions stored on a computer readable medium. Computer readable instructions may be executed by a processor of a mobile unit, a network element, and / or any other computing device.

  There is little difference between the hardware implementation and the software implementation in terms of the system. The use of hardware or software is generally a design choice that represents a cost-effective tradeoff (although not always, but in some situations the choice between hardware and software can be important) . There are a variety of means (eg, hardware, software, and / or firmware) that can provide the processes and / or systems and / or other techniques described herein, and the preferred means of And / or depending on the circumstances in which the system and / or other technologies are introduced. For example, if the implementer determines that speed and accuracy are most important, the implementer can primarily select a hardware and / or firmware achievement means. If flexibility is paramount, implementers can primarily choose software implementations. Or, as yet another alternative, the implementer can select any combination of hardware, software, and / or firmware.

  In the foregoing detailed description, various embodiments of apparatus and / or processes have been described through the use of block diagrams, flowcharts, and / or examples. As long as such a block diagram, flowchart, and / or example includes one or more functions and / or operations, each function and / or operation in such a block diagram, flowchart, or example may include: Those skilled in the art will appreciate that a wide range of hardware, software, firmware, or virtually any combination thereof can be implemented individually and / or collectively. In certain embodiments, some portions of the subject matter described herein include application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or other integration schemes. Can be implemented. However, some aspects of the embodiments disclosed herein may be in whole or in part as one or more computer programs (eg, one or more) running on one or more computers. As one or more programs running on one computer system) as one or more programs running on one or more processors (eg, one or more running on one or more microprocessors) Those skilled in the art will recognize that they can be equivalently implemented in an integrated circuit (as multiple programs), as firmware, or virtually any combination thereof, as well as electrical circuit design and / or software and / or Or the firmware coding is in light of this disclosure. That Te is well within the abilities of those skilled in the art, those skilled in the art will recognize. Further, those skilled in the art will appreciate that the mechanisms of the subject matter described herein can be distributed as various types of program products, and exemplary embodiments of the subject matter described herein. Will be understood regardless of the specific type of signaling medium used to actually perform the distribution. Examples of signal transmission media include recordable types of media such as floppy disks, hard disk drives, CDs, DVDs, digital tapes, computer memory, and digital and / or analog communication media (eg, fiber optic cables). Communication type media such as, but not limited to, waveguides, wired communication links, wireless communication links, etc.).

  It is known in the art to describe an apparatus and / or process in the manner described herein and then use an engineering scheme to integrate the apparatus and / or process so described into a data processing system. Those skilled in the art will recognize that That is, at least some of the devices and / or processes described herein can be integrated into a data processing system with a reasonable number of experiments. Conventional data processing systems generally include system unit housings, video display devices, memories such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computing entities such as operating systems, drivers, graphical user interfaces , And one or more of the application programs, one or more interactive devices such as a touchpad or screen, and / or a control system including a feedback loop and a control motor (eg, position sensing and / or speed sensing) Those skilled in the art will understand that it includes control motors for feedback, component movement and / or quantity adjustment).A typical data processing system may be implemented utilizing suitable commercially available components, such as those typically found in data computing / communication systems and / or network computing / communication systems.

  The subject matter described herein often illustrates various components, which are encompassed by or otherwise connected to various other components. It will be appreciated that the architecture so illustrated is merely exemplary and in fact many other architectures that implement the same functionality can be implemented. In a conceptual sense, any configuration of components that achieve the same function is effectively “associated” to achieve the desired function. Thus, any two components herein combined to achieve a particular function are “associated” with each other so that the desired function is achieved, regardless of architecture or intermediate components. You can see that. Similarly, any two components so associated may be considered “operably connected” or “operably coupled” to each other to achieve the desired functionality, and as such Any two components that can be associated with can also be considered "operably coupled" to each other to achieve the desired functionality. Examples where it can be operatively coupled include physically interlockable and / or physically interacting components, and / or wirelessly interacting and / or wirelessly interacting components, And / or components that interact logically and / or logically interact with each other.

  FIG. 8 is a block diagram illustrating an example computing device 800 arranged to read a fiducial marker and generate an expanded image using data read from the fiducial marker in accordance with the present invention. In a very basic configuration 802, the computing device 800 typically includes one or more processors 804 and system memory 806. Memory bus 808 can be used for communication between processor 804 and system memory 806.

  Depending on the desired configuration, processor 804 can be any type including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. . The processor 804 can include another level of caching, such as a level 1 cache 810 and a level 2 cache 812, a processor core 814, and a register 816. The example processor core 814 may include an arithmetic logic unit (ALU), a floating point decimal unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 818 can be used with the processor 804, or in some implementations the memory controller 818 can be an internal part of the processor 804.

  Depending on the desired configuration, system memory 806 may be any type of memory including, but not limited to, volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. It can be. The system memory 806 can include an operating system 820, one or more applications 822, and program data 824. Application 822 can include an augmented reality application 826 that is arranged to create and display content that is read from or made accessible by reading fiducial markers. Program data 824 may include fiducial marker data 828 read from fiducial markers that may be useful for generating an expanded image as described herein. The fiducial marker data 828 can also include other data that can be used in generating the expanded image. In some embodiments, the application 822 may be arranged to operate with the program data 824 on the operating system 820 such that an expanded image is generated and displayed. This described basic configuration 802 is illustrated in FIG. 8 by the components within the inner dashed line.

  The computing device 800 may have additional features or functionality and additional interfaces to facilitate communication between the basic configuration 802 and all necessary devices and interfaces. For example, the bus / interface controller 830 can be used to facilitate communication between the basic configuration 802 and one or more data storage devices 832 via the storage interface bus 834. The data storage device 832 can be a removable storage device 836, a non-removable storage device 838, or a combination thereof. Examples of removable storage devices and non-removable storage devices include, for example, magnetic disk devices such as flexible disk drives and hard disk drives (HDD), compact disk (CD) drives, or digital versatile disks (DVDs). ) Optical disk drives such as drives, solid state drives (SSD), and tape drives. Example computer storage media are volatile and non-volatile, removable and non-implemented in any method or technique for storing information such as computer readable instructions, data structures, program modules, or other data. Removable media can be included.

  System memory 806, removable storage device 836, and non-removable storage device 838 are examples of computer storage media. Computer storage media can be RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disc (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage, or other Including, but not limited to, any magnetic storage device or any other medium that can be used to store desired information and that can be accessed by computing device 800. Any such computer storage media may be part of computing device 800.

  The computing device 800 includes an interface bus 840 that facilitates communication via the bus / interface controller 830 from various interface devices (eg, output device 842, peripheral interface 844, and communication device 846) to the basic configuration 802. It can also be included. The example output device 842 includes a graphics processing unit 848 and an audio processing unit 850 that can be configured to communicate to various external devices such as a display or speakers via one or more A / V ports 852. The example peripheral interface 844 can be an input device (eg, keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral device (eg, printer, via one or more I / O ports 858). A serial interface controller 854 or a parallel interface controller 856 that can be configured to communicate with an external device such as a scanner. The example communication device 846 includes a network controller 860 that can be arranged to facilitate communication with one or more other computing devices 862 over a network communication link via one or more communication ports 864. Including.

  A network communication link may be an example of a communication medium. Communication media typically can be implemented by computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, which can be any information A delivery medium can be included. A “modulated data signal” can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR), and other wireless media. be able to. As used herein, the term computer readable media can include both storage media and communication media.

  The computing device 800 can be a small form such as a cell phone, personal digital assistant (PDA), personal media player device, wireless web watch device, personal headset device, application specific device, or a hybrid device including any of the above functions. It can be implemented as part of a factor portable (or mobile) electronic device. The computing device 800 may also be implemented as a personal computer that includes both laptop computer and non-laptop computer configurations.

  For the use of substantially all plural and / or singular terms herein, those skilled in the art will recognize from the plural to the singular and / or singular as appropriate to the situation and / or application. You can convert from shape to plural. Various singular / plural permutations can be clearly described herein for ease of understanding.

  In general, the terms used herein, particularly in the appended claims (eg, the body of the appended claims), are intended throughout as “open” terms. Will be understood by those skilled in the art (eg, the term “including” should be construed as “including but not limited to” and the term “having”). Should be interpreted as “having at least,” and the term “includes” should be interpreted as “including but not limited to”. ,Such). Where a specific number of statements is intended in the claims to be introduced, such intentions will be explicitly stated in the claims, and in the absence of such statements, such intentions It will be further appreciated by those skilled in the art that is not present. For example, as an aid to understanding, the appended claims use the introductory phrases “at least one” and “one or more” to guide the claims. May include that. However, the use of such phrases may be used even if the same claim contains indefinite articles such as the introductory phrases “one or more” or “at least one” and “a” or “an”. Embodiments in which the introduction of a claim statement by the indefinite article "a" or "an" includes any particular claim, including the claim description so introduced, is merely one such description. (Eg, “a” and / or “an” should be construed to mean “at least one” or “one or more”). Should be). The same applies to the use of definite articles used to introduce claim recitations. Further, even if a specific number is explicitly stated in the description of the claim to be introduced, it should be understood that such a description should be interpreted to mean at least the number stated. (For example, the mere description of “two descriptions” without other modifiers means at least two descriptions, or two or more descriptions). Further, in cases where a conventional expression similar to “at least one of A, B and C, etc.” is used, such syntax usually means that one skilled in the art would understand the conventional expression. Contemplated (eg, “a system having at least one of A, B, and C” includes A only, B only, C only, A and B together, A and C together, B and C together And / or systems having both A, B, and C together, etc.). In cases where a customary expression similar to “at least one of A, B, or C, etc.” is used, such syntax is usually intended in the sense that one skilled in the art would understand the customary expression. (Eg, “a system having at least one of A, B, or C” includes A only, B only, C only, A and B together, A and C together, B and C together, And / or systems having both A, B, and C together, etc.). Any disjunctive word and / or phrase that presents two or more alternative terms may be either one of the terms, anywhere in the specification, claims, or drawings. It will be further understood by those skilled in the art that it should be understood that the possibility of including either of the terms (both terms), or both of them. For example, it will be understood that the phrase “A or B” includes the possibilities of “A” or “B” or “A and B”.

  Further, if a feature or aspect of the present disclosure is described with respect to a Markush group, those skilled in the art will appreciate that the present disclosure is thereby described with respect to any individual member or sub-group of members. Will.

  In all respects, all ranges disclosed herein are intended to include any and all possible subranges and combinations of subranges, such as with respect to providing written descriptions, as will be appreciated by those skilled in the art Is also included. Easily recognize all listed ranges as fully describing and enabling the same range divided into at least equal halves, 1/3, 1/4, 1/5, 1/10, etc. Can do. As a non-limiting example, each range described herein can be easily broken down into lower 1/3, central 1/3, upper 1/3, and the like. As will also be appreciated by those skilled in the art, all terms such as “up to”, “at least”, and the like include the number indicated and are then described above. Reference is made to a range that can be divided into subranges. Finally, as will be appreciated by those skilled in the art, the range includes each individual member. Thus, for example, a group having 1-3 cells refers to groups having 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers to a group having 1, 2, 3, 4, or 5 cells, and so forth.

  From the foregoing, it should be understood that various embodiments of the disclosure have been described herein for purposes of illustration and that various changes can be made without departing from the scope and spirit of the disclosure. Accordingly, the various embodiments disclosed herein are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims (20)

A sheet of material attached to an object, the material comprising a first portion and a second portion, the material configured to reflect electromagnetic radiation;
A mask applied to the material, the mask obscuring the first portion of the material and not obscuring the second portion of the material;
The first part and the second part store data read by a device;
The mask is made separately from the sheet and can be rearranged to store new data in the first part and the second part,
Reference marker.   The fiducial marker of claim 1, wherein the material comprises a plurality of corner cubes.   The fiducial marker of claim 1, wherein the material comprises at least one photonic crystal.   The reference marker according to claim 3, wherein the photonic crystal includes a multidimensional photonic crystal.   The first portion of the material and the second portion of the material are arranged to convey data to the device when read by the device, and the light emitted by the device is data stored in the material The fiducial marker of claim 1, wherein the reference marker is reflected back to the device according to the mask so that can be read by the device. The first and second portions of material are a description of the object , a reference for scaling the object, a correlator for the object to a second object, and comparing the location of the reference marker with a second reference marker 2. Stores data that is at least one of a URL (Uniform Resource Locator) that sometimes provides a relative scale and that allows the device to access computer-generated content over a network. Reference marker according to.   The fiducial marker of claim 1, wherein the electromagnetic radiation includes light and the second portion of the material includes a retroreflector that reflects the light back toward the source of light. A method for extending a real-world image using a computer-generated image,
Emitting a light beam toward a tangible fiducial marker disposed on a real world object, the fiducial marker including a mask covering a portion of the fiducial marker , the mask providing a pattern, the mask Emitting a light beam, which can be replaced with a new mask so that the pattern is changed ;
Reading a light beam reflected from the tangible fiducial marker, wherein the reflected light beam includes data stored by the tangible fiducial marker;
Generating computer generated content using the data;
Displaying an augmented image on a display of a device, the augmented image comprising displaying the computer-generated content and a view of the real world object.   The method of claim 8, wherein the light beam comprises a laser beam, and emitting the light beam further comprises scanning the tangible fiducial marker with the laser beam.   9. The method of claim 8, wherein the tangible fiducial marker comprises a plurality of retroreflectors arranged to store the data and arranged to generate the reflected light beam. The data includes a URL (Uniform Resource Locator),
The method of claim 8, further comprising accessing a server using the URL for the computer generated content.   9. The reading of the light beam reflected from the tangible reference marker comprises demodulating a signal generated from the reflected light beam to extract the stored data. the method of.   9. The method of claim 8, further comprising using an additional component of the device to generate the computer generated content, the additional component including a global positioning sensor and a compass. The tangible fiducial marker includes a plurality of corner cubes arranged in an array;
A portion of the plurality of corner cubes is obscured such that the array of the plurality of corner cubes stores data that can be read by the light beam, or a portion of the array is The method of claim 8, wherein there is no corner cube, such that the corner cube and the portion of the array without the corner cube are arranged to be read by the light beam. Reference markers of the tangible comprises a photonic crystal including the mask formed thereon to form a partially reflecting portion and reflective are lower, or non-reflective, the reflective portion and the reflected 9. A method according to claim 8, wherein a less prone or non-reflective portion is arranged to store the data. A material attached to an object in the environment,
A first portion including a retroreflector configured to reflect electromagnetic radiation received from the device back toward the device;
A second part that is less reflective or non-reflective, wherein the first part and the second part are arranged to form a pattern for storing data, the second part A material including a second portion, wherein data stored by changing the pattern is changed by replacement of one mask made separately including
Electromagnetic radiation reflected by said material is modulated according to the pattern, the reflected electromagnetic radiation is detected by the device emitting the electromagnetic radiation towards said first portion and said second portion, be read Can be a reference marker.   The reference marker according to claim 16, wherein the retroreflector includes at least one of a plurality of corner cubes or photonic crystals arranged in an array.   The fiducial marker of claim 17, wherein the second portion includes a mask applied to the material to a portion of the plurality of corner cubes or to the photonic crystal.   18. A fiducial marker according to claim 17, wherein the second part is free of retroreflectors. The data is
A description of the object,
A URL (Uniform Resource Locator) that allows the device to access computer generated content for inclusion in an augmented image displayed on the device over a network, or the computer generated content 17. A fiducial marker according to claim 16, which is at least one of those used to generate.

Images