TWI771086B - Method and system for showing objects in augmented reality environment - Google Patents

Abstract

An embodiment of the invention provides a method for showing objects in augmented reality environment, which includes the following steps: Augmented Reality (AR) device obtains spatial information and object identification (ID) from an anchor device. The AR device obtains parameters of object from a cloud system. The parameters of object include the image of the object, the audio of the object and the first spatial relationship between the object and an anchor device. The AR device obtains a second spatial relationship between the AR device and the anchor device. A third spatial relationship between the object and the AR device can be obtained according to the first spatial relationship and the second spatial relationship. The image of the object and the audio of the object shows in the AR device according to the third spatial relationship.

Description

A method and system for displaying objects in an augmented reality environment

The present invention relates to augmented reality technology, and in particular, to a method and system for displaying objects in an augmented reality environment.

In the display of objects in the existing augmented reality, the shared user needs to move the camera to scan the physical anchor point, then download the object information according to the characteristics of the physical anchor point, and continuously calculate the image information of the camera to confirm the space between the anchor point and the object It consumes a lot of computing power and electricity.

In view of this, the present invention provides a method for displaying objects in an augmented reality environment. An anchor device is used to actively broadcast and provide spatial information and related information of objects in the augmented reality environment. After the anchor device is installed, the spatial information of the objects in the augmented reality can be received, and the objects in the augmented reality can be displayed without scanning or any operation.

An embodiment of the present invention provides a method for displaying objects in an augmented reality environment. The method includes the following steps: an augmented reality (Augmented Reality, AR) device obtains spatial information and an object identification code of an anchor device; The identification code obtains the object parameters corresponding to the object identification code from the anchor point cloud. The object parameters include the object image, the object audio, and the first spatial relationship of the object relative to the anchor point device. The AR device obtains the first space relationship relative to the anchor point device according to the spatial information. Two spatial relations, according to the first empty The interrelationship and the second spatial relationship obtain a third spatial relationship of the object relative to the AR device, and the AR device displays the object image and the object audio in the third spatial relationship.

An embodiment of the present invention further provides a display system for objects in an augmented reality environment, including: an augmented reality (AR) device that acquires spatial information and an object identification code from an anchor device, and the AR device obtains the spatial information and the object identification code according to the object identification code. , obtain the object parameters corresponding to the object identification code from the anchor point cloud. The object parameters include the object image, the object audio, and the first spatial relationship of the object relative to the anchor point device. According to the spatial information, the AR device relative to the anchor point device is obtained. Spatial relationship, according to the object identification code, obtain the object parameters from the anchor point cloud, obtain the third spatial relationship of the object relative to the AR device according to the first spatial relationship and the second spatial relationship, and the AR device displays the object image and the object in the third spatial relationship audio.

The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention.

210, 300, 410, 510: AR installations

220, 520, 650, 750, 780: Anchor point devices

230, 530, 630, 730: Anchor cloud

240,640,740,790:Object

250, 260, 270: distance

251, 261, 271: Horizontal angle

301,411: First Speaker

302,412: Second speaker

303: Display unit

311~318: Location

421,540: first object

422,550: Second Object

431: First Distance

432: Second Distance

433: Third Distance

610, 710: Speakers

620, 720, 770: Photographic installations

660:Limited Viewers

670: Average viewer

760: Viewer

S101~S105: Steps

FIG. 1 is a flow chart showing a method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

FIG. 2 is a diagram showing the structure of a display system for objects in an augmented reality environment according to an embodiment of the present invention.

FIG. 3 is a schematic diagram showing when objects are located at different positions in the augmented reality environment according to the embodiment of the present invention.

FIG. 4 is a schematic diagram showing the display of a plurality of objects in an augmented reality environment according to an embodiment of the present invention.

FIG. 5 is a schematic diagram showing the display of multiple objects in an augmented reality environment according to another embodiment of the present invention.

FIG. 6 is a schematic diagram showing a one-way application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

FIG. 7 is a schematic diagram showing a bidirectional application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.

In the following description, many specific details are set forth in order to facilitate a full understanding of the present invention, and the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

It should be noted that the descriptions involving "first", "second", etc. in the present invention are only for the purpose of description, and should not be construed as indicating or implying their relative importance or implying the number of indicated technical features . Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In addition, the technical solutions between the various embodiments can be combined with each other, but must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of such technical solutions does not exist. , is not within the protection scope of the present invention.

FIG. 1 is a flow chart showing a method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

Step S101 , in the augmented reality environment, the anchor device continues to send out spatial information and object identification codes. When the user wears the augmented reality (AR) device and enters the augmented reality environment, the user can self-define the device from the anchor point. The device obtains spatial information and object identifiers.

Step S102, the AR device obtains the object parameters corresponding to the object identification code from the anchor point cloud according to the object identification code obtained from the anchor point device, and the object parameters include the image information of the object, the object audio information, and the relative position of the object relative to the anchor point device. The first spatial relationship, wherein the image information includes the shape, color, size, and orientation of the object, and the first spatial relationship includes the orientation, distance, and horizontal angle of the object relative to the anchor device.

Step S103 , the AR device obtains a second spatial relationship between the AR device and the anchor device according to the spatial information. The second spatial relationship includes the azimuth, distance, and horizontal angle of the AR device relative to the anchor device. The spatial information includes the azimuth, the horizontal angle, the first spatial information packet sent by the anchor device through the first communication technology and the second spatial information packet sent by the second communication technology, the first spatial information packet and the second spatial information packet. Send out at the same time, calculate the distance between the AR device and the anchor device through the known propagation rate of the first communication technology and the known second communication technology and the time difference between the AR device receiving the first spatial information packet and the second spatial information packet. Distance, the distance is the distance in the second spatial relationship, such as in Taiwan Patent Application No. 110119736, entitled "METHOD AND SYSTEM FOR ESTABLISHING ANCHORS IN AUGMENTED REALITY )" revealed.

Step S104, taking the anchor device as the origin in the augmented reality environment, the first position of the object in the augmented reality environment can be obtained according to the first spatial relationship, and the AR device in the augmented reality environment can be obtained according to the second spatial relationship. The second position in the real environment. Therefore, the third spatial relationship of the object relative to the AR device is calculated and obtained according to the first position and the second position. The third spatial relationship includes the position, distance and level of the object relative to the AR device. angle.

Step S105, the AR device displays the object according to the image information of the object and the audio of the object, and adjusts the size of the image of the object, the volume and directionality of the audio of the object according to the distance in the third spatial relationship to display the object, so as to create the object in the augmented reality environment Closer to the real sense of space.

FIG. 2 is a diagram showing the structure of a display system for objects in an augmented reality environment according to an embodiment of the present invention. As shown in FIG. 2 , the display system includes an AR device 210 , an anchor point device 220 , an anchor point cloud 230 , and an object 240 . The AR device 210 obtains the spatial information and the object identification code of the object 240 from the anchor point device 220 , the AR device 210 obtains the object parameters of the object 240 from the anchor point cloud 230 according to the object identification code, and the anchor point cloud 230 is used to store and provide the object 240 related object parameters, the object parameters include the object image, the object audio and the first spatial relationship of the object 240 relative to the anchor point device 220, the first spatial relationship includes the position, distance 250 and 250 of the object 240 relative to the anchor point device 220. Horizontal angle 251.

The second spatial relationship of the AR device 210 relative to the anchor device 220 is obtained according to the spatial information provided by the anchor device 220 . The second spatial relationship includes the position, distance 260 and horizontal angle 261 of the AR device 210 relative to the anchor device 220 . . The spatial information includes the position of the AR device 210 relative to the anchor device 220 , the horizontal included angle 261 , the first spatial information packet sent by the anchor device 220 through the first communication technology, and the second spatial information sent by the second communication technology. packets, the first spatial information packet and the second spatial information packet are sent simultaneously, and the first spatial information packet and the second spatial information are received by the AR device 210 through the known first communication technology and the known propagation rate of the second communication technology and the AR device 210 The time difference of the packets calculates the distance 260 between the AR device 210 and the anchor device 230 , and the distance 260 is the distance 260 in the second spatial relationship.

The AR device 210 obtains the object parameters of the object 240 from the anchor point cloud 230 according to the object identification code obtained from the anchor point device 220 , and takes the anchor point device 220 as the origin in the augmented reality environment, and can be obtained according to the first spatial relationship For the first position of the object 240 in the augmented reality environment, the second position of the AR device 210 in the augmented reality environment can be obtained according to the second spatial relationship. Calculate the third spatial relationship of the object 240 relative to the AR device 210 . The third spatial relationship includes the orientation, distance 270 and horizontal angle 271 of the object 240 relative to the AR device 210 . The AR device 210 displays the object image according to the third spatial relationship. and object audio.

The AR device 210 further includes an inertial measurement unit for obtaining the moving direction, moving distance, and moving height of the AR device 210 when the AR device 210 moves. The AR device 210 will instantly update the second For the spatial relationship and the second position, the third spatial relationship is updated according to the first position and the updated second position, so that the AR device 210 can display the object image and the object audio according to the updated third spatial relationship.

FIG. 3 is a schematic diagram showing that objects are displayed at different positions in an augmented reality environment according to an embodiment of the present invention. In one embodiment, the AR device 300 includes a first speaker 301, a second speaker 302, and a display unit 303. When the user wears the AR device 300, the first speaker 301 is close to the user's left ear, and the second speaker 302 is close to the user's right ear. Ear, when the AR device 300 displays the object, the object image and the object audio will adjust the outputs of the first speaker 301 , the second speaker 302 and the display unit 303 according to the third spatial relationship.

As shown in FIG. 3 , the AR device 300 obtains the azimuth, distance and horizontal angle of the object relative to the AR device 300 according to the third spatial relationship. Taking the position 311 directly in front of the AR device 300 as an example, the display unit 303 is located at the position 311 The object image is displayed, and the size of the object image and the object audio are adjusted according to the distance and the horizontal included angle in the third spatial relationship. The output time and volume of the speaker 302 are consistent.

Taking the AR device 300 obtaining the object according to the third spatial relationship and displaying it at the position 312 as an example, the display unit 303 displays the object image at the position 312 according to the distance and the horizontal angle in the third spatial relationship. To adjust the size of the object image and the audio of the object, the first speaker 301 outputs the audio of the object before the second speaker 302 , and the output volume of the first speaker 301 is also greater than that of the second speaker 302 .

Taking the AR device 300 obtaining the object according to the third spatial relationship and displaying it at position 313 as an example, since the object is located on the left side of the AR device 300, the display unit 303 does not display the object image, and the object audio is performed according to the distance and the horizontal angle in the third spatial relationship. The volume of the object audio output by the second speaker 302 is lower or silent than that of the first speaker 301 , and the output time of the second speaker 302 is later than the output time of the first speaker 301 .

By analogy, according to the third spatial relationship, when the object is displayed at the position 311 - the position 318 , the AR device 300 adjusts the object image and the object audio as shown in Table 1.

FIG. 4 is a schematic diagram showing the representation of a plurality of objects in an augmented reality environment according to an embodiment of the present invention. As shown in FIG. 4 , this embodiment includes an AR device 410 , a first object 421 and a second object 422 . The AR device 410 includes a first speaker 411 and a second speaker 412. When the user wears the AR device 410, the first speaker 411 is close to the user's left ear, and the second speaker 412 is close to the user's right ear. The first object 421 includes the audio of the object, and the second object 422 is a large-scale object in the shape of a house as shown in FIG. The position of the second object 422 and the position of the second object 422 relative to the AR device 410 can be obtained by obtaining a second distance 432 between the left wall of the first object 421 and the second object 422, and a third distance 433 between the AR device 410 and the left wall of the second object 422 .

The time for the first speaker 411 to output the object audio of the first object 421 will be adjusted according to "(the second distance 432 + the third distance 433)/speed of sound", and the time for the second speaker 412 to output the object audio of the first object 421 will be adjusted according to "First distance 431/speed of sound" is adjusted. The volume of the object audio output by the second speaker 412 of the first object 421 will be based on "system volume setting value - 20log first distance 431/reference distance", wherein the system volume setting value is the initial volume of the sound emitted by the creator to the object Definition, the reference distance is defined by the creator, the volume of the object audio of the first object 421 output by the first speaker 411 will be based on "the volume output by the second speaker-20log* (the second distance 432 + the third distance 433) / the first Distance 431" to adjust.

FIG. 5 is a schematic diagram showing the display of multiple objects in the augmented reality environment according to the embodiment of the present invention. As shown in FIG. 5 , the AR device 510 obtains the second spatial relationship of the AR device 510 relative to the anchor device 520 and the object identification code through the anchor device 520 , and obtains the corresponding object parameters from the anchor point cloud 530 through the object identification code. In this embodiment, the object identifier includes the object identifiers of the first object 540 and the second object 550 Don't code. As shown in FIG. 5 , the size of the object image can be as small as the first object 540 displayed in the vicinity of the user, or as large as the second object 550 so as to encompass the user. In one embodiment, the object parameters further include time parameters, and the AR device 510 further sets the display time of the first object 540 and the second object 550 according to the time parameters of the first object 540 and the second object 550, respectively. For example, the first object 550 can be displayed permanently, and the second object 540 can be displayed for a specific time period. Specifically, multiple objects can be divided into fixed objects and real-time objects through time parameters. For example, the object image of the second object 550 is a house, and the time parameter in the object parameters is permanent. When the AR device 510 obtains the second object from the anchor point cloud After the object 550, it can be fixedly displayed in the augmented reality system. In actual implementation, the second object may also include advertising billboards, road signs, buildings, specific scenes, and the like. If the augmented reality environment is a live broadcast, a speech, or a ball game, since these environments are activities in a specific time period, the objects in these environments belong to the objects in a specific time period. Through the setting of time parameters, You can display objects in real time, or set the display time of objects to display them in a specific time period.

FIG. 6 is a schematic diagram showing a one-way application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention. As shown in FIG. 6 , the speaker 610 uses the camera device 620 to use the captured image as an object, sets the object parameters, and sends it to the anchor point cloud 630 , so that the object 640 of the speaker 610 can be displayed in the remote scene. The AR device locates the position of the object 640 in the remote scene through the anchor device 650 . In one embodiment, the object parameters further include encryption parameters, and the encryption parameters include encryption and non-encryption. Only the AR device that can decrypt the encryption parameters of the object 640 can display the encrypted object, and the AR device that cannot be decrypted cannot display the encrypted object. Encrypted Object 640. When the encryption parameter of the object 640 is set to have encryption, only the AR device that can decrypt the object can display the object 640, and the viewer 660 is limited to view through the AR device that can decrypt. Generally, the AR device of the viewer 670 cannot decrypt, so the object cannot be seen. 640. When the encryption parameter of the object 640 is no encryption, anyone can view the object 640 through the AR device.

FIG. 7 is a schematic diagram showing a bidirectional application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention. As shown in FIG. 7 , the speaker 710 uses the camera device 720 to use the captured image as an object 740 and sets the object parameters and sends it to the anchor point cloud 730, so that the object 740 of the speaker 710 can be displayed in the remote scene for viewing The operator 760 locates the position of the object 740 in the remote scene through the anchor point device 750 . At the same time, the viewer 760 sends the captured image to the anchor point cloud 730 through the camera device 770 as the object 790 and after setting the object parameters, so that the object 790 of the viewer 760 can be displayed in the scene where the speaker 710 is located, through the anchor point device 780 locates the position of the object 790 in the scene, thereby completing the two-way interaction of multiple objects.

The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced. without departing from the spirit and scope of the technical solutions of the present invention.

S101~S105: Steps

Claims (8)

A method for displaying objects in an augmented reality environment, the method comprising the following steps: an augmented reality (Augmented Reality, AR) device obtains spatial information and an object identification code sent from an anchor device; the AR device identifies the object according to the above The code obtains the object parameter corresponding to the object identification code from the anchor point cloud, and the object parameter includes the object image, the object audio, and the first spatial relationship of the object relative to the anchor point device; the AR device obtains the object relative to the above-mentioned anchor point device according to the above-mentioned spatial information. The second spatial relationship of the anchor point device; obtaining the first position of the object in the augmented reality environment according to the first spatial relationship, and obtaining the AR device in the augmented reality environment according to the second spatial relationship For the second position, a third spatial relationship of the object relative to the AR device is obtained according to the first position and the second position; the AR device displays the object image and the object audio according to the third spatial relationship. The method for displaying objects in an augmented reality environment according to claim 1, wherein the AR device further includes an inertial measurement unit for acquiring the moving direction, moving distance, and moving height of the AR device when moving, and the AR device is based on the above The moving direction, the moving distance, and the moving height update the second spatial relationship and the third spatial relationship in real time. The method for displaying an object in an augmented reality environment according to claim 1, wherein the object parameter further includes a time parameter, and the AR device sets the display time of the object according to the time parameter. The method for displaying objects in an augmented reality environment according to claim 1, wherein the object parameters further include encryption parameters, and the encryption parameters include encryption and non-encryption; the objects displayed by the AR device include that the encryption parameters are non-encryption The object and the above encryption parameters are encrypted and can be decrypted by the above AR device. A display system for objects in an augmented reality environment, comprising: an augmented reality (Augmented Reality, AR) device for acquiring spatial information and object identification codes sent from an anchor device; The AR device obtains the object parameters corresponding to the object identification code from the anchor point cloud according to the object identification code, and the object parameters include the object image, the object audio, and the first spatial relationship of the object relative to the anchor point device; according to the space obtaining the second spatial relationship of the AR device relative to the anchor point device according to the information; obtaining the object parameter from the anchor point cloud according to the object identification code; obtaining the object in the augmented reality environment according to the first spatial relationship The first position of the AR device in the augmented reality environment is obtained according to the second spatial relationship, and the third space of the object relative to the AR device is obtained according to the first position and the second position relationship; displaying the image of the object and the audio of the object on the AR device according to the third spatial relationship. The display system for objects in an augmented reality environment according to claim 5, wherein the AR device further includes an inertial measurement unit for acquiring the moving direction, moving distance, and moving height of the AR device when moving, and the AR device is based on the above The moving direction, the moving distance, and the moving height instantly update the second spatial relationship and the third spatial relationship. The display system of an object in an augmented reality environment according to claim 5, wherein the object parameter further includes a time parameter, and the AR device sets the display time of the object according to the time parameter. The display system for objects in an augmented reality environment according to claim 5, wherein the object parameters further include encryption parameters, and the encryption parameters include encryption and non-encryption; the objects displayed by the AR device include that the encryption parameters are non-encryption The object and the above encryption parameters are encrypted and can be decrypted by the above AR device.

Images