TWI777801B - Augmented reality display method - Google Patents

Abstract

An augmented reality display method is implemented by a mobile device, and includes: storing a normal map corresponding to a predetermined area on an object surface, and a color spectrum; each drawing of the predetermined area on the object surface pixel, calculate a viewing position and an observer's perspective of the pixel; convert the normal vectors when the normal map is on the predetermined area into a plurality of world normals in a world space relative to the viewing position vector; calculate the inner product of the world normal vector of each pixel of the predetermined area and the viewing angle of the observer to obtain a dispersion value, and obtain a dispersion result according to the color spectrum; in an augmented reality manner The corresponding dispersion result is displayed on each pixel of the predetermined area on the surface of the object to generate colored glare.

Description

Augmented reality display method

The present invention relates to a display method, especially an augmented reality display method.

In the real world, some game cards, player cards or other similar cards will be coated with multiple layers on the surface of the card, so that when the surface of the card is illuminated, diamonds or colorful glare will be produced according to the viewing angle of the viewer. Effect. However, in a virtual world such as an augmented reality (AR) display environment, how to generate the same or similar glare effect on the surface of cards or other objects has become a problem to be solved.

Therefore, the purpose of the present invention is to provide an augmented reality display method that produces a glare effect on the surface of an object.

Therefore, the present invention provides an augmented reality display method, which is suitable for and implemented by a mobile device, and includes steps (A) to (G).

In step (A), a normal map corresponding to a predetermined area of an object surface and a color spectrum are stored. The normal map includes a normal vector corresponding to each pixel of the predetermined area in the tangent space, and the color spectrum includes the correspondence between a plurality of colors and a plurality of terminal dispersion values.

In step (B), each pixel of the predetermined area on the surface of the object is calculated from a viewing position of the mobile device and an observer's viewing angle of the pixel.

In step (C), the normal vectors when the normal map is on the predetermined area are converted into a plurality of world normal vectors in a world space relative to the viewing position.

In step (D), calculate the inner product of the world normal vector of each pixel in the predetermined area and the viewing angle of the observer to obtain a dispersion value, and determine that the terminal dispersion value is equal to the color spectrum according to the color spectrum. The color corresponding to the dispersion value, and define the color as a dispersion result.

In step (E), the corresponding dispersion result is displayed on each pixel of the predetermined area on the surface of the object in an augmented reality manner to generate color glare.

In some implementation aspects, wherein, in step (A), the terminal dispersion values are between a first value and a second value, the first value and the second value are both positive numbers, and the The first value is less than the second value.

In some implementation aspects, wherein, in step (A), the first value is equal to 0, the second value is equal to 1, and the terminal dispersion values are respectively equal to 0, 0.166, 0.332, 0.498, 0.664, 0.830, and The colors corresponding to 1 are red, orange, yellow, green, blue, indigo, and violet, respectively.

In other embodiments, wherein, in step (A), the first value is equal to 0, the second value is equal to 1, the terminal dispersion values are between 0 and a third value, and a fourth value and The corresponding colors between 1 are all gray, and the colors corresponding to the terminal dispersion values between the third value and the fourth value include red, orange, yellow, green, blue, indigo, and violet .

In some implementation aspects, wherein, in step (A), the third value is 0.056, the fourth value is 0.944, and the terminal dispersion values are respectively equal to 0.167, 0.278, 0.389, 0.5, 0.612, 0.723, and The colors corresponding to 0.834 are red, orange, yellow, green, blue, indigo, and violet.

In some other implementations, in step (D), when the calculated dispersion value is greater than or equal to the second value, the dispersion value is equal to the second value, and when the calculated dispersion value is When less than or equal to the first value, the dispersion value is equal to the first value.

In some other implementations, in step (B), the observation position is a camera lens position of the mobile device, and the observer angle of view corresponding to each pixel=normalize(the camera lens position−the pixel position), normalize is a normalization function.

In other embodiments, in step (A), the normal map includes a plurality of sub-regions, and the normal vectors corresponding to the pixels of each sub-region are the same and adjacent to each other. The two normal vectors corresponding to the pixels of any two of the sub-regions are different.

In some implementation aspects, in step (A), the two normal vectors corresponding to the pixels of any two adjacent sub-regions are only different in one of the three dimensions .

The effect of the present invention is that: according to the normal map, the mobile device firstly calculates the world normal vector corresponding to each pixel in the predetermined area of the object surface, and then calculates the corresponding world normal vector for each pixel. The dispersion value is determined according to the color spectrum, and the dispersion result corresponding to each pixel is determined according to the color spectrum, and the visual effect of color glare can be generated in the predetermined area of the surface of the object in an augmented reality manner.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are designated by the same reference numerals.

Referring to FIG. 1 and FIG. 2 , an embodiment of an augmented reality display method of the present invention is applicable to a mobile device 1 and includes steps S1 to S5 . The mobile device 1 has an augmented reality function, and is, for example, a smart phone, and includes a processing unit 11 , a camera lens 12 , a storage unit 14 , and a display unit 13 electrically connected to the processing unit 11 . . The storage unit 14 stores, for example, an application program (APP). When the processing unit 11 executes the application program, the processing unit 11 controls the photographing lens 12 to photograph an object, and displays it on the display unit 13 in an augmented reality manner. In this embodiment, the object is a card, such as a game card.

In step S1, the storage unit 14 stores a normal map corresponding to a predetermined area of an object surface of the object, and a color spectrum. The normal map includes a normal vector corresponding to each pixel of the predetermined area in a tangent space, and the color spectrum includes correspondences between a plurality of colors and a plurality of terminal dispersion values.

Referring to FIG. 3 again, for example, the storage unit 14 also stores a card 3D model 8 corresponding to the object in the augmented reality. The card 3D model 8 includes a first part model 81 and a second part model 82 , FIG. 3 only shows a front view of the three-dimensional model 8 of the card, and the predetermined area of the surface of the object is the visible area of the second part of the model 82 shown in FIG. 3 . For example, the predetermined area includes 180*64 pixels, and the normal map corresponding to the 180*64 pixels in the predetermined area has 180*64 normal vectors.

The terminal dispersion values are between a first value and a second value, the first value and the second value are both positive numbers, and the first value is smaller than the second value. In this embodiment, the first value is equal to 0, the second value is equal to 1, the terminal dispersion values are respectively equal to 0, 0.166, 0.332, 0.498, 0.664, 0.830, and the colors corresponding to 1 are red respectively , orange, yellow, green, blue, indigo, and purple. And the terminal dispersion values are, for example, between 0 and 0.166, and the corresponding colors gradually change from red to orange.

In other embodiments, the terminal dispersion values between 0 and a third value, and between a fourth value and 1 are all gray. The colors corresponding to the terminal dispersion values between the third value and the fourth value include red, orange, yellow, green, blue, indigo, and violet. The third numerical value is, for example, 0.056, and the fourth numerical value is, for example, 0.944, but not limited thereto. The colors corresponding to the terminal dispersion values equal to 0.167, 0.278, 0.389, 0.5, 0.612, 0.723, and 0.834 are red, orange, yellow, green, blue, indigo, and violet, respectively.

In addition, in other embodiments, the first numerical value and the second numerical value may also be other numerical values, and the various colors included in the color spectrum may also not include red, orange, yellow, green, blue, and indigo. , and some of the seven colors of purple, that is, the types of colors included in the color spectrum are not limited to this embodiment.

In step S2, the processing unit 11 calculates an observation position of the mobile device and the distance of the pixel for each pixel of the predetermined area on the surface of the object according to the image captured by the photographic lens 12 of the object. An observer's point of view. More specifically, the observation position is a camera lens position of the camera lens 12 of the mobile device 1, and the observer angle of view corresponding to each pixel=normalize(the camera lens position−the position of the pixel) , normalize is a normalization function. For example, the spatial coordinate of the camera lens position is (0, 0, 0), and one of the spatial coordinates of the pixel is (3, 3, 3), then the observer's angle of view = normalize(-3,-3 ,-3)=(-0.6,-0.6,-0.6).

In step S3, the processing unit 11 converts the normal vectors when the normal map is on the predetermined area into a plurality of world normal vectors in a world space relative to the viewing position. Known graphics have defined the tangent space and the world space, that is, each of the world normal vectors is the coordinate of each of the normal vectors transformed from the coordinate system of the tangent space to the coordinate system of the world space conversion result.

In step S4, the processing unit 11 calculates the inner product of the world normal vector of each pixel in the predetermined area and the viewing angle of the observer to obtain a dispersion value. In addition, when the calculated dispersion value is greater than or equal to the second value, the dispersion value is equal to the second value, for example, when the calculated dispersion value is 1.1, it is equal to 1, and when the calculated dispersion value is less than or equal to the first value When it is a value, the dispersion value is equal to the first value. The processing unit 11 then determines the color corresponding to when the terminal dispersion value is equal to the dispersion value according to the color spectrum, and defines the color as a dispersion result.

In step S5, the processing unit 11 displays the corresponding dispersion result on each pixel in the predetermined area of the object surface through the display unit 13 in an augmented reality manner to generate color glare. More specifically, following the previous example, the processing unit 11 displays the three-dimensional card model 8 on the object in an augmented reality manner, and the second part of the model 82 of the three-dimensional card model 8 is shown in FIG. 3 The corresponding dispersion result is displayed on each pixel of the visible area (ie, the predetermined area) displayed in , so as to generate colored glare. In addition, the processing unit 11 can also display other image effects in the visible area of the first part model 81 of the card 3D model 8 shown in FIG. 3 , such as an image or avatar of a character of the card.

Referring to FIGS. 1 , 3 , and 4 , FIG. 4 illustrates the normal map 83 corresponding to the predetermined area shown in FIG. 3 . The normal map 83 includes, for example, 20 sub-regions of the same size, and each of the sub-regions includes the same 36*16 normal vector as the number of pixels. In addition, the normal vectors corresponding to the pixels of each sub-region are the same, that is, the 36*16 normal vectors of the same sub-region are the same, and any two adjacent sub-regions have the same normal vectors. The two normal vectors corresponding to the pixels of the region are different. For example, the normal vector of the sub-region 831 is different from the four normal vectors of the adjacent sub-regions 832 to 835 . And the two normal vectors corresponding to the pixels of any two adjacent sub-regions are only different in one of the three dimensions (such as X-axis, Y-axis, Z-axis). The two normal vectors of the region 831 and the sub-region 832 are (0, 0, 1) and (0, 0, 0.7), respectively. In this way, the processing unit 11 can display the color glare effect of 20 rectangular squares in the predetermined area of the card in the augmented reality with less computation when the two or three dimensions are different. .

Referring to Figure 1, Figure 5, and Figure 6, another visual effect to be displayed by the card is illustrated, wherein Figure 5 is another three-dimensional model 9 of the card corresponding to the object, and the three-dimensional model of the card 9 includes a first part Model 91 , a second partial model 92 , and a third partial model 93 . FIG. 5 only shows a front view of the three-dimensional model 9 of the card, and the predetermined area of the surface of the object is the visible area of the second partial model 92 and the third partial model 93 shown in FIG. 5 . FIG. 6 illustrates two of the normal maps 94, 95 corresponding to the predetermined area shown in FIG. 5, the normal map 94 includes, for example, 12 sub-regions in the shape of a cow spot, and the normal map 95 includes, for example, 16 sub-regions of the same size sub-areas, the processing unit 11 displays 12 cow spot-shaped colored glare effects and 16 rectangular square colored glare effects at the frame position in the predetermined area of the card in the augmented reality. That is to say, the augmented reality display method of the present invention can also be applied to a plurality of different normal map implementations.

To sum up, the present invention uses the mobile device 1 to first calculate the world normal vector corresponding to each pixel in the predetermined area of the object surface according to the normal map, and then calculate each pixel The corresponding dispersion value, and the dispersion result corresponding to each pixel is determined according to the color spectrum, so that colorful glare and diamond glare can be generated in the predetermined area on the surface of the object in an augmented reality manner , or other visual effects of various color glare, so it can indeed achieve the purpose of the present invention.

However, the above are only examples of the present invention, and should not limit the scope of implementation of the present invention. Any simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the patent specification are still included in the scope of the present invention. within the scope of the invention patent.

1: Mobile device 11: Processing unit 12: Camera Lens 13: Display unit 14: Storage unit 8, 9: 3D model of the card 81, 91: The first part of the model 82, 92: The second part of the model 83, 94, 95: normal maps 831~835: Sub area 93: The third part of the model S1~S5: Steps

Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: FIG. 1 is a block diagram illustrating a mobile device to which the augmented reality display method of the present invention is applicable; FIG. 2 is a flow chart illustrating an embodiment of the display method of augmented reality of the present invention; Figure 3 is a schematic diagram illustrating a three-dimensional model of a card of this embodiment; FIG. 4 is a schematic diagram to assist FIG. 3 to illustrate a normal map; Fig. 5 is a schematic diagram illustrating another three-dimensional model of the card of this embodiment; and FIG. 6 is a schematic diagram that assists FIG. 5 to illustrate two other normal maps.

S1~S5: Steps

Claims (9)

A display method of augmented reality, applicable to and implemented by a mobile device, comprising: (A) store a normal map corresponding to a predetermined area of an object surface, and a color spectrum, the normal map includes a normal vector corresponding to each pixel of the predetermined area in the tangent space, the color The color spectrum includes the correspondence between a plurality of colors and a plurality of terminal dispersion values; (B) for each pixel of the predetermined area on the surface of the object, calculate from a viewing position of the mobile device and an observer's viewing angle of the pixel; (C) converting the normal vectors when the normal map is on the predetermined area into a plurality of world normal vectors in a world space relative to the viewing position; (D) Calculate the inner product of the world normal vector of each pixel in the predetermined area and the viewing angle of the observer to obtain a dispersion value, and determine that the terminal dispersion value is equal to the dispersion value according to the color spectrum corresponds to the color, and defines the color as a dispersion result; and (E) displaying the corresponding dispersion result on each pixel of the predetermined area of the object surface in an augmented reality manner to generate colored glare. The method for displaying augmented reality as claimed in claim 1, wherein, in step (A), the terminal dispersion values are between a first value and a second value, the first value and the first value Both values are positive, and the first value is less than the second value. The method for displaying augmented reality according to claim 2, wherein, in step (A), the first value is equal to 0, the second value is equal to 1, and the terminal dispersion values are respectively equal to 0, 0.166, and 0.332 The colors corresponding to , 0.498, 0.664, 0.830, and 1 are red, orange, yellow, green, blue, indigo, and violet, respectively. The method for displaying augmented reality as claimed in claim 2, wherein, in step (A), the first value is equal to 0, the second value is equal to 1, and the terminal dispersion values are between 0 and a third value , and between a fourth value and 1 are all gray, and the colors corresponding to the terminal dispersion values between the third value and the fourth value include red, orange, yellow, green, blue color, indigo, and purple. The method for displaying augmented reality according to claim 4, wherein, in step (A), the third numerical value is 0.056, the fourth numerical value is 0.944, and the terminal dispersion values are respectively equal to 0.167, 0.278, and 0.389 The colors corresponding to , 0.5, 0.612, 0.723, and 0.834 are red, orange, yellow, green, blue, indigo, and violet, respectively. The method for displaying augmented reality according to claim 2, wherein, in step (D), when the calculated dispersion value is greater than or equal to the second value, the dispersion value is equal to the second value, and When the calculated dispersion value is less than or equal to the first value, the dispersion value is equal to the first value. The method for displaying augmented reality as claimed in claim 1, wherein, in step (B), the observation position is a camera lens position of the mobile device, and the observer's angle of view corresponding to each pixel=normalize (the camera lens position - the pixel position), normalize is a normalization function. The method for displaying augmented reality as claimed in claim 1, wherein, in step (A), the normal map includes a plurality of sub-regions, and the normals corresponding to the pixels in each sub-region The vectors are the same, and the two normal vectors corresponding to the pixels of any two adjacent sub-regions are different. The method for displaying augmented reality according to claim 8, wherein, in step (A), the two normal vectors corresponding to the pixels of any two adjacent sub-regions are only within three One of the dimensions is not the same.

Images