TWI590189B - Augmented reality method, system and computer-readable non-transitory storage medium - Google Patents

Description

Augmented reality methods, systems and computer readable non-transitory storage media

This case is about an augmented reality method, system and computer readable non-transitory storage medium.

Through the physical house makeup, the housekeeper can see the furniture placement in the house, so that the transaction rate of the house can be improved, and to understand the preferences of consumers. Since the introduction of Augment Reality (AR), designers/vendors have instantly uploaded design styles and virtual furniture to the augmented reality platform, allowing consumers to see virtual and physical homes on mobile smart devices. The combination of objects.

At present, the market for ready-to-assemble furniture continues to grow in terms of augmented reality applications. If you can improve the interactive and friendly look-and-see effect, you can speed up the transaction.

Therefore, how to enable users to edit and quickly preview the overall effect of augmented reality for appropriate augmented reality display is one of the focuses in this field.

This case is about an augmented reality method, system and computer readable A non-transitory storage medium is taken to correct the initial coordinate system of the augmented reality based on the depth information.

According to an embodiment of the present invention, an augmented reality method is proposed. Choose a virtual landscape that is related to a real environment. Configuring at least one virtual object in a virtual placement position in the virtual landscape. Initialize and correct a standard system. In the calibration coordinate system, the augmented reality is displayed.

According to another embodiment of the present invention, an augmented reality system is provided, including: a three-dimensional sensing unit; an image capturing unit; and an augmented reality application module, and a sensing result of the three-dimensional sensing unit is An image captured by the image capturing unit is transmitted to the augmented reality application module. In response to a user selection, the augmented reality application module selects a virtual landscape associated with a real environment. The augmented reality application module configures at least one virtual object in a virtual placement position in the virtual landscape. The augmented reality application module performs initialization and correction of a standard system. In the calibration coordinate system, the augmented reality application module displays the augmented reality.

According to still another embodiment of the present invention, a computer readable non-transitory storage medium is provided. When read by a computer, the computer performs the augmented reality method as described above.

In order to better understand the above and other aspects of the present invention, the following specific embodiments, together with the drawings, are described in detail below:

110-140‧‧‧Steps

210-240‧‧‧Steps

310-330, 310’-330’‧‧‧ virtual objects

M‧‧‧Virtual Augmentation Reality Mark

L‧‧‧Virtual Reference Line

M’‧‧‧Real Augmented Reality Mark

L’‧‧· true reference line

410-455‧‧‧Steps

510-550‧‧‧Steps

1000‧‧‧Augmented Reality System

1010‧‧‧3D sensing unit

1015‧‧‧Gravity sensing unit

1020‧‧‧Image capture unit

1030‧‧‧Augmented Reality Application Module

1040‧‧‧Display unit

Figure 1 shows a flow chart of an augmented reality (AR) method in accordance with an embodiment of the present invention.

Figure 2 shows a flow chart of an augmented reality (AR) method in accordance with another embodiment of the present invention.

FIG. 3A is a schematic diagram showing the virtual layout according to an embodiment of the present invention.

Figure 3B shows a schematic diagram of a virtual reference line L in accordance with an embodiment of the present invention.

Figure 3C shows a schematic diagram of a true reference line L' in accordance with an embodiment of the present invention.

Figure 3D shows the augmented reality displayed by the augmented reality system in accordance with an embodiment of the present invention.

Figure 4 is a flow chart showing the coordinate system initialization and correction architecture in accordance with an embodiment of the present invention.

Figure 5 shows the flow of the Z-axis reference point distance estimation according to an embodiment of the present invention.

Figure 6 is a schematic diagram of the fast-moving image restoration technique.

Figure 7 is a diagram showing the viewing angle correction according to an embodiment of the present invention.

Figure 8 is a diagram showing the actual reference line length estimation according to an embodiment of the present invention.

Figure 9 shows an example of a coordinate system established in accordance with an embodiment of the present invention.

Figure 10 shows a functional block diagram of an augmented reality system in accordance with an embodiment of the present invention.

The technical terms of the present specification refer to the idioms in the technical field, and some of the terms are explained or defined in the specification, and the explanation of the terms is based on the description or definition of the specification. Various embodiments of the present disclosure each have one or more of the technical features. Subject to the possible implementation, those skilled in the art can selectively implement some or all of the technical features of any embodiment. The features, or some or all of the technical features of these embodiments, are selectively combined.

Figure 1 shows a flow chart of an augmented reality (AR) method in accordance with an embodiment of the present invention. The Augmented Reality (AR) method of Figure 1 can be applied to augmented reality systems. In step 110, the user selects a virtual landscape that is related to the real environment in which they are located. That is, in response to a user selection, the augmented reality system selects a virtual landscape that is related to a real environment. For example, if the user wants to operate the augmented reality system in the room to preview the placement of the furniture, the user can operate on the augmented reality system to select a virtual one suitable for the current room. Pattern (such as room shape, room function, design style, etc.). In an embodiment of the present invention, the augmented reality system and method may be performed by a mobile smart device (such as, but not limited to, a smart phone, a tablet, etc.) or a head mounted device (including a display).

In step 120, a virtual placement position of at least one virtual object in the virtual landscape is configured by the augmented reality system, which may be input by a user operation or input by other input modes. Here, the virtual object is, for example but not limited to, furniture, furniture, and the like. In detail, in step 120, according to the virtual pattern selected by the user (this virtual pattern is related to the real environment), the augmented reality system selects at least one virtual one from the database in the augmented reality system. An object to place the selected at least one virtual object in a virtual pattern selected by the user.

In step 130, the coordinate system initialization and correction is performed. The details of step 130 will be explained below.

In step 140, on the corrected coordinate system, display amplification The reality is on the display unit of the device.

Figure 2 shows a flow chart of an augmented reality (AR) method in accordance with another embodiment of the present invention. Steps 210, 220, 230 and 240 of Figure 2 are identical or similar to steps 110, 120, 130 and 140 of Figure 1.

In step 210, the user selects a virtual landscape that is relevant to the real environment in which they are located.

In step 215, the augmented reality system displays the virtual pattern selected by the user on the interactive interface.

In step 220, the user operates an interactive interface of the augmented reality system (ie, in response to the operation of the interactive interface) to configure the virtual placement of the virtual object in the real environment. Here, the virtual placement position of the virtual object is displayed in a plan view. In an embodiment of the present invention, the plan view is composed of the selected virtual pattern and the virtual object. The plan view may be a layout constructed by a perspective method including, but not limited to, one or any combination of the following: a bird's eye view, a parallel perspective view, a two-point perspective view, a beveled projection view, and the like. The virtual landscape can be represented by geometry. The virtual object may be selected from a virtual object database within the augmented reality system or customized by the user. The user can display the virtual placement position of the virtual object by dragging or clicking on the interactive interface. FIG. 3A is a schematic diagram showing the virtual layout according to an embodiment of the present invention. In Figure 3A, objects 310-330 represent virtual objects. The user can configure and/or change the virtual placement of the virtual objects 310-330 on the interactive interface.

In the embodiment of the present case, the function of the interactive interface further includes a reference value. set up. The reference value setting refers to the setting of the correspondence between the floor plan reference object and the real environment. The interactive interface automatically assigns any one or more virtual objects in the floor plan as a reference. The foregoing correspondence relationship is the comparison, calibration, and setting between the specified floor plan reference object and the real environment information; wherein the real environment information is obtained by one or more sensors, and the real environment information includes but is not limited to the image signal and depth. Information, etc.

In step 222, the augmented reality system prompts the user to select a virtual reference line for coordinate initialization and correction in the selected virtual landscape. The augmented reality system generates a cue signal to the user, in response to the user selection, the augmented reality system selects the virtual reference line used for coordinate initialization and correction in the selected virtual landscape. After the user selects the virtual reference line, the augmented reality system turns on the three-dimensional sensing unit of the mobile smart device and other related sensors (such as, but not limited to, a camera, a gravity sensing unit (g-sensor), The gyroscope or the like is configured to detect the corresponding real reference line (step 224). In an embodiment of the present disclosure, the three-dimensional sensing unit is, for example, a three-dimensional depth sensing unit.

In the present embodiment, the virtual reference line is defined as a line segment extending from a virtual augmented reality marker M to a boundary (usually a wall) of the virtual pattern selected by the user in a plan view. Figure 3B shows a schematic diagram of a virtual reference line L extending from the virtual augmented reality mark M to the virtual pattern boundary, in accordance with an embodiment of the present invention. The length of the virtual reference line L is known. That is, since the size of the virtual pattern selected by the user is known and the virtual augmented reality mark M is located at the center or other position of the virtual pattern, the length of the virtual reference line L can be known. In an embodiment of the present invention, the location of the virtual augmented reality marker M can be specified by an interactive interface.

As for the true reference line corresponding to this virtual reference line, the true augmented reality mark M' placed in the real environment extends to the line segment of the boundary (usually the wall) of the real environment. Figure 3C shows a schematic diagram of a true reference line L' according to an embodiment of the present invention, which is extended from the true augmented reality mark M' to the boundary of the real environment.

For example, the real augmented reality mark M' may be a real object of a known size, such as an A4 size white paper placed in a central location of the real environment in which the user is located (for example, but not limited to the central position). Therefore, the augmented reality system must successfully detect the boundary of the real augmented reality marker M' and the real environment.

However, in practice, the mobile intelligence device held by the user is inevitably subject to viewing angle deviation or displacement. Therefore, in an embodiment of the present invention, if the mobile intelligence device has a viewing angle deviation or displacement, the coordinate system initialization and correction is performed according to the depth information (step 230) to estimate the length of the real reference line L', thereby deriving the plan view. The virtual object is placed in a virtual environment in a real environment for the user to experience augmented reality content.

In step 235, the scaling of the virtual object can be estimated by the real environment information detected by the three-dimensional sensing unit and other related sensors (such as, but not limited to, a camera, a gravity sensing unit, a gyroscope, etc.). . The relative position of the virtual object in the real environment can be estimated from the ratio between the length of the virtual reference line L and the length of the real reference line L'. It can be known from the above description that the embodiment of the present invention is known The length of the virtual reference line L can also detect the length of the real reference line L', and can be scaled according to the ratio between the length of the virtual reference line L and the length of the real reference line L', and the scaling ratio can be Apply to the virtual placement of all virtual objects in the real world.

In step 240, the augmented reality is displayed on the augmented reality system. Figure 3D shows an augmented reality displayed by an augmented reality system in accordance with an embodiment of the present invention. For example, for convenience of explanation, it is assumed that the distance between the virtual object 310 of FIG. 3A and the boundary of the virtual pattern is A. After the scaling adjustment, the distance between the virtual object 310' displayed in the 3D map and the boundary of the real environment is A*(L'/L).

As shown in Fig. 3D, the user can see the placement of the virtual objects 310', 320' and 330' in the real environment on the screen of the mobile smart device.

When the augmented reality is displayed, the position of the user's mobile intelligence device and the camera angle are continuously tracked to continuously correct the displayed augmented reality.

It will now be explained how the embodiment of the present invention performs coordinate system initialization and correction. Figure 4 is a flow chart showing the coordinate system initialization and correction architecture in accordance with an embodiment of the present invention.

In step 410, the depth information is obtained by using the three-dimensional sensing unit. For example, three-dimensional depth sensing technology is used to obtain depth information. Examples of three-dimensional depth sensing technologies include, but are not limited to, ultrasonic sensing, time-of-flight (ToF), structured lighting, and stereoscopic Vision (stereo vision), etc. The embodiment of the present invention uses the time difference ranging technology to obtain depth information as an example to illustrate, by preprocessing the depth information to estimate the Z-axis reference point distance and Contribute to the coordinate system initialization and correction.

In step 420, an environmental image is captured. Step 420 is, for example but not limited to, performed by a camera of the mobile smart device. In step 425, the displacement information is detected. Step 425 is, for example but not limited to, being performed by a gravity sensing unit of the mobile smart device to detect displacement information of the mobile smart device.

In step 430, a Z value pre-processing is performed. The Z value pre-processing of step 430 may be based on the depth information obtained in step 410, the environmental image captured in step 420, and the displacement information obtained in step 425, the details of which will be described below.

In step 435, reference point pre-processing is performed. The reference point pre-processing of step 435 may be based on the depth information obtained in step 410, the environmental image captured in step 420, and the displacement information obtained in step 425, the details of which will be described below.

In step 440, the distance is estimated based on the Z value pre-processing result of step 430 and the reference point pre-processing result of step 435. The estimated distance is, for example, the length of the real reference line and the like. The details of step 440 will be explained below.

At step 445, a standard system is established. In step 450, the virtual object is placed in the established coordinate system.

In step 455, the coordinate system initialization and correction is performed.

When the coordinate system is initialized, the distance pushback and scaling conversion can be imported, and the coordinate system conversion matrix can be obtained.

When the distance is pushed back, the depth information sensed by the three-dimensional sensing unit and the image signal captured by the camera can be used to estimate the length of the real reference line. Auxiliary positioning, or assisted positioning by Bluetooth signal, radio frequency, visible light communication signal, etc. Distance pushback includes, but is not limited to, noise processing, reference point positioning, distance pushback matrix operations, and fault tolerant control.

The scaling is based on the known value and the scale factor, and the object size and the relative relationship between the objects in the known coordinate system are scaled according to the proportional coefficient to the corresponding size and relative relationship.

Coordinate system transformation matrices include, but are not limited to, mapping coordinates, virtual 3D model coordinates, camera coordinates, 3D sensing unit coordinates, and world coordinates such as conversion between coordinate systems.

For coordinate system calibration, information obtained by cameras, gravity sensing units, gyroscopes, and three-dimensional sensing units can be utilized. In addition, the virtual three-dimensional model used for coordinate system correction includes, but is not limited to, the placement coordinates of the virtual object and the three-dimensional model rendering effect setting.

When the coordinate system is calibrated, the information detected/captured by the sensors can be integrated, and the consistency of the information can be compared with each other to estimate the error value, and the error value is used to estimate the original output result.

Figure 5 shows the flow of the Z-axis reference point distance estimation according to an embodiment of the present invention. In the embodiment of the present invention, in order to use the image repair technology to repair the depth information sensed by the time difference ranging method, the depth information obtained in step 410 is converted into a two-dimensional grayscale image to visually present the current scene. Distance (distance) relationship. The measured distance is converted into a floating point number (in the range of 0.0 to 1.0) based on the farthest and closest sensing distance limit of the three-dimensional sensing unit, and then mapped to A grayscale value between 0-255 (step 520). Wherein, the depth information within the trusted range (set by the estimation program) for the sensing distance will be reduced to the area to be repaired (step 510), the area to be repaired contains the depth information to be corrected and the sensor is due to the edge of the object or The return value (depth information) caused by the material problem is missing.

3D and 2D information mapping (step 520), in an embodiment, but not limited thereto, dividing the depth information detected by the three-dimensional sensing unit by the farth sensing distance limit, and converting into a floating point number, Then multiply the result by 255 and take the integer, and set it to the grayscale value. Assuming that the measured depth information is d and the farthest sensing distance limit is d _MAX , the mapped grayscale value is . In this way, all the depth information obtained by the detection is calculated, and the result of the two-dimensional information mapping can be obtained.

The two-dimensional image restoration (step 530) can adopt the image restoration technology of the Fast Marching Method, and the sixth figure is the schematic diagram of the image technique of the fast marching method. The two-dimensional image is divided into a pixel area ε and a region to be repaired Ω, assuming that the contour of the region to be repaired Ω is δΩ, and the pixel contour δΩ is repaired from the outside to the inside by patching.

For each pixel p to be repaired in δΩ, the pixel value is determined by equation (1):

Where B _i ( p ) ε is a pixel region point group centered on p within a given radius, I(.) is the output pixel value, and ▽I(.) is the gradient value function. w(p,q) is the weight of pixel point q in B _i (P) to be repaired pixel point p, w(p,q) is determined by three factors in formula (2): w(p,q)=dir (p, q). Dst(p,q). Lev(p,q) (2)

The direction factor dir(p,q) represents the larger the normal vector weight of the tangential direction of the pixel p to be repaired, and the direction factor dir(p,q) is as shown in formula (3):

The geometric distance factor dst(p,q) represents the greater the weight closer to the pixel p to be repaired, and the geometric distance factor dst(p,q) is as in equation (4):

The horizontal set distance factor lev(p,q) represents the greater the weight closer to the contour line δΩ, and the horizontal set distance factor lev(p,q) is as shown in equation (5):

Where d ₀ is the geometric distance parameter, T represents the distance from the pixel point to δ Ω, and T ₀ is the level set parameter. The use of image restoration techniques can reduce the absence of pixel values in the reduced depth information in step 510.

The angle of view correction (step 540) is as shown in FIG. The left image of Figure 7 illustrates common viewing angle deviations, where M' represents a true augmented reality marker of known size and shape (for example, an image of A4), and r' represents the detected Z-axis reference. Point distance (that is, the wall surface, which corresponds to the virtual pattern boundary). When the user stands at a position that deviates from the true augmented reality mark M' (y-axis rotation) and senses the device forward (x-axis rotation) in a natural hand-held habit, the detected r' is longer than the ideal Z-axis. Reference point distance r (r represents the distance between the user and the ideal Z-axis reference point), which will affect the coordinate system establishment (step 445) The accuracy with the placement of objects (step 450). In order to reduce the estimation of the length of the real reference line L caused by the distance estimation error (caused by the angle of view deviation), the image restoration result map is rotated by the projection transformation matrix R, as shown in the right diagram of FIG. Show.

Detecting the edge and apex of the real augmented reality marker M' by augmented reality marker recognition technology, and using the deformation result of the captured image (such as the real augmented reality marker on the left of Fig. 7, for example It is the deformation result of the A4 image) and the ideal shape (such as the true augmented reality mark M' ideal shape on the right in Fig. 7, for example, the isosceles trapezoid) reversely projects the projection transformation matrix R, as shown in equation (6):

For the rotation matrix, define the degree of rotation and scaling of the graph.

For the translation vector, define the degree of displacement of each point.

[ c ₃₁ c ₃₂ ] is a projection vector that defines the degree of perspective distortion.

By rotating the points on the image restoration result through the projection transformation matrix R, the angle of view of the user standing at the midpoint of the real augmented reality marker can be simulated. At this point, the deviation has been corrected to the Z-axis reference point corresponding to the midpoint of the true augmented reality marker, the distance being r" (as shown in the right image of Figure 7). Then using the gravity sensing unit or The angular displacement information sensed by the gyroscope or the like is calculated as the forward tilt angle θ of the mobile smart device, and r"cos θ is the ideal Z-axis reference point distance (step 550).

As for the distance estimation unit 440, the length of the real reference line L' is estimated as shown in FIG. In Fig. 8, the parameter X represents the length of the real reference line L' (i.e., the distance from the true augmented reality mark M' to the wall). It can be seen from Fig. 8 that x=rd _r , wherein the horizontal distance d _{r of the} mobile smart device (held by the user) to the real augmented reality mark M′ can be expressed as . Since the program has been detected real augmented reality marker M 'position, therefore it is preferable that a depth value as the position of the mobile device intelligence (held by the user) to the true augmented reality marker M' distance d _m. The distance s from the mobile smart device to the corner of the wall needs to first locate the junction through the edge detection algorithm, and then take the depth value as the distance s.

In general, in the present embodiment, the z-value pre-processing (i.e., depth information pre-processing) and the reference point pre-processing can complement each other to derive the length of the real reference line L'. The z-value pre-processing (depth information pre-processing) of step 430 includes a reduction of depth information (step 510), mapping (step 520), repair (step 530), perspective correction operation (step 540), and obtaining a z-axis reference point distance ( Step 550) and the like. The reference point pre-processing of step 435 includes sensor signal processing (such as image angle correction operation), augmented reality mark detection, edge detection, and angular displacement information output. Of course, the method, program, information, and signal source for calculating the length of the real reference line L' are not limited to the contents disclosed in this embodiment.

After obtaining the length of the real reference line L' (step 440), according to the scaling ratio, the virtual environment selected by the user is used to calculate the real environment of the user. The size is roughly sized to establish a coordinate system (step 445).

Figure 9 shows an example of a coordinate system established in accordance with an embodiment of the present invention. As shown in Figure 9, a cubic space model of 30 inches long, 15 inches wide, and 10 inches high (for example, when the case is not limited to this) is based on the detected true reference line. The length and the selected virtual pattern are derived by proportionally, and the ratio can also be used to convert the relative position of the virtual object in the plan view (such as the virtual object 310-330 of FIG. 3A) to the origin of the coordinate system (ie, the virtual object). The coordinates of the centroid) to complete the coordinate system initialization and correction. Once the coordinate system is established, the placement of the virtual object can be determined (as shown in Figure 9).

Another embodiment of the present disclosure discloses an augmented reality system executable on an electronic device. Electronic devices such as, but not limited to, mobile smart devices and head mounted devices. Figure 10 shows a functional block diagram of an augmented reality system in accordance with an embodiment of the present invention. The augmented reality system 1000 includes at least a three-dimensional sensing unit 1010, a gravity sensing unit 1015, an image capturing unit 1020, an augmented reality application module 1030, and a display unit 1040. The sensing result of the three-dimensional sensing unit 1010 and the gravity sensing unit 1015 and the image captured by the image capturing unit 1020 can be transmitted to the augmented reality application module 1030. The gravity sensing unit 1015 is configured to sense displacement information, and may also be referred to as a displacement sensing unit. When the augmented reality application module 1030 is executed, the electronic device may perform the augmented reality method described above and display the augmented reality on the display unit 1040. The augmented reality application module 1030 can be implemented as a hardware circuit or software. The augmented reality application module 1030 can be implemented by an integrated circuit or a processing unit. The presentation of subsequent augmented reality content can rely on instant location technology to find out the user in the real environment. The coordinates in the camera and the camera angle are detected to correctly fit the virtual object to the specified position to achieve a sense of experience in the augmented reality.

Another embodiment of the present disclosure discloses a computer readable non-transitory storage medium in which the above-described augmented reality method is stored.

The embodiment of the present invention corrects the initial coordinate system of the augmented reality based on the depth information, in order to solve how to find the scaling of the augmented reality content and how to find the coordinate system conversion when converting the plan to the virtual three-dimensional model. The problem with the matrix. It can be seen from the above embodiments that the embodiment of the present invention can easily set the virtual reference line and detect the length of the real reference line by using the three-dimensional sensing unit in combination with other technologies. It is complemented by an interactive interface that allows users to edit and preview augmented reality experiences. The embodiment of the present invention allows the user to immediately experience the overall configuration effect and increase the user experience.

The embodiment of the present invention utilizes a three-dimensional sensing unit to obtain depth information, and combines the depth information with the real augmented reality mark to find a proportional conversion relationship. In this way, the user can augment the reality preview of its overall design on the interactive interface, increasing user experience.

In summary, although the present invention has been disclosed above by way of example, it is not intended to limit the present invention. Those who have ordinary knowledge in the technical field of the present invention can make various changes and refinements without departing from the spirit and scope of the present case. Therefore, the scope of protection of this case is subject to the definition of the scope of the patent application attached.

210-240‧‧‧Steps

Claims (17)

An augmented reality method, comprising: selecting a virtual pattern related to a real environment; configuring a virtual placement position of at least one virtual object in the virtual pattern; performing initialization and correction of a standard system; and performing the correction In the coordinate system, the augmented reality is displayed, wherein the step of initializing and correcting the coordinate system comprises: obtaining a depth information; capturing an environmental image; obtaining a displacement information; and according to the depth information, the environmental image and The displacement information is subjected to Z value pre-processing and reference point pre-processing; estimating a length of a real reference line of the real environment according to a Z-value pre-processing result and a reference point pre-processing result; establishing a bidding system; The at least one virtual object is placed in the established coordinate system. The augmented reality method of claim 1, further comprising: displaying the selected virtual pattern on an interactive interface; and configuring the real environment in response to one of the interactive interfaces The virtual placement position of the at least one virtual object, wherein the virtual placement position of the at least one virtual object is displayed in a plan view. The augmented reality method of claim 2, wherein the plan view comprises the virtual pattern and the at least one virtual object; the plan view is formed by a perspective method, including one or any combination of the following: a bird's-eye view, a parallel perspective view, a two-point perspective view, and an oblique projection view; the virtual pattern is represented by a geometric figure; and the at least one virtual object is selected from a virtual object database by the interactive interface. Or be customized by a user. The augmented reality method of claim 3, further comprising: prompting to select a virtual reference line, and after selecting the virtual reference line, turning on at least one three-dimensional sensing unit to detect the corresponding a virtual reference line extending from a virtual augmented reality marker to a boundary of the virtual pattern, a length of the virtual reference line being known; and the real reference line being placed in the real environment A true augmented reality marker extends to a boundary of the real environment. For example, in the augmented reality method described in claim 4, after the real reference line is detected, initialization of the coordinate system is performed; wherein if there is a viewing angle deviation or a displacement, the coordinate system is corrected to Estimating the length of the real reference line, and inferring the virtual placement position of the at least one virtual object of the plan view in the real environment. The augmented reality method of claim 5, further comprising: after performing initialization and correction of the coordinate system, estimating a scaling ratio of the at least one virtual object, and converting the at least one virtual object in the a relative position in the real environment, wherein the relative position is obtained based on a ratio between the length of the virtual reference line and the length of the real reference line. The augmented reality method of claim 1, wherein the step of estimating the length of the real reference line comprises: clipping the depth information; mapping the three-dimensional information with the two-dimensional information; repairing the two-dimensional Obtaining a viewing angle; and obtaining a Z-axis reference point distance, wherein after obtaining the length of the real reference line, the size of the real environment is derived from the selected virtual pattern according to the scaling ratio, thereby establishing The coordinate system. The augmented reality method of claim 1, further comprising: continuously tracking a position and a camera angle when displaying the augmented reality to continuously correct the displayed augmented reality. An augmented reality system comprising: a three-dimensional sensing unit; an image capturing unit; a displacement sensing unit; and an augmented reality application module, a sensing result of the three-dimensional sensing unit, an image captured by the image capturing unit, and a displacement information of the displacement sensing unit are transmitted to The augmented reality application module, the augmented reality application module executes: in response to a user selection, selecting a virtual pattern related to a real environment; configuring at least one virtual object in the virtual pattern to be a virtual Positioning; performing initialization and correction of a standard system; and displaying the augmented reality in the calibration coordinate system, wherein the augmented reality application module is obtained during initialization and correction of the coordinate system: a depth information; capturing an environmental image; obtaining a displacement information; performing Z value pre-processing and reference point pre-processing according to the depth information, the environment image and the displacement information; and processing a result according to a Z value and a reference point Pre-processing results, estimating a length of a real reference line of the real environment; establishing a landmark system; and placing the at least one virtual object in the established coordinate system. An augmented reality system as described in claim 9 of the patent application, wherein Augmented reality application module: displaying the selected virtual landscape on an interactive interface; and in response to one of the interactive interfaces, configuring the virtual pendulum of the at least one virtual object in the real environment a placement position, wherein the virtual placement position of the at least one virtual object is displayed in a plan view. The augmented reality system of claim 10, wherein the plan view comprises the virtual pattern and the at least one virtual object; the plan view is formed by a perspective method, including one or any combination of the following: a bird's-eye view, a parallel perspective view, a two-point perspective view, and an oblique projection view; the virtual pattern is represented by a geometric figure; and the at least one virtual object is selected from a virtual object database by the interactive interface. Or be customized by a user. The augmented reality system of claim 11, wherein the augmented reality application module: generates a prompt signal; selects a virtual reference line in response to a user selection, and selects the virtual After the reference line, the three-dimensional sensing unit is turned on to detect the corresponding real reference line; wherein the virtual reference line extends from a virtual augmented reality mark to a boundary of the virtual pattern, and one of the virtual reference lines The length is known; The real reference line extends from a true augmented reality marker placed in the real environment to a boundary of the real environment. The augmented reality system of claim 12, wherein if there is a viewing angle deviation or a displacement, the augmented reality application module performs initialization and correction of the coordinate system to estimate the reality. The length of the reference line further infers the virtual placement position of the at least one virtual object of the plan view in the real environment. The augmented reality system of claim 13, wherein the augmented reality application module: after performing initialization and correction of the coordinate system, estimating a scaling ratio of the at least one virtual object, and Converting a relative position of the at least one virtual object in the real environment, wherein the relative position is obtained according to a ratio between the length of the virtual reference line and the length of the real reference line. The augmented reality system of claim 9, wherein, in estimating the length of the real reference line, the augmented reality application module: reducing the depth information; for three-dimensional information and two-dimensional information Mapping the information; repairing the two-dimensional image; correcting the viewing angle; and obtaining the Z-axis reference point distance, wherein after obtaining the length of the real reference line, according to the scaling ratio, The size of the real environment is derived from the selected virtual pattern, and the coordinate system is established accordingly. The augmented reality system of claim 9, wherein the augmented reality application module continuously tracks a position and a camera angle when displaying the augmented reality to continuously correct the displayed Augmented reality; the augmented reality application module is implemented by an integrated circuit or a processor. A computer readable non-transitory storage medium that, when read by a computer, performs the augmented reality method as described in claim 1 of the patent application.