WO2021237952A1

WO2021237952A1 - Augmented reality display system and method

Info

Publication number: WO2021237952A1
Application number: PCT/CN2020/109366
Authority: WO
Inventors: 张元�; 钟正杰
Original assignee: 上海鸿臣互动传媒有限公司
Priority date: 2020-05-29
Filing date: 2020-08-14
Publication date: 2021-12-02

Abstract

The present invention relates to the field of augmented reality, and relates to an augmented reality display system and method. The system comprises: a head-mounted display frame, a recess, a lens, and a portable terminal. When the portable terminal is placed into the recess, a first surface of the portable terminal is enabled to face toward the lens. A processing unit specifically comprises: an inertia measurement module; a pose processing module; and an image processing module, connected to the pose processing module and used for generating a virtual visual range and a virtual screen according to the current pose of the portable terminal and reflecting the virtual screen by means of a display unit and the lens to a user for viewing. The technical solution has the beneficial effects that positioning can be quickly implemented, the virtual screen is generated, and spatial adjustment is implemented.

Description

Display system and method for augmented reality

Technical field

The invention relates to the field of augmented reality, and relates to an augmented reality display system and method.

Background technique

Augmented reality technology is a way of identifying and locating scenes and objects in the real world, and placing virtual three-dimensional objects in the real scene in real time. The goal of this technology is to integrate and interact with the virtual world and the real world. Augmented reality mainly relies on two key technologies: one is the real-time rendering and display of three-dimensional models, and the other is the perception of the shape and position of real objects.

However, current augmented reality devices usually have two ways to perceive location:

(1) The mobile phone’s three-free gyroscope is used for positioning, but the user cannot move closer to or away from the virtual three-dimensional object;

(2) Positioning is performed by calculating the relative position between the front camera of the mobile phone and the preset picture. However, the user needs the picture to realize the six-degree-of-freedom positioning. The technical stability of the picture augmented reality is not high, and the front camera cannot directly When you see the picture, you see the deformed picture through the front lens. The picture must be within the visible range of the front camera to display the 3D model, which makes the tracking and positioning very unstable, and the user's moving range is determined by the location of the picture. The limitations.

It can be seen that there is still a lack of a display device and display method that can quickly perform position positioning, generate virtual images, and perform spatial correction.

Summary of the invention

In view of the above-mentioned defects in the prior art, the present invention provides an augmented reality display system, which is characterized in that it includes:

A head-mounted display frame, the head-mounted display frame has a circular ring shape and is used for wearing by the user;

A groove, the opening of the groove is inclined upward, and one side of the groove is connected with the headset frame;

A lens, the lens is arranged below the groove and connected to the other side of the groove, and the lens is made of a semi-reflective and semi-transparent material;

A portable terminal, the portable terminal has a first side provided with a display unit and a second side provided with an image acquisition unit, the first side and the second side are arranged opposite to each other, the portable terminal is also Comprising a processing unit for processing real-time images collected by the image collecting unit and displaying them on the display unit;

The size of the portable terminal is adapted to the size of the groove, and when the portable terminal is put into the groove, the first side of the portable terminal faces the lens;

The processing unit specifically includes:

Inertial measurement module, used to collect and output real-time motion data;

A pose processing module, connected to the inertial measurement module, and configured to determine the current pose of the portable terminal according to the real-time image collected by the image acquisition unit and the real-time motion data at the corresponding time;

An image processing module, connected to the pose processing module, for generating a virtual visual range and a virtual screen according to the current pose of the portable terminal, and reflecting the virtual screen to the display unit and the lens User view.

Preferably, the image acquisition unit obtains the real-time image by collecting a feature point area including a plurality of feature points;

The processing unit also includes:

A feature point processing module, which is respectively connected to the image acquisition unit and the pose processing module, and is used to obtain the feature point in the real-time image and analyze the location of the feature point area, according to Outputting the characteristic points to the pose processing module as a result of the analysis;

The pose processing module uses the position of the feature point area as a reference, and determines the current pose of the portable terminal according to the real-time motion data at the corresponding time.

Preferably, the virtual visual range includes virtual angle information;

The image processing module includes:

A first processing component, connected to the pose processing module, and configured to construct a spatial rectangular coordinate system with the image acquisition unit as the origin according to the current pose, and determine the spatial rotation angle of the image acquisition unit, And the included angle between the spatial rotation angle and the user, and the virtual angle information is generated according to the spatial rotation angle and the included angle and included in the virtual visual range for output.

Preferably, the virtual visual range includes virtual location information;

The image processing module includes:

A second processing component, connected to the pose processing module, configured to construct a spatial rectangular coordinate system with the image acquisition unit as the origin according to the current pose, and select the center of the user's brow as a preset reference point And generating the virtual position information according to the offset between the portable terminal and the preset reference point and the interpupillary distance of the user and including the virtual position information for output in the virtual visual range.

Preferably, the virtual visual range includes virtual field angle information;

The image processing module includes:

A third processing component, connected to the pose processing module, for generating a preset virtual screen, and generating the virtual field of view according to the position difference between the curved edge of the preset virtual screen and the user The angle information is also included in the virtual visual range and output.

An augmented reality display method, applied to the display system according to any one of the above, characterized in that a head-mounted display frame, a groove, a lens and a portable terminal are provided in the display device;

The display method includes:

Step S1, the image acquisition unit acquires a real-time image of a plane directly above the portable terminal, and

The inertial measurement module collects real-time motion data;

Step S2, the pose processing module processes the current pose of the portable terminal according to the real-time image and the real-time motion data;

Step S3, the image processing module generates a virtual visual range and a virtual screen according to the current pose, and sends the virtual screen to the display unit for display;

Step S4, the virtual screen displayed on the display unit is reflected by the lens to the user for viewing.

Preferably, the step S1 includes:

Step S11, the image acquisition unit obtains the real-time image by collecting a feature point region including a plurality of feature points;

Step S12, the image acquisition unit acquires the feature points in the real-time image and analyzes whether the location of the feature point area meets the viewing angle requirement:

If yes, go to step S2;

If not, go to step S13;

In step S13, the image acquisition unit generates a prompt instruction for indicating that the feature points are too few and feeds it back to the user, and then returns to the step S11.

Preferably, in the step S2, the image processing module determines the current pose through a vSLAM algorithm.

Preferably, the virtual viewing range includes virtual angle information;

The step S3 includes a first process of generating the virtual angle information;

The first process includes:

Step S31A, the image processing module constructs a spatial rectangular coordinate system with the image acquisition unit as the origin according to the current pose, and determines the spatial rotation angle of the image acquisition unit in the spatial rectangular coordinate system;

Step S32A, acquiring the angle between the image acquisition unit and the user;

Step S33A, generating the virtual angle information according to the spatial rotation angle and the included angle.

Preferably, the virtual angle information is expressed by the following formula:

θ=(θ _X ,θ _Y -α,θ _Z )

in,

θ is used to express the virtual angle information;

θ _{X is} used to represent the pitch angle in the space rotation angle;

θ _{Y is} used to represent the yaw angle in the space rotation angle;

α is used to indicate the included angle;

θ _{Z is} used to represent the roll angle in the spatial rotation angle.

Preferably, the virtual visual range includes virtual location information;

The step S3 includes a second process of generating the virtual location information;

The second process includes:

In step S31B, the image processing module constructs a spatial rectangular coordinate system with the image acquisition unit as the origin according to the current pose, and selects the center of the user’s brow as a preset reference point. The offset between the preset reference points to generate first position information;

Step S32B: The image processing module adjusts the first position information according to the interpupillary distance of the user, generates second position information, and outputs the second position information as the virtual position information.

Preferably, the first location information is expressed by the following formula:

χ′=(-B _X ,-B _Y ,-B _Z )

in,

χ′ is used to express the first position information;

B _{X is} used to indicate the projection of the offset on the X axis;

B _{Y is} used to represent the projection of the offset on the Y axis;

B _{Z is} used to indicate the projection of the offset on the Z axis.

Preferably, the second position information includes left-eye position information and right-eye position information;

The second location information is expressed by the following formula:

in,

χ" _{1 is} used to express the left eye position information;

χ" _{2 is} used to express the right eye position information;

B _{X is} used to indicate the projection of the offset on the X axis;

B _{Y is} used to represent the projection of the offset on the Y axis;

I is used to represent the interpupillary distance of the user;

B _{Z is} used to indicate the projection of the offset on the Z axis.

Preferably, the virtual visual range includes virtual field of view angle information;

The step S3 includes a third process of generating the virtual field of view information;

The third process includes:

Step S31C, the image processing module generates a preset virtual screen and displays it on the lens;

Step S32C, the image processing module calculates the position difference between the curved edge of the preset virtual screen and the user;

In step S33C, the image processing module determines the virtual field of view information according to the position difference.

The beneficial effects of the above technical solutions are: position positioning can be performed quickly, virtual images can be generated, and spatial correction can be performed.

Description of the drawings

FIG. 1 is a schematic diagram of the structure in a preferred embodiment of the present invention;

2 is a schematic diagram of a portable terminal before being placed in a preferred embodiment of the present invention;

FIG. 3 is a schematic diagram of a portable terminal in a preferred embodiment of the present invention after being placed;

Figure 4 is a schematic diagram of the overall flow in a preferred embodiment of the present invention;

FIG. 5 is a schematic flowchart of step S1 in a preferred embodiment of the present invention;

FIG. 6 is a schematic flowchart of the first process in a preferred embodiment of the present invention;

FIG. 7 is a schematic flowchart of the second process in a preferred embodiment of the present invention;

FIG. 8 is a schematic flowchart of the third process in a preferred embodiment of the present invention;

FIG. 9 is a schematic structural diagram of the third process in a preferred embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other if there is no conflict.

An augmented reality display system, as shown in Figure 1 to Figure 3, includes:

A headset frame 1, which has a circular ring shape and is used for users to wear;

A groove 2, the opening of the groove 2 is inclined upward, and one side of the groove 2 is connected with the headset frame 1;

A lens 3, the lens 3 is arranged under the groove 2 and connected to the other side of the groove 2, and the lens 3 is made of semi-reflective and semi-transparent material;

A portable terminal 4, the portable terminal 4 has a first side provided with a display unit and a second side provided with an image acquisition unit 41, the first side and the second side are arranged opposite to each other, the portable terminal 4 further includes a processing unit , Used to process the real-time images collected by the image collection unit 41 and display them on the display unit;

The size of the portable terminal 4 is adapted to the size of the groove 2. When the portable terminal 4 is placed in the groove 2, the first side of the portable terminal 4 faces the lens 3;

The processing unit specifically includes:

Inertial measurement module, used to collect and output real-time motion data;

The pose processing module, connected to the inertial measurement module, is used to determine the current pose of the portable terminal 4 according to the real-time image collected by the image acquisition unit 41 and the real-time motion data at the corresponding time;

The image processing module, connected to the pose processing module, is used to generate a virtual visual range and a virtual screen according to the current pose of the portable terminal 4, and reflect the virtual screen to the user through the display unit and the lens 3 for viewing.

Specifically, the display device in the prior art often sets the processing unit to perform data interaction in the display device, and uses the three-free gyroscope of the mobile phone or preset pictures for positioning, which results in the user being unable to move closer to or away from the virtual The tracking and positioning of three-dimensional objects is very unstable, and the user's range of movement is limited by the location of the picture.

This technical solution provides an augmented reality display system. There is no data interaction between the portable terminal 4 and the headset frame 1, the groove 2 and the lens 3. The portable terminal 4 collects real-time images through the image acquisition unit 41 and the inertial measurement module. Real-time motion data, then the current pose is determined by the pose processing module and the image processing module, and the virtual visual range and virtual screen are generated according to the current pose. Finally, the virtual screen is reflected to the user through the display unit and the lens 3, and the user The virtual picture and the real environment are observed through the lens 3, and the superposition and fusion between the virtual picture and the real environment is realized, and the purpose of augmented reality is achieved. Here, a mobile phone can be selected as the portable terminal 4, so as to realize the rapid acquisition of real-time images and real-time motion data, and the generation of virtual visual ranges and virtual images.

Further, in the process of generating the virtual visual range, the image processing unit needs to obtain the current pose of the portable terminal 4 and construct a spatial rectangular coordinate system, and determine the virtual visual range based on the user’s real visual range to generate an appropriate virtual visual range. The screen is displayed on the lens 3.

Further, when the portable terminal 4 is placed in the groove 2, the first side of the portable terminal 4 faces the lens 3, and the area in the groove 2 that fits the first side of the portable terminal 4 can be either a hollow design or a hollow design. It is made of light-transmitting materials to display a virtual screen on the lens 3 for easy viewing by the user.

Further, a hook 21 matching fixing device, such as a fixing rope, can be provided on one side of the groove to assist in fixing the portable terminal 4, so as to avoid the position and angle of the portable terminal 4 that may be caused by the user's posture changes during use. Change.

In a preferred embodiment of the present invention, the image collection unit 41 obtains a real-time image by collecting a feature point area including a plurality of feature points;

The processing unit also includes:

The feature point processing module is connected to the image acquisition unit 41 and the pose processing module respectively, and is used to obtain the feature points in the real-time image and analyze the location of the feature point area, and output the feature points to the pose processing module according to the analysis result ；

The pose processing module uses the position of the feature point area as a reference, and determines the current pose of the portable terminal 4 according to the real-time motion data at the corresponding time.

Specifically, the image acquisition unit 41 acquires real-time images and outputs the real-time images to the image processing unit. The image processing unit extracts feature points in the image and analyzes the regions corresponding to the feature points, and generates an instruction according to the analysis result to enable the image acquisition unit 41 collects more feature points with spatial recognition, so as to finally determine the corresponding area of the virtual screen.

In a preferred embodiment of the present invention, the virtual visual range includes virtual angle information;

The image processing module includes:

A first processing component, connected to the pose processing module, is used to construct a spatial rectangular coordinate system with the image acquisition unit 41 as the origin according to the current pose, determine the spatial rotation angle of the image acquisition unit 41, and the spatial rotation angle with the user The included angle between the virtual angle information is generated according to the space rotation angle and the included angle and included in the virtual visual range for output.

In a preferred embodiment of the present invention, the virtual visual range includes virtual location information;

The image processing module includes:

A second processing component, connected to the pose processing module, is used to construct a spatial rectangular coordinate system with the image acquisition unit 41 as the origin according to the current pose, and select the center of the user’s brow as the preset reference point, according to the portable terminal 4 and the preset reference point. It is assumed that the offset between the reference points and the interpupillary distance of the user generates virtual position information and is included in the virtual visual range for output.

In a preferred embodiment of the present invention, the virtual visual range includes virtual field of view information;

The image processing module includes:

A third processing component, connected to the pose processing module, is used to generate a preset virtual screen, and generate virtual field of view information according to the position difference between the curved edge of the preset virtual screen and the user, and include it in the virtual visual Output in the range.

An augmented reality display method, applied to any one of the above-mentioned display systems, as shown in FIG. 4, a head-mounted display frame 1, a groove 2, a lens 3, and a portable terminal 4 are arranged in the display device;

The display methods include:

Step S1, the image acquisition unit 41 acquires a real-time image of the plane directly above the portable terminal 4, and

Inertial measurement module collects real-time motion data;

Step S2, the pose processing module processes the current pose of the portable terminal 4 according to the real-time image and real-time motion data;

Step S3, the image processing module generates a virtual visual range and a virtual picture according to the current pose, and sends the virtual picture to the display unit for display;

Step S4, the virtual picture displayed on the display unit is reflected by the lens 3 to the user for viewing.

Specifically, in order to realize the fusion of virtual images and real images, an augmented reality display method is provided. In step S1-step S2, the user wears the head-mounted display frame 1, the user's eyes are facing the lens 3 of the display device, the image acquisition unit 41 of the portable terminal 4 faces upward, and the real-time image above the portable terminal 4 is collected. The inertial measurement module measures the portable The real-time motion data of the terminal 4 finally determines the current pose of the portable terminal 4.

Augmented reality needs to present a virtual screen on the lens 3 to realize the superposition and fusion between the virtual screen and the real image that the user sees through the lens 3. Therefore, the image processing module must ensure that the final virtual screen can be accurately incident on the user's Eye. Therefore, in step S3, the actual visual range and the virtual visual range of the user are determined according to the current pose of the portable terminal 4. The real visual range here refers to the visual range of the user’s human eyes to observe the real environment ahead through the lens 3, and the virtual visual range refers to the simulation in the virtual space by the image processing module in the process of generating the virtual screen. The sight range corresponding to the virtual human eye. Then use the virtual visual range to generate a suitable virtual screen. When the virtual screen is generated, the virtual visual range is determined according to the actual visual range.

The virtual visual range here includes virtual position information, virtual angle information, and virtual field of view angle information. When the virtual position information, virtual angle information, and virtual field angle information in the virtual visual range correspond to the actual position information, real angle information, and real field angle information in the display visual range, it can ensure that the virtual space The virtual human eyes overlap with the user’s eyes in the real environment, and the virtual visual range corresponds to the user’s real visual range. After the virtual screen is reflected on the lens 3, it can be reflected in the user’s eyes together with the real environment, allowing the user to observe The virtual picture is more in line with the ideal effect.

In a preferred embodiment of the present invention, as shown in FIG. 5, step S1 includes:

Step S11, the image acquisition unit 41 obtains a real-time image by collecting a feature point region including a plurality of feature points;

Step S12, the image acquisition unit 41 acquires the feature points in the real-time image and analyzes whether the location of the feature point area meets the viewing angle requirement:

If yes, go to step S2;

If not, go to step S13;

In step S13, the image acquisition unit 41 generates a prompt instruction for indicating that there are too few feature points and feeds it back to the user, and then returns to step S11.

Specifically, after the user wears the headset frame 1, the preset program in the portable terminal 4 will guide the user to walk around in the space, and the corresponding image acquisition unit 41 starts and collects the upper image, extracts the feature points in the real-time image, and analyzes the features Whether the corresponding area covered by the point, the robustness of the feature point and other parameters meet the viewing angle requirements, if yes, go to step S2, if not, then generate a guidance instruction to guide the user to change the current position, travel direction and line of sight direction to fill the collected In the image corresponding to the area with fewer feature points, the specific position of the virtual picture in the real scene is finally determined.

Further, in order to control the specific location information of the virtual screen, feature points can also be preset through an external feature point device. The feature point device includes a launch unit and a diffusion unit. The diffusion unit is arranged above the launch unit. A transparent disc can be selected from the diffusion unit and multiple feature points can be added to the transparent disc. The launch unit can select a laser emission The device projects the characteristic points of the diffusion unit above the space where the display device is located, and the image processing module then performs characteristic point analysis according to the upper image collected by the image acquisition unit 41.

In a preferred embodiment of the present invention, in step S2, the image processing module determines the current pose through the vSLAM algorithm.

In a preferred embodiment of the present invention, the virtual viewing range includes virtual angle information;

Step S3 includes the first process of generating virtual angle information;

The first process, as shown in Figure 6, includes:

In step S31A, the image processing module constructs a spatial rectangular coordinate system with the image acquisition unit 41 as the origin according to the current pose, and determines the spatial rotation angle of the image acquisition unit 41 in the spatial rectangular coordinate system;

Step S32A, acquiring the angle between the image acquisition unit 41 and the user;

Step S33A, generating virtual angle information according to the spatial rotation angle and the included angle.

In a preferred embodiment of the present invention, the virtual angle information is expressed by the following formula:

θ=(θ _X ,θ _Y -α,θ _Z ) (1)

in,

θ is used to express virtual angle information;

θ _{X is} used to represent the pitch angle in the space rotation angle;

θ _{Y is} used to represent the yaw angle in the space rotation angle;

α is used to indicate the included angle;

θ _{Z is} used to represent the roll angle in the space rotation angle.

Step S3 includes a second process of generating virtual location information;

The second process, as shown in Figure 7, includes:

In step S31B, the image processing module constructs a spatial rectangular coordinate system with the image acquisition unit 41 as the origin according to the current pose, and selects the center of the user's brow as a preset reference point, according to the offset between the image acquisition unit 41 and the preset reference point Quantities, generate the first position information;

In step S32B, the image processing module adjusts the first position information according to the user's interpupillary distance, generates second position information, and outputs the second position information as virtual position information.

Specifically, when the user turns the head up and down, the roll angle of the image acquisition unit 41 also changes accordingly, and the angle of the up and down rotation corresponds to the roll angle. When the user turns the head back and forth, the pitch angle of the image acquisition unit 41 also changes accordingly. Change, the angle of the front and back rotation corresponds to the pitch angle, and when the user rotates the head left and right, the image acquisition unit 41 performs a circular rotation at the current position, and there is a deviation between the angle of the left and right rotation and the yaw angle.

Therefore, the space coordinate system is constructed with the image acquisition unit 41 at this time as the origin, and the space rotation angle of the image acquisition unit 41 in the space coordinate system is determined. Here, Euler angles are used to express as (θ _X , θ _Y , θ _Z ) Then, the angle α between the image acquisition unit 41 and the user’s line of sight on the y-axis is calculated by formula (1), thereby determining the virtual angle information θ in the virtual visual range, and according to the virtual angle information, to ensure that the virtual space The angle information of the virtual human eye overlaps with the angle information of the user’s eye in the real environment, and the virtual visual range corresponds to the user’s real visual range, so as to realize the superimposition and fusion of the virtual screen and the real scene.

In a preferred embodiment of the present invention, the first location information is expressed by the following formula:

χ′=(-B _X ,-B _Y ,-B _Z ) (2)

in,

χ′ is used to express the first position information;

B _{X is} used to indicate the projection of the offset on the X axis;

B _{Y is} used to indicate the projection of the offset on the Y axis;

B _{Z is} used to indicate the projection of the offset on the Z axis.

In a preferred embodiment of the present invention, the second position information includes left-eye position information and right-eye position information;

The second location information is expressed by the following formula:

in,

χ″ _{1 is} used to express the position information of the left eye;

χ″ _{2 is} used to express the position information of the right eye;

B _{X is} used to indicate the projection of the offset on the X axis;

B _{Y is} used to indicate the projection of the offset on the Y axis;

I is used to represent the interpupillary distance of the user;

B _{Z is} used to indicate the projection of the offset on the Z axis.

Specifically, in the prior art, the vSLAM algorithm is often used to fuse the image collected by the image acquisition unit 41 with the data collected by the sensor to calculate the six degrees of freedom information of the device, and the determined position information in the virtual visual range is the image acquisition unit 41 current position. Due to the large deviation between the user's eyes in the real environment and the position information of the image acquisition unit 41, when the user wears the display device to observe the virtual screen, obvious screen misalignment may occur.

Therefore, a second process of determining virtual position information is set in step S3, considering that the distance between the lens 3 and the center of the eyebrows, and the distance between the center of the eyebrows and the left and right eyes are basically The center of the brow is selected as the reference point. In step S31B, it is first determined that the position information of the image acquisition unit 41 in the spatial rectangular coordinate system is (0, 0, 0), and then the position between the image acquisition unit 41 and the center of the brow is acquired. The relative position, thereby determining the first position information, that is, the position information (-B _X , B _Y , B _Z ) of the center of the eyebrows in the spatial rectangular coordinate system, which can be regarded as the generated virtual human eye position from the image The acquisition unit 41 turns to the center of the eyebrows, and then obtains the pupil distance I of the real user's eyes in step S32B, and then adjusts the first position information to generate second position information, that is, the position information of the left and right eyes in the spatial rectangular coordinate system (-B _X , -B _Y -I/2, -B _Z ) and (-B _X , -B _Y +I/2, -B _Z ) to ensure that the position information of the virtual human eye in the virtual space is consistent with that in the real environment The position information of the user's eyes overlaps, and the virtual visual range corresponds to the user's real visual range, so as to realize the superimposition and fusion of the virtual screen and the real scene.

In a preferred embodiment of the present invention, the virtual viewing range includes virtual viewing angle information;

Step S3 includes a third process of generating virtual field of view information;

The third process, as shown in Figure 8, includes:

Step S31C, the image processing module generates a preset virtual picture and displays it on the lens 3;

Specifically, as shown in FIG. 9, in the process of determining the virtual field of view information, the portable terminal 4 forms a preset virtual screen on the lens 3 through the display unit, and respectively calculates the curved edge of the preset virtual screen to the corresponding user person. The distance of the eye finally determines the virtual field of view information.

The above are only preferred embodiments of the present invention, and do not therefore limit the implementation and protection scope of the present invention. For those skilled in the art, they should be aware of equivalent replacements and equivalents made by using the description and illustrations of the present invention. All solutions obtained by obvious changes should be included in the protection scope of the present invention.

Claims

An augmented reality display system is characterized in that it comprises:

A head-mounted display frame, the head-mounted display frame has a circular ring shape and is used for wearing by the user;

A groove, the opening of the groove is inclined upward, and one side of the groove is connected with the headset frame;

A lens, the lens is arranged below the groove and connected to the other side of the groove, and the lens is made of a semi-reflective and semi-transparent material;

A portable terminal, the portable terminal has a first side provided with a display unit and a second side provided with an image acquisition unit, the first side and the second side are arranged opposite to each other, the portable terminal is also Comprising a processing unit for processing real-time images collected by the image collecting unit and displaying them on the display unit;

The size of the portable terminal is adapted to the size of the groove, and when the portable terminal is put into the groove, the first side of the portable terminal faces the lens;

The processing unit specifically includes:

Inertial measurement module, used to collect and output real-time motion data;

A pose processing module, connected to the inertial measurement module, and configured to determine the current pose of the portable terminal according to the real-time image collected by the image acquisition unit and the real-time motion data at the corresponding time;

An image processing module, connected to the pose processing module, for generating a virtual visual range and a virtual screen according to the current pose of the portable terminal, and reflecting the virtual screen to the display unit and the lens User view.
The augmented reality display system according to claim 1, wherein the real-time image includes feature points;

The processing unit also includes:

A feature point processing module, which is respectively connected to the image acquisition unit and the pose processing module, and is used to obtain the feature points in the real-time image and analyze the area corresponding to the feature points, according to Outputting the characteristic points to the pose processing module as a result of the analysis;

The pose processing module determines the current pose of the portable terminal according to the area corresponding to the feature point and the real-time motion data at the corresponding time.
The augmented reality display system according to claim 1, wherein the virtual visual range includes virtual angle information;

The image processing module includes:

A first processing component, connected to the pose processing module, and configured to construct a spatial rectangular coordinate system with the image acquisition unit as the origin according to the current pose, and determine the spatial rotation angle of the image acquisition unit , And the angle between the space rotation angle and the user, and generating the virtual angle information according to the space rotation angle and the included angle;

An image generating component, connected to the first processing component, for generating the virtual image according to the virtual angle information, and outputting the virtual screen to the display unit, through the display unit and the lens Reflect to the user for viewing.
The augmented reality display system according to claim 1, wherein the virtual visual range includes virtual position information;

The image processing module includes:

A second processing component, connected to the pose processing module, configured to construct a spatial rectangular coordinate system with the image acquisition unit as the origin according to the current pose, and select the center of the user's brow as a preset reference Point, generating the virtual position information according to the offset between the portable terminal and the preset reference point and the interpupillary distance of the user;

An image generating component, connected to the second processing component, for generating the virtual image according to the virtual position information, and outputting the virtual screen to the display unit, through the display unit and the lens Reflect to the user for viewing.
The augmented reality display system according to claim 1, wherein the virtual visual range includes virtual field of view information;

The image processing module includes:

A third processing component, connected to the pose processing module, for generating a preset virtual screen, and generating the virtual field of view according to the position difference between the curved edge of the preset virtual screen and the user Angle information

An image generating component, connected to the third processing component, for generating the virtual image according to the virtual field of view information, and outputting the virtual screen to the display unit, through the display unit and the The lens reflects for the user to view.
An augmented reality display method, applied to the display system according to any one of claims 1-5, characterized in that a head-mounted display frame, a groove, a lens, and a portable display device are provided in the display device terminal;

The display method includes:

Step S1, the image acquisition unit acquires a real-time image of a plane directly above the portable terminal, and

The inertial measurement module collects real-time motion data;

Step S2, the pose processing module processes the current pose of the portable terminal according to the real-time image and the real-time motion data;

Step S3, the image processing module generates a virtual visual range and a virtual screen according to the current pose, and sends the virtual screen to the display unit for display;

Step S4, the virtual screen displayed on the display unit is reflected by the lens to the user for viewing.
The display method according to claim 6, wherein the step S1 includes:

Step S11, the image acquisition unit obtains the real-time image by collecting a feature point region including a plurality of feature points;

Step S12, the image acquisition unit extracts feature points in the real-time image and analyzes whether the area corresponding to the feature point meets the viewing angle requirement:

If yes, go to step S2;

If not, go to step S13;

In step S13, the image acquisition unit generates a prompt instruction for indicating that the feature points are too few and feeds it back to the user, and then returns to the step S11.
The display method according to claim 6, wherein in the step S2, the image processing module determines the current pose through a vSLAM algorithm.
The display method according to claim 6, wherein the virtual viewing range includes virtual angle information;

The step S3 includes a first process of generating the virtual angle information;

The first process includes:

Step S31A, the image processing module constructs a spatial rectangular coordinate system centered on the image acquisition unit according to the current pose, and determines the spatial rotation angle of the image acquisition unit in the spatial rectangular coordinate system;

Step S32A, acquiring the angle between the image acquisition unit and the user;

Step S33A, generating the virtual angle information according to the spatial rotation angle and the included angle.
The display method according to claim 9, wherein the virtual angle information is expressed by the following formula:

θ=(θ X ,θ Y -α,θ Z )

in,

θ is used to express the virtual angle information;

θ X is used to represent the pitch angle in the space rotation angle;

θ Y is used to represent the yaw angle in the space rotation angle;

α is used to indicate the included angle;

θ Z is used to represent the roll angle in the spatial rotation angle.
The display method according to claim 6, wherein the virtual visual range includes virtual position information;

The step S3 includes a second process of generating the virtual location information;

The second process includes:

In step S31B, the image processing module constructs a spatial rectangular coordinate system centered on the image acquisition unit according to the current pose, and selects the center of the user's brow as a preset reference point, and according to the image acquisition unit and Generating the first position information by the offset between the preset reference points;

Step S32B: The image processing module adjusts the first position information according to the interpupillary distance of the user, generates second position information, and outputs the second position information as the virtual position information.
The display method according to claim 11, wherein the first position information is expressed by the following formula:

χ′=(-B X ,-B Y ,-B Z )

in,

χ′ is used to express the first position information;

B X is used to indicate the projection of the offset on the X axis;

B Y is used to represent the projection of the offset on the Y axis;

B Z is used to indicate the projection of the offset on the Z axis.
The display method according to claim 11, wherein the second position information includes left-eye position information and right-eye position information;

The second location information is expressed by the following formula:

in,

χ" 1 is used to express the left eye position information;

χ" 2 is used to express the right eye position information;

B X is used to indicate the projection of the offset on the X axis;

B Y is used to represent the projection of the offset on the Y axis;

I is used to represent the interpupillary distance of the user;

B Z is used to indicate the projection of the offset on the Z axis.
The display method according to claim 6, wherein the virtual visual range includes virtual field of view information;

The step S3 includes a third process of generating the virtual field of view information;

The third process includes:

Step S31C, the image processing module generates a preset virtual screen and displays it on the lens;

Step S32C, the image processing module calculates the position difference between the curved edge of the preset virtual screen and the user;

In step S33C, the image processing module determines the virtual field of view information according to the position difference.