WO2023226693A1

WO2023226693A1 - Display module and display method thereof, display apparatus, and virtual display device

Info

Publication number: WO2023226693A1
Application number: PCT/CN2023/091507
Authority: WO
Inventors: 李治富; 汪志强; 苗京花
Original assignee: 京东方科技集团股份有限公司; 北京京东方显示技术有限公司
Priority date: 2022-05-26
Filing date: 2023-04-28
Publication date: 2023-11-30
Also published as: WO2023226693A9; CN115002444A

Abstract

Embodiments of the present disclosure provide a display method, comprising: splitting a frame of color picture into n picture sub-frames; sequentially displaying the n picture sub-frames, n being greater than or equal to 3, and n being an integer; obtaining an eye image, processing the eye image, and determining a current pupil position corresponding to a first picture sub-frame, the first picture sub-frame being a picture sub-frame which is firstly displayed during sequentially displaying the n picture sub-frames; according to the current pupil position corresponding to the first picture sub-frame and pupil positions corresponding to the first picture sub-frame obtained after previous m measurements, calculating a current pupil position corresponding to each subsequent picture sub-frame of the first picture sub-frame in the n picture sub-frames, m being greater than or equal to 2, and m being an integer; calculating an actual offset of an original position for display of each subsequent picture sub-frame relative to the current position of the first picture sub-frame; calculating an actual display position of each subsequent picture sub-frame according to the actual offset; and sequentially displaying the subsequent picture sub-frames according to the actual display positions of the subsequent picture sub-frames.

Description

Display module and display method thereof, display device, virtual display device

Technical field

The present disclosure relates to the field of display technology, and specifically to a display module and a display method thereof, a display device, and a virtual display device.

Background technique

Virtual Reality (VR for short) display equipment uses VR display equipment to close people's vision and hearing to the outside world, guiding users to have a feeling of being in a virtual environment. Its display principle is to use computer technology to simulate a three-dimensional and highly simulated 3D space. When the user wears the VR head-mounted display device, it will create the illusion of being in reality. In this space, operators can use controllers or keyboards to navigate or interact in this virtual environment.

Contents of the invention

On the one hand, embodiments of the present disclosure provide a display method, which includes:

Disassemble one frame of color picture into n sub-frame pictures; the n sub-frame pictures are displayed in sequence, n≥3, and n is an integer;

Obtain an eye image, process the eye image, and determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is the sub-frame picture that is displayed first when the n sub-frame pictures are displayed in sequence. ;

According to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements, calculate each subsequent pupil position of the first sub-frame picture in the n sub-frame pictures. The current pupil position corresponding to the sub-frame picture; m≥2, and m is an integer;

Calculate the original to-be-displayed position of each subsequent subframe relative to the current pupil position corresponding to the first subframe based on the offset of the current pupil position corresponding to each subsequent subframe in the n subframes. The actual offset of the current position of the first sub-frame picture; and based on the original to-be-displayed position of each subsequent sub-frame picture and its current position relative to the first sub-frame picture The actual offset is used to calculate the actual display position of each subsequent sub-frame;

Each subsequent sub-frame picture is displayed in sequence according to the actual display position of each subsequent sub-frame picture.

In some embodiments, obtaining the eye image, processing the eye image, and determining the current pupil position corresponding to the first subframe includes:

When the first sub-frame is displayed, capture an eye image;

Convert eye images to grayscale images;

In the grayscale image, the left and right corner position points of the eye are detected based on the eye corner feature points; the left and right corner position points are connected as the X-axis, the axis perpendicular to the X-axis is used as the Y-axis, and the origin of the intersection of the X-axis and the Y-axis is The midpoint of the line connecting the left and right eye corners;

The grayscale image is processed in the coordinate plane formed by the X-axis and the Y-axis to determine the pupil area of the eye; the center of the pupil area is determined as the current pupil position corresponding to the first sub-frame picture.

In some embodiments, the grayscale image is processed in the coordinate plane constituted by the The current pupil position, including:

Binarize the grayscale image to obtain the binarized image of the eye;

Using a connected region labeling method to detect candidate pupil connected regions on the binary image of the eye;

Based on geometric constraints and distance constraint algorithms, filter out the pupil area of the eye from the candidate pupil connected areas;

Cover the pupil area with a circle with the smallest diameter, and determine the center of the circle as the center of the pupil area.

In some embodiments, n=3, m=2;

Subsequent subframe pictures of the first subframe picture include a second subframe picture and a third subframe picture; the first subframe picture, the second subframe picture and the third subframe picture are performed in sequence. show;

According to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements, calculate the first sub-frame picture in the n sub-frame pictures. The current pupil position corresponding to each subsequent sub-frame includes:

The current pupil position corresponding to the second sub-frame is calculated according to formula (1), and the current pupil position corresponding to the third sub-frame is calculated according to formula (2);
pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta ² ; (1)

pos_curr_b＝pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta) ² ; (2)

in,

a_curr is the current acceleration of eyeball rotation; v_curr is the current speed of eyeball rotation; pos_1 and pos_2 are respectively the pupil positions corresponding to the first sub-frame obtained by the previous two measurements; pos_curr is the corresponding pupil position of the first sub-frame. The current pupil position; pos_curr_g is the current pupil position corresponding to the second sub-frame picture; pos_curr_b is the current pupil position corresponding to the third sub-frame picture; delta is the refresh duration of one of the sub-frame pictures, delta=1/ The refresh frequency of one of the sub-frame pictures.

In some embodiments, the subsequent sub-frames are calculated based on the offset of the current pupil position corresponding to the subsequent sub-frames in the n sub-frames relative to the current pupil position corresponding to the first sub-frame. The actual offset of the original to-be-displayed position relative to the current position of the first sub-frame picture; and based on the original to-be-displayed position of each subsequent sub-frame picture and its relative to the current position of the first sub-frame picture The actual offset is used to calculate the actual display position of each subsequent sub-frame, including:

The original to-be-displayed position of the second sub-frame relative to the first sub-frame is calculated based on the offset of the current pupil position corresponding to the second sub-frame relative to the current pupil position corresponding to the first sub-frame. The first offset matrix of the current position of the sub-frame picture;

The original to-be-displayed position of the third sub-frame relative to the first sub-frame is calculated based on the offset of the current pupil position corresponding to the third sub-frame relative to the current pupil position corresponding to the first sub-frame. the second offset matrix of the current position of the sub-frame picture;

Multiply the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture;

Multiply the original to-be-displayed position of the third sub-frame picture by the second offset matrix to obtain the actual display position of the third sub-frame picture.

In some embodiments,

The first offset matrix is:

The second offset matrix is:

Among them, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original to-be-displayed position of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx1, Ty1, Tz1 respectively represents the translation amplitude of the original to-be-displayed position of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture;

Rx2, Ry2, and Rz2 respectively represent the rotation amplitude of the original to-be-displayed position of the third sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx2, Ty2, and Tz2 respectively Represents the translation amplitude of the original to-be-displayed position of the third sub-frame picture relative to the current position of the first sub-frame picture along the X-axis, Y-axis, and Z-axis;

The Z-axis is perpendicular to the two-dimensional coordinate plane formed by the intersection of the X-axis and the Y-axis, and the Z-axis intersects the X-axis and Y-axis at the origin.

In some embodiments, the method further includes: storing the current pupil position corresponding to the first sub-frame and the current pupil position corresponding to each subsequent sub-frame.

In a second aspect, an embodiment of the present disclosure also provides a display module, which includes:

The disassembly module is configured to disassemble a frame of color picture into n sub-frame pictures; n≥3, and n is an integer;

A display module configured to display the n sub-frame pictures in sequence;

A processing module configured to acquire an eye image, process the eye image, and determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is the first to be displayed among the n sub-frame pictures. sub-frame picture;

The first prediction module is configured to calculate the pupil position corresponding to the first sub-frame picture based on the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements. The current pupil position corresponding to each subsequent sub-frame of the first sub-frame; m≥2, and m is an integer;

The second prediction module is configured to calculate each subsequent subframe picture based on the offset of the current pupil position corresponding to each subsequent subframe picture in the n subframe pictures relative to the current pupil position corresponding to the first subframe picture. The actual offset of the original to-be-displayed position relative to the current position of the first sub-frame picture; and based on the original to-be-displayed position of each subsequent sub-frame picture and its relative to the current position of the first sub-frame picture The actual offset is used to calculate the actual display position of each subsequent sub-frame;

The display module is further configured to display subsequent sub-frame pictures in sequence according to the actual display position of each subsequent sub-frame picture.

In some embodiments, the display module includes a display panel and a lens, the lens being located on a display side of the display panel;

The processing module includes an infrared transmitter and an infrared camera,

The infrared emitter is located on a side of the lens away from the display panel and distributed around the edges of the lens for emitting infrared light to the eyes;

The infrared camera is located on a side of the lens away from the display panel and on the edge of the lens for capturing eye images.

In some embodiments, the processing module further includes an image processing unit configured to The head image is converted into a grayscale image; in the grayscale image, the left and right corner position points of the eye are detected based on the eye corner feature points; the left and right corner position points are connected and used as the X-axis, and the axis perpendicular to the X-axis is used as the Y-axis, The origin of the intersection of the axis and the Y-axis is the midpoint of the line connecting the left and right eye corner points;

The image processing unit is further configured to process the grayscale image of the eye, determine the pupil area of the eye, and determine the center of the pupil area as the current pupil position corresponding to the first sub-frame.

In some embodiments, the second prediction module is configured to calculate the first subframe based on an offset of the current pupil position corresponding to the second subframe relative to the current pupil position corresponding to the first subframe. The first offset matrix of the original to-be-displayed position of the second sub-frame picture relative to the current position of the first sub-frame picture;

The second prediction module is further configured to calculate the offset of the current pupil position corresponding to the third sub-frame picture from the current pupil position corresponding to the first sub-frame picture. a second offset matrix of the original to-be-displayed position relative to the current position of the first sub-frame picture;

The second prediction module is further configured to multiply the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture; The original to-be-displayed position of the third sub-frame picture is multiplied by the second offset matrix to obtain the actual display position of the third sub-frame picture.

In some embodiments, a storage module is further included, configured to store the current pupil position corresponding to the first sub-frame and the current pupil position corresponding to each subsequent sub-frame.

In a third aspect, an embodiment of the present disclosure further provides a display device, which includes the above-mentioned display module.

In a fourth aspect, an embodiment of the present disclosure further provides a virtual display device, which includes the above display device.

Description of the drawings

The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure, and constitute a part of the specification. points, together with the embodiments of the present disclosure, are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

Figure 1 is a schematic diagram of the principle of color separation when VR displays a color picture in the public technology.

Figure 2 is a picture of a scene where color separation occurs in the disclosed technology.

Figure 3 is a functional block diagram of a display module in an embodiment of the present disclosure.

FIG. 4 is a schematic top view of the arrangement of pixels and light sources in the display module according to the embodiment of the present disclosure.

Figure 5 is a schematic diagram of disassembling a color picture into three sub-frames in an embodiment of the present disclosure.

FIG. 6 is another schematic diagram of disassembling a color picture into three sub-frames in an embodiment of the present disclosure.

Figure 7 is a schematic diagram of each sub-frame picture being refreshed sequentially and the light sources of different colors in the backlight module being sequentially lit during the display process of the display module according to the embodiment of the present disclosure.

Figure 8 is a schematic disassembly of the structure of a display module in an embodiment of the present disclosure.

Figure 9a is a schematic diagram of a grayscale image of an eye pattern.

Figure 9b is a schematic diagram of a binarized image of the eye obtained through processing by the image processing unit in an embodiment of the present disclosure.

Figure 10 is a schematic diagram of the detection frequency of the pupil position corresponding to the first sub-frame in an embodiment of the present disclosure.

FIG. 11 is a schematic diagram of the process of predicting the actual display positions of the second subframe picture and the third subframe picture in an embodiment of the present disclosure.

Figure 12 is a flow chart of a display module display method in an embodiment of the present disclosure.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions of the embodiments of the present disclosure, the display module and its display method, display device, and virtual display device provided by the embodiments of the present disclosure are further described in detail below in conjunction with the drawings and specific implementation modes. .

Embodiments of the disclosure will be described more fully hereinafter with reference to the accompanying drawings, but the illustrated embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth in the disclosure. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully understand the scope of the disclosure to those skilled in the art.

The disclosed embodiments are not limited to the embodiments shown in the drawings but include modifications of configurations formed based on the manufacturing process. Accordingly, the regions illustrated in the figures are schematic in nature, and the shapes of the regions shown in the figures are illustrative of the specific shapes of the regions and are not intended to be limiting.

Hereinafter, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

In public technology, for VR display screens, users will have an immersive feeling when the screen refresh rate reaches 90Hz. When using a field-sequential screen as a VR screen, in order to ensure the above effects, the refresh rate needs to reach 270Hz. The field sequence screen divides the previous frame of color image (RGB image) into three sub-frames (that is, images backlit by red, green, and blue light sources respectively) for display. Each sub-frame is displayed in sequence, and The backlight used for the display of each sub-frame picture is also provided in sequence.

Referring to Figure 1, there is a schematic diagram of the principle of color separation when VR displays a color picture in the public technology; as the human eye rotates, pixels at the same position on the screen display red, green, and blue light sources respectively to provide backlight. The green and blue images fall on different locations on the retina of the human eye, causing the human eye to experience color separation. Referring to Figure 2, there is a picture of a scene where color separation occurs in the disclosed technology; from Figure 2, it can be seen that there are red, green and blue stripes on the window frame. Analyzing the scene in which color separation occurs, it is found that the greater the difference between the three subframes (that is, when the display grayscale of the three subframes is the same and the display grayscale is not 0), the more obvious the color separation is. For example: three subframes (i.e., a subframe with backlight provided by a red light source, a subframe with backlight provided by a green light source, and a subframe with backlight provided by a blue light source) are displayed respectively. When displaying 255 gray scale, the backlight of each sub-frame lights up sequentially. At this time, color separation is most likely to occur when the eye scans.

In order to solve the above-mentioned problems existing in the disclosed technology, in the first aspect, embodiments of the present disclosure provide a display method, which includes: disassembling a frame of color pictures into n sub-frame pictures; displaying the n sub-frame pictures in sequence, n ≥3, and n is an integer; obtain the eye image, process the eye image, and determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is displayed first when the n sub-frame pictures are displayed in sequence subframe picture; based on the current pupil position corresponding to the first subframe picture and the pupil position corresponding to the first subframe picture obtained by the previous m measurements, calculate subsequent subframe pictures of the first subframe picture in n subframe pictures Corresponding current pupil position; m ≥ 2, and m is an integer; calculate subsequent pupil positions based on the offset of the current pupil position corresponding to each subsequent subframe in n subframes relative to the current pupil position corresponding to the first subframe. The actual offset of the original to-be-displayed position of the sub-frame picture relative to the current position of the first sub-frame picture; and based on the original to-be-displayed position of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture The offset calculates the actual display position of each subsequent sub-frame picture; the subsequent sub-frame pictures are displayed in sequence according to the actual display position of each subsequent sub-frame picture.

The display method of the display module provided in the embodiment of the present disclosure determines the current pupil position corresponding to the first sub-frame picture displayed first among the n sub-frame pictures through processing; calculates the corresponding corresponding sub-frame pictures of the first sub-frame picture The current pupil position of the first sub-frame picture; calculate the actual display position of each subsequent sub-frame picture of the first sub-frame picture based on the calculation results of the current pupil position corresponding to each subsequent sub-frame picture of the first sub-frame picture; be able to obtain the rotation trajectory of the human eye information in order to capture and track the rotation of the human eye; by adjusting the subsequent sub-frames of the first sub-frame from the original to-be-displayed position to the actual display position, it can be achieved that the subsequent sub-frames of the first sub-frame are The actual display position of the frame picture catches up with the position of the eyeball rotation, so that the picture displayed by the pixels with the same position on the subsequent sub-frame pictures of the first sub-frame picture is projected to the same position in the pupil area of the human eye (that is, on the retina). Three pictures with backlights provided by different color light sources are fused here to eliminate color separation and ensure that the picture perceived by the human eye is a complete picture.

In a second aspect, an embodiment of the present disclosure provides a display module. Refer to FIG. 3 , which is a functional block diagram of the display module in an embodiment of the present disclosure. The display module includes: a disassembly module configured to convert a frame of color The picture is disassembled into n sub-frame pictures; n≥3, and n is an integer; the display module is configured to display the n sub-frame pictures in sequence; the processing module is configured to obtain the eye image and process the eye image , determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is the sub-frame picture that is displayed first when n sub-frame pictures are displayed in sequence; the first prediction module is configured to be based on the first sub-frame picture corresponding to The current pupil position and the pupil position corresponding to the first sub-frame obtained by the previous m measurements are used to calculate the current pupil position corresponding to the subsequent sub-frames of the first sub-frame in n sub-frames; m≥2, and m is integer; the second prediction module is configured to calculate the original value of each subsequent sub-frame based on the offset of the current pupil position corresponding to the subsequent sub-frame in the n sub-frame relative to the current pupil position corresponding to the first sub-frame. The actual offset of the position to be displayed relative to the current position of the first sub-frame picture; and the subsequent calculation is based on the original to-be-displayed position of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture. The actual display position of each sub-frame picture; the display module is also configured to display each subsequent sub-frame picture in sequence according to the actual display position of each subsequent sub-frame picture.

In some embodiments, refer to FIG. 4 , which is a schematic top view of the arrangement of pixels and light sources in a display module according to an embodiment of the present disclosure; the display module includes a display panel 1 , and the display panel 1 includes a plurality of pixels 10 , and a plurality of pixels 10 Arranged in an array. In this embodiment, refer to Figures 5 and 6. Figure 5 is a schematic diagram of disassembling a frame of color picture into three sub-frames in an embodiment of the present disclosure; Figure 6 is a schematic diagram of disassembling a frame of color picture into three sub-frames in an embodiment of the present disclosure. Another schematic diagram of disassembling a color image into three sub-frame images; disassembling a frame of color image into three sub-frame images, and the three sub-frame images are sequentially displayed by an array of 10 pixels.

In some embodiments, the display module further includes a lens located on the display side of the display panel 1 . In some embodiments, referring to Figure 4, the display module also includes a backlight module 5, located on the side of the display panel 1 away from the lens, for providing backlight for the display of the display panel 1; the backlight module 5 includes light sources of n colors. ; The display panel 1 includes an upper substrate and a lower substrate, and the upper substrate and the lower substrate pair The cells form a cell-to-cell gap, and the cell-to-cell gap is filled with liquid crystal.

In some embodiments, the backlight module 5 includes three colors of light sources, namely a red light source 51 , a green light source 52 and a blue light source 53 . The red light source 51, the green light source 52 and the blue light source 53 respectively provide backlight for the three sub-frame images displayed in sequence.

In some embodiments, the backlight module 5 provides direct backlight for the display of the display panel 1 , that is, the light exit surface of the light source faces the display surface of the display panel 1 .

In some embodiments, referring to FIG. 4 , the display area of the display panel 1 includes multiple sub-areas 100 , and multiple pixels 10 are distributed in each sub-area 100 ; the multiple pixels 10 are arranged in an array; the backlight module 5 includes multiple groups Light source 50 , each group of light sources 50 includes one light source of n colors; multiple groups of light sources 50 are arranged in one-to-one correspondence with multiple sub-regions 100 .

In this embodiment, refer to FIG. 4 and FIG. 7 . FIG. 7 is a schematic diagram of each sub-frame picture being refreshed sequentially and the light sources of different colors in the backlight module being sequentially lit during the display process of the display module according to the embodiment of the present disclosure; each sub-area The backlight of 100 is provided by a set of light sources 50 composed of three color light sources (such as LED lights): red, green, and blue. Light sources of different colors in the backlight module can be lit in sequence, such as red, green, and blue light sources, to provide backlight for the three sub-frames displayed in sequence. The brightness of different color light sources in the backlight module can be adjusted. Different from the liquid crystal panel in the public technology, which is composed of three color sub-pixels of red, green and blue, and one pixel controls the color display, each pixel 10 of the field sequential display module in the embodiment of the present disclosure corresponds to a In the overall pixel area, the pixel 10 can only display a grayscale image through the deflection of the liquid crystal in the display of each sub-frame; the pixel 10 array in the display module cooperates with the sequential refreshing of the three sub-frames and the backlight module. The three color light sources are lit in sequence to realize the display of color images. The design of the pixel 10 in this embodiment increases the display aperture ratio and reduces display power consumption.

In some embodiments, referring to Figure 7, the display module generates an original color picture of 90Hz. In this embodiment, the field sequential display of the display module disassembles each frame of the original color picture into The three sub-frame pictures actually extract the pixel values (i.e. gray scale values) of the corresponding colors from the original color picture, that is, they extract the backlight from one frame of the original color picture and are provided by the red light source 51 The pixel value of the first sub-frame picture (for example, the pixel value is 167), the pixel value of the second sub-frame picture (for example, the pixel value is 145) provided by the green light source 52 and the third sub-frame picture with the backlight provided by the blue light source 53 The pixel value of the sub-frame picture (for example, the pixel value is 189); refer to Figure 6 for the three corresponding sub-frame pictures after splitting the original one-frame color picture, that is, the R, G, and B three-channel pictures. The three-channel images of R, G, and B are displayed in sequence, and the three color light sources in the backlight module are also lit in sequence to provide different color backlights for the three sub-frame images.

In some embodiments, the refresh frequencies of the three sub-frame images are 270 Hz respectively, and the lighting frequencies of the three color light sources in the backlight module are 270 Hz respectively.

In some embodiments, refer to FIG. 8 , which is a schematic disassembly of the structure of a display module in an embodiment of the present disclosure; the display module also includes a lens 2 , which is located on the display side of the display panel 1 ; the processing module includes an infrared emitter 3 and infrared camera 4. The infrared emitter 3 is located on the side of the lens 2 away from the display panel 1 and is distributed around the edges of the lens 2 for emitting infrared light to the eyes; the infrared camera 4 is located on the side of the lens 2 away from the display panel 1. One side of the lens 2 is located at the edge of the lens 2 for capturing eye images.

Among them, the function of lens 2 is to introduce distortion to achieve virtual reality display. There are multiple infrared emitters 3 , and the plurality of infrared emitters 3 surround the surrounding edges of the lens 2 . The IR Camera 4 setup has one. The infrared emitter 3 emits infrared light to the eye, enabling the infrared camera 4 to capture a clear infrared image of the eye.

In some embodiments, referring to Figure 3, the processing module further includes an image processing unit configured to convert the eye image into a grayscale image; detect the left and right corner position points of the eye according to the eye corner feature points in the grayscale image; The line connecting the left and right eye corner position points is used as the X axis, the axis perpendicular to the The grayscale image of the eye is processed to determine the pupil area of the eye; the center of the pupil area is determined as the current pupil position corresponding to the first sub-frame.

Among them, refer to Figure 9a, which is a schematic diagram of a grayscale image of an eye pattern; the eye image captured by the infrared camera is a color picture; because the eye image is not required in the process of determining the pupil area of the eye Therefore, the eye image is converted into a grayscale image, that is, only the brightness value (grayscale value) of the eye image is included; for example: the calculation formula of the grayscale image can be: gray=0.30R+0.60G +0.1B; where gray is the gray value of each pixel in the grayscale image; assuming that the pixels include R (red) sub-pixels, G (green) sub-pixels and B (blue) sub-pixels, then in the grayscale image The gray value calculation of each pixel takes into account 30% of the R sub-pixel gray value, 60% of the G sub-pixel gray value and 10% of the B sub-pixel gray value to obtain the gray value of the entire eye image. image. Referring to Figure 9a, the left and right corner position points eye_edeg_l and eye_edge_r of the eye are detected in the grayscale image based on the eye corner feature points, and the grayscale image is cropped using these two position points as the left and right sides; the left and right corner positions of the eye are The line connecting eye_edeg_l and eye_edeg_r is used as the X-axis, and the axis perpendicular to the X-axis is used as the Y-axis. The origin of the intersection of the X-axis and the Y-axis is the midpoint of the line connecting the left and right eye corner points eye_edeg_l and eye_edeg_r.

In some embodiments, refer to Figure 9b, which is a schematic diagram of the pupil area of the eye obtained by processing by the image processing unit in an embodiment of the present disclosure; wherein the image processing unit is also configured to process the grayscale image of the eye to determine The pupil area of the eye; the center of the pupil area is determined as the current pupil position corresponding to the first sub-frame. The specific processing process of the image processing unit is: first, binarize the grayscale image to obtain the binary image of the eye. value image; then, the connected region labeling method is used to detect candidate pupil connected areas on the binarized image of the eye; then, based on geometric constraints and distance constraint algorithms, the pupil area of the eye is screened out from the candidate pupil connected areas; Finally, cover the pupil area with a circle with the smallest diameter, and determine the center of the circle as the center O_t of the pupil area.

In some embodiments, binarizing a grayscale image includes: setting a grayscale threshold t, and judging the grayscale value of each sub-pixel in the grayscale image. If it is greater than the grayscale threshold t, the grayscale value of the subpixel will be The gray value is set to 0. If it is less than or equal to the gray threshold t, the gray value of the sub-pixel is set to 1. After binary processing, a binary image of the eye (such as black and white) is formed as shown in Figure 9b. image).

In some embodiments, the processing module may be an image processing chip in the display module. An image processing unit is integrated into the processing chip.

In some embodiments, n=3, m=2; subsequent subframe pictures of the first subframe picture include a second subframe picture and a third subframe picture; the display module is configured to combine the first subframe picture, the third subframe picture The second subframe picture and the third subframe picture are displayed in sequence; the first prediction module is configured to calculate the current pupil position corresponding to the second subframe picture according to formula (1), and calculate the corresponding position of the third subframe picture according to formula (2) the current pupil position.
pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta ² ; (1)
pos_curr_b=pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta) ² ;
(2)

in,

a_curr is the current acceleration of the eyeball rotation; v_curr is the current speed of the eyeball rotation; pos_1 and pos_2 are the pupil positions corresponding to the first subframe obtained in the first two measurements respectively; pos_curr is the current pupil position corresponding to the first subframe; pos_curr_g is the current pupil position corresponding to the second subframe; pos_curr_b is the current pupil position corresponding to the third subframe; delta is the refresh duration of one subframe, delta=1/the refresh frequency of one subframe.

In some embodiments, the first sub-frame picture is a sub-frame picture with a backlight provided by a red light source; the second sub-frame picture is a sub-frame picture with a backlight provided by a green light source; and the third sub-frame picture is a sub-frame picture with a backlight provided by a blue light source. subframe picture.

In some embodiments, refer to FIG. 10 , which is a schematic diagram of the detection frequency of the pupil position corresponding to the first sub-frame picture in the embodiment of the present disclosure; where n=3, that is, the detection frequency of the eye pupil position is performed every two sub-frame pictures. Detection and prediction of actual display positions of the second sub-frame picture and the third sub-frame picture.

In some embodiments, the refresh frequency of a sub-frame picture is 270 Hz, and the refresh duration of a sub-frame picture is delta=1/270=0.0037s. The backlight module provides backlight for each sub-frame. The lighting frequency of the source is 270Hz.

In some embodiments, the pupil position corresponding to the first sub-frame obtained by the first m measurements is included in the calculation of the current pupil position corresponding to the subsequent sub-frames of the first sub-frame in the n sub-frames, so that it can Obtain the rotation trajectory information of the human eye, so as to facilitate the capture and tracking of the rotation of the human eye.

In some embodiments, the first prediction module is a hardware structure with computing functions in the display module, such as an algorithm.

In some embodiments, refer to FIG. 11 , which is a schematic diagram of the process of predicting the actual display positions of the second subframe picture and the third subframe picture in an embodiment of the present disclosure; wherein, the second prediction module is configured to predict the actual display position of the second subframe picture according to the second subframe picture. The offset of the current pupil position corresponding to the picture relative to the current pupil position corresponding to the first sub-frame picture calculates the first offset matrix of the original to-be-displayed position of the second sub-frame picture relative to the current position of the first sub-frame picture; The second prediction module is further configured to calculate the original to-be-displayed position of the third sub-frame picture relative to the first sub-frame picture based on the offset of the current pupil position corresponding to the third sub-frame picture relative to the current pupil position corresponding to the first sub-frame picture. the second offset matrix of the current position of the sub-frame picture; the second prediction module is also configured to multiply the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display of the second sub-frame picture position; multiply the original to-be-displayed position of the third sub-frame picture by the second offset matrix to obtain the actual display position of the third sub-frame picture.

Wherein, the current position of the first sub-frame picture refers to the current display position of the first sub-frame picture. The current pupil position corresponding to the first sub-frame is obtained by detecting the currently established coordinate system. The current pupil position corresponding to the second subframe and the current pupil position corresponding to the third subframe are calculated based on the current pupil position corresponding to the first subframe, the speed and acceleration of eyeball rotation, and the refresh frequency of one subframe. of. The position of the original to-be-displayed position of the second sub-frame picture in the human eye corresponds to the current pupil position corresponding to the second sub-frame picture, and the position of the original to-be-displayed position of the third sub-frame picture in the human eye corresponds to the position of the third sub-frame picture corresponding to the human eye. The current pupil position; the actual display position of the second sub-frame picture in the human eye corresponds to the current pupil position corresponding to the first sub-frame picture; the actual display position of the third sub-frame picture in the human eye corresponds to the first sub-frame picture Corresponding current pupil position; that is, after the above calculation of the second prediction module, the positions of the first subframe, the second subframe and the third subframe in the human eye can be located at the current pupil position corresponding to the first subframe, This eliminates color separation caused by eye movement.

In some embodiments, the first offset matrix is:

The second offset matrix is:

Among them, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original to-be-displayed position of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx1, Ty1, and Tz1 respectively represent the rotation amplitude of the second sub-frame picture. The translation amplitude of the original to-be-displayed position of the second sub-frame picture relative to the current position of the first sub-frame picture along the X-axis, Y-axis, and Z-axis; Rx2, Ry2, and Rz2 respectively represent the original to-be-displayed position of the third sub-frame picture. The rotation amplitude of the position along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx2, Ty2, and Tz2 respectively represent the original to-be-displayed position of the third sub-frame picture along the X-axis, Y-axis, The translation amplitude of the Z-axis relative to the current position of the first sub-frame; the Z-axis is perpendicular to the two-dimensional coordinate plane formed by the intersection of the X-axis and the Y-axis, and the Z-axis intersects the X-axis and the Y-axis at the origin.

In some embodiments, the second prediction module is a hardware structure with computing functions in the display module, such as an algorithm.

In some embodiments, referring to FIG. 5 , the display module is configured to sequentially display the second subframe picture and the third subframe picture according to their actual display positions. Specifically: the second sub-frame picture and the third sub-frame picture are sequentially displayed to their corresponding actual display positions on the display module, so that the first sub-frame picture, the second sub-frame picture and the third sub-frame picture can be The picture displayed by the pixels 10 with the same position is projected to the same position in the pupil area of the human eye (i.e., on the retina). Three pictures with backlight provided by different color light sources are fused here, thereby eliminating the color separation phenomenon and ensuring that the human eye can feel The picture is a complete picture.

In some embodiments, referring to Figure 3, the display module further includes a storage module configured to store The current pupil position corresponding to the first sub-frame and the current pupil position corresponding to each subsequent sub-frame. This facilitates the current pupil position data corresponding to each sub-frame to participate in the prediction of the pupil position corresponding to each subsequent sub-frame.

In some embodiments, the storage module is a hardware structure with storage function in the display module, such as a memory.

The display module provided in the embodiment of the present disclosure determines the current pupil position corresponding to the first sub-frame picture displayed first among the n sub-frame pictures through processing by the processing module; the first prediction module calculates the subsequent sub-frame pictures of the first sub-frame picture. The current pupil position corresponding to the frame; the second prediction module calculates the actual display position of each subsequent sub-frame of the first sub-frame based on the calculation results of the first prediction module; and can obtain the rotation trajectory information of the human eye in order to predict the human eye. Capture and track the rotation of the eyeballs; by adjusting the subsequent sub-frames of the first sub-frame from the original to-be-displayed position to the actual display position, the actual display of subsequent sub-frames of the first sub-frame can be achieved position to catch up with the position of the eyeball rotation, so that the picture displayed by the pixels with the same position on the subsequent sub-frames of the first sub-frame is projected to the same position in the pupil area of the human eye (i.e. on the retina). Three images of different colors are The backlit images provided by the light source are fused here to eliminate color separation and ensure that the image perceived by the human eye is a complete image.

Based on the structure of the display module in the above embodiment, the embodiment of the present disclosure also provides a display method of the display module. Refer to Figure 12, which is a flow chart of the display method of the display module in the embodiment of the present disclosure; wherein, the display The method includes: step S101: disassemble a frame of color picture into n sub-frame pictures; n sub-frame pictures are displayed in sequence, n≥3, and n is an integer.

Step S102: Obtain an eye image, process the eye image, and determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is the sub-frame that is displayed first when the n sub-frame pictures are displayed sequentially. picture.

This step specifically includes: taking an eye image when the first sub-frame is displayed; converting the eye image into a grayscale image; detecting the left and right corner position points of the eye based on the eye corner feature points in the grayscale image; The line connecting the eye corner position points is used as the X-axis, and the axis perpendicular to the X-axis is used as the Y-axis. The origin of the intersection of the axis and the Y-axis is the midpoint of the line connecting the left and right eye corners; the grayscale image is processed in the coordinate plane formed by the X-axis and the Y-axis to determine the pupil area of the eye; the center of the pupil area is determined as The current pupil position corresponding to the first subframe.

In some embodiments, the grayscale image is processed within the coordinate plane formed by the X-axis and the Y-axis to determine the pupil area of the eye; the center of the pupil area is determined as the current pupil position corresponding to the first sub-frame, specifically It includes: binarizing the grayscale image to obtain a binary image of the eye; using the connected region labeling method to detect candidate pupil connected areas on the binary image of the eye; based on geometric constraints and distance constraint algorithms, The pupil area of the eye is screened out from the candidate pupil connected areas; the pupil area is covered with a circle with the smallest diameter, and the center of the circle is determined as the center of the pupil area.

Step S103: Based on the current pupil position corresponding to the first sub-frame and the pupil position corresponding to the first sub-frame obtained by the previous m measurements, calculate the corresponding sub-frames corresponding to the subsequent sub-frames of the first sub-frame in the n sub-frames. Current pupil position; m≥2, and m is an integer.

In some embodiments, n=3, m=2; subsequent subframe pictures of the first subframe picture include the second subframe picture and the third subframe picture; the first subframe picture, the second subframe picture and the third subframe picture. The three sub-frames are displayed in sequence; step S103 includes: calculating the current pupil position corresponding to the second sub-frame according to formula (1), and calculating the current pupil position corresponding to the third sub-frame according to formula (2).

pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta ² ; (1)

pos_curr_b＝pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta) ² ; (2)

in,

a_curr is the current acceleration of the eyeball rotation; v_curr is the current speed of the eyeball rotation; pos_1 and pos_2 are the pupil positions corresponding to the first subframe obtained in the first two measurements respectively; pos_curr is the current pupil position corresponding to the first subframe; pos_curr_g is the current pupil corresponding to the second sub-frame picture hole position; pos_curr_b is the current pupil position corresponding to the third sub-frame picture; delta is the refresh duration of one sub-frame picture, delta=1/the refresh frequency of one sub-frame picture.

In some embodiments, the display method further includes: storing the current pupil position corresponding to the first sub-frame and the current pupil position corresponding to each subsequent sub-frame.

Step S104: Calculate the original to-be-displayed position of each subsequent sub-frame picture relative to the first sub-frame picture based on the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture. The actual offset of the current position of the sub-frame picture; and calculate the actual display of each subsequent sub-frame picture based on the original to-be-displayed position of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture Location.

This step specifically includes: calculating the original to-be-displayed position of the second sub-frame relative to the first sub-frame based on the offset of the current pupil position corresponding to the second sub-frame relative to the current pupil position corresponding to the first sub-frame. The first offset matrix of the current position. Calculate the original to-be-displayed position of the third sub-frame relative to the current position of the first sub-frame based on the offset of the current pupil position corresponding to the third sub-frame relative to the current pupil position corresponding to the first sub-frame. Two offset matrices. Multiply the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture. Multiply the original to-be-displayed position of the third subframe picture by the second offset matrix to obtain the actual display position of the third subframe picture.

In some embodiments, the first offset matrix is:

The second offset matrix is:

Among them, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original to-be-displayed position of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx1, Ty1, and Tz1 respectively represent the rotation amplitude of the second sub-frame picture. The translation amplitude of the original to-be-displayed position of the second sub-frame picture relative to the current position of the first sub-frame picture along the X-axis, Y-axis, and Z-axis; Rx2, Ry2, and Rz2 respectively represent the original to-be-displayed position of the third sub-frame picture. Indicates the rotation amplitude of the position along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx2, Ty2, and Tz2 respectively represent the original to-be-displayed position of the third sub-frame picture along the X-axis, Y-axis, and Z-axis. The translation amplitude of the axis relative to the current position of the first subframe; the Z axis is perpendicular to the two-dimensional coordinate plane formed by the intersection of the X axis and the Y axis, and the Z axis intersects the X axis and the Y axis at the origin.

Step S105: Display subsequent sub-frame pictures in sequence according to the actual display position of each subsequent sub-frame picture.

In a third aspect, an embodiment of the present disclosure further provides a display device, which includes the display module in the above embodiment.

By using the display module in the above embodiment, the display device will not have color separation when performing virtual reality display, thereby improving the virtual reality display effect of the display device.

The display device can be: VR glasses, VR panels, VR TVs, mobile phones, tablets, laptops, monitors, digital photo frames, navigators and other products or components with VR display functions.

In a fourth aspect, an embodiment of the present disclosure also provides a virtual display device, including the display device in the above embodiment.

By using the display device in the above embodiment, the virtual display device will not have color separation when performing virtual reality display, thereby improving the virtual reality display effect of the virtual display device.

The virtual display device can be: VR glasses, VR panels, VR TVs, mobile phones, tablets, laptops, monitors, digital photo frames, navigators and other products or components with VR display functions.

It can be understood that the above embodiments are only exemplary embodiments adopted to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the disclosure, and these modifications and improvements are also regarded as the protection scope of the disclosure.

Claims

A display method, including:

Disassemble one frame of color picture into n sub-frame pictures; the n sub-frame pictures are displayed in sequence, n≥3, and n is an integer;

Obtain an eye image, process the eye image, and determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is the sub-frame picture that is displayed first when the n sub-frame pictures are displayed in sequence. ;

According to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements, calculate each subsequent pupil position of the first sub-frame picture in the n sub-frame pictures. The current pupil position corresponding to the sub-frame picture; m≥2, and m is an integer;

Calculate the original to-be-displayed position of each subsequent subframe relative to the current pupil position corresponding to the first subframe based on the offset of the current pupil position corresponding to each subsequent subframe in the n subframes. The actual offset of the current position of the first sub-frame picture; and calculating the subsequent sub-frame pictures based on the original to-be-displayed position of each subsequent sub-frame picture and its actual offset relative to the current position of the first sub-frame picture. The actual display position of the frame;

Each subsequent sub-frame picture is displayed in sequence according to the actual display position of each subsequent sub-frame picture.
The display method according to claim 1, wherein said obtaining the eye image, processing the eye image, and determining the current pupil position corresponding to the first sub-frame picture includes:

When the first sub-frame is displayed, capture an eye image;

Convert eye images to grayscale images;

In the grayscale image, the left and right corner position points of the eye are detected based on the eye corner feature points; the left and right corner position points are connected as the X-axis, the axis perpendicular to the X-axis is used as the Y-axis, and the origin of the intersection of the X-axis and the Y-axis is The midpoint of the line connecting the left and right eye corners;

Process the grayscale image in the coordinate plane composed of the X-axis and the Y-axis to determine the pupil of the eye. pupil area; determine the center of the pupil area as the current pupil position corresponding to the first sub-frame picture.
The display method according to claim 2, wherein the grayscale image is processed in the coordinate plane formed by the X-axis and the Y-axis to determine the pupil area of the eye; the center of the pupil area is determined as the The current pupil position corresponding to the first subframe includes:

Binarize the grayscale image to obtain the binarized image of the eye;

Using a connected region labeling method to detect candidate pupil connected regions on the binary image of the eye;

Based on geometric constraints and distance constraint algorithms, filter out the pupil area of the eye from the candidate pupil connected areas;

Cover the pupil area with a circle with the smallest diameter, and determine the center of the circle as the center of the pupil area.
The display method according to claim 2, wherein n=3, m=2;

Subsequent subframe pictures of the first subframe picture include a second subframe picture and a third subframe picture; the first subframe picture, the second subframe picture and the third subframe picture are performed in sequence. show;

According to the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements, calculate the first sub-frame picture in the n sub-frame pictures. The current pupil position corresponding to each subsequent sub-frame includes:

The current pupil position corresponding to the second sub-frame is calculated according to formula (1), and the current pupil position corresponding to the third sub-frame is calculated according to formula (2);
pos_curr_g＝pos_curr+v_curr×delta+1/2×a_curr×delta 2 ; (1)
pos_curr_b=pos_curr+v_curr×delta×2+1/2×a_curr×(2×delta) 2 ;
(2)

in,

a_curr is the current acceleration of eyeball rotation; v_curr is the current speed of eyeball rotation; pos_1 and pos_2 are respectively the pupil positions corresponding to the first sub-frame obtained by the previous two measurements; pos_curr is the corresponding pupil position of the first sub-frame. The current pupil position; pos_curr_g is the current pupil position corresponding to the second sub-frame picture; pos_curr_b is the current pupil position corresponding to the third sub-frame picture; delta is the refresh duration of one of the sub-frame pictures, delta=1/ The refresh frequency of one of the sub-frame pictures.
The display method according to claim 4, wherein the offset of the current pupil position corresponding to each subsequent sub-frame picture in the n sub-frame pictures relative to the current pupil position corresponding to the first sub-frame picture is Calculate the actual offset of the original to-be-displayed position of each subsequent sub-frame picture relative to the current position of the first sub-frame picture; and based on the original to-be-displayed position of each subsequent sub-frame picture and its relative to the first sub-frame picture The actual offset of the current position of the frame is used to calculate the actual display position of each subsequent sub-frame, including:

The original to-be-displayed position of the second sub-frame relative to the first sub-frame is calculated based on the offset of the current pupil position corresponding to the second sub-frame relative to the current pupil position corresponding to the first sub-frame. The first offset matrix of the current position of the sub-frame picture;

The original to-be-displayed position of the third sub-frame relative to the first sub-frame is calculated based on the offset of the current pupil position corresponding to the third sub-frame relative to the current pupil position corresponding to the first sub-frame. the second offset matrix of the current position of the sub-frame picture;

Multiply the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture;

Multiply the original to-be-displayed position of the third sub-frame picture by the second offset matrix to obtain the actual display position of the third sub-frame picture.
The display method according to claim 5, wherein,

The first offset matrix is:

The second offset matrix is:

Among them, Rx1, Ry1, and Rz1 respectively represent the rotation amplitude of the original to-be-displayed position of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx1, Ty1, Tz1 respectively represents the translation amplitude of the original to-be-displayed position of the second sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture;

Rx2, Ry2, and Rz2 respectively represent the rotation amplitude of the original to-be-displayed position of the third sub-frame picture along the X-axis, Y-axis, and Z-axis relative to the current position of the first sub-frame picture; Tx2, Ty2, and Tz2 respectively Represents the translation amplitude of the original to-be-displayed position of the third sub-frame picture relative to the current position of the first sub-frame picture along the X-axis, Y-axis, and Z-axis;

The Z-axis is perpendicular to the two-dimensional coordinate plane formed by the intersection of the X-axis and the Y-axis, and the Z-axis intersects the X-axis and Y-axis at the origin.
The display method according to any one of claims 1 to 6, further comprising: storing the current pupil position corresponding to the first sub-frame picture and the current pupil position corresponding to each subsequent sub-frame picture.
A display module, including:

The disassembly module is configured to disassemble a frame of color picture into n sub-frame pictures; n≥3, and n is an integer;

A display module configured to display the n sub-frame pictures in sequence;

A processing module configured to acquire an eye image, process the eye image, and determine the current pupil position corresponding to the first sub-frame picture; the first sub-frame picture is the first to be displayed among the n sub-frame pictures. sub-frame picture;

The first prediction module is configured to calculate the pupil position corresponding to the first sub-frame picture based on the current pupil position corresponding to the first sub-frame picture and the pupil position corresponding to the first sub-frame picture obtained by the previous m measurements. The current pupil position corresponding to each subsequent sub-frame of the first sub-frame; m≥2, and m is an integer;

The second prediction module is configured to calculate each subsequent subframe picture based on the offset of the current pupil position corresponding to each subsequent subframe picture in the n subframe pictures relative to the current pupil position corresponding to the first subframe picture. The actual offset of the original to-be-displayed position relative to the current position of the first sub-frame picture; and based on the original to-be-displayed position of each subsequent sub-frame picture and its relative to the current position of the first sub-frame picture The actual offset is used to calculate the actual display position of each subsequent sub-frame;

The display module is further configured to display subsequent sub-frame pictures in sequence according to the actual display position of each subsequent sub-frame picture.
The display module according to claim 8, wherein the display module includes a display panel and a lens, and the lens is located on the display side of the display panel;

The processing module includes an infrared transmitter and an infrared camera,

The infrared emitter is located on a side of the lens away from the display panel and distributed around the edges of the lens for emitting infrared light to the eyes;

The infrared camera is located on a side of the lens away from the display panel and on the edge of the lens for capturing eye images.
The display module according to claim 9, wherein the processing module further comprises an image processing unit configured to convert the eye image into a grayscale image; in the grayscale image according to the eye The corner feature points detect the left and right corner position points of the eye; connect the left and right corner position points as the X axis, and the axis perpendicular to the X axis as the Y axis. The origin of the intersection of the X axis and the Y axis is the line connecting the left and right corner position points the midpoint;

The image processing unit is further configured to process the grayscale image of the eye, determine the pupil area of the eye, and determine the center of the pupil area as the current pupil position corresponding to the first sub-frame.
The display module according to claim 10, wherein the second prediction module is configured to calculate the current pupil position corresponding to the second sub-frame picture relative to the current pupil position corresponding to the first sub-frame picture. The offset calculates a first offset matrix of the original to-be-displayed position of the second sub-frame picture relative to the current position of the first sub-frame picture;

The second prediction module is further configured to calculate the offset of the current pupil position corresponding to the third sub-frame picture from the current pupil position corresponding to the first sub-frame picture. The second offset matrix of the original to-be-displayed position relative to the current position of the first sub-frame picture;

The second prediction module is further configured to multiply the original to-be-displayed position of the second sub-frame picture by the first offset matrix to obtain the actual display position of the second sub-frame picture; The original to-be-displayed position of the third sub-frame picture is multiplied by the second offset matrix to obtain the actual display position of the third sub-frame picture.
The display module according to any one of claims 8-11, further comprising a storage module configured to store the current pupil position corresponding to the first sub-frame and the current pupil position corresponding to each subsequent sub-frame. .
A display device, comprising the display module according to any one of claims 8-12.
A virtual display device, comprising the display device according to claim 13.