CN117631274A

CN117631274A - Display module, imaging control method and related device

Info

Publication number: CN117631274A
Application number: CN202210967100.6A
Authority: CN
Inventors: 赵飞; 陈智单; 何小宇; 赵嘉卉
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2022-08-11
Filing date: 2022-08-11
Publication date: 2024-03-01

Abstract

The application discloses a display module, an imaging control method and a related device, which are used for providing a super-resolution scheme of display equipment. The control component decomposes the high-resolution image to obtain a plurality of frames of low-resolution images, then displays the low-resolution images through the display component in a time-sharing manner, and makes each frame of images displayed by the display component displace through time-sharing control of rotation and vibration of the vibrating mirror, namely, displays the plurality of frames of low-resolution images in a time-sharing manner, and utilizes the persistence of vision and vision synthesis function of human eyes, the plurality of frames of low-resolution images are overlapped in the human eyes, so that the human eyes see the high-resolution images.

Description

Display module, imaging control method and related device

Technical Field

The embodiment of the application relates to the technical field of optics, in particular to a display module, an imaging control method and a related device.

Background

A display device, such as a Virtual Reality (VR) device, achieves depth immersion by satisfying a large angle of view (FOV) and high resolution. The content watched in VR devices currently employed only reaches 10-20 angular resolution (points per degree, PPD), and cannot meet the resolution limit of human eye 1' (60 PPD), so that the screen window effect exists in the image seen by the user. In order to solve the screen window effect of VR devices, on the one hand, a high-resolution display screen may be used, but in order to meet the size requirement of VR devices, a display screen adopting a silicon-based electro-mechanical laser display (micro-OLED) technology is adopted, but the cost of the high-resolution micro-OLED is higher. On the other hand, the resolution of the display screen can be indirectly improved by a resolution enhancement technology, but no possible resolution enhancement scheme applied to the display device is currently available.

Disclosure of Invention

The embodiment of the application provides a display module, an imaging control method and a related device, which are used for providing an enhancement scheme applied to resolution of VR equipment.

In a first aspect, an embodiment of the present application provides a display module, including a display module, a resolution enhancing module and a display light path module disposed on a light path of the display module, and a control module respectively connected with the display module and the resolution enhancing module. The resolution enhancement assembly specifically may include a galvanometer and a driving assembly, where an image exiting the display assembly may be incident to the display light path assembly after passing through the galvanometer. And the display component is used for displaying the image. And the display light path component is used for adjusting the display position of each frame of image displayed by the display component. For example, the display device is a wearable device, and the display light path component can adjust the position of each frame of image displayed by the display component on a virtual image surface at a certain distance from the display component. The control component is used for receiving the image to be displayed, decomposing the image to be displayed to obtain a multi-frame image, and the display component can display the multi-frame image in a time sharing mode under the control of the control component. And the vibrating mirror is used for generating rotary vibration under the control of the driving assembly so as to enable any image in the multi-frame images displayed by the display assembly to be displaced.

In some embodiments, the multi-frame image may be obtained by downsampling an image to be displayed by the control component, where a resolution of each of the multi-frame images is less than a resolution of the image to be displayed. For example, the control component splits the high-resolution image to be displayed into a plurality of low-resolution images, the resolution of the low-resolution images is generally the same as the physical resolution of the display component, and the control component sends the plurality of low-resolution images to the display component for display in a time-sharing manner.

In some embodiments, the time at which the display assembly displays any of the plurality of images may be synchronized with the time at which the galvanometer displaces any of the plurality of images displayed by the display assembly.

In the display module provided by the embodiment of the application, the control component can obtain multiple frames of low-resolution images by decomposing the high-resolution images, then can display the low-resolution images through the display component in a time-sharing manner, and can adjust the position of each frame of images displayed by the display component through the time-sharing control resolution lifting component, namely, the multiple frames of low-resolution images are displayed in a time-sharing manner, and the multiple frames of low-resolution images are overlapped in human eyes by utilizing the vision persistence and the vision synthesis function of the human eyes, so that the high-resolution images seen by the human eyes are obtained.

In some embodiments, the drive assembly may be a piezoelectric drive assembly or a motor drive assembly, among others. The motor drive assembly may be in particular an ultrasonic motor or a servomotor or the like. The control component can control the driving component, and then the driving component can perform rotary vibration through the time-sharing control vibrating mirror, so that each frame of image displayed by the display component is adjusted in a time-sharing mode to perform displacement, and multiple frames of low-resolution images displayed by the display component are overlapped in human eyes by utilizing the persistence of vision and the vision synthesis function of human eyes, so that the high-resolution images seen by the human eyes are realized. In order to achieve the time that the display component displays any one of the multi-frame images can be synchronous with the time that the galvanometer adjusts any one of the multi-frame images, the frequency of the rotation vibration of the galvanometer can be the same as the switching frequency of the display component displaying each frame of images.

In some embodiments, the display component may be a liquid crystal display (liquid crystal display, LCD), a micro organic light emitting diode display (micro organic light emitting diode, micro OLED), a silicon-based OLED, a micro light emitting diode (micro light emitting diode, micro LED), or other display device, as not specifically limited herein.

In some embodiments, the control component may be a processor, microprocessor, controller, or the like control component, such as a general purpose central processing unit (central processing unit, CPU), general purpose processor, digital signal processing (digital signal processing, DSP), application specific integrated circuit (application specific integrated circuits, ASIC), field programmable gate array (field programmable gate array, PGA), or other programmable logic device, transistor logic device, hardware component, or any combination thereof.

In some embodiments, the galvanometer may be positioned proximate the display assembly, i.e., the galvanometer may be positioned in the optical path between the display optical path assembly and the display assembly. Therefore, after each frame of image emitted from the display component in a time-sharing way passes through the vibrating mirror which rotates and vibrates, the displacement of each frame of image in space can be realized, then each frame of image is adjusted to a set position through the display light path component for display, and multiple frames of low-resolution images are overlapped in human eyes by utilizing the persistence of vision and the vision synthesis function of human eyes, so that the high-resolution images seen by the human eyes are improved in order to realize resolution improvement.

Alternatively, in some embodiments, when the display light path assembly includes at least two lenses, the galvanometer may also be disposed on the light path between the at least two lenses, i.e., the galvanometer may also be disposed inside the display light path assembly. In this way, each frame of image emitted from the display component in a time-sharing way is firstly refracted by part of lenses in the display light path component and then is incident to the vibrating mirror which rotates and vibrates, so that the displacement of each frame of image in space is realized, then the images are adjusted to the set position through the rest lenses in the display light path component for display, and the vision persistence and the vision synthesis function of human eyes are utilized, and multiple frames of low-resolution images are overlapped in the human eyes, so that the high-resolution images seen by the human eyes are improved, and the resolution is improved.

In some embodiments, the galvanometer may realize time-sharing adjustment of displacement of adjacent two frames of images displayed by the display assembly by outputting the adjacent two frames of images of the multiple frames of images at different positions at adjacent two times. The time when the display assembly displays two adjacent frames of images in the multi-frame images is referred to by two adjacent times, for example, the display assembly displays one frame of image in the multi-frame images at time 1, and the display assembly displays the next frame of image in the multi-frame images at time 2. After the vibrating mirror is positioned at the position 1 at the moment 1 and the rotating vibration of the vibrating mirror is controlled, the vibrating mirror is positioned at the position 2 at the moment 2, and the vibrating mirror rotates at a set angle relative to the position 1 at the position 2. Because the vibrating mirror has a certain thickness and refractive index different from that of air, the position of emergent light can be moved after the same beam of incident light passes through the vibrating mirror at different positions, namely the position of the next frame of image emergent from the vibrating mirror can be displaced. The superposition effect of the multiple frames of images can be seen according to the persistence effect of human eyes, and although the resolution of each frame of image in the multiple frames of images is smaller than that of the image to be displayed, most of information of the image to be displayed is reserved in each frame of image in the multiple frames of images, so that the image finally seen by the human eyes is similar to the original image to be displayed, and the effect that the display component with low resolution displays high resolution is realized.

In some embodiments, in order to ensure that the superposition effect of multiple frames of images can be seen according to the persistence effect of human eyes, the displacement Δy between two adjacent frames of images can be adjusted to be smaller than a pixel distance by controlling the rotation angle θ of the galvanometer, and the pixels are pixels in the images displayed by the display component. In some embodiments, the distance between two adjacent frames of images may be one half of a pixel distance, for example, the pixel size is 16.5um, the displacement may be 8.3um, and the displacement direction may be along any one or more of a horizontal direction, a vertical direction, or a diagonal direction.

In some embodiments, the rotation angle θ of the galvanometer may satisfy the following relationship:

wherein deltay is the displacement of the output image of the vibrating mirror before and after rotation, theta is the rotation angle of the vibrating mirror, t is the thickness of the vibrating mirror, and n is the refractive index of the vibrating mirror.

In some embodiments, the material of the galvanometer may be a light-transmitting material such as glass, polycarbonate (PC), polymethyl methacrylate (polymethyl methacrylate, PMMA), or Polystyrene (PS). The refractive index of the selected material of the vibrating mirror is generally 1.3-2, the error range is less than 0.01, and for example, a light-transmitting material with the refractive index of 1.5 can be selected. Under the same condition, the larger the thickness of the vibrating mirror is, the larger the displacement is, but the larger the thickness of the vibrating mirror is, the whole volume of the display module is increased, the thickness range of the vibrating mirror is generally 0.5mm-6mm, the error range is smaller than 0.1mm, and the thickness of the vibrating mirror can be selected to be 2mm.

In some embodiments, the control component may obtain the rotation angle of the galvanometer based on the thickness value of the galvanometer, the refractive index value of the galvanometer, and the displacement of the output image of the galvanometer before and after rotation. The driving component can allow an error within +/-10% of the actual rotation angle of the vibrating mirror compared with the rotation angle of the vibrating mirror when controlling the rotation angle of the vibrating mirror to rotate.

In the embodiment of the application, the improvement multiple of the resolution is related to the resolution of the image to be displayed and the physical resolution of the display component. For example, if the resolution of the image to be displayed is twice the physical resolution of the display assembly, for example, the image to be displayed is 4k, the physical resolution of the display assembly is 2k, the pixel density unit (PPI) of the display assembly is about 1200-1500, the angular resolution (points per degree, PPD) is about 20-30, by splitting the image to be displayed into multiple frame images and rotating the vibrating mirror, the superimposed display effect can be finally presented to the human eye as a near 4k image, the PPI of the display effect is about 2400-3000, and the PPD is about 40-60.

In some embodiments, taking the resolution of the image to be displayed as 4k 60Hz, and the physical resolution of the display assembly as 2k as an example, the control assembly may divide the image to be displayed into two frames of 2k120Hz images by an algorithm, i.e. the multi-frame image comprises a first image of 2k120Hz and a second image of 2k120 Hz. The vibrating mirror can output a first image at a first position at a first moment, and output a second image at a second position at a second moment after rotating and vibrating; wherein the first time and the second time are adjacent in time.

Specifically, any one or more of horizontal superdivision, vertical superdivision, or diagonal superdivision can be achieved by using rotational vibration of the galvanometer. The display assembly has a first surface nearest to the galvanometer, defines a vertical axis perpendicular to the horizontal plane, defines a horizontal axis perpendicular to the vertical axis, defines a diagonal axis in a plane formed by the vertical axis and the horizontal axis, and is parallel to a diagonal direction of the pixels in the first surface of the display assembly. Also, the vertical axis, the horizontal axis, and the diagonal axis may each pass through the center point of the galvanometer, or the vertical axis, the horizontal axis, and the diagonal axis may have a distance from the center point of the galvanometer, which may be within a range, for example. The horizontal super-resolution is understood to be that two frames of images formed by light beams output by the vibrating mirror have a spacing distance in the horizontal direction, and taking the spacing distance as Px/2 as an example, px represents the distance between adjacent pixels of any one frame of images in the two frames of images in the horizontal direction, when the horizontal super-resolution is adopted, the vibrating mirror needs to rotate by a first set angle or a second set angle relative to the first moment by taking a vertical axis as a rotation axis at the second moment, so that an offset vector of the second position relative to the first position is (Px/2, 0) or (-Px/2, 0), that is, the first position and the second position can be separated by a half pixel distance in the horizontal direction, thereby realizing doubling of resolution in the horizontal direction. The vertical superdivision can be understood that two frames of images output by the vibrating mirror have a spacing distance in the vertical direction, and taking the spacing distance as Py/2 as an example, py represents the distance between adjacent pixels of any one frame of images in the two frames of images in the vertical direction, when the vertical superdivision is adopted, the vibrating mirror needs to rotate by a third set angle or a fourth set angle relative to the first moment by taking a horizontal axis as a rotation axis at a second moment, so that the offset vector of the second position relative to the first position is (0, py/2) or (0, -Py/2), namely the first position and the second position can be separated by one half pixel distance in the vertical direction, thereby realizing doubling of resolution in the vertical direction. The diagonal superdivision can be understood that two frames of images output by the vibrating mirror have a spacing distance in the vertical direction, and have a spacing distance in the horizontal direction, and when the diagonal superdivision is adopted, the vibrating mirror needs to rotate at a second moment by a fifth set angle or a sixth set angle relative to the first moment by taking a diagonal axis as a rotation axis, so that an offset vector of the second position relative to the first position is (Px/2, py/2), (-Px/2, py/2), (Px/2, -Py/2) or (-Px/2), and taking the vertical spacing distance as Px/2 as an example, namely, the first position and the second position can be respectively spaced by one half pixel distance in the horizontal direction and the vertical direction, thereby realizing resolution doubling.

Taking 4*4 pixel array as an example, assuming that the galvanometer can realize offset with offset vector (Px/2, -Py/2), the effect of doubling the equivalent display pixel number can be realized by a time division multiplexing mode. When the offset vector of the galvanometer is (+/-Px/2, 0), the resolution in the horizontal direction can be doubled. When the offset vector of the galvanometer is (0, ±py/2), the resolution multiplication in the vertical direction can be realized.

Splitting the image to be displayed into two frames of images, and seeing the superposition effect of the two frames of images according to the persistence effect of human eyes, wherein when the image to be displayed is split, most of the original information of the image to be displayed can be ensured to be reserved through reasonable algorithm processing, so that the image finally seen by the human eyes is close to the original image to be displayed, and the improvement from the 2k physical resolution of the display module to the image with the display effect of 4k resolution is realized.

In some embodiments, taking the resolution of the image to be displayed as 4k 60Hz and the physical resolution of the display assembly as 2k as an example, the control assembly may algorithmically split the image to be displayed into four frames of 2k 240Hz images, i.e., a multi-frame image comprising a first image of 2k120Hz, a second image of 2k120Hz, a third image of 2k120Hz, and a fourth image of 2k120 Hz. The vibrating mirror can output a first image at a first position at a first moment, can output a second image at a second position at a second moment after generating first rotary vibration, can output a third image at a third position at a third moment after generating second rotary vibration, can output a fourth image at a fourth position at a fourth moment after generating third rotary vibration, and can output a first image in a multi-frame image obtained by decomposing a next frame of image to be displayed at the first position at a fifth moment after generating fourth rotary vibration; the first time, the second time, the third time, the fourth time and the fifth time are adjacent in time. According to the visual persistence effect of the human eyes, the superposition effect of four frames of images can be seen, when the images to be displayed are split, the four frames of images can be ensured to keep most of the information of the original images to be displayed through reasonable algorithm processing, so that the image finally seen by the human eyes is the image to be displayed close to the original image, and the improvement from the 2k physical resolution of the display module to the image with the display effect of 4k resolution is realized.

Specifically, the rotation vibration of the vibrating mirror along different directions can realize the mutual conversion of images between four positions at four moments, and the two frames of images output by the vibrating mirror can be separated by a distance Px/2 in the horizontal direction, wherein Px represents the distance between adjacent pixels of the two frames of images in the horizontal direction, and the resolution in the horizontal direction is doubled at the moment; the distance Py/2 between two frames of images output by the vibrating mirror in the vertical direction can be used for representing the distance between adjacent pixels of the two frames of images in the vertical direction, and the resolution in the vertical direction is doubled; the two frames of images output by the vibrating mirror are separated by a distance Py/2 in the vertical direction and separated by a distance Px/2 in the horizontal direction, and at the moment, the resolution is doubled.

For example, the galvanometer may output a first image at a first position at a first time, the first position may be spaced apart from the second position by a half of a pixel distance in a vertical direction, and when the galvanometer is rotated at a second time by a third set angle with respect to the first time about a horizontal axis as a rotation axis, the galvanometer may implement an offset vector (0, py/2) such that a second image may be output at the second position at the second time. When the galvanometer rotates at a first set angle with respect to the second time with the vertical axis as a rotation axis at the third time, the galvanometer can realize an offset vector (Px/2, 0) such that a third image can be output at a third position at the third time, and the second position and the third position can be spaced apart by a half pixel distance in the horizontal direction. When the galvanometer rotates by a fourth set angle with respect to the third moment by taking the horizontal axis as a rotation axis at the fourth moment, the galvanometer can realize an offset vector (0, -Py/2) so that a third image can be output at a fourth position at the fourth moment, and the third position and the fourth position can be separated by a half pixel distance in the vertical direction. When the vibrating mirror rotates a second set angle relative to a fourth moment by taking a vertical axis as a rotation axis, the vibrating mirror can realize an offset vector (-Px/2, 0), so that a first image in a multi-frame image obtained by decomposing a next frame of image to be displayed can be output at the fifth moment back to a first position, and the fourth position and the first position can be separated by a half pixel distance in the horizontal direction.

It should be noted that, in the embodiment of the present application, the order in which the vibrating mirror outputs the first image, the second image, the third image or the fourth image is not specifically limited, and by way of example, the vibrating mirror may also output the first image at the first position at the first time, the vibrating mirror may output the fourth image at the fourth position at the second time after rotating and vibrating, the vibrating mirror may output the third image at the third position at the third time after rotating and vibrating, and the vibrating mirror may output the second image at the second position at the fourth time after rotating and vibrating; the vibrating mirror can also output a first image at a first position at a first moment, can output a third image at a third position at a second moment after rotating and vibrating, can output a fourth image at a fourth position at a third moment after rotating and vibrating, and can output a second image at a second position at a fourth moment after rotating and vibrating, and the like. Wherein the first time, the second time, the third time and the fourth time are adjacent in time.

The above is merely illustrative of the rotational vibration operation of the vibrating mirror, and is not limited thereto. The image to be displayed may be split into multiple frames of images with other values, such as three frames of images, five frames of images, six frames of images, etc., which will not be described in detail herein. In order to ensure the display effect, each frame of image should be ensured to be displayed at different positions as much as possible.

In some embodiments, the display module may support operation in two modes. Supersplit mode, and normal mode. The control component executes superprocessing when the superdivision mode is enabled; the control component maintains the resolution unchanged and the frame rate is not reduced when it determines that the normal mode is enabled. In the normal mode, the vibrating mirror does not generate rotational vibration. In some embodiments, the control component may resample the image source to the physical resolution of the display component, output directly to the display component, and this normal mode may be applicable to some high frame rate scenarios, such as games, etc.

In a second aspect, an embodiment of the present application provides an imaging control method, where the method is applied to a display device, where the display device includes a display component, and a resolution enhancing component and a display light path component that are disposed on a light path of the display component, where the resolution enhancing component may include a galvanometer and a driving component, and an image exiting from the display component may be incident on the display light path component after passing through the galvanometer. The imaging control method may specifically include: receiving an image to be displayed, and decomposing the image to be displayed to obtain a multi-frame image; the display component is controlled to display multi-frame images in a time sharing way; the driving component controls the vibrating mirror to generate rotary vibration, so that any one of the multi-frame images displayed by the display component is displaced.

In some embodiments, the multi-frame image may be downsampled to the image to be displayed, such that the resolution of each of the multi-frame images is less than the resolution of the image to be displayed. For example, the high-resolution image to be displayed can be split into a plurality of low-resolution images, the resolution of the low-resolution images is generally the same as the physical resolution of the display assembly, and the plurality of low-resolution images are sent to the display assembly for display in a time-sharing manner.

In the imaging control method provided by the embodiment of the application, the multi-frame low-resolution image can be obtained by decomposing the high-resolution image, then the low-resolution image can be displayed in a time-sharing manner through the display component, and the position of each frame of image displayed by the display component can be adjusted through the time-sharing control resolution lifting component, namely, the multi-frame low-resolution image is displayed in a time-sharing manner, and the multi-frame low-resolution image is overlapped in human eyes by utilizing the vision persistence and the vision synthesis function of human eyes, so that the high-resolution image seen by the human eyes is obtained.

In some embodiments, the driving component controls the vibrating mirror to generate rotary vibration, so that each frame of image displayed by the display component is displaced, and multiple frames of low-resolution images displayed by the display component are overlapped in human eyes by utilizing the persistence of vision and the vision synthesis function of human eyes, so that the high-resolution images are seen by the human eyes. In order to achieve the time that the display component displays any one of the multiple images can be synchronous with the time that the resolution enhancing component adjusts any one of the multiple images, the frequency of the rotation vibration of the vibrating mirror needs to be the same as the switching frequency of the display component displaying each frame of image.

In some embodiments, the driving component controls the vibrating mirror to generate rotation vibration so as to enable each frame of image displayed by the display component to generate displacement, which may specifically include: the driving assembly controls the vibrating mirror to output two adjacent frames of images in the multi-frame images at different positions at two adjacent moments, so that the displacement of the two adjacent frames of images displayed by the time-sharing adjustment display assembly is realized. The time when the display assembly displays two adjacent frames of images in the multi-frame images is referred to by two adjacent times, for example, the display assembly displays one frame of image in the multi-frame images at time 1, and the display assembly displays the next frame of image in the multi-frame images at time 2. After the vibrating mirror is positioned at the position 1 at the moment 1 and the rotating vibration of the vibrating mirror is controlled, the vibrating mirror is positioned at the position 2 at the moment 2, and the vibrating mirror rotates at a set angle relative to the position 1 at the position 2. Because the vibrating mirror has a certain thickness and refractive index different from that of air, the position of emergent light can be moved after the same beam of incident light passes through the vibrating mirror at different positions, namely the position of the next frame of image emergent from the vibrating mirror can be displaced. The superposition effect of the multiple frames of images can be seen according to the persistence effect of human eyes, and although the resolution of each frame of image in the multiple frames of images is smaller than that of the image to be displayed, most of information of the image to be displayed is reserved in each frame of image in the multiple frames of images, so that the image finally seen by the human eyes is similar to the original image to be displayed, and the effect that the display component with low resolution displays high resolution is realized.

Specifically, in order to ensure that the superposition effect of multiple frames of images can be seen according to the persistence effect of human eyes, the displacement between two adjacent frames of images can be adjusted to be smaller than the distance of one pixel by controlling the rotation angle of the vibrating mirror, and the pixel is a pixel in the image displayed by the display component. In some embodiments, the distance between two adjacent frames of images may be one half of a pixel distance, for example, the pixel size is 16.5um, the displacement may be 8.3um, and the displacement direction may be along any one or more of a horizontal direction, a vertical direction, or a diagonal direction.

In some embodiments, the image to be displayed may be split into two frames of images by an algorithm, i.e. the multi-frame image may comprise a first image and a second image; the driving assembly controls the galvanometer to output two adjacent frames of images in the multi-frame images at different positions at two adjacent moments, and the driving assembly can specifically comprise: controlling the vibrating mirror to output a first image at a first position at a first moment, and outputting a second image at a second position at a second moment after controlling the vibrating mirror to generate rotary vibration; the first time and the second time are adjacent in time.

In some embodiments, the first and second locations may be separated by one-half pixel distance in the diagonal direction, i.e., the first and second locations are separated by one-half pixel distance in the horizontal direction, while being separated by one-half pixel distance in the vertical direction, thereby achieving resolution doubling.

In some embodiments, controlling the vibrating mirror to generate rotational vibration may specifically include: the vibrating mirror is controlled to rotate at a set angle relative to the first moment by taking the diagonal axis as a rotation axis at the second moment.

Specifically, the image to be displayed is split into two frames of images, the superposition effect of the two frames of images can be seen according to the persistence effect of human eyes, when the image to be displayed is split, the two frames of images can be ensured to keep most of the original information of the image to be displayed through reasonable algorithm processing, and then the image finally seen by the human eyes is the image to be displayed close to the original image to be displayed, so that the improvement from the 2k physical resolution of the display module to the image with the display effect of 4k resolution is realized.

In some embodiments, the image to be displayed may be split into four frames of images by an algorithm, and the multiple frames of images may include a first image, a second image, a third image, and a fourth image; the driving assembly controls the galvanometer to output two adjacent frames of images in the multi-frame images at different positions at two adjacent moments, and the driving assembly can specifically comprise: the method comprises the steps that a first image is output at a first position by controlling a vibrating mirror at a first moment, a second image can be output at a second position at a second moment after the first rotating vibration of the vibrating mirror is controlled, a third image can be output at a third position at a third moment after the second rotating vibration of the vibrating mirror is controlled, a fourth image can be output at a fourth position at a fourth moment after the third rotating vibration of the vibrating mirror is controlled, and a first image in a multi-frame image obtained by decomposing a next frame of image to be displayed can be output at the first position at a fifth moment after the fourth rotating vibration of the vibrating mirror is controlled; the first time, the second time, the third time, the fourth time and the fifth time are sequentially adjacent in time, and the pixels are pixels in the display assembly. According to the visual persistence effect of the human eyes, the superposition effect of four frames of images can be seen, when the images to be displayed are split, the four frames of images can be ensured to keep most of the information of the original images to be displayed through reasonable algorithm processing, so that the image finally seen by the human eyes is the image to be displayed close to the original image, and the improvement from the 2k physical resolution of the display module to the image with the display effect of 4k resolution is realized.

In some embodiments, the first position may be spaced apart from the second position by one-half of a pixel distance in a vertical direction, the second position may be spaced apart from the third position by one-half of a pixel distance in a horizontal direction, the third position may be spaced apart from the fourth position by one-half of a pixel distance in a vertical direction, and the fourth position may be spaced apart from the first position by one-half of a pixel distance in a horizontal direction.

In some embodiments, controlling the vibrating mirror to generate rotational vibration may specifically include: the vibrating mirror is controlled to rotate at a third set angle relative to the first moment by taking the horizontal axis as a rotation axis, the vibrating mirror is controlled to rotate at a first set angle relative to the second moment by taking the vertical axis as a rotation axis, the vibrating mirror is controlled to rotate at a fourth set angle relative to the third moment by taking the horizontal axis as a rotation axis, and the vibrating mirror is controlled to rotate at a fifth set angle relative to the fourth moment by taking the vertical axis as a rotation axis.

In some embodiments, the rotation angle of the galvanometer may satisfy the following relationship:

In some embodiments, the rotation angle of the galvanometer can be obtained based on the value of the thickness of the galvanometer, the value of the refractive index and the displacement of the output image of the galvanometer before and after rotation, and the actual rotation angle of the galvanometer can be allowed to have an error within +/-10% compared with the rotation angle of the galvanometer when the driving component controls the rotation angle of the galvanometer to rotate.

In a third aspect, the present application provides an electronic device comprising at least one processor coupled to at least one memory, the at least one processor configured to read a computer program stored by the at least one memory to perform any one of the imaging control methods of the second aspect described above. The functions may be realized by hardware, or may be realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.

In a fourth aspect, the present application provides a computer readable storage medium having stored therein a computer program or instructions which, when executed by a computer, cause the computer to perform the method of any of the possible implementations of the second aspect described above.

In a fifth aspect, the present application provides a computer program product comprising a computer program or instructions which, when executed by a computer, implement the method of any possible implementation of the second aspect described above.

Drawings

Fig. 1 is a schematic structural diagram of a display assembly according to an embodiment of the present disclosure;

fig. 2 is a schematic diagram of a specific structure of a display module according to an embodiment of the present application;

fig. 3 is a schematic diagram of a specific optical path structure of a display module according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram of another specific optical path structure of the display module provided in the embodiment of the present application;

fig. 5 is a schematic diagram of a specific optical path structure of a display module provided in an embodiment of the present application;

fig. 6 is a schematic diagram of a specific optical path structure of a display module according to an embodiment of the present disclosure;

fig. 7a is a schematic perspective view of a vibrating mirror in a display module according to an embodiment of the disclosure;

fig. 7b is a schematic diagram of rotational vibration of a vibrating mirror in a display module according to an embodiment of the present disclosure;

fig. 8 is a schematic diagram of a diagonal super-resolution principle of two frames of images according to an embodiment of the present application;

fig. 9 is a schematic diagram of two-frame image diagonal super-resolution image superposition provided in the embodiment of the present application;

Fig. 10 is a schematic diagram of a diagonal super-resolution effect of two frames of images according to an embodiment of the present application;

FIG. 11 is a schematic diagram of resolution enhancement implemented by pixel shifting of a four-frame image according to an embodiment of the present application;

fig. 12 is a flowchart of an imaging control method according to an embodiment of the present application.

Reference numerals:

100-display assembly, 200-resolution enhancement assembly, 300-display light path assembly, 400-control assembly, 100 a-first surface, 210-galvanometer, 220-drive assembly, 310-first lens, 320-second lens, 330-1/4 slide, 340-concave mirror, 350-spectroscope, x-horizontal axis, y-vertical axis, z-diagonal axis.

Detailed Description

Embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Hereinafter, some terms in the present application will be explained. It should be noted that these explanations are for the convenience of those skilled in the art, and do not limit the scope of protection claimed in the present application.

(1) Near-eye display:

the display near the eye is one display mode of an AR display device or a VR display device.

(2) The angular resolution may also be referred to as spatial resolution, which refers to the number of pixel points filled per 1 degree included angle on average of the field of view. Before the clear sense reaches the limit of the resolution of human eyes, the more the number of pixel points filled in a picture of a unit area in the visual field of the human eyes is, the clearer and finer the visual sense is. The larger the PPD, the more the number of pixel points filled in the picture of the unit area in the human eye field of view, and the clearer the user's feeling of displaying the picture.

(3) A pixel:

the smallest picture element on the display screen is a pixel. One pixel is composed of three sub-pixels of different colors. The sub-pixels may also be referred to as sub-pixels (sub-pixels). For example, a red (R) subpixel, a green (G) subpixel, and a blue (B) subpixel constitute one pixel. The pixel arrangement mode adopted by the display screen can comprise RGB stripe (stripe) arrangement, pentile RGBG arrangement, pentile RGBW arrangement, delta RGB arrangement and the like.

The embodiment of the application is applied to the display equipment with the resolution required to be improved. Such as a terminal device with a display screen, e.g. a mobile phone, a display, a television, a head-up display system (HUD), etc. The display device may also be a wearable device, which may be a Near Eye Display (NED) device, such as VR glasses, or VR helmets, or the like. For example, a user wears a NED device to play a game, read, watch a movie (or a television show), attend a virtual meeting, attend video education, or video shopping, etc.

In order to realize the effect of using the low-resolution display screen to realize the effect that human eyes see the high-resolution image, a resolution enhancement mode, which can also be called super resolution, can be used for improving the resolution of the low-resolution display screen. Referring to fig. 1, a display module provided in an embodiment of the present application may include a display module 100, a resolution enhancing module 200 and a display light path module 300 disposed on a light path of the display module 100, and a control module 400 respectively connected with the display module 100 and the resolution enhancing module 200 in a signal manner. And a display assembly 100 for displaying an image. The display light path assembly 300 is used for adjusting the display position of each frame of image displayed by the display assembly 100. For example, the display device is a wearable device, and the display light path component 300 may adjust a position on a virtual image plane that is a distance from the display component 100 for each frame of image displayed by the display component 100. The control component 400 is configured to receive an image to be displayed, decompose the image to be displayed to obtain a multi-frame image, and the display component 100 may display the multi-frame image in a time-sharing manner under the control of the control component 400. The resolution enhancing component is configured to time-share adjust the position of each frame of image displayed by the display component 100 under the control of the control component 400.

In this embodiment of the present application, the multi-frame image may be obtained by downsampling the image to be displayed by the control module 400, where the resolution of each frame of image in the multi-frame image is smaller than the resolution of the image to be displayed. For example, the control component 400 splits the high resolution image to be displayed into a plurality of low resolution images, the resolution of the low resolution images is generally the same as the physical resolution of the display component 100, and the control component 400 time-sharing transmits the plurality of low resolution images to the display component 100 for display.

In embodiments of the present application, the time at which the display assembly 100 displays any one of the multiple images may be synchronized with the time at which the galvanometer 210 displaces any one of the multiple images displayed by the display assembly.

In the display module provided in this embodiment of the present application, the control component 400 may obtain multiple frames of low resolution images by decomposing the high resolution image, and then may display the low resolution images by the display component 100 in a time-sharing manner, and may adjust the position of each frame of image displayed by the display component 100 by the time-sharing control resolution enhancing component 200, that is, display multiple frames of low resolution images in a time-sharing manner, and utilize the persistence of vision and the vision synthesis function of human eyes, the multiple frames of low resolution images are superimposed in the human eyes, so that the human eyes see the high resolution images.

Referring to fig. 2, in the embodiment of the present application, the resolution enhancing component 200 may specifically include a galvanometer 210 and a driving component 220, where an image exiting from the display component 100 may enter the display light path component 300 after passing through the galvanometer 210. The driving assembly 220 may be a piezoelectric driving assembly 220 or a motor driving assembly 220, etc. The motor drive assembly 220 may be, in particular, an ultrasonic motor or a servo motor, etc. The control component 400 can control the driving component 220, so that the driving component 220 can perform rotational vibration through time-sharing control on the vibrating mirror 210, displacement of each frame of image displayed by the time-sharing adjustment display component 100 is realized, and multiple frames of low-resolution images displayed by the display component 100 are overlapped in human eyes by utilizing the persistence of vision and the vision synthesis function of human eyes, so that the high-resolution images are seen by human eyes. In order to achieve the time for the display assembly 100 to display any one of the multiple images may be synchronized with the time for the galvanometer 210 to adjust any one of the multiple images, the frequency of the rotational vibration of the galvanometer 210 may be made the same as the switching frequency for the display assembly 100 to display each image.

In the embodiment of the present application, the display assembly 100 may be a liquid crystal display (liquid crystal display, LCD), a micro organic light emitting diode display (micro organic light emitting diode, micro OLED), a silicon-based OLED, a micro light emitting diode (micro light emitting diode, micro LED), or other display device, which is not particularly limited herein.

In the embodiment of the present application, the control component 400 may be a processor, a microprocessor, a controller, or the like, and the control component 400 may be, for example, a general-purpose central processing unit (central processing unit, CPU), a general-purpose processor, a digital signal processing unit (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuits, ASIC), a field programmable gate array (field programmable gate array, PGA), or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof.

Referring to fig. 3 to 6, in an embodiment of the present application, the galvanometer 210 may be disposed near the display assembly 100, i.e., the galvanometer 210 may be disposed on an optical path between the display optical path assembly 300 and the display assembly 100. In this way, after each frame of image emitted from the display assembly 100 in a time-sharing manner passes through the vibrating mirror 210 that rotates and vibrates, the displacement of each frame of image in space can be realized, and then each frame of image is adjusted to a set position for display through the display light path assembly 300, and multiple frames of low-resolution images are overlapped in human eyes by utilizing the persistence of vision and the vision synthesis function of human eyes, so that the high-resolution images seen by human eyes can be realized, and the resolution can be improved.

In some embodiments, when the display light path assembly 300 includes at least two lenses, the galvanometer 210 may also be disposed on the light path between the at least two lenses, i.e., the galvanometer 210 may also be disposed inside the display light path assembly 300. In this way, each frame of image emitted from the display component 100 in a time-sharing manner is firstly refracted by a part of lenses in the display light path component 300, then is incident to the vibrating mirror 210 which rotates and vibrates, so that displacement of each frame of image in space is realized, then the images are adjusted to a set position for display through the rest of lenses in the display light path component 300, and multiple frames of low-resolution images are overlapped in human eyes by utilizing the persistence of vision and the vision synthesis function of human eyes, so that the high-resolution images seen by human eyes are improved.

In the embodiment of the present application, the main function of the display light path assembly 300 is to change the transmission direction of the light in the image and to converge the light of the image, and the display light path assembly 300 may have various specific implementations.

In some embodiments, referring to fig. 3, the display light path assembly 300 may include refractive lenses (refractive lenses). The image emitted from the display assembly 100 is incident on the refractive lens after passing through the galvanometer 210, and the refractive lens converges the image to a set position. When resolution is required to be improved, the display assembly 100 outputs multiple frames of images in a time-sharing manner, and the vibrating mirror 210 rotates and vibrates along with the multiple frames of images, so that each frame of images is ensured to displace after passing through the vibrating mirror 210.

In some embodiments, referring to fig. 4, the display light path assembly 300 may include a first lens 310 and a second lens 320, and a 1/4 glass slide 330 between the first lens 310 and the second lens 320, wherein the first lens 310 is disposed between the 1/4 glass slide 330 and the galvanometer 210, and a surface of the first lens 310 near the galvanometer 210 may be provided with a semi-transparent and semi-reflective film. The second lens 320 is disposed between the viewing surface and the 1/4 glass slide 330, and a polarization beam-splitting coating film can be disposed on a surface of the second lens 320 close to the 1/4 glass slide 330, where the coating film can realize reflection of S-polarized light by transmitting P-polarized light. For example, the display assembly 100 emits left circularly polarized light L, the left circularly polarized light L is incident to the first lens 310 in the display light path assembly 300 after passing through the galvanometer 210, and is converted into S-polarized light after passing through the 1/4 glass slide 330, the S-polarized light is reflected by the second lens 320, is converted into left circularly polarized light L after passing through the 1/4 glass slide 330, is converted into right circularly polarized light R after passing through the reflection of the first lens 310, is converted into P-polarized light after passing through the 1/4 glass slide 330, and is emitted to the set position after being refracted by the second lens 320. In order to ensure that the display device 100 emits left circularly polarized light L, a phase film may be added to the light emitting surface of the display device 100. When resolution is required to be improved, the display assembly 100 can output multiple frames of images in a time-sharing manner, and the vibrating mirror 210 rotates and vibrates along with the multiple frames of images, so that each frame of images is ensured to displace after passing through the vibrating mirror 210. Further, in some examples, the position of galvanometer 210 in FIG. 4 may be altered and placed in the optical path between first lens 310, second lens 320, and 1/4 glass slide 330.

In some embodiments, referring to fig. 5, the display light path assembly 300 may include a curved mirror. The image emitted from the display assembly 100 is incident on a curved mirror after passing through the galvanometer 210, and the curved mirror reflects and converges the image to a set position. When resolution is required to be improved, the display assembly 100 can output multiple frames of images in a time-sharing manner, and the vibrating mirror 210 rotates and vibrates along with the multiple frames of images, so that each frame of images is ensured to displace after passing through the vibrating mirror 210.

In some embodiments, referring to FIG. 6, the display light path assembly 300 may include a concave mirror 340 and a beam splitter 350. The image emitted from the display assembly 100 is incident to the spectroscope 350 after passing through the galvanometer 210, the spectroscope 350 reflects the image to the concave mirror 340, and the image is emitted from the spectroscope 350 to the set position after being converged by the concave mirror 340. When resolution is required to be improved, the display assembly 100 can output multiple frames of images in a time-sharing manner, and the vibrating mirror 210 rotates and vibrates along with the multiple frames of images, so that each frame of images is ensured to displace after passing through the vibrating mirror 210. Furthermore, in some embodiments, the position of galvanometer 210 in FIG. 6 may be altered and placed in the optical path between the beam splitter and the concave mirror.

Next, for convenience of description, first, a horizontal axis x, a vertical axis y, and a diagonal axis z according to an embodiment of the present application will be described. Referring to fig. 7a, the display assembly 100 has a first surface 100a nearest to the galvanometer 210 defining a vertical axis y perpendicular to the horizontal plane or an angle of 85 degrees to 95 degrees from the horizontal plane; defining the angle of the horizontal axis x perpendicular to the vertical axis y or the included angle between the vertical axis y and the horizontal axis x to be 85-95 degrees; the diagonal axis z is defined to lie in a plane formed by the vertical axis y and the horizontal axis x, and is parallel to the diagonal direction of one or more pixels included in the first surface 100a of the display assembly 100, or the angle of the diagonal axis z to the diagonal of the pixels is 85 degrees to 95 degrees. Also, the vertical axis y, the horizontal axis x, and the diagonal axis z may each pass through the center point of the galvanometer 210, or the vertical axis y, the horizontal axis x, and the diagonal axis z may be spaced apart from the center point of the galvanometer 210, and the vertical distance from the center point to any one or more of the vertical axis y, the horizontal axis x, and the diagonal axis z may be within 5mm, as an example. It should be noted that, in fig. 7a, each surface of the galvanometer 210 is taken as a plane for illustration, and in practical application, one or more surfaces of the galvanometer 210 may be curved, which is not limited herein.

Referring to fig. 7b, in the embodiment of the present application, the galvanometer 210 may implement time-sharing adjustment of displacement of two adjacent frame images displayed by the display assembly 100 by outputting the two adjacent frame images of the multi-frame images at different positions at two adjacent times. The time when the display module 100 displays two adjacent frames of images in the multi-frame image is referred to as two adjacent times, for example, the display module 100 displays one frame of image in the multi-frame image at time 1 and the display module 100 displays the next frame of image in the multi-frame image at time 2. At time 1, the galvanometer 210 is at position 1, and after the rotational vibration of the galvanometer 210 is controlled, at time 2, the galvanometer 210 is at position 2, and the galvanometer 210 is rotated at a set angle with respect to position 1 at position 2. The galvanometer 210 shown in fig. 7b rotates at a position 2 by an angle θ about a horizontal axis x as a rotation axis with respect to a position 1. Since the galvanometer 210 has a certain thickness t and a refractive index n different from that of air, the position of the emergent light will be moved after the incident light passes through the galvanometer 210 at different positions, i.e. the position of the next frame of image emergent from the galvanometer 210 will be displaced by Δy. The superposition effect of the multiple frames of images can be seen according to the persistence effect of human eyes, and although the resolution of each frame of image in the multiple frames of images is smaller than that of the image to be displayed, most of information of the image to be displayed is reserved in each frame of image in the multiple frames of images, so that the image finally seen by the human eyes is similar to the original image to be displayed, and the effect that the display assembly 100 with low resolution displays high resolution is realized.

Specifically, in order to ensure that the superposition effect of the multi-frame images can be seen according to the persistence effect of human eyes, the displacement Δy between two adjacent frames of images can be adjusted to be smaller than a pixel distance by controlling the rotation angle θ of the galvanometer 210, and the pixels are pixels in the image displayed by the display assembly 100. In some embodiments, the distance between two adjacent frames of images may be one half of a pixel distance, for example, the pixel size is 16.5um, the displacement may be 8.3um, and the displacement direction may be along any one or more of a horizontal direction, a vertical direction, or a diagonal direction. The rotation angle θ of the galvanometer 210 may satisfy the following relationship:

where Δy is the displacement of the output image of the galvanometer 210 before and after rotation, θ is the rotation angle of the galvanometer 210, t is the thickness of the galvanometer 210, and n is the refractive index of the galvanometer 210.

In the embodiment of the present application, the material of the galvanometer 210 may be a light-transmitting material such as glass, polycarbonate (PC), polymethyl methacrylate (polymethyl methacrylate, PMMA), or Polystyrene (PS). The refractive index of the material selected for the galvanometer 210 is generally 1.3-2, the error range is less than 0.01, for example, a light-transmitting material with a refractive index of 1.5 can be selected. Under the same condition, the larger the thickness of the vibrating mirror 210 is, the larger the displacement is, but the larger the thickness of the vibrating mirror 210 is, the whole volume of the display module is increased, the thickness range of the vibrating mirror 210 is generally 0.5mm-6mm, the error range is smaller than 0.1mm, and for example, the thickness of the vibrating mirror 210 can be selected to be 2mm. Specifically, the control unit 400 may obtain the rotation angle θ of the galvanometer 210 based on the thickness value of the galvanometer 210, the refractive index value of the galvanometer, and the displacement Δy of the output image of the galvanometer 210 before and after rotation. The driving assembly 220 may allow an error within + -10% of the actual rotation angle theta' of the galvanometer 210 compared to the rotation angle theta when controlling the rotation angle theta of the galvanometer 210.

In the present embodiment, the resolution improvement factor is related to the resolution of the image to be displayed and the physical resolution of the display assembly 100. For example, if the resolution of the image to be displayed is twice the physical resolution of the display assembly 100, for example, the image to be displayed is 4k, the physical resolution of the display assembly 100 is 2k, the pixel density unit (PPI) of the display assembly 100 is about 1200-1500, the angular resolution (points per degree, PPD) is about 20-30, by splitting the image to be displayed into multiple frame images and rotating the vibrating mirror 210, the superimposed display effect can be finally presented to the human eye as a 4k image, the PPI of the display effect is about 2400-3000, and the PPD of the display effect is about 40-60.

The operation of the galvanometer 210 of the present application will be described in detail with reference to the resolution of the image to be displayed being 4k 60hz and the physical resolution of the display assembly 100 being 2 k.

Referring to fig. 8, in the embodiment of the present application, the control component 400 may divide the image to be displayed into two frames of 2k 120hz images by an algorithm, i.e., a multi-frame image including a first image of 2k 120hz and a second image of 2k 120 hz. The galvanometer 210 can output a first image at a first position A at a first moment, and can output a second image at a second position B at a second moment after the galvanometer 210 generates rotary vibration; wherein the first time and the second time are adjacent in time.

Specifically, any one or more of horizontal, vertical, or diagonal superdivision may be achieved using rotational vibration of the galvanometer 210. It is understood that the two frame images formed by the beams output by the galvanometer 210 have a separation distance in the horizontal direction, and the separation distance is Px/2, where Px represents a distance between adjacent pixels of any one of the two frame images in the horizontal direction. When the horizontal super-resolution is adopted, the galvanometer 210 needs to rotate by a first set angle or a second set angle relative to the first moment by taking the vertical axis y as a rotation axis, so that the offset vector of the second position B relative to the first position A is (Px/2, 0) or (-Px/2, 0), that is, the first position A and the second position B can be separated by a half pixel distance in the horizontal direction, thereby doubling the resolution in the horizontal direction. It can be understood that the two frames of images output by the galvanometer 210 have a separation distance in the vertical direction, taking the separation distance as Py/2 as an example, py represents the interval between adjacent pixels of any one frame of images in the two frames of images in the vertical direction, when the vertical direction is exceeded, the galvanometer 210 needs to rotate by a third set angle or a fourth set angle with respect to the first moment by taking the horizontal axis x as the rotation axis at the second moment, so as to realize that the offset vector of the second position B relative to the first position a is (0, py/2) or (0, -Py/2), that is, the first position a and the second position B can be separated by a half pixel distance in the vertical direction, thereby realizing the doubling of the resolution in the vertical direction. It is understood that the two frames of images output by the galvanometer 210 have a separation distance in the vertical direction and a separation distance in the horizontal direction, and taking the vertical separation distance as Py/2 and the horizontal separation distance as Px/2 as an example, when the diagonal direction is exceeded, the galvanometer 210 needs to rotate at the second moment by a fifth set angle or a sixth set angle with respect to the first moment by taking the diagonal axis z as the rotation axis, so as to realize that the offset vector of the second position B relative to the first position a is (Px/2, py/2), (-Px/2, py/2), (Px/2, -Py/2) or (-Px/2, -Py/2), that is, the first position a and the second position B may be respectively separated by one half pixel distance in the horizontal direction and the vertical direction, thereby realizing resolution doubling. In fig. 8, taking a pixel as an example, the center of the pixel is used as a mark point of the first position a and the second position B, and the case of superdivision in the diagonal direction is illustrated.

Referring to fig. 9, taking 4*4 pixel array as an example, assuming that the galvanometer 210 can realize offset with offset vector (Px/2, -Py/2), the effect of doubling the equivalent number of display pixels can be realized by time division multiplexing. When the offset vector of the galvanometer 210 is (±px/2, 0), the resolution in the horizontal direction can be doubled. When the offset vector of the galvanometer 210 is (0, ±py/2), resolution multiplication in the vertical direction can be achieved.

Referring to fig. 10, the image to be displayed is split into two frames of images, the superposition effect of the two frames of images is seen according to the persistence effect of human eyes, when the image to be displayed is split, most of the original information of the image to be displayed can be ensured to be remained through reasonable algorithm processing, and then the image finally seen by the human eyes is the image to be displayed close to the original image to be displayed, so that the improvement from the 2k physical resolution of the display module to the image with the display effect of 4k resolution is realized.

Referring to fig. 11, in the embodiment of the present application, the control component 400 may divide the image to be displayed into four frames of 2k 240hz images by an algorithm, that is, the multi-frame image includes a first image of 2k 120hz, a second image of 2k 120hz, a third image of 2k 120hz, and a fourth image of 2k 120 hz. The galvanometer 210 can output a first image at a first position a at a first moment, the galvanometer 210 can output a second image at a second position B at a second moment after generating first rotation vibration, the galvanometer 210 can output a third image at a third position C at a third moment after generating second rotation vibration, the galvanometer 210 can output a fourth image at a fourth position D at a fourth moment after generating third rotation vibration, and the galvanometer 210 can output a first image in a multi-frame image obtained by decomposing a next frame of image to be displayed at the first position a at a fifth moment after generating fourth rotation vibration; the first time, the second time, the third time, the fourth time and the fifth time are adjacent in time. According to the visual persistence effect of the human eyes, the superposition effect of four frames of images can be seen, when the images to be displayed are split, the four frames of images can be ensured to keep most of the information of the original images to be displayed through reasonable algorithm processing, so that the image finally seen by the human eyes is the image to be displayed close to the original image, and the improvement from the 2k physical resolution of the display module to the image with the display effect of 4k resolution is realized.

Specifically, the rotation vibration of the galvanometer 210 along different directions can realize the mutual conversion of the images between four positions at four moments, and by way of example, two frames of images output by the galvanometer 210 can be separated by a distance Px/2 in the horizontal direction, and Px represents the distance between adjacent pixels of the two frames of images in the horizontal direction, and at this time, the resolution in the horizontal direction is doubled; the two frames of images output by the vibrating mirror 210 can be vertically spaced by a distance Py/2, py represents the vertical distance between adjacent pixels of the two frames of images, and at this time, the resolution in the vertical direction is doubled; the two frames of images output by the galvanometer 210 are vertically spaced apart by a distance Py/2 and horizontally spaced apart by a distance Px/2, at which time resolution doubling is achieved.

Referring to fig. 11, for example, the galvanometer 210 may output a first image at a first position a at a first time, the first position a may be spaced apart from the second position B by a half pixel distance in a vertical direction, and when the galvanometer 210 rotates at a second time by a third set angle with respect to the first time about a horizontal axis x as a rotation axis, the galvanometer 210 may implement an offset vector (0, py/2) such that a second image may be output at the second position B at the second time. When the galvanometer 210 rotates at a first set angle with respect to the second time with the vertical axis y as a rotation axis at the third time, the galvanometer 210 may implement an offset vector (Px/2, 0) such that a third image may be output at a third position C at the third time, and the second position B may be spaced apart from the third position C by a half pixel distance in the horizontal direction. When the galvanometer 210 rotates a fourth set angle with respect to the third timing along the rotation axis of the horizontal axis x at the fourth timing, the galvanometer 210 may implement an offset vector (0, -Py/2) such that a third image may be output at a fourth position D at the fourth timing, and the third position C may be spaced apart from the fourth position D by a half pixel distance in the vertical direction. When the galvanometer 210 rotates at the fifth moment by a second set angle relative to the fourth moment along the rotation axis with the vertical axis y, the galvanometer 210 can realize an offset vector (-Px/2, 0), so that the galvanometer can return to the first position a at the fifth moment to output a first image in the multi-frame image obtained by decomposing the next frame of image to be displayed, and the fourth position D and the first position a can be separated by a half pixel distance in the horizontal direction. In fig. 11, taking one pixel as an example, the center pixel of the image frame is used as the mark points of the first position a, the second position B, the third position C, and the fourth position D.

It should be noted that, in the embodiment of the present application, the order of outputting the first image, the second image, the third image or the fourth image by the galvanometer 210 is not specifically limited, and by way of example, the galvanometer 210 may also output the first image at the first position a at the first time, the fourth image may be output at the fourth position D at the second time after the galvanometer 210 generates the rotational vibration, the third image may be output at the third position C at the third time after the galvanometer 210 generates the rotational vibration, and the second image may be output at the second position B at the fourth time after the galvanometer 210 generates the rotational vibration; the galvanometer 210 may also output a first image at the first position a at a first time, may output a third image at the third position C at a second time after the galvanometer 210 generates rotational vibration, may output a fourth image at the fourth position D at a third time after the galvanometer 210 generates rotational vibration, may output a second image at the second position B at a fourth time after the galvanometer 210 generates rotational vibration, and so on. Wherein the first time, the second time, the third time and the fourth time are adjacent in time.

The above is merely illustrative of the rotational vibration operation of the galvanometer 210, and is not limited thereto. In addition, the image to be displayed may be split into more numerical multi-frame images, such as three-frame images, five-frame images, six-frame images, etc., which will not be described in detail herein. In order to ensure the display effect, each frame of image can be ensured to be displayed at different positions as much as possible.

In some embodiments of the present application, the display module may support operation in two modes. Supersplit mode, and normal mode. The control component 400 performs superprocessing upon determining that supersplit mode is enabled; the control component 400 maintains the resolution unchanged and the frame rate is not reduced when it determines that the normal mode is enabled. In the normal mode, the galvanometer 210 does not generate rotational vibration. In some embodiments, the control component 400 may resample the image source to the physical resolution of the display component 100, outputting directly to the display component 100, which normal mode may be applicable to some high frame rate scenarios, such as games, etc.

Based on the above and the same technical ideas, the embodiments of the present application also provide an imaging control method, which is applied to a wearable device. The wearable device comprises a display component, a resolution improving component and a display light path component, wherein the resolution improving component and the display light path component are arranged on a light path of the display component, the resolution improving component can comprise a vibrating mirror and a driving component, and an image emitted by the display component can be incident to the display light path component after passing through the vibrating mirror.

Referring to fig. 12, the imaging control method may specifically include the steps of:

s1, receiving an image to be displayed, and decomposing the image to be displayed to obtain a multi-frame image.

In this embodiment of the present application, the multiple frame images may be obtained by downsampling an image to be displayed, so that the resolution of each frame of image in the multiple frame images may be smaller than the resolution of the image to be displayed. For example, the high-resolution image to be displayed can be split into a plurality of low-resolution images, the resolution of the low-resolution images is generally the same as the physical resolution of the display assembly, and the plurality of low-resolution images are sent to the display assembly for display in a time-sharing manner.

S2, controlling the display assembly to display multi-frame images in a time sharing mode.

S3, the driving assembly controls the vibrating mirror to generate rotary vibration, so that any one of the multi-frame images displayed by the display assembly is displaced.

In the embodiment of the application, the time when the display component displays any one of the multiple images can be synchronous with the time when the galvanometer shifts any one of the multiple images displayed by the display component.

In this embodiment of the present application, in order to achieve that the time when the display component displays any one of the multiple frames of images may be synchronized with the time when the galvanometer displaces any one of the multiple frames of images displayed by the display component, the frequency of rotational vibration of the galvanometer needs to be the same as the switching frequency of each frame of images displayed by the display component.

In the embodiment of the application, the driving component can control the vibrating mirror to output two adjacent frames of images in the multi-frame images at different positions at two adjacent moments, so that the displacement of the two adjacent frames of images displayed by the time-sharing adjustment display component is realized. The display components of two adjacent time points display the time of two adjacent frames of images in the multi-frame images.

Specifically, in order to ensure that the superposition effect of the multi-frame images can be seen according to the persistence effect of human eyes, the displacement between two adjacent frames of images can be adjusted to be smaller than a pixel distance by controlling the rotation angle of the vibrating mirror, the pixel is a pixel in the image displayed by the display component, and the displacement direction can be any one or more directions along the horizontal direction, the vertical direction or the diagonal direction. For example, the image to be displayed may be split into two frames of images by an algorithm, i.e. the multi-frame image may comprise a first image and a second image; the driving component can control the vibrating mirror to output a first image at a first position at a first moment, and control the vibrating mirror to generate rotary vibration at a second moment and then output a second image at a second position; the first time and the second time are adjacent in time, and the first position and the second position can be separated by one half of a pixel distance in the diagonal direction, namely, the first position and the second position are separated by one half of a pixel distance in the horizontal direction, and are separated by one half of a pixel distance in the vertical direction, so that the resolution is doubled.

In the embodiment of the application, the image to be displayed can be split into four frames of images through an algorithm, and the multiple frames of images can comprise a first image, a second image, a third image and a fourth image; the driving assembly controls the vibrating mirror to output a first image at a first position at a first moment, controls the vibrating mirror to output a second image at a second position at a second moment after the vibrating mirror generates first rotation vibration, controls the vibrating mirror to output a third image at a third position at a third moment after the vibrating mirror generates second rotation vibration, controls the vibrating mirror to output a fourth image at a fourth moment after the vibrating mirror generates third rotation vibration, controls the vibrating mirror to output a first image in a multi-frame image obtained by decomposing a next frame of image to be displayed at the first position at a fifth moment, for example, the first position and the second position can be separated by one half pixel distance in the vertical direction, the second position and the third position can be separated by one half pixel distance in the horizontal direction, the third position and the fourth position can be separated by one half pixel distance in the vertical direction, and the fourth position and the first position can be separated by one half pixel distance in the horizontal direction; the first time, the second time, the third time, the fourth time and the fifth time are sequentially adjacent in time. According to the visual persistence effect of the human eyes, the superposition effect of four frames of images can be seen, when the images to be displayed are split, the four frames of images can be ensured to keep most of the information of the original images to be displayed through reasonable algorithm processing, so that the image finally seen by the human eyes is the image to be displayed close to the original image, and the improvement from the 2k physical resolution of the display module to the image with the display effect of 4k resolution is realized.

In the embodiment of the present application, the rotation angle and the displacement amount of the galvanometer may satisfy the following relationship:

In this application embodiment, based on the displacement volume of galvanometer thickness numerical value, refracting index numerical value and galvanometer output image around the rotation, can obtain the rotation angle of galvanometer, when the rotation angle of this galvanometer is rotated to the drive assembly control galvanometer, the actual rotation angle of galvanometer compares the rotation angle of this galvanometer can allow to have within + -10% error.

In the imaging control method provided by the embodiment of the application, the multi-frame low-resolution image is obtained by decomposing the high-resolution image, then the low-resolution image can be displayed in a time-sharing manner through the display component, and the position of each frame of image displayed by the display component can be adjusted through the time-sharing control resolution lifting component, namely, the multi-frame low-resolution image is displayed in a time-sharing manner, and the multi-frame low-resolution image is overlapped in human eyes by utilizing the vision persistence and the vision synthesis function of human eyes, so that the high-resolution image seen by the human eyes is obtained.

Based on the above embodiments, the present application further provides an electronic device, where the electronic device includes at least one processor and at least one memory, where the at least one memory stores computer program instructions, and when the electronic device is running, the at least one processor performs functions performed by the electronic device in the methods described in the embodiments of the present application.

Based on the above embodiments, the present application also provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the methods described in the embodiments of the present application.

Based on the above embodiments, the present application also provides a computer-readable storage medium having stored therein a computer program which, when executed by a computer, causes the computer to perform the methods described in the embodiments of the present application.

The method steps in the embodiments of the present application may be implemented by hardware, or may be implemented by a processor executing software instructions. The software instructions may be comprised of corresponding software modules that may be stored in random access memory (random access memory, RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. In addition, the ASIC may be located in a head mounted display device or a terminal device. It is also possible that the processor and the storage medium reside as discrete components in a head mounted display device or a terminal device.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are performed in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, a network device, a user device, or other programmable apparatus. The computer program or instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer program or instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired or wireless means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that integrates one or more available media. The usable medium may be a magnetic medium, e.g., floppy disk, hard disk, tape; optical media, such as digital video discs (digital video disc, DVD); but also semiconductor media such as solid state disks (solid state drive, SSD).

In the various embodiments of the application, if there is no specific description or logical conflict, terms and/or descriptions between the various embodiments are consistent and may reference each other, and features of the various embodiments may be combined to form new embodiments according to their inherent logical relationships.

In the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a alone, a and B together, and B alone, wherein a, B may be singular or plural. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural. In the text description of the present application, the character "/", generally indicates that the associated object is an or relationship. In the formulas of the present application, the character "/" indicates that the front and rear associated objects are a "division" relationship. In the present application, the symbol "(a, b)" represents an open interval, the range being greater than a and less than b; "[ a, b ]" means a closed interval in a range of greater than or equal to a and less than or equal to b; in addition, in this application, the term "exemplary" is used to denote as an example, illustration, or description any embodiment or design described as "exemplary" in this application should not be construed as being preferred or advantageous over other embodiments or designs, or it is understood that the use of the term "exemplary" is intended to present concepts in a concrete fashion and is not limiting to this application.

It will be appreciated that the various numerical numbers referred to in this application are merely descriptive convenience and are not intended to limit the scope of embodiments of this application. The sequence number of each process does not mean the sequence of the execution sequence, and the execution sequence of each process should be determined according to the function and the internal logic. The terms "first," "second," and the like, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion, such as a series of steps or elements. The method, system, article, or apparatus is not necessarily limited to those explicitly listed but may include other steps or elements not explicitly listed or inherent to such process, method, article, or apparatus.

Although the present application has been described in connection with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely exemplary of the arrangements defined in the appended claims and are to be construed as covering any and all modifications, variations, combinations, or equivalents that are within the scope of the application.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the embodiments of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to encompass such modifications and variations.

Claims

1. A display module, comprising: the display component is arranged on the optical path of the display component, comprises a resolution enhancement component and a display optical path component and a control component which are respectively connected with the display component and the resolution enhancement component in a signal manner;

the resolution ratio improving assembly comprises a vibrating mirror and a driving assembly, wherein an image emitted by the display assembly is incident to the display light path assembly after passing through the vibrating mirror;

the control component is used for receiving an image to be displayed, and decomposing the image to be displayed to obtain a multi-frame image;

the display component is used for displaying the multi-frame images in a time sharing mode under the control of the control component;

the vibrating mirror is used for generating rotary vibration under the control of the driving assembly so as to enable any image in the multi-frame images displayed by the display assembly to be displaced;

The display light path component is used for adjusting the display position of each frame of image displayed by the display component.

2. The display module of claim 1, wherein a resolution of each of the plurality of images is less than a resolution of the image to be displayed, and wherein the resolution of each of the plurality of images is the same as a physical resolution of the display assembly.

3. The display module according to claim 1 or 2, wherein a time at which the display component displays any one of the plurality of images is synchronized with a time at which the galvanometer displaces any one of the plurality of images displayed by the display component;

the frequency of the rotation vibration of the vibrating mirror is the same as the switching frequency of the display component for displaying each frame of image.

4. A display module according to any one of claims 1 to 3, wherein the galvanometer is specifically configured to output two adjacent frames of images of the multi-frame image at different positions at two adjacent times, and displacement between the two adjacent frames of images is less than a pixel distance, where the pixel is a pixel in the image displayed by the display module.

5. The display module assembly of claim 4, wherein the multi-frame image comprises a first image and a second image;

The vibrating mirror is specifically configured to output the first image at a first position at a first time, and output the second image at a second position at a second time after rotational vibration occurs; the first time and the second time are adjacent in time.

6. The display module of claim 5, wherein the first location is diagonally spaced from the second location by one-half a pixel distance.

7. The display module of claim 6, wherein the galvanometer rotates at a set angle with respect to the first moment about a diagonal axis at the second moment.

8. The display module of claim 4, wherein the multi-frame image comprises a first image, a second image, a third image, and a fourth image;

the vibrating mirror is specifically configured to output the first image at a first position at a first time, output the second image at a second position at a second time after a first rotational vibration occurs, output the third image at a third position at a third time after a second rotational vibration occurs, output the fourth image at a fourth position at a fourth time after a third rotational vibration occurs, and output a first image of a multi-frame image obtained by decomposing a next frame of image to be displayed at the first position at a fifth time after a fourth rotational vibration occurs;

The first time, the second time, the third time, the fourth time and the fifth time are sequentially adjacent in time.

9. The display module of claim 8, wherein the first position is vertically spaced from the second position by one-half a pixel distance, the second position is horizontally spaced from the third position by one-half a pixel distance, the third position is vertically spaced from the fourth position by one-half a pixel distance, and the fourth position is horizontally spaced from the first position by one-half a pixel distance.

10. The display module of claim 9, wherein the galvanometer rotates a third set angle with respect to the first time about a horizontal axis as a rotation axis, the galvanometer rotates a first set angle with respect to the second time about a vertical axis as a rotation axis, the galvanometer rotates a fourth set angle with respect to the third time about the horizontal axis as a rotation axis, and the galvanometer rotates a second set angle with respect to the fourth time about the vertical axis as a rotation axis.

11. A display module according to any one of claims 1-10, wherein the rotation angle of the vibrating mirror satisfies the following relationship:

12. The display module of claim 11, wherein the thickness of the galvanometer ranges from 0.5mm to 6mm, and the refractive index of the galvanometer ranges from 1.3 to 2;

the control component is also used for obtaining the rotation angle of the vibrating mirror based on the thickness of the vibrating mirror, the refractive index of the vibrating mirror and the displacement of the vibrating mirror output images before and after rotation;

when the driving assembly controls the vibrating mirror to rotate the rotating angle of the vibrating mirror, the error of the actual rotating angle of the vibrating mirror compared with the rotating angle of the vibrating mirror is within +/-10%.

13. A display module according to any one of claims 1 to 12, wherein the display light path assembly comprises at least two lenses, the vibrating mirror being disposed in the light path between the at least two lenses.

14. A display module according to any one of claims 1 to 13, wherein the drive assembly is a piezo-electric drive assembly or a motor drive assembly.

15. The display module of any one of claims 1-14, wherein the material of the vibrating mirror is glass, PC, PMMA, or PS.

16. The imaging control method is characterized by being applied to a display module, wherein the display module comprises a display component, a resolution enhancement component and a display light path component, wherein the resolution enhancement component and the display light path component are arranged on an optical path of the display component; the resolution ratio improving assembly comprises a vibrating mirror and a driving assembly, wherein an image emitted by the display assembly is incident to the display light path assembly after passing through the vibrating mirror; the method comprises the following steps:

receiving an image to be displayed, and decomposing the image to be displayed to obtain a multi-frame image;

controlling the display component to display the multi-frame images in a time-sharing way;

the driving component controls the vibrating mirror to generate rotary vibration, so that any one of the multi-frame images displayed by the display component is displaced.

17. The method of claim 16, wherein the resolution of each of the plurality of frames of images obtained by decomposition is less than the resolution of the image to be displayed, and the resolution of each of the plurality of frames of images is the same as the physical resolution of the display assembly.

18. The method of claim 16 or 17, further comprising: controlling the time of the display assembly to display any one of the multiple frames of images to be synchronous with the time of the vibrating mirror to enable any one of the multiple frames of images displayed by the display assembly to displace;

19. The method of any of claims 16-18, wherein the driving assembly controlling the galvanometer to rotationally vibrate to displace any of the plurality of frames of images displayed by the display assembly comprises:

the driving assembly controls the vibrating mirror to output two adjacent frames of images in the multi-frame images at different positions at two adjacent moments, the displacement between the two adjacent frames of images is smaller than a pixel distance, and the pixels are pixels in the images displayed by the display assembly.

20. The method of claim 19, wherein the multi-frame image comprises a first image and a second image;

the driving assembly controls the vibrating mirror to output two adjacent frames of images in the multi-frame images at different positions at two adjacent moments, and the driving assembly comprises:

Controlling the vibrating mirror to output the first image at a first position at a first moment, and outputting the second image at a second position at a second moment after controlling the vibrating mirror to generate rotary vibration; the first time and the second time are adjacent in time.

21. The method of claim 20, wherein the first location is diagonally spaced from the second location by one-half a pixel distance.

22. The method of claim 21, wherein said controlling said galvanometer to rotationally vibrate comprises:

and controlling the vibrating mirror to rotate at a set angle by taking a diagonal axis as a rotation axis at the second moment relative to the first moment.

23. The method of claim 19, wherein the multi-frame image comprises a first image, a second image, a third image, and a fourth image;

the method comprises the steps of controlling the vibrating mirror to output a first image at a first position at a first moment, controlling the vibrating mirror to output a second image at a second position at a second moment after first rotation vibration occurs, controlling the vibrating mirror to output a third image at a third position at a third moment after second rotation vibration occurs, controlling the vibrating mirror to output a fourth image at a fourth position at a fourth moment after third rotation vibration occurs, controlling the vibrating mirror to output a first image in a multi-frame image obtained by decomposing a next frame of image to be displayed at a first position at a fifth moment after fourth rotation vibration occurs;

24. The method of claim 23, wherein the first position is vertically spaced from the second position by one-half a pixel distance, the second position is horizontally spaced from the third position by one-half a pixel distance, the third position is vertically spaced from the fourth position by one-half a pixel distance, and the fourth position is horizontally spaced from the first position by one-half a pixel distance.

25. The method of claim 24, wherein said controlling said galvanometer to rotationally vibrate comprises:

the vibrating mirror is controlled to rotate at a third set angle relative to the first moment by taking a horizontal axis as a rotation axis, the vibrating mirror is controlled to rotate at a first set angle relative to the second moment by taking a vertical axis as a rotation axis, the vibrating mirror is controlled to rotate at a fourth set angle relative to the third moment by taking the horizontal axis as a rotation axis, and the vibrating mirror is controlled to rotate at a fifth set angle relative to the fourth moment by taking the vertical axis as a rotation axis.

26. The method of any one of claims 16-25, wherein the rotation angle of the galvanometer satisfies the relationship:

27. The method as recited in claim 26, further comprising: obtaining the rotation angle of the vibrating mirror based on the thickness of the vibrating mirror, the refractive index of the vibrating mirror and the displacement of the output image of the vibrating mirror before and after rotation;

and when the vibrating mirror is controlled to rotate the rotating angle of the vibrating mirror, the error of the actual rotating angle of the vibrating mirror compared with the rotating angle of the vibrating mirror is within +/-10 percent.

28. An electronic device comprising at least one processor coupled to at least one memory, the at least one processor configured to read a computer program stored in the at least one memory to perform the imaging control method of any of claims 16-27.

29. A computer program product comprising instructions, characterized in that the computer program product comprises a computer program or instructions which, when run on a computer, cause the computer to perform the imaging control method as claimed in any one of claims 16-27.

30. A computer-readable storage medium, in which a computer program or instructions are stored which, when run on a computer, cause the computer to perform the imaging control method as claimed in any one of claims 16 to 27.