WO2022121260A1

WO2022121260A1 - Projection picture anti-shake method and apparatus, projection device, and storage medium

Info

Publication number: WO2022121260A1
Application number: PCT/CN2021/098494
Authority: WO
Inventors: 苏博文; 王鑫
Original assignee: 成都极米科技股份有限公司
Priority date: 2020-12-09
Filing date: 2021-06-05
Publication date: 2022-06-16
Also published as: CN112565725A; CN112565725B

Abstract

The present application relates to the technical field of projection display. Disclosed are a projection picture anti-shake method and apparatus, a projection device, and a storage medium. The projection picture anti-shake method comprises: acquiring gyroscope data; calculating a rotation angle according to the gyroscope data; converting the rotation angle into a first quaternion q; calculating a difference value Δq between the first quaternion q and a second quaternion Q, the second quaternion Q being used for reflecting a target three-dimensional attitude; and remapping each pixel according to the difference value Δq to obtain a stabilized image. According to the present application, the problem of picture projection shake in a projector application scene is solved by using a gyroscope quaternion method, response is fast, and costs are low. In addition, according to the present application, a gyroscope is mounted at the position of a projector ray machine instead of the inside of the machine, so that the attitude change of the ray machine can be estimated more accurately.

Description

A projection image stabilization method, device, projection equipment and storage medium

technical field

The present application relates to the technical field of projection display, and in particular, to an anti-shake method, device, projection device and storage medium for a projection image.

Background technique

When the projector is unstable, it tends to produce a flickering picture due to its own shaking or shaking. In many usage scenarios of the projector, when the plane on which the projector is placed, such as the table top of the projector or the suspended ceiling vibrates, it will cause the projector to shake, resulting in the instability of the projected image. The instability of the projected image often causes viewers an uncomfortable experience in watching movies.

SUMMARY OF THE INVENTION

At present, most of the mainstream projectors do not have anti-shake measures. Even if there are anti-shake measures, most of them are based on mechanical anti-shake. This method has higher requirements on structural design and also increases the corresponding cost. In view of this, the present application proposes an electronic digital anti-shake method on software, which uses the rotation angle data of the gyroscope inside the projector to calculate the real three-dimensional attitude of the projector at each frame moment, and simultaneously sets a stable three-dimensional attitude. , calculate the compensation difference from the real 3D attitude to the stable 3D attitude, and compensate each frame of the projected picture of the projector optical machine according to the compensation difference, so as to achieve the effect of picture stabilization. This method can quickly and effectively stabilize the projected image, and is more cost-effective than mechanical image stabilization.

In a first aspect, the present application provides a method for image stabilization of a projection image, including:

Collect gyroscope data;

According to the gyroscope data, calculate the rotation angle;

converting the rotation angle into a first quaternion q;

Calculate the difference value Δq between the first quaternion q and the second quaternion Q, and the second quaternion Q is used to reflect the three-dimensional attitude of the target;

According to the difference value Δq, each pixel is remapped to obtain a stabilized image.

In a possible implementation manner, the collecting gyroscope data includes:

The angular velocities w _x , w _y and w _z of the gyroscope in the x, y and z directions are acquired according to the acquisition interval Δt.

In a possible implementation manner, the collecting gyroscope data further includes:

Record the corresponding system time when the gyroscope data is collected.

In a possible implementation manner, the acquisition interval Δt is determined according to the frame rate p of the projected picture, and Δt=1/p.

In a possible implementation manner, the calculating the rotation angle according to the gyroscope data includes:

θ=w _x1 ×Δt,

ψ=w _z1 ×Δt, or,

where θ,

and ψ are the rotation angles in the x, y and z directions, respectively, w _x1 , w _y1 and w _z1 are the angular velocities of the gyroscope in the x, y and z directions of the previous acquisition in the two adjacent acquisitions, respectively, w _x2 , w _y2 and w _z2 are the angular velocities of the gyroscope in the x, y and z directions of the last acquisition in two adjacent acquisitions, respectively.

θ=w _x1 ×(t ₂ -t ₁ ),

ψ=w _z1 ×(t ₂ -t ₁ ), or,

where θ,

and ψ are the rotation angles in the x, y and z directions, respectively, w _x1 , w _y1 and w _z1 are the angular velocities of the gyroscope in the x, y and z directions of the previous acquisition in the two adjacent acquisitions, respectively, w _x2 , w _y2 and w _z2 are the angular velocities of the gyroscope in the x, y and z directions of the last acquisition in the two adjacent acquisitions, respectively, t ₂ and t ₁ are the latter acquisition in the two adjacent acquisitions, respectively The system time corresponding to the previous acquisition.

In a possible implementation, the converting the rotation angle into the first quaternion q includes calculating the first quaternion q according to the following formula:

where a, b, c and d are the four components of the first quaternion q.

In one possible implementation, the difference value

In a possible implementation manner, the remapping of each pixel to obtain a stabilized image according to the difference value Δq includes:

Transform each pixel as follows: X'=Δq ^-1 XΔq,

Among them, X is the pixel coordinates of the original image, and X' is the pixel coordinates of the stabilized image.

converting the difference value Δq into a rotation matrix R;

Transform each pixel as follows: X'=RX,

In a possible implementation, the rotation matrix

where Δa, Δb, Δc and Δd are the four components of the difference value Δq.

In a possible implementation manner, the target three-dimensional posture is the three-dimensional posture of the projector in a stable state, and the second quaternion Q is calculated after the projector is placed.

In a possible implementation manner, after obtaining a stabilized image by remapping each pixel according to the compensation amount, the method further includes:

Crop the projected image and scale the cropped projected image to the target display size.

In a possible implementation manner, the gyroscope is installed at the structural connection with the optomechanical body.

In a second aspect, the present application provides a projection image stabilization device, comprising:

The gyroscope data acquisition module is used to collect gyroscope data;

a rotation angle calculation module for calculating the rotation angle according to the gyroscope data;

a first quaternion conversion module for converting the rotation angle into a first quaternion q;

a difference value calculation module, configured to calculate the difference value Δq between the first quaternion q and the second quaternion Q, and the second quaternion Q is used to reflect the three-dimensional attitude of the target;

The stabilized image transformation module is configured to remap each pixel to obtain a stabilized image according to the difference value Δq.

In a possible implementation, it also includes:

The second quaternion setting module is used to calculate the second quaternion Q.

In a possible implementation, it also includes:

The picture adjustment module is used for cropping the projection picture and scaling the cropped projection picture to the target display size.

In a third aspect, the present application provides a projection device, the projection device includes a processor and a memory, the memory stores at least one piece of program code, and the at least one piece of program code is loaded and executed by the processor to implement The projection image stabilization method as described in the first aspect or a possible implementation manner of the first aspect.

In a fourth aspect, the present application provides a storage medium, where at least one piece of program code is stored in the storage medium, and the at least one piece of program code is loaded and executed by a processor to implement the first aspect or the possibility of the first aspect The anti-shake method for the projection image described in the implementation manner is implemented.

It should be noted that, in the present application, the projection image stabilization device described in the second aspect, the projection device described in the third aspect, and the storage medium described in the fourth aspect are used to implement the method provided in the above-mentioned first aspect, Therefore, the same beneficial effects as the method described in the first aspect can be achieved, and details are not repeated in the embodiments of the present application.

The present application adopts the method of the gyroscope quaternion to solve the problem of image projection jitter in the application scene of the projector, with fast response and low cost. In addition, the present application installs the gyroscope at the position of the projector opto-mechanical, not just inside the machine, so that the attitude transformation of the opto-mechanical can be estimated more accurately.

Description of drawings

The present application will be described by way of example and with reference to the accompanying drawings, wherein:

FIG. 1 is a flowchart of a method for anti-shake of a projection image provided by an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described The embodiments are only a part of the embodiments of the present application, but not all of the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of the present application. In addition, although the disclosures in this application are presented in terms of one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution individually. The embodiments described below and features in the embodiments may be combined with each other without conflict.

In the embodiments of the present application, words such as "exemplarily" and "for example" are used to represent examples, illustrations or illustrations. Any embodiment or design described in this application as "exemplary" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the word example is intended to present a concept in a concrete way.

Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. "First", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish descriptions, and the meanings of corresponding terms may be the same or different. "Comprises" or "comprising" and similar words mean that the elements or things appearing before the word encompass the elements or things recited after the word and their equivalents, but do not exclude other elements or things.

The technical solutions in the present application will be described below with reference to the accompanying drawings.

The embodiment of the present application mainly uses the gyroscope data as input, and uses the gyroscope data to calculate the motion state of the projector at each moment. A stable state is set, and the motion state of each frame is aligned to the stable state, that is, the difference between the two states is calculated, and the difference is the compensation amount. According to the compensation amount, each frame of projection picture is corrected and displayed, so as to achieve the effect of anti-shake. As shown in FIG. 1 , the anti-shake method for a projection image provided by an embodiment of the present application includes the following steps:

S101. Collect gyroscope data.

When collecting gyroscope data, the collection interval Δt can be selected according to the actual situation. For example, a collection interval may be artificially set according to experience, or the collection interval Δt may be determined according to the frame rate p of the projection image, and Δt=1/p. For example, if the video playback frame rate is 30fps, that is, one frame is played every 33ms, then one frame of gyroscope data is collected every 33ms. The gyroscope data can be the angular velocity w _x , w of the gyroscope in the three directions of x, y and z. _y and w _z . Since the time interval between two adjacent acquisitions may fluctuate near the acquisition interval, when collecting gyroscope data, the system time t corresponding to the gyroscope data at this time can also be recorded to improve the accuracy of the time interval.

In some embodiments, the gyroscope is installed at the structural connection with the optical machine body, which can ensure that the gyroscope can accurately record the motion state of the optical machine in real time when the optical machine sends jitter.

S102. Calculate the rotation angle according to the gyroscope data.

After obtaining the gyroscope data corresponding to each frame of the projected picture in step S101, the angle that the projector has turned for two adjacent acquisitions can be calculated according to the following formula:

θ=w _x1 ×Δt,

ψ=w _z1 ×Δt, or,

If the system time corresponding to the acquisition of the gyroscope data is recorded in step S101, the angle that the projector has turned for two adjacent acquisitions is calculated according to the following formula:

θ=w _x1 ×(t ₂ -t ₁ ),

ψ=w _z1 ×(t ₂ -t ₁ ), or,

where, where θ,

S103. Convert the rotation angle into a first quaternion q.

Calculate the rotation angle θ of two adjacent frames of gyroscope data according to step S102,

and ψ, which can be converted into the corresponding first quaternion q according to the following formula. The first quaternion q represents the pose of the projector in the three-dimensional space at the current moment. The specific conversion formula is as follows:

where a, b, c and d are the four components of the first quaternion q.

S104. Calculate the difference value Δq between the first quaternion q and the second quaternion Q.

A fixed second quaternion Q=(a',b',c',d') is set, and the second quaternion Q is used to reflect the three-dimensional attitude of the target, that is, a stable three-dimensional attitude. It can be set to the three-dimensional posture of the projector in a stable state. After the projector is placed, the calculation can be performed. If the placement position of the projector remains unchanged, the quaternion Q remains unchanged. If the position changes, the quaternion Q needs to be recalculated.

Step S103 can obtain the first quaternion q at each acquisition moment, such as the pose of the projector in the three-dimensional space every 33 ms. This set of quaternions reflects the actual jitter of the projector at each acquisition moment. In order to achieve the anti-shake effect, it is hoped that the pose of the projector at each acquisition moment will change from q to Q, so that the actual pose of the projector at each acquisition moment will be the same value, thus changing from jitter to stability .

The difference value Δq between the first quaternion q and the second quaternion Q represents the compensation amount required to change the projector from the actual rotation attitude q to the ideal and stable rotation attitude Q at each acquisition moment, the calculation The formula is:

in,

Represents quaternion multiplication. Δq means the difference between q and Q, that is, when the frame is in the q posture, it will reach the Q posture after rotating Δq.

S105. According to the difference value Δq, remap each pixel to obtain a stabilized image.

The projected image is compensated by using the quaternion compensation amount Δq calculated in step S104, so as to achieve the effect of image stabilization. Illustratively, each pixel can be transformed in the following two ways:

Method 1: X'=Δq ^-1 XΔq,

Among them, X=(u,r,1,1), u, r are the pixel coordinates of the original image, for example, the coordinates of the upper left point of the image are (0,0), and the coordinates of the lower right corner are (1920, 1080), u, r The value range is between 0-1920 and 0-1080, respectively. After transforming X with the above formula, a new coordinate X' is obtained, that is, the pixel at the position of the final image X' corresponds to the position pixel of the original image X. The final image obtained is the image after compensation, so as to achieve the effect of anti-shake of the projected image.

Method 2: First convert the difference value Δq into the corresponding rotation matrix R, the conversion formula is as follows:

Among them, Δa, Δb, Δc and Δd are the four components of the difference value Δq.

Then use the rotation matrix R to correct the projection picture: X'=RX, at this time, X=(u, r, 1).

After the image is transformed in step S105, the screen will be adjusted, and after the adjustment, a non-visible area outside the screen will appear on the border. This is unacceptable to the user, so the non-visible area displayed by the border needs to be cropped, and only the middle picture is retained. The specific method is as follows: after step S105, the projection picture is cropped, and the cropped projection picture is scaled to the target display size. For example, a picture with a length and width of 80% from the picture center (w/2, h/2) (the value depends on demand) is extracted, and the extracted picture is scaled to the projector display size.

Embodiments of the present application further provide a projection image stabilization device, which is used to implement the projection image stabilization method involved in the above embodiments, which can be implemented by hardware or by executing corresponding software in hardware. The hardware or software includes one or more modules corresponding to the above-mentioned functions, for example, a gyroscope data acquisition module for collecting gyroscope data; a rotation angle calculation module for calculating the rotation angle according to the gyroscope data; A first quaternion conversion module for converting the rotation angle into a first quaternion q; a difference value calculation module for calculating the difference value Δq between the first quaternion q and the second quaternion Q , the second quaternion Q is used to reflect the three-dimensional attitude of the target; it is used to remap each pixel to obtain a stabilized image transformation module of the stabilized image according to the difference value Δq.

In some embodiments, the projection image stabilization apparatus may further include a second quaternion setting module for calculating the second quaternion Q, and may also include a second quaternion setting module for cropping the projection picture, and for cropping the cropped image. A screen adjustment module that scales the projected screen to the target display size.

An embodiment of the present application further provides a projection device, the projection device includes a processor and a memory, the memory stores at least one piece of program code, and the at least one piece of program code is loaded and executed by the processor to implement the above The anti-shake method of the projection picture involved in the embodiment.

Embodiments of the present application further provide a storage medium, where at least one piece of program code is stored in the storage medium, and the at least one piece of program code is loaded and executed by a processor to implement the projection image stabilization method involved in the foregoing embodiments.

It should be understood that, in various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not imply the sequence of execution, some or all of the steps may be executed in parallel or sequentially, and the execution sequence of each process should be based on its functions and It is determined by the internal logic and should not constitute any limitation on the implementation process of the embodiments of the present application.

Those of ordinary skill in the art can realize that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described devices and modules, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. For example, the flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function(s) executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated in one processing unit, or each module may exist physically alone, or two or more modules may be integrated in one unit.

If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution, and the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, a network device or a terminal device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, removable hard disk, ROM, RAM) disk or optical disk and other media that can store program codes.

The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the embodiments of this application and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. The character "/" in this article generally indicates that the related objects are an "or" relationship.

Depending on the context, the words "if" or "if" as used herein may be interpreted as "at" or "when" or "in response to determining" or "in response to detecting." Similarly, the phrases "if determined" or "if detected (the stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detected (the stated condition or event)," depending on the context )" or "in response to detection (a stated condition or event)".

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

A method for image stabilization of projection images, comprising:

Collect gyroscope data;

According to the gyroscope data, calculate the rotation angle;

converting the rotation angle into a first quaternion q;

Calculate the difference value Δq between the first quaternion q and the second quaternion Q, and the second quaternion Q is used to reflect the three-dimensional attitude of the target;

According to the difference value Δq, each pixel is remapped to obtain a stabilized image.
The anti-shake method of a projection image according to claim 1, wherein the collecting gyroscope data comprises:

The angular velocities w x , w y and w z of the gyroscope in the x, y and z directions are acquired according to the acquisition interval Δt.
The anti-shake method for a projection image according to claim 2, wherein the collecting gyroscope data further comprises:

Record the corresponding system time when the gyroscope data is collected.
The anti-shake method for a projection image according to claim 2, wherein the acquisition interval Δt is determined according to the frame rate p of the projection image, and Δt=1/p.
The anti-shake method for a projection image according to claim 2, wherein the calculating the rotation angle according to the gyroscope data comprises:

θ=w x1 ×Δt,
ψ=w z1 ×Δt, or,

where θ,
and ψ are the rotation angles in the x, y and z directions, respectively, w x1 , w y1 and w z1 are the angular velocities of the gyroscope in the x, y and z directions of the previous acquisition in the two adjacent acquisitions, respectively, w x2 , w y2 and w z2 are the angular velocities of the gyroscope in the x, y and z directions of the last acquisition in two adjacent acquisitions, respectively.
The anti-shake method for a projection image according to claim 3, wherein the calculating the rotation angle according to the gyroscope data comprises:

θ=w x1 ×(t 2 -t 1 ),
ψ=w z1 ×(t 2 -t 1 ), or,

where θ,
and ψ are the rotation angles in the x, y and z directions, respectively, w x1 , w y1 and w z1 are the angular velocities of the gyroscope in the x, y and z directions of the previous acquisition in the two adjacent acquisitions, respectively, w x2 , w y2 and w z2 are the angular velocities of the gyroscope in the x, y and z directions of the last acquisition in the two adjacent acquisitions, respectively, t 2 and t 1 are the latter acquisition in the two adjacent acquisitions, respectively The system time corresponding to the previous acquisition.
A projection image stabilization method according to claim 5 or 6, wherein the converting the rotation angle into the first quaternion q comprises calculating the first quaternion q according to the following formula:

where a, b, c and d are the four components of the first quaternion q.
The anti-shake method of a projection image according to claim 1, wherein the difference value is
The anti-shake method for a projection image according to claim 1, wherein, according to the difference value Δq, remapping each pixel to obtain a stabilized image comprises:

Transform each pixel as follows: X'=Δq -1 XΔq,

Among them, X is the pixel coordinates of the original image, and X' is the pixel coordinates of the stabilized image.
The anti-shake method for a projection image according to claim 1, wherein, according to the difference value Δq, remapping each pixel to obtain a stabilized image comprises:

converting the difference value Δq into a rotation matrix R;

Transform each pixel as follows: X'=RX,

Among them, X is the pixel coordinates of the original image, and X' is the pixel coordinates of the stabilized image.
The anti-shake method of a projection image according to claim 10, wherein the rotation matrix
where Δa, Δb, Δc and Δd are the four components of the difference value Δq.
The anti-shake method of a projection image according to claim 1, wherein the three-dimensional posture of the target is the three-dimensional posture of the projector in a stable state, and the second quaternion Q is after the projector is placed Calculated.
The anti-shake method for projection images according to claim 1, characterized in that, after obtaining a stabilized image by remapping each pixel according to the compensation amount, the method further comprises:

Crop the projected image and scale the cropped projected image to the target display size.
The anti-shake method of a projection image according to claim 1, wherein the gyroscope is installed at a structural connection with the optical machine body.
A projection image stabilization device, characterized in that it includes:

The gyroscope data acquisition module is used to collect gyroscope data;

a rotation angle calculation module for calculating the rotation angle according to the gyroscope data;

a first quaternion conversion module for converting the rotation angle into a first quaternion q;

a difference value calculation module, configured to calculate the difference value Δq between the first quaternion q and the second quaternion Q, and the second quaternion Q is used to reflect the three-dimensional attitude of the target;

The stabilized image transformation module is configured to remap each pixel to obtain a stabilized image according to the difference value Δq.
A projection image stabilization device according to claim 15, further comprising:

The second quaternion setting module is used to calculate the second quaternion Q.
A projection image stabilization device according to claim 15, further comprising:

The picture adjustment module is used for cropping the projection picture and scaling the cropped projection picture to the target display size.
A projection device, characterized in that the projection device includes a processor and a memory, and the memory stores at least one piece of program code, and the at least one piece of program code is loaded and executed by the processor, so as to realize the method as claimed in the claims. The anti-shake method for a projection screen according to any one of 1-14.
A storage medium, characterized in that, at least one piece of program code is stored in the storage medium, and the at least one piece of program code is loaded and executed by a processor to realize the projection according to any one of claims 1-14 Image stabilization method.