CN112598728B - Projector attitude estimation, trapezoidal correction method and device, projector and medium - Google Patents
Projector attitude estimation, trapezoidal correction method and device, projector and medium Download PDFInfo
- Publication number
- CN112598728B CN112598728B CN202011542663.8A CN202011542663A CN112598728B CN 112598728 B CN112598728 B CN 112598728B CN 202011542663 A CN202011542663 A CN 202011542663A CN 112598728 B CN112598728 B CN 112598728B
- Authority
- CN
- China
- Prior art keywords
- projector
- image
- posture
- parameters
- projection picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 83
- 238000012937 correction Methods 0.000 title claims abstract description 46
- 230000036544 posture Effects 0.000 claims abstract description 93
- 238000012549 training Methods 0.000 claims abstract description 32
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 238000013527 convolutional neural network Methods 0.000 claims description 31
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 abstract description 19
- 238000013528 artificial neural network Methods 0.000 abstract description 6
- 238000013135 deep learning Methods 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 12
- 230000003287 optical effect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005266 casting Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3185—Geometric adjustment, e.g. keystone or convergence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The application discloses a projector attitude estimation method, a trapezoidal correction method, a device, a projector and a medium. The projector attitude estimation method comprises the following steps: acquiring an image of a projection picture; and obtaining the posture parameters of the projector according to the images and the posture estimation model, wherein the posture estimation model is obtained by training the projection picture images under different postures and the posture parameters of the projector corresponding to the projection picture images. And obtaining the posture parameters of the projector according to the posture estimation method of the projector, and performing trapezoidal correction according to the posture parameters of the projector. The method utilizes the strong nonlinear fitting capability of the neural network and other deep learning methods to obtain a more accurate model, has simple training process, establishes a direct calculation process from an image end to a gesture end, improves the precision of projector gesture estimation, further improves the precision of automatic trapezoid correction, and has simple calculation process. In addition, the model can be automatically corrected in the later period of the user, so that the automatic trapezoid correction precision is further improved, and the user experience is improved.
Description
Technical Field
The present disclosure relates to the field of projection display technologies, and in particular, to a method and apparatus for estimating a projector pose and correcting a trapezoid, a projector, and a medium.
Background
At present, projectors all start to have an automatic trapezoid correction function, the core flow of the projector is usually that a designed feature map is projected, and then the posture to be corrected is calculated based on an optical principle so as to realize automatic trapezoid correction, namely the whole process is perceivable by a user. Meanwhile, the correction based on the optical structure is affected by various factors, so that the calibrated optical structure is invalid, and the situation that a user cannot normally perform automatic trapezoid correction may occur.
In the prior art, a pre-calibrated optical model is used for trapezoidal correction, and feature matching and calculation are performed by matching with a specific feature map. The accuracy of the calibration model and the extraction and calculation accuracy of the features directly influence the accuracy of the final automatic trapezoidal correction. And the calibration and calculation processes are complex, and meanwhile, the model change can directly lead to the failure of the pre-calibration model.
Disclosure of Invention
The existing trapezoidal correction technology is usually realized by matching with a professional characteristic diagram, and the effect and the precision of trapezoidal correction depend on the precision of a characteristic algorithm and the calibration precision of an optical model. The automatic trapezoid correction process mainly comprises the steps of extracting and positioning characteristic points, and then calculating based on a calibrated optical model, wherein the accuracy and algorithm robustness of each step in the process can influence the final effect. In view of this, the present application provides a method, a device, a projector and a medium for realizing projector attitude estimation based on deep learning such as a deep convolutional neural network, and further realizing automatic trapezoidal correction according to projector attitude, wherein the core is to fit a nonlinear optical model by utilizing strong nonlinear fitting capability of a deep learning model such as a neural network, and realize automatic trapezoidal correction.
In a first aspect, the present application provides a projector attitude estimation method, including:
acquiring an image of a projection picture;
and obtaining the posture parameters of the projector according to the images and the posture estimation model, wherein the posture estimation model is obtained by training the projection picture images under different postures and the posture parameters of the projector corresponding to the projection picture images.
In one possible implementation manner, the obtaining the pose parameter of the projector according to the image and the pose estimation model includes:
inputting the image into the attitude estimation model, and outputting the attitude parameters of the projector by the attitude estimation model.
In one possible implementation manner, the obtaining the pose parameter of the projector according to the image and the pose estimation model includes:
binarizing the image to generate a binary image of the image;
and inputting the binary image of the image into the attitude estimation model, and outputting the attitude parameters of the projector by the attitude estimation model.
In one possible implementation manner, the training method of the gesture estimation model includes:
acquiring a projection picture image and corresponding projector posture parameters during forward projection of a projector;
acquiring a projection picture image and corresponding projector posture parameters when the projector is laterally projected;
acquiring a projection picture image when only a pitch angle exists in a projector and corresponding projector attitude parameters;
obtaining a projection picture image when a horizontal rotation angle and a pitch angle exist in a projector and corresponding projector attitude parameters;
and forming a training set by using the obtained projection picture image and the corresponding projector posture parameter to train the convolutional neural network, so as to obtain the posture estimation model.
In one possible implementation manner, the training convolutional neural network using the acquired projection picture image and the corresponding projector pose parameters includes:
and training the convolutional neural network by taking the obtained projection picture image as input data and taking projector posture parameters corresponding to the projection picture image as supervision data.
In one possible implementation manner, the training convolutional neural network using the acquired projection picture image and the corresponding projector pose parameters includes:
binarizing the obtained projection picture image to generate a binary image of the projection picture image;
and training the convolutional neural network by taking the binary image of the projection picture image as input data and taking the projector posture parameter corresponding to the projection picture image as supervision data.
In one possible implementation manner, the binarization method includes:
preprocessing an image to be binarized to obtain a binary image with the same resolution as the image to be binarized.
In one possible implementation manner, the preprocessing the image to be binarized to obtain a binary image with the same resolution as the image to be binarized includes:
graying the image to be binarized to obtain a gray image;
denoising the gray level image;
carrying out gray histogram statistics on the denoised gray image;
and carrying out fixed threshold binarization according to gray level histogram statistics to generate a binary image with the same resolution as the image to be binarized.
In one possible implementation, the fixed threshold is a gray value corresponding to a median value of the gray histogram statistics.
In one possible implementation, the pose estimation model is a trained convolutional neural network.
In one possible implementation, the attitude parameters include a horizontal rotation angle and a vertical pitch angle.
In a second aspect, the present application provides a projector trapezoidal correction method, including:
acquiring attitude parameters of a projector, wherein the attitude parameters are obtained according to the first aspect or the projector attitude estimation method in a possible implementation manner of the first aspect;
and carrying out trapezoidal correction according to the attitude parameters.
In one possible implementation, the method further includes:
receiving a gesture estimation model retraining instruction sent by a user;
obtaining projection picture images and corresponding projector posture parameters when a user puts the projector according to different postures;
and retraining the posture estimation model according to the acquired projection picture images when the projector is placed according to different postures and the corresponding projector posture parameters.
In a third aspect, the present application provides a projector attitude estimation apparatus, comprising:
the image acquisition module is used for acquiring an image of the projection picture;
and the gesture parameter calculation module is used for obtaining gesture parameters of the projector according to the images and gesture estimation models, wherein the gesture estimation models are obtained by training projection picture images under different gestures and corresponding gesture parameters of the projector.
In one possible implementation manner, the obtaining the pose parameter of the projector according to the image and the pose estimation model includes:
inputting the image into the attitude estimation model, and outputting the attitude parameters of the projector by the attitude estimation model.
In one possible implementation manner, the obtaining the pose parameter of the projector according to the image and the pose estimation model includes:
binarizing the image to generate a binary image of the image;
and inputting the binary image of the image into the attitude estimation model, and outputting the attitude parameters of the projector by the attitude estimation model.
In a fourth aspect, the present application provides a projector trapezoidal correction device, comprising:
the system comprises a gesture parameter acquisition module, a gesture parameter estimation module and a gesture parameter analysis module, wherein the gesture parameter acquisition module is used for acquiring gesture parameters of a projector, and the gesture parameters are obtained according to the first aspect or a gesture estimation method of the projector in a possible implementation manner of the first aspect;
and the trapezoid correction module is used for performing trapezoid correction according to the attitude parameters.
In one possible implementation, the method further includes:
the retraining instruction receiving module is used for receiving an attitude estimation model retraining instruction sent by a user;
the user parameter acquisition module is used for acquiring projection picture images and corresponding projector posture parameters when a user puts the projector according to different postures;
and the model retraining module is used for retraining the posture estimation model according to the acquired projection picture images and corresponding projector posture parameters when the projector is placed according to different postures by the user.
In a fifth aspect, the present application provides a projector comprising a processor and a memory, the memory having stored therein at least one piece of program code, the at least one piece of program code being loaded and executed by the processor to implement the projector pose estimation method as described in the first aspect or a possible implementation of the first aspect.
In a sixth aspect, the present application provides a projector comprising a processor and a memory, the memory having stored therein at least one piece of program code, the at least one piece of program code being loaded and executed by the processor to implement the projector keystone correction method as described in the second aspect or a possible implementation of the second aspect.
In a seventh aspect, the present application provides a storage medium having stored therein at least one program code loaded and executed by a processor to implement the projector pose estimation method as described in the first aspect or a possible implementation of the first aspect or the projector trapezoidal correction method as described in the second aspect or a possible time slot manner of the second aspect.
It should be noted that, the projector posture estimating apparatus according to the third aspect and the projector according to the fifth aspect of the present application are configured to implement the method provided by the first aspect, the projector trapezoidal correction apparatus according to the fourth aspect and the projector according to the sixth aspect are configured to implement the method provided by the second aspect, and the storage medium according to the seventh aspect is configured to implement the method provided by the first aspect or the second aspect, so that the same beneficial effects as those of the method provided by the first aspect or the second aspect can be achieved, and embodiments of the present application are not repeated.
According to the method, projector posture estimation is achieved based on the deep learning method such as the convolutional neural network, automatic trapezoid correction is achieved according to the projector posture, a more accurate model can be obtained by utilizing the strong nonlinear fitting capacity of the deep learning method such as the neural network, the training process is simple, the direct calculation process from an image end to a posture end is established, the accuracy of projector posture estimation is improved, the accuracy of automatic trapezoid correction is improved, and the calculation process is simple. In addition, the model can be automatically corrected in the later period of the user, so that the automatic trapezoid correction precision is further improved, and the user experience is improved.
Drawings
The present application will now be described by way of example and with reference to the accompanying drawings, in which:
fig. 1 is a flowchart of a projector attitude estimation method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a convolutional neural network according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions in the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application. Furthermore, while the disclosure has been presented in terms of an exemplary embodiment or embodiments, it should be understood that various aspects of the disclosure can be practiced separately from the disclosure in a complete subject matter. The following embodiments and features of the embodiments may be combined with each other without conflict.
In the embodiments of the present application, words such as "exemplary," "for example," and the like are used to indicate by way of example, illustration, or description. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the term use of an example is intended to present concepts in a concrete fashion.
Unless otherwise defined, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. The terms "first," "second," and the like, as used herein, do not denote any order, quantity, or importance, but rather are used to distinguish one from another, and the corresponding terms may or may not have the same meaning. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items.
The technical solutions in the present application will be described below with reference to the accompanying drawings.
In the following embodiments, the convolutional neural network is taken as an example for the detailed description, but the present application is not limited to the construction of a model by using the convolutional neural network, and other deep learning methods can be used.
Fig. 1 is a flowchart of a projector attitude estimation method according to an embodiment of the present application. As shown in fig. 1, the projector attitude estimation method includes the steps of:
s101, acquiring an image of a projection picture.
In some embodiments, the image of the projector screen in the actual use situation of the user can be acquired through a camera, and the camera can be arranged inside the projector or independent of the outside of the projector.
S102, acquiring the posture parameters of the projector according to the images and the posture estimation model, wherein the posture estimation model is trained by utilizing projection picture images under different postures and corresponding posture parameters of the projector.
The construction of the pose estimation model by using the convolutional neural network is described in detail herein as an example.
First, a convolutional neural network is built and trained.
1) A convolutional neural network is constructed, as shown in fig. 2, and comprises an input layer, an intermediate layer and an output layer. The input layer can be the obtained projection picture image or a binary image obtained by binarizing the obtained projector picture image, and the binary image is used as input, so that the training of a model can be simplified; the middle layer is formed by alternately forming a convolution layer and a pooling layer, the convolution kernel size of the convolution layer can be selected according to the requirement, the pooling layer adopts a maximum pooling MaxPool mode, the layer number of the middle layer is selected to consider the balance between training efficiency and a final result, for example, 5-9 layers (one convolution layer and one pooling layer are one middle layer), and finally two full-connection layers are connected; the output layer is the pose parameter of the projector. Illustratively, the pose parameters of the projector include a horizontal rotation angle α and a vertical pitch angle β of the projector.
2) Collecting a training set: in order to enable the trained neural network model to have better nonlinear fitting capability and generalization capability, as many real-time images of the projector under different postures as possible are acquired, and current posture parameters are correspondingly acquired, and the acquisition of the posture parameters is realized through a built-in sensor of the projector (such as a six-way gyroscope or a three-axis inclination sensor), and the application takes the gyroscope as an example. The specific implementation process comprises the following steps:
a. forward projection image dataset acquisition: and (3) forward projecting a projector picture onto a white wall, synchronously acquiring images by a built-in camera of the projector, and simultaneously acquiring the attitude parameters of the current projector by a built-in gyroscope of the projector. Repeating the step a, and collecting images at different distances.
b. Side projection image dataset acquisition: and obliquely casting the projector picture on a white wall, synchronously acquiring images by a built-in camera of the projector, and simultaneously acquiring the attitude parameters of the current projector by a built-in gyroscope of the projector. And b, repeating the step b, and collecting images at different distances and different oblique projection angles.
c. Pitch image dataset acquisition: and projecting the picture of the projector onto the white wall at a certain pitching angle, synchronously acquiring images by a built-in camera of the projector, and simultaneously acquiring the attitude parameters of the current projector by a built-in gyroscope of the projector. And c, repeating the step, and collecting images at different distances and different pitching angles.
d. Free six-way angle dataset acquisition: and projecting the picture of the projector onto the white wall at any pitching and rotating angle, synchronously acquiring images by a built-in camera of the projector, and simultaneously acquiring the attitude parameters of the current projector by a built-in gyroscope of the projector. And d, repeating the step, and collecting images at different distances and any postures.
And (3) acquiring multiple groups of data (such as 20-50 groups, more data and better model pre-training effect) from each group of a, b, c and d to form a training set.
If the input layer of the convolutional neural network is a projection picture image, the acquired projection picture image is directly used as input data, projector posture parameters corresponding to the projection picture image are used as supervision data, and the convolutional neural network is trained. If the input layer of the convolutional neural network is a binary image of a projector picture image, the binary image of the projection picture image is generated by binarizing the projection picture image, then the binary image of the projection picture image is used as input data, projector posture parameters corresponding to the projection picture image are used as supervision data, and the convolutional neural network is trained.
3) And (5) preprocessing an image. Preferably, the binarization process includes a certain preprocessing of the projected picture image to obtain a binary image with the same resolution as the projector picture image, and the binarization process can be adapted to cameras with any resolution without up-down sampling operation.
A. Graying: the grey scale of the projection picture image can be realized by using the opencv:cvtColor () function.
B. Denoising an image: since most of the actually acquired images exist as gaussian white noise, gaussian filtering can be used to remove the image noise.
C. Gray histogram statistics: and carrying out gray level histogram statistics on the denoised gray level image, for example, dividing the gray level step length into 255 gray level histograms, and using opencv: calcHist () to realize the statistics and calculation of the image gray level histograms.
D. Binarization: because the projector is an active light source, the brightness of a projection picture is higher than the background brightness in theory, so that the gray value corresponding to the median of the histogram statistics can be selected as the gray threshold value thresd for binarization according to the histogram statistics obtained by the calculation in the step C, the binarization algorithm is faster, and the stability and the noise resistance are better. I.e.
thresd＝I median
Wherein thresd is a binary gray threshold, I median And counting the gray values corresponding to the median for the histogram.
The binarization is directly realized by using a fixed threshold binarization mode:
where I is the gray image pixel gray value.
Through the steps, the acquired projection picture image is processed into a binary image representing a projection area with the same resolution, and the binary image is used as the input of the convolutional neural network.
4) Convolutional neural network training. Here, the input of the convolutional neural network is exemplified as a binary image.
aa. the binary image obtained after preprocessing and the corresponding posture parameters are used as training sets to carry out neural network training. The loss function employs a standard Mean Square Error (MSE) loss function, i.e
Wherein r is j Is the expected output (labeling the tag, i.e. the actual gesture parameters), a j Is the predicted output of the neural network (i.e., the attitude parameter of the network output), and E is the residual value.
The bb. network training adopts traditional back propagation to iterate and optimize parameters, and adopts a learning rate attenuation method, for example, the attenuation period is 1000 times, and the learning rate is changed into 1/2 of the original one; setting certain residual and iteration conditions, e.g. setting residual termination conditions to e < 10 -5 The iteration number is 10000, and the neural network training is completed as long as any one condition is satisfied.
Secondly, carrying out projector attitude estimation by using the convolutional neural network after training to obtain attitude parameters.
The trained convolutional neural network can be used as a posture estimation model to estimate the posture of the projector. If the input of the convolutional neural network is a projection picture image, the image is input into the gesture estimation model, and the gesture estimation model outputs the gesture parameters of the projector. If the input of the convolutional neural network is a binary image, binarizing the projection picture image, wherein the process is as in the step 3), and generating a binary image of the projection picture image; and inputting the binary image into the attitude estimation model, and outputting the attitude parameters of the projector by the attitude estimation model.
After the posture parameters of the projector are obtained by the projector posture estimation method, trapezoidal correction can be carried out according to the obtained projector posture parameters. If the obtained attitude parameters are transmitted into a projector trapezoidal correction module, trapezoidal correction of the projector picture can be completed, and a final trapezoidal correction effect is obtained.
In some embodiments, the interface for retraining the posture estimation model can be opened to the user, and if the user is not satisfied with the trapezoidal correction effect, the retraining of the posture estimation model can be started, and the user only needs to put the projector for a plurality of times at will. After receiving a gesture estimation model retraining instruction sent by a user, the projector acquires a projection picture image and corresponding projector gesture parameters when the user puts the projector according to different gestures; and retraining the posture estimation model according to the acquired projection picture images when the projector is placed according to different postures and the corresponding projector posture parameters. The specific method of training may refer to the above training steps, and will not be described herein.
The embodiment of the application also provides a projector posture estimation device, which is used for realizing the projector posture estimation method related to the embodiment, and can be realized through hardware or can be realized through executing corresponding software through hardware. The hardware or software includes one or more modules corresponding to the above functions, for example, an image acquisition module for acquiring an image of a projection screen; and a posture parameter calculation module used for obtaining the posture parameters of the projector according to the image and the posture estimation model.
The embodiment of the application also provides a projector trapezoidal correction device, which is used for realizing the trapezoidal correction method related to the embodiment, and can be realized by hardware or can be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above functions, for example, an attitude parameter acquisition module for acquiring an attitude parameter of the projector; and a trapezoidal correction module for trapezoidal correction according to the attitude parameters.
The embodiment of the application also provides a projector, which comprises a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to realize the projector posture estimation method related to the embodiment.
The embodiment of the application also provides a projector, which comprises a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to realize the projector trapezoidal correction method related to the embodiment.
The present embodiment also provides a storage medium having at least one program code stored therein, the at least one program code being loaded and executed by a processor to implement the projector attitude estimation method or the projector trapezoidal correction method according to the above embodiment.
It should be understood that, in various embodiments of the present application, the sequence number of each process does not mean that the sequence of execution is sequential, and some or all of the steps may be executed in parallel or sequentially, where the execution sequence of each process should be determined by its functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus and modules described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. For example, the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in each embodiment of the present application may be integrated in one processing unit, or each module may exist alone physically, or two or more modules may be integrated in one unit.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device or a terminal device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: u disk, removable hard disk, ROM, RAM) disk or optical disk, etc.
The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items. The character "/" herein generally indicates that the associated object is an "or" relationship.
The word "if" or "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to detection", depending on the context. Similarly, the phrase "if determined" or "if detected (stated condition or event)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event)" or "in response to detection (stated condition or event), depending on the context.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (14)
1. A projector attitude estimation method, characterized by comprising:
acquiring an image of a projection picture, wherein the projection picture is a projection picture under the actual use condition of a user;
obtaining the posture parameters of a projector according to the images and a posture estimation model, wherein the posture estimation model is obtained by training the projection picture images under different postures and the posture parameters of the projector corresponding to the projection picture images, and the posture parameters comprise a horizontal rotation angle and a vertical pitching angle;
the obtaining the posture parameters of the projector according to the image and the posture estimation model comprises the following steps:
inputting the image into the gesture estimation model, and outputting gesture parameters of a projector by the gesture estimation model; or,
binarizing the image to generate a binary image of the image;
inputting the binary image of the image into the attitude estimation model, and outputting the attitude parameters of a projector by the attitude estimation model;
the method further comprises the steps of:
receiving a gesture estimation model retraining instruction sent by a user;
obtaining projection picture images and corresponding projector posture parameters when a user puts the projector according to different postures;
and retraining the posture estimation model according to the acquired projection picture images when the projector is placed according to different postures and the corresponding projector posture parameters.
2. The projector pose estimation method according to claim 1, wherein said pose estimation model training method comprises:
acquiring a projection picture image and corresponding projector posture parameters during forward projection of a projector;
acquiring a projection picture image and corresponding projector posture parameters when the projector is laterally projected;
acquiring a projection picture image when only a pitch angle exists in a projector and corresponding projector attitude parameters;
obtaining a projection picture image when a horizontal rotation angle and a pitch angle exist in a projector and corresponding projector attitude parameters;
and forming a training set by using the obtained projection picture image and the corresponding projector posture parameter to train the convolutional neural network, so as to obtain the posture estimation model.
3. The projector attitude estimation method according to claim 2, wherein training the convolutional neural network using the acquired projection screen image and the corresponding projector attitude parameters comprises:
and training the convolutional neural network by taking the obtained projection picture image as input data and taking projector posture parameters corresponding to the projection picture image as supervision data.
4. The projector attitude estimation method according to claim 2, wherein training the convolutional neural network using the acquired projection screen image and the corresponding projector attitude parameters comprises:
binarizing the obtained projection picture image to generate a binary image of the projection picture image;
and training the convolutional neural network by taking the binary image of the projection picture image as input data and taking the projector posture parameter corresponding to the projection picture image as supervision data.
5. The projector attitude estimation method according to claim 1 or 4, characterized in that the binarization method comprises:
preprocessing an image to be binarized to obtain a binary image with the same resolution as the image to be binarized.
6. The projector attitude estimation method according to claim 5, wherein preprocessing the image to be binarized to obtain a binary image of the same resolution as the image to be binarized comprises:
graying the image to be binarized to obtain a gray image;
denoising the gray level image;
carrying out gray histogram statistics on the denoised gray image;
and carrying out fixed threshold binarization according to gray level histogram statistics to generate a binary image with the same resolution as the image to be binarized.
7. The projector pose estimation method according to claim 6, wherein said fixed threshold is a gray value corresponding to a median value of said gray histogram statistics.
8. The projector pose estimation method of claim 1 wherein said pose estimation model is a trained convolutional neural network.
9. A projector trapezoidal correction method, characterized by comprising:
acquiring attitude parameters of a projector, wherein the attitude parameters are obtained according to the projector attitude estimation method of any one of claims 1 to 8;
and carrying out trapezoidal correction according to the attitude parameters.
10. A projector attitude estimation apparatus, characterized by comprising:
the image acquisition module is used for acquiring an image of a projection picture, wherein the projection picture is a projection picture under the actual use condition of a user;
the gesture parameter calculation module is used for obtaining gesture parameters of the projector according to the images and gesture estimation models, the gesture estimation models are obtained by training projection picture images under different gestures and corresponding gesture parameters of the projector, and the gesture parameters comprise a horizontal rotation angle and a vertical pitching angle;
the obtaining the posture parameters of the projector according to the image and the posture estimation model comprises the following steps:
inputting the image into the gesture estimation model, and outputting gesture parameters of a projector by the gesture estimation model; or,
binarizing the image to generate a binary image of the image;
inputting the binary image of the image into the attitude estimation model, and outputting the attitude parameters of a projector by the attitude estimation model;
the apparatus further comprises:
the retraining instruction receiving module is used for receiving an attitude estimation model retraining instruction sent by a user;
the user parameter acquisition module is used for acquiring projection picture images and corresponding projector posture parameters when a user puts the projector according to different postures;
and the model retraining module is used for retraining the posture estimation model according to the acquired projection picture images and corresponding projector posture parameters when the projector is placed according to different postures by the user.
11. A projector trapezoidal correction device, characterized by comprising:
a posture parameter acquisition module for acquiring a posture parameter of a projector, the posture parameter being obtained according to the projector posture estimation method of any one of claims 1 to 8;
and the trapezoid correction module is used for performing trapezoid correction according to the attitude parameters.
12. A projector comprising a processor and a memory, wherein the memory has stored therein at least one program code that is loaded and executed by the processor to implement the projector pose estimation method according to any of claims 1-8.
13. A projector comprising a processor and a memory, wherein the memory has stored therein at least one program code that is loaded and executed by the processor to implement the projector keystone correction method of claim 9.
14. A storage medium having stored therein at least one program code loaded and executed by a processor to implement the projector pose estimation method according to any of claims 1-8 or the projector trapezoidal correction method according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011542663.8A CN112598728B (en) | 2020-12-23 | 2020-12-23 | Projector attitude estimation, trapezoidal correction method and device, projector and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011542663.8A CN112598728B (en) | 2020-12-23 | 2020-12-23 | Projector attitude estimation, trapezoidal correction method and device, projector and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112598728A CN112598728A (en) | 2021-04-02 |
| CN112598728B true CN112598728B (en) | 2024-02-13 |
Family
ID=75200564
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011542663.8A Active CN112598728B (en) | 2020-12-23 | 2020-12-23 | Projector attitude estimation, trapezoidal correction method and device, projector and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112598728B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793250A (en) * | 2021-08-13 | 2021-12-14 | 北京迈格威科技有限公司 | Pose evaluation method, pose determination method, corresponding device and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105144196A (en) * | 2013-02-22 | 2015-12-09 | 微软技术许可有限责任公司 | Method and device for calculating a camera or object pose |
| CN109099915A (en) * | 2018-06-27 | 2018-12-28 | 未来机器人（深圳）有限公司 | Method for positioning mobile robot, device, computer equipment and storage medium |
| CN109698944A (en) * | 2017-10-23 | 2019-04-30 | 深圳市Tcl高新技术开发有限公司 | View field's bearing calibration, projection device and computer readable storage medium |
| CN110225321A (en) * | 2019-05-27 | 2019-09-10 | 广州市讯码通讯科技有限公司 | The training sample data of keystone obtain system and method |
| CN110334605A (en) * | 2019-06-06 | 2019-10-15 | 平安科技（深圳）有限公司 | Gesture identification method, device, storage medium and equipment neural network based |
| CN110503689A (en) * | 2019-08-30 | 2019-11-26 | 清华大学 | Attitude prediction method, model training method and device |
| CN110859630A (en) * | 2019-11-26 | 2020-03-06 | 塔普翊海（上海）智能科技有限公司 | Posture corrector based on AR technology and correction method thereof |
| CN111033524A (en) * | 2017-09-20 | 2020-04-17 | 奇跃公司 | Personalized neural network for eye tracking |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10235771B2 (en) * | 2016-11-11 | 2019-03-19 | Qualcomm Incorporated | Methods and systems of performing object pose estimation |
| US20200021787A1 (en) * | 2017-03-23 | 2020-01-16 | Sony Corporation | Projector, projection method, image processing system, and method |
| US20200380723A1 (en) * | 2019-05-30 | 2020-12-03 | Seiko Epson Corporation | Online learning for 3d pose estimation |
-
2020
- 2020-12-23 CN CN202011542663.8A patent/CN112598728B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105144196A (en) * | 2013-02-22 | 2015-12-09 | 微软技术许可有限责任公司 | Method and device for calculating a camera or object pose |
| CN111033524A (en) * | 2017-09-20 | 2020-04-17 | 奇跃公司 | Personalized neural network for eye tracking |
| CN109698944A (en) * | 2017-10-23 | 2019-04-30 | 深圳市Tcl高新技术开发有限公司 | View field's bearing calibration, projection device and computer readable storage medium |
| CN109099915A (en) * | 2018-06-27 | 2018-12-28 | 未来机器人（深圳）有限公司 | Method for positioning mobile robot, device, computer equipment and storage medium |
| CN110225321A (en) * | 2019-05-27 | 2019-09-10 | 广州市讯码通讯科技有限公司 | The training sample data of keystone obtain system and method |
| CN110334605A (en) * | 2019-06-06 | 2019-10-15 | 平安科技（深圳）有限公司 | Gesture identification method, device, storage medium and equipment neural network based |
| CN110503689A (en) * | 2019-08-30 | 2019-11-26 | 清华大学 | Attitude prediction method, model training method and device |
| CN110859630A (en) * | 2019-11-26 | 2020-03-06 | 塔普翊海（上海）智能科技有限公司 | Posture corrector based on AR technology and correction method thereof |
Non-Patent Citations (2)
| Title |
|---|
| X-Ray PoseNet: 6 DoF Pose Estimation for Mobile X-Ray Devices;Mai Bui等;《2017 IEEE Winter Conference on Applications of Computer Vision》;第1036、1038页 * |
| 可估量曲面的自主感知与多投影校正技术研究;杨帆;《中国博士学位论文全文数据库 信息科技辑》(第3期);第I138-30页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112598728A (en) | 2021-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11798132B2 (en) | Image inpainting method and apparatus, computer device, and storage medium | |
| CN111667520B (en) | Registration method and device for infrared image and visible light image and readable storage medium | |
| WO2021174939A1 (en) | Facial image acquisition method and system | |
| CN109376631B (en) | Loop detection method and device based on neural network | |
| CN110866871A (en) | Text image correction method and device, computer equipment and storage medium | |
| CN109640066B (en) | Method and device for generating high-precision dense depth image | |
| CN112927279A (en) | Image depth information generation method, device and storage medium | |
| CN111680573B (en) | Face recognition method, device, electronic equipment and storage medium | |
| CN110795975B (en) | Face false detection optimization method and device | |
| CN112598728B (en) | Projector attitude estimation, trapezoidal correction method and device, projector and medium | |
| CN113888431A (en) | Training method and device of image restoration model, computer equipment and storage medium | |
| CN117197388A (en) | Live-action three-dimensional virtual reality scene construction method and system based on generation of antagonistic neural network and oblique photography | |
| CN112070181B (en) | Image stream-based cooperative detection method and device and storage medium | |
| CN111784660B (en) | Method and system for analyzing frontal face degree of face image | |
| CN111353325A (en) | Key point detection model training method and device | |
| CN113706472A (en) | Method, device and equipment for detecting road surface diseases and storage medium | |
| CN115937991A (en) | Human body tumbling identification method and device, computer equipment and storage medium | |
| CN114638921B (en) | Motion capture method, terminal device, and storage medium | |
| CN113658274B (en) | Automatic individual spacing calculation method for primate population behavior analysis | |
| CN115393423A (en) | Target detection method and device | |
| CN114663683A (en) | Underwater target detection method based on spatial feature self-supervision | |
| CN110971895B (en) | Video jitter detection method and device | |
| CN111127327B (en) | Picture inclination detection method and device | |
| CN113920196A (en) | Visual positioning method and device and computer equipment | |
| CN108921805B (en) | Image and video haze removing method, computer device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 610000 building 4, zone a, Tianfu Software Park, No. 1129, shijicheng Road, high tech Zone, Chengdu, Sichuan Applicant after: Jimi Technology Co.,Ltd. Address before: No.2, floor 2, unit 1, building 4, Tianfu Software Park, no.1129, shijicheng Road, hi tech Zone, Chengdu, Sichuan 610041 Applicant before: Chengdu Jimi Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |