CN110244840A - Image processing method, relevant device and computer storage medium - Google Patents
Image processing method, relevant device and computer storage medium Download PDFInfo
- Publication number
- CN110244840A CN110244840A CN201910443434.1A CN201910443434A CN110244840A CN 110244840 A CN110244840 A CN 110244840A CN 201910443434 A CN201910443434 A CN 201910443434A CN 110244840 A CN110244840 A CN 110244840A
- Authority
- CN
- China
- Prior art keywords
- moment
- image
- electronic equipment
- wear
- type electronic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Human Computer Interaction (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The embodiment of the invention discloses image processing method, relevant device and computer storage mediums, the method comprise the steps that wear-type electronic equipment obtains the true picture that camera acquires at the first moment, re-projection matrix is determined according to the posture information at the first moment, then true picture is handled using the re-projection matrix, to obtain wear-type electronic equipment in the image to be displayed at the second moment.Using the embodiment of the present invention, it is able to solve the problems such as time delay present in conventional images displaying scheme is larger, user experience is poor.
Description
Technical field
The present invention relates to field of communication technology more particularly to image processing methods, relevant device and computer storage medium.
Background technique
It wears in electronic equipment, augmented reality (augmented reality, AR) device can watch real generation in user
While boundary's scene, virtual image is displayed for a user.User can also interact to realize augmented reality with virtual image
Effect.
Currently, wear-type electronic equipment realizes the display of composograph using frame as shown in Figure 1.Such as Fig. 1, wear
Formula electronic equipment acquires true picture in the real world by camera, to true picture and is worn by algorithm processing module
The current posture information of formula electronic equipment is handled, and to generate virtual image, then will really be schemed by image synthesis module
Picture and virtual image synthesize composograph to be shown, and then the composograph is projected to the display of wear-type electronic equipment
So that user checks on device.
In practice, it has been found that the display that collects composograph of the wear-type electronic equipment from true picture, whole process
Consumed time delay is larger, and user experience is bad.
Summary of the invention
The embodiment of the invention discloses image processing method, relevant device and computer storage mediums, are able to solve existing
Image show present in time delay it is larger the problems such as.
In a first aspect, the embodiment of the present invention, which discloses, provides a kind of image processing method, it is applied to wear-type electronic equipment,
The described method includes: wear-type electronic equipment obtains the true picture that camera acquires at the first moment, which can be
The image for the required display that camera acquires under real-world coordinates system, the visual angle of the true picture are typically larger than wear-type electricity
The visual angle of the image shown needed for sub- equipment.In other words, the visual angle of camera collection image is greater than wear-type electronic equipment institute
It need to show the visual angle of image.Further, wear-type electronic equipment can be obtained from the posture information in the first moment, posture letter
The athletic posture for reflecting wear-type electronic equipment at the first moment is ceased, which includes but is not limited to wear-type electricity
Attitude angle (such as the angle of depression, elevation angle etc.), position, movement velocity, angular velocity of satellite motion or other letters relevant to posture of sub- equipment
Breath.Wear-type electronic equipment can calculate according to the posture information at the first moment and obtain re-projection matrix, and then utilize the re-projection
Matrix is handled true picture to obtain wear-type electronic equipment in the image to be displayed at the second moment.Wherein, re-projection
Matrix can be used for reflecting the image of wear-type electronic equipment display needed for the first moment and the image of display needed for the second moment
Between mapping relations, which was later than for the first moment.
By implementing the embodiment of the present invention, it is able to solve and exists in the prior art that time delay is larger, user experience is bad etc. and ask
Topic promotes user experience so as to promote the efficiency that image is shown.
With reference to first aspect, in some possible embodiments, wear-type electronic equipment can be according to the posture at the first moment
Information predicts the posture information at the second moment, and then according to the first moment and the second moment respective posture information, calculates and obtain
Re-projection matrix.
With reference to first aspect, in some possible embodiments, the posture information of wear-type electronic equipment may include movement
Speed and angular velocity of satellite motion.Wear-type electronic equipment can calculate second according to the movement velocity and angular velocity of satellite motion at the first moment
The relative rotation angle of wear-type electronic equipment and relative translation vector when moment is relative to the first moment.Further, wear-type
Electronic equipment calculates according to the relative rotation angle and relative translation vector of calculating and obtains re-projection matrix.
With reference to first aspect, in some possible embodiments, wear-type electronic equipment can according to relative rotation angle and
The direction of rotation of relative rotation angle, calculates and obtains spin matrix, which was used to indicate for the second moment relative to first
For moment, the relative rotation situation of wear-type electronic equipment.Further, wear-type electronic equipment is according to the relative translation vector
And spin matrix, it calculates and obtains re-projection matrix.
With reference to first aspect, in some possible embodiments, wear-type electronic equipment can carry out true picture multi-thread
Journey processing, to obtain processing image.Further wear-type electronic equipment can utilize re-projection matrix, carry out figure to the processing image
As mapping, to obtain wear-type electronic equipment in the image to be displayed at the second moment.
With reference to first aspect, in some possible embodiments, wear-type electronic equipment carries out multithreading to true picture
The specific embodiment of processing is as follows: wear-type electronic equipment pre-processes true picture in main thread, pre- to obtain
Treated true picture, the pretreatment include but is not limited to distortion processing, correction process, translation processing, scaling processing or
Other image procossings etc..Further wear-type electronic equipment can obtain the second image, example based on the pretreated true picture
Dimension-reduction treatment such as is carried out to pretreated true picture, which may include but be not limited to drop port number processing (such as list
Channel image is in advance etc.), resolution decreasing processing (such as down-sampling of image etc.) or other processing etc..In turn, wear-type electronics
Equipment can carry out synthesis processing by pretreated true picture and each from the respective virtual image of thread, to obtain processing figure
Picture.
With reference to first aspect, in some possible embodiments, main thread and arbitrarily from mutually indepedent between thread, any
Two run from also mutually indepedent between thread.In other words, main thread and multi-threading parallel process is each supported from thread,
Independently of each other, it is independent of each other.
With reference to first aspect, in some possible embodiments, the second moment is after the first moment of distance preset duration
Time point, the preset duration are that wear-type electronic equipment is handled true picture the duration to show on a display screen.It changes
Sentence is talked about, and the when a length of true picture between the second moment and the first moment is processed into support and shows on wear-type electronic equipment
The duration of the image shown.
Second aspect, the embodiment of the present invention, which discloses, provides a kind of wear-type electronic equipment, including for executing above-mentioned the
The functional unit of the method for one side.
The third aspect, the embodiment of the invention provides another wear-type electronic equipment, including memory, communication interface and
The processor coupled with the memory and communication interface；For storing instruction, the processor is for executing for the memory
Described instruction, the communication interface are used under the control of the processor and other equipment (concretely user equipment, vehicle-mounted
Equipment or other network equipments, such as server) it is communicated；Wherein, it is executed when the processor executes described instruction above-mentioned
The method of first aspect description.
Fourth aspect, provides a kind of computer readable storage medium, and the computer-readable recording medium storage is used
In the program code of image procossing.Said program code includes the instruction for executing the method for above-mentioned first aspect description.
5th aspect, provides a kind of computer program product including instruction, when run on a computer, so that
The method that computer executes above-mentioned first aspect description.
6th aspect, provides a kind of chip product, to execute any possible of above-mentioned first aspect or first aspect
Method in embodiment.
The present invention can also be further combined on the basis of the implementation that above-mentioned various aspects provide to provide more
More implementations.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described.
Fig. 1 is a kind of block schematic illustration for wear-type electronic equipment processing that the prior art provides.
Fig. 2 is a kind of flow diagram of image processing method provided in an embodiment of the present invention.
Fig. 3 is a kind of image schematic diagram provided in an embodiment of the present invention.
Fig. 4 is a kind of image operation chart provided in an embodiment of the present invention.
Fig. 5 is a kind of multithreading operation schematic diagram provided in an embodiment of the present invention.
Fig. 6 is a kind of structural schematic diagram of wear-type electronic equipment provided in an embodiment of the present invention.
Fig. 7 is the structural schematic diagram of another wear-type electronic equipment provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing of the invention, technical solution in the embodiment of the present invention is described in detail.
Applicant has found during proposing the application: wear-type electronic equipment is realized true using frame shown in Fig. 1
The synthesis of image and virtual image shows can there is biggish time delay in whole process.These time delays are mainly derived from camera
The processing of the modules such as acquisition imaging, algorithm processing module and image synthesis module, and the processing category between modules or device
In serial process, the processing delay of each module/device is cumulative, and it is larger to will lead to whole time delay, reduces user experience.
To solve the above problems, the application spy proposes that a kind of image processing method and the applicable correlation of the method are set
It is standby.Fig. 2 is referred to, is a kind of flow diagram of image processing method provided in an embodiment of the present invention.Method as shown in Figure 2
Including step is implemented as follows:
Step S201, wear-type electronic equipment obtains the true picture at the first moment acquired by camera.This is true
Image refers to the image for the required display that camera acquires under real-world coordinates system.
In the application, when attitudes vibration occurs for wear-type electronic equipment, wear-type electronic equipment can acquire for the first moment
(such as moment T) true picture in the real world, according to the second moment (such as moment T+ after true picture prediction attitudes vibration
X the image shown needed for), the image and true picture in the real world for avoiding (T+X) moment wear-type electronic equipment from showing
Between time delay it is larger, be easy that user is allowed to cause dizziness, influence user's perception experience.
In practical applications, wear-type electronic equipment can obtain the true picture at the first moment of camera acquisition.Specifically
, when camera is controlled by wear-type electronic equipment, such as the camera is deployed in wear-type electronic equipment internal or outside
When, wear-type electronic equipment can call camera in the true of the first moment acquisition real world (i.e. under real-world coordinates system)
Image.The real-world coordinates system is positioned at reference frame in the real world, which is usually artificial or is
It unites customized setting.When camera is not controlled by wear-type electronic equipment, wear-type electronic equipment can be sent to camera
Signal is controlled, to call camera to acquire true picture at the first moment.
Optionally, the visual angle of the true picture of camera acquisition is typically larger than the view that wear-type electronic equipment shows image
Angle, such as the visual angle of true picture can be 120 °, wear-type electronic equipment supports that the visual angle of display image can be 45 ° etc..It please join
See that Fig. 3 shows a kind of image display schematic diagram.As shown in figure 3, the true picture of camera acquisition is supported to cover entire picture,
Including the objects such as doggie, trees and basketball in diagram.It is limited to the display view angle of wear-type electronic equipment, wear-type electronics is set
The standby picture (being illustrated as display image) for supporting display is the part picture in true picture, concretely illustrates dotted line frame institute
Show, including doggie and trees.
Step S202, wear-type electronic equipment obtains the posture information at the first moment, according to the posture information at the first moment
Determine re-projection matrix.The re-projection matrix is for reflecting that wear-type electronic equipment is respectively required at the first moment and the second moment
Show the mapping relations between image.Second moment was later than for the first moment.
Step S203, wear-type electronic equipment is handled true picture according to the re-projection matrix, to be worn
Image to be displayed of the formula electronic equipment at the second moment shows the figure to be shown convenient for wear-type electronic equipment on a display screen
Picture.
In the application, wear-type electronic equipment can acquire the posture information at the first moment, the posture by motion sensor
Information is that may include but be not limited to wear for describing wear-type electronic equipment in the information of athletic posture in the real world
Pitch angle, attitude angle, movement velocity, angular velocity of satellite motion or other information relevant to posture of formula electronic equipment.The fortune
Dynamic sensor include but is not limited to velocity sensor, gyroscope, pressure sensor or other be used for measuring device posture sensing
Device, the application and without limitation.
Further wear-type electronic equipment can predict the posture letter for obtaining for the second moment according to the posture information at the first moment
Breath, and the posture information according to the second moment calculates the figure to be shown of wear-type electronic equipment required display at the second moment
Picture.Specifically, wear-type electronic equipment obtain the second moment posture information after, can be each according to the first moment and the second moment
From posture information, calculate relative attitude information of second moment relative to the first moment, the relative attitude information include but
Be not limited to relative rotation angle, the direction of rotation of relative rotation angle, relative translation vector (including displacement and direction) or other
Information.Wear-type electronic equipment can calculate re-projection matrix according to the relative attitude information, which is used to indicate
Mapping relations between the image of wear-type electronic equipment display needed for the first moment and the image of display needed for the second moment.
After obtaining re-projection matrix, wear-type electronic equipment carries out the true picture that the first moment acquired using the re-projection matrix
Image mapping handles (alternatively referred to as scalloping), with the image to be displayed of display needed for the second moment of acquisition.Optionally, to mention
Image display effect is risen, wear-type electronic equipment carries out image mapping processing to processing image using the re-projection matrix, with
Obtain the image to be displayed at the second moment.The processing image be the true picture that acquires to the first moment of wear-type electronic equipment into
The image obtained after row intermediate treatment, the specific implementation which is related to can be the customized setting of system, such as image
The operation processings such as distortion, correction, scaling and synthesis are gone, are specifically set forth below in the application.
Optionally, the first moment can be denoted as the T moment in the application, and the second moment can be denoted as (T+X) moment.Wherein, X is indicated
The time delay that since wear-type electronic equipment consumed to processing by required display image acquiring true picture.The time delay can specifically lead to
The acquisition of dimension survey means is crossed, tests and obtains for example, by using dedicated delay testing equipment.In the application, time delay X is mainly by following three
Part forms: the time delay (example that time delay (i.e. the time delay of true picture acquisition imaging), the intermediate treatment of camera acquisition imaging are related to
As image mapping etc.) and wear-type electronic equipment display image needed for time delay.
As a kind of possible embodiment, the posture information of wear-type electronic equipment includes movement velocity and movement angle speed
Degree.For example, movement velocity of the wear-type electronic equipment at the first moment isAngular velocity of satellite motion isOptionally, fortune here
Dynamic speed and angular velocity of satellite motion indicate the vector of three-dimensional space, they have direction and size.Correspondingly, wear-type electronic equipment
Following formula (1) can be used and calculate acquisition relative rotation angleWith relative translation vector
Wherein, X indicates the time delay (alternatively referred to as duration) between the second moment and the first moment.
Further, wear-type electronic equipment can (specifically may include the size θ of relative rotation angle according to relative rotation angle
With its direction) calculate acquisition spin matrix.For example, by relative rotation angleIt brings into calculate in rodrigues formula and obtain
Spin matrix R, shown in formula (2) specific as follows.
R=cos θ+(1-cos θ) n × nT+ sin θ × n^ formula (2)
Wherein, n indicates rotary shaft, specifically can be by relative rotation angleIt determines, indicates to have rotated the angle θ around rotary shaft n
Degree.T indicates transposition.
Wear-type electronic equipment can be calculated and be weighed according to the translational movement L (L=δ) of spin matrix R and relative translation vector
Projection matrix.About the specific calculation of re-projection matrix, without limitation.For example, wear-type electronic equipment can according to R, L with
And the parameters such as resolution ratio of true picture (or processing image), it can COMPREHENSIVE CALCULATING acquisition re-projection matrix.The re-projection matrix is used
In mapping relations of the second moment of reflection relative to the first moment wear-type electronic equipment between respective display image.
After calculating acquisition re-projection matrix, wear-type electronic equipment will handle image (example using the re-projection matrix
The image after intermediate treatment is such as carried out to the true picture at the first moment) it is mapped as the image to be displayed at the second moment, it should
Image to be displayed is shown on the display screen of wear-type electronic equipment, is watched for user.
For example, the effect diagram that Fig. 4 shows a kind of wear-type electronic equipment display image is referred to.In Fig. 4,
Wear-type electronic equipment is the first image in the image that the first moment at moment T showed, is in the image of the second moment (T+X) display
Second image (i.e. image to be displayed).Wherein, the picture in the first image including doggie and trees includes in the second image
Picture including data and basketball.Specifically, wear-type electronic equipment can acquire the true picture at T moment by camera, it should
True picture is as shown, picture including doggie, trees and basketball.Wear-type electronic equipment is adopted by motion sensor
Collect the exercise data (i.e. posture information) at T moment itself, and then true picture is carried out in advance according to the posture information at the first moment
It surveys, with the image to be displayed (i.e. the second image) shown needed for acquisition (T+X) moment.For details, reference can be made to previous embodiments, wear
Formula electronic equipment can be handled true picture, example according to the visual angle of prediction and picture range (being embodied in re-projection matrix)
Such as scaling, rotation, translation or distortion, with the second image of display needed for obtaining the T+X moment, which is not described herein again.
The related embodiment that image intermediate treatment is related to is described below.Specifically, wear-type electronic equipment can be used it is multi-thread
Journey parallel mode handles the true picture at the first moment, obtains processing image, is convenient for subsequent wear-type electronic equipment base
The image to be displayed at the second moment is obtained in the processing image prediction.For compared with the prior art, using multi-threaded parallel
Image procossing scheme can evade the larger problem of serial process consumption time delay, reduce the time delay of image intermediate treatment consumption, promoted
Image processing efficiency.
As a kind of possible embodiment, when wear-type electronic equipment can be to the first of camera acquisition in main thread
The true picture at quarter is pre-processed, to obtain pretreated true picture.The pretreatment can be the customized setting of system
Image procossing may include but be not limited to image flame detection, image scaling, image translation and image and go the processing such as distortion.Into one
Step, in one or more, wear-type electronic equipment can carry out dummy synthesis to intermediate image respectively from thread, each to obtain
From the corresponding virtual image of thread.The intermediate image is to be obtained based on pretreated true picture, such as can be to pre-
Treated, and true picture carries out dimension-reduction treatment acquisition, which includes but is not limited to drop port number processing and resolution decreasing
Processing etc..Illustratively, it is assumed that true picture is RGB color image, and wear-type electronic equipment can be to pretreated true
Image carries out single-channel data extraction, and RGB image is converted to black and white (single channel) image, further can also be to black white image
Down-sampling processing is carried out to reduce the resolution ratio of black white image, thus the intermediate image that obtains that treated.
Optionally, wear-type electronic equipment can store intermediate image to preset data buffer area, so as to other threads
It uses.So as to reduce the operational data amount of other threads, the computational complexity and power consumption of other threads are reduced.In addition, more
Any two thread is mutually indepedent in thread, is independent of each other, for example, main thread and any one from thread or any two from line
It is run independently of each other between journey, image processing efficiency can be promoted in this way, avoid a large amount of time delay consumption of serial process bring.
The specific embodiment of dummy synthesis in the application, the application is also without limitation.For example, wear-type electronic equipment can
Using pre-set image composition algorithm, such as SLAM algorithm, it optionally may also be combined with the data of other sensors acquisition (such as wait close
At dummy object relevant data acquisition sensor etc.), to intermediate image carry out dummy synthesis, in intermediate image increase newly
The dummy object displayed for a user, to obtain the virtual image of synthesis.Optionally, wear-type electronic equipment can be by the void of synthesis
Quasi- image is stored to correspondingly data buffer area.
Further, wear-type electronic equipment is in the pretreated true picture of acquisition main thread and each from the void of thread
After quasi- image, wear-type electronic equipment can also be carried out by pretreated true picture and each from the respective virtual image of thread
Synthesis, to obtain processing image.
In practical applications, locking mechanisms can also be increased between multithreading, to ensure the safety of data access.Specifically
, when some thread accesses data, the data can be protected by locking mechanisms, prevent other thread accesses number
According to, i.e. only operation current thread access at this time after current thread uses and discharges the data, can notify other thread accesses,
It avoids multithreading while leading to occur that data are tampered, data are inconsistent or data contamination when accessing.For example, certain from
When the intermediate image stored in thread accesses data storage area, the intermediate image usually utilized locking mechanisms lock, at this time its
He can not support to access the intermediate image from thread or main thread.Only when this is from thread UNLOCKS intermediate image, and use in this
Between after image, can notify other thread accesses or use the intermediate image.
For example, by taking the time delay of wear-type electronic equipment is X as an example, every part of time delay is run for a thread.Such as Fig. 5
The parallel processing of (X+1) a thread is shown to obtain the image to be displayed of the second moment (T+X) moment wear-type electronic equipment.Such as
In Fig. 5, thread 0 is expressed as main thread, and wear-type electronic equipment can obtain camera at the first moment (T moment) in main thread
The nth frame image of acquisition can further carry out pretreatment to nth frame image and obtain pretreated nth frame image.Optionally,
Wear-type electronic equipment can carry out dimension-reduction treatment to pretreated nth frame image, such as extract single-channel data, down-sampling
Deng will handle image and be stored in data buffer storage.
In figure, 1~thread of thread X is belonged to from thread.It is each similar from the process flow of thread, by taking thread 1 as an example
Wear-type electronic equipment can obtain N+1 frame image from data buffer area, then virtually be closed to the N+1 frame image
At, such as handled in diagram using SLAM algorithm, to generate correspondingly virtual image.Wherein N+1 frame image is T+1
The image shown needed for moment wear-type electronic equipment, can be for some or all of in image after dimension-reduction treatment, without limitation.
Optionally, which can be stored in data buffer area by wear-type electronic equipment, for subsequent use.Similarly, it wears
Formula electronic equipment is obtaining each after the respective virtual image of thread, in conjunction with the pretreated nth frame image of main thread,
They are synthesized into processing image.Scalloping is carried out to the processing image convenient for subsequent wear-type electronic equipment, with acquisition (T+
X) the image to be displayed of moment wear-type electronic equipment.About scalloping, for details, reference can be made to the correlations in previous embodiment to explain
It states, which is not described herein again.
It should be noted that wear-type electronic equipment belongs to one kind of electronic equipment, the application is only set with wear-type electronics
It is illustrated for standby, but does not constitute and limit, can also be other electronic equipments.User, which can wear, wears electronic equipment reality
The different-effects such as existing virtual reality (virtual reality, VR), AR, mixed reality (mixed reality, MR).For example,
Wear electronic equipment and can be glasses, wear electronic equipment, goggles etc..Electronic equipment can also be other comprising display screen
Equipment, such as the automatic driving vehicle comprising display screen etc..
By implementing the embodiment of the present invention, it is able to solve that time delay present in conventional images displaying scheme is larger, user's body
Test the problems such as bad.
The relevant device that the application is applicable in is described below in the embodiment in conjunction with described in earlier figures 1- Fig. 5.Fig. 6 is referred to, is this
A kind of structural schematic diagram for wear-type electronic equipment that inventive embodiments provide.Wear-type electronic equipment 600 as shown in FIG. 6 wraps
Include communication unit 602 and processing unit 604.Wherein,
For communication unit 602 for obtaining the true picture that camera acquires at the first moment, the true picture is described
The image for the required display that camera acquires under real-world coordinates system；
Processing unit 604 is used to determine re-projection in the posture information at first moment according to wear-type electronic equipment
Matrix, the re-projection matrix are used to reflect the image that shows needed for the wear-type electronic equipment from first moment to the
Mapping relations between two moment, second moment are later than first moment；
Processing unit 604 is also used to handle the true picture according to the re-projection matrix, described in obtaining
Image to be displayed of the wear-type electronic equipment at second moment.
In some possible embodiments, the processing unit 604 is specifically used for being existed according to the wear-type electronic equipment
The posture information at first moment predicts the posture information at second moment；According to the posture information at first moment
With the posture information at second moment, re-projection matrix is calculated.
In some possible embodiments, the posture information includes the movement velocity and fortune of the wear-type electronic equipment
Dynamic angular speed, the processing unit 604 are specifically used for the movement speed according to the wear-type electronic equipment at first moment
Degree and angular velocity of satellite motion, the opposite rotation of wear-type electronic equipment when calculating second moment relative to first moment
Gyration and relative translation vector；According to the relative rotation angle and relative translation vector, calculates and obtain the re-projection square
Battle array.
In some possible embodiments, the processing unit 604 is specifically used for carrying out multithreading to the true picture
Processing, to obtain processing image；According to the re-projection matrix, image mapping is carried out to the processing image, described in obtaining
Image to be displayed of the wear-type electronic equipment at second moment.
In some possible embodiments, the processing unit 604 also particularly useful in main thread to the true figure
As being pre-processed, to obtain pretreated true picture；The second image is carried out respectively from thread at least one empty
It is fitted to, each described from the corresponding virtual image of thread to obtain, second image is based on described pretreated true
Image obtains；By the pretreated true picture and it is described at least one close from the corresponding virtual image of thread
At, to obtain the processing image, wherein the main thread and it is described at least one from thread all support multi-threading parallel process.
Optionally, wear-type electronic equipment 600 may also include storage unit 606, which wears for storing
The program code and data of formula electronic equipment 600, such as storage are used for the program code of image procossing.Processing unit 604 is used for
The program code in the storage unit 606 is called to realize such as the content in figure 2 above the method embodiment.
In practical applications, each module or unit involved in the wear-type electronic equipment 600 of the embodiment of the present invention are specific
Software program or hardware realization can be passed through.When being realized by software program, each module involved in wear-type electronic equipment 600 or
Unit is software module or software unit, when by hardware realization, each module or unit that wear-type electronic equipment 600 is related to
It can be realized by specific integrated circuit (application-specific integrated circuit, ASIC), or can
Programmed logic device (programmable logic device, PLD) realizes that above-mentioned PLD can be complicated process logical device
(complex programmable logical device, CPLD), field programmable gate array (field-
Programmable gate array, FPGA), Universal Array Logic (generic array logic, GAL) or its any group
It closes, the present invention is without limitation.
It should be noted that Fig. 6 is only a kind of possible implementation of the embodiment of the present application, in practical application, wear
Formula electronic equipment can also include more or fewer components, here with no restriction.About being not shown in the embodiment of the present invention or
The content not described, reference can be made to the related elaboration in preceding method embodiment, which is not described herein again.
Fig. 7 is referred to, is the structural schematic diagram of another wear-type electronic equipment provided in an embodiment of the present invention.When wearing
When electronic equipment is mounted on in account, eyes of user can see the image for wearing electronic equipment display screen presentation.Such as Fig. 7 institute
Show, wearing electronic equipment 100 may include processor 110, memory 120, sensor module 130, microphone 140, key
150, input/output interface 160, communication module 170, camera 180, battery 190 and display screen 1100 etc..Wherein sensor
Module 130 may include motion sensor 131, for detecting the posture information for obtaining wear-type electronic equipment.Sensor module
130 can also include other sensors, such as sound-detection gear, and close to optical sensor, range sensor, focal length detects optics and passes
Sensor, ambient light sensor, acceleration transducer and temperature sensor etc..
It is understood that the structure of the embodiment of the present application signal does not constitute the specific limit of correct wearing electronic equipment 100
It is fixed.In other embodiments of the application, wearing electronic equipment 100 may include components more more or fewer than diagram, or
Certain components are combined, certain components or different component layouts are perhaps split.The component of diagram can with hardware, software or
The combination of software and hardware is realized.
Processor 110 may include one or more processing units, such as: processor 110 may include application processor
(application processor, AP), modem processor, graphics processor (graphics processing
Unit, GPU), image-signal processor (image signal processor, ISP), video processing unit (video
Processing unit, VPU) controller, memory, Video Codec, digital signal processor (digital signal
Processor, DSP), baseband processor and/or neural network processor (neural-network processing unit,
NPU) etc..Wherein, different processing units can be independent device, also can integrate in one or more processors.
Wherein, controller can be the nerve center for wearing electronic equipment 100 and command centre.Controller can be according to finger
Operation code and clock signal are enabled, operating control signal is generated, the control completing instruction fetch and executing instruction.
Memory can also be set in processor 110, for storing instruction and data.In some embodiments, processor
Memory in 110 is cache memory.The memory can save the instruction that processor 110 is just used or is recycled
Or data.If processor 110 needs to reuse the instruction or data, can be called directly from the memory.It avoids
Repeated access, reduces the waiting time of processor 110, thus improves the efficiency of system.
In some embodiments, processor 110 may include one or more interfaces.Interface may include integrated circuit
(inter-integrated circuit, I2C) interface, universal asynchronous receiving-transmitting transmitter (universal asynchronous
Receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor
Interface, MIPI), universal input exports (general-purpose input/output, GPIO) interface, user identifier
Module (subscriber identity module, SIM) interface and/or universal serial bus (universal serial
Bus, USB) interface, Serial Peripheral Interface (SPI) (serial peripheral interface, SPI) interface etc..
I2C interface is a kind of bi-directional synchronization universal serial bus, including serial data line (serial data line,
SDA) He Yigen serial time clock line (derail clock line, SCL).In some embodiments, processor 110 may include
Multiple groups I2C bus.Processor 110 can by different I2C bus interface distinguish coupled motions sensor 131, battery 190,
Camera 180 etc..Such as: processor 110 can make processor 110 and movement by I2C interface coupled motions sensor 131
Sensor 131 is communicated by I2C bus interface, to obtain the posture information (i.e. exercise data) of wear-type electronic equipment.SPI connects
Mouth can be used for the connection between processor and sensor.
UART interface is a kind of Universal Serial Bus, is used for asynchronous communication.The bus can be bidirectional communications bus.
The data that it will be transmitted are converted between serial communication and parallel communications.In some embodiments, UART interface usually by with
In connection processor 110 and communication module 170.Such as: processor 110 passes through the bluetooth in UART interface and communication module 170
Module communication, realizes Bluetooth function.
MIPI interface can be used to connect the peripheral components such as processor 110 and display screen 1100, camera 180.MIPI
Interface includes camera serial line interface (camera serial interface, CSI), display screen serial line interface (display
Serial interface, DSI) etc..In some embodiments, processor 110 and camera 180 are communicated by CSI interface, real
Now wear the shooting function of electronic equipment 100.By DSI interface communication, electronics is worn in realization for processor 110 and display screen 1100
The display function of equipment 100.
GPIO interface can pass through software configuration.GPIO interface can be configured as control signal, may be alternatively configured as counting
It is believed that number.In some embodiments, GPIO interface can be used for connecting processor 110 and camera 180, and display screen 1100 leads to
Believe module 170, sensor module 130, microphone 140 etc..GPIO interface can be additionally configured to I2C interface, I2S interface,
UART interface, MIPI interface etc..
Usb 1 30 is the interface for meeting USB standard specification, specifically can be Mini USB interface, and Micro USB connects
Mouthful, USB Type C interface etc..It is to wear the charging of electronic equipment 100 that usb 1 30, which can be used for connecting charger, can also be with
Data are transmitted between electronic equipment 100 and peripheral equipment for wearing.It can be used for connection earphone, sound played by earphone
Frequently.The interface can be also used for connecting other electronic equipments, such as mobile phone etc..USB interface can be USB3.0, for compatible high
Fast display interface (display port, DP) signal transmission, can transmit video and audio high-speed data.
It is understood that the interface connection relationship of each intermodule of the embodiment of the present application signal, only schematically illustrates,
The structure qualification of correct wearing electronic equipment 100 is not constituted.In other embodiments of the application, electronic equipment 100 is worn
It can be using the combination of interface connection type or multiple interfaces connection type different in above-described embodiment.
In addition, wearing electronic equipment 100 may include wireless communication function.Communication module 170 may include wireless communication
Module and mobile communication module.Wireless communication function can be modulated by antenna (not shown), mobile communication module (not shown)
Demodulation processor (not shown) and baseband processor (not shown) etc. are realized.
Antenna electromagnetic wave signal for transmitting and receiving.It wears and may include mutiple antennas, Mei Getian in electronic equipment 100
Line can be used for covering single or multiple communication bands.Different antennas can also be multiplexed, to improve the utilization rate of antenna.Such as:
Antenna 1 can be multiplexed with to the diversity antenna of WLAN.In other embodiments, antenna can be with tuning switch knot
It closes and uses.
Mobile communication module, which can provide to apply, is wearing the channel radios such as including 2G/3G/4G/5G on electronic equipment 100
The solution of letter.Mobile communication module may include at least one filter, switch, power amplifier, low-noise amplifier
(low noise amplifier, LNA) etc..Mobile communication module can be received electromagnetic wave by antenna, and to received electromagnetic wave
It is filtered, the processing such as amplification is sent to modem processor and is demodulated.Mobile communication module can also be to modulated solution
The modulated signal amplification of processor is adjusted, switchs to electromagenetic wave radiation through antenna and goes out.In some embodiments, mobile communication module
At least partly functional module can be arranged in processor 110.In some embodiments, at least portion of mobile communication module
Point functional module can be arranged in the same device at least partly module of processor 110.
Modem processor may include modulator and demodulator.Wherein, modulator is used for low frequency base to be sent
Band signal is modulated into high frequency signal.Demodulator is used to received electromagnetic wave signal being demodulated into low frequency baseband signal.Then solution
Adjust device that the low frequency baseband signal that demodulation obtains is sent to baseband processor.Low frequency baseband signal is through baseband processor
Afterwards, it is delivered to application processor.Application processor exports voice signal by audio frequency apparatus (being not limited to loudspeaker etc.), or logical
It crosses display screen 1100 and shows image or video.In some embodiments, modem processor can be independent device.Another
In some embodiments, modem processor can be set independently of processor 110 with mobile communication module or other function module
It sets in the same device.
It includes WLAN (wireless that wireless communication module, which can be provided and be applied wearing on electronic equipment 100,
Local area networks, WLAN) (such as Wireless Fidelity (wireless fidelity, Wi-Fi) network), bluetooth
(bluetooth, BT), Global Navigation Satellite System (global navigation satellite system, GNSS), frequency modulation
(frequency modulation, FM), the short distance wireless communication technology (near field communication, NFC) are red
The solution of the wireless communications such as outer technology (infrared, IR).Wireless communication module can be integrated into a few mailing address
Manage one or more devices of module.Wireless communication module receives electromagnetic wave via antenna, by electromagnetic wave signal frequency modulation and filter
Wave processing, by treated, signal is sent to processor 110.Wireless communication module can also receive to be sent from processor 110
Signal, frequency modulation is carried out to it, amplify, switch to electromagenetic wave radiation through antenna and go out.
In some embodiments, antenna and the mobile communication module coupling for wearing electronic equipment 100, set so that wearing electronics
Standby 100 technology can communicate with network and other equipment by wireless communication.The wireless communication technique may include the whole world
Mobile communcations system (global system for mobile communications, GSM), general packet radio service
(general packet radio service, GPRS), CDMA access (code division multiple
Access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), the time-division
CDMA (time-division code division multiple access, TD-SCDMA), long term evolution (long
Term evolution, LTE), BT, GNSS, WLAN, NFC, FM and/or IR technology etc..The GNSS may include that the whole world is defended
Star positioning system (global positioning system, GPS), Global Navigation Satellite System (global navigation
Satellite system, GLONASS), Beidou satellite navigation system (beidou navigation satellite
System, BDS), quasi- zenith satellite system (quasi-zenith satellite system, QZSS) and/or star base enhancing system
It unites (satellite based augmentation systems, SBAS).
Electronic equipment 100 is worn by GPU, display screen 1100 and application processor etc. realize display function.GPU is
The microprocessor of image procossing connects display screen 1100 and application processor.GPU is calculated for executing mathematics and geometry, is used for
Figure rendering.Processor 110 may include one or more GPU, execute program instructions to generate or change display information.
The quantity for wearing display screen 1100 in electronic equipment 100 can be two, respectively correspond two eyes of user 200
Ball.The content of the two display screen displays can independently be shown.It can come in the different image of the two display screen displays
Improve the three-dimensional sense of image.In some possible embodiments, the quantity for wearing display screen 1100 in electronic equipment 100 can also be with
It is one, to correspond to two eyeballs of user 200.
Wearing electronic equipment 100 can be by ISP, camera 180, Video Codec, GPU, display screen 1100 and
Application processor etc. realizes shooting function.
ISP is used to handle the data of the feedback of camera 180.For example, opening shutter when taking pictures, light is passed by camera lens
It is delivered on camera photosensitive element, optical signal is converted to electric signal, and camera photosensitive element passes to the electric signal at ISP
Reason, is converted into macroscopic image.ISP can also be to the noise of image, brightness, colour of skin progress algorithm optimization.ISP can be with
Exposure to photographed scene, the parameter optimizations such as colour temperature.In some embodiments, ISP can be set in camera 180.
Camera 180 is for capturing still image or video.Object generates optical imagery by camera lens and projects photosensitive member
Part.Photosensitive element can be charge-coupled device (charge coupled device, CCD) or complementary metal oxide is partly led
Body (complementary metal-oxide-semiconductor, CMOS) phototransistor.Photosensitive element turns optical signal
It changes electric signal into, electric signal is passed into ISP later and is converted into data image signal.Data image signal is output to DSP by ISP
Working process.Data image signal is converted into the RGB of standard, the picture signal of the formats such as YUV by DSP.In some embodiments,
Wearing electronic equipment 100 may include 1 or N number of camera 180, and N is the positive integer greater than 1.
As shown in fig. 7, camera 180 may be mounted at the side for wearing electronic equipment 100, it can also be mounted on and wear electricity
Position in sub- equipment 100 between two display screens.Camera 180 is for the image and view in 200 visual angle of real-time capture user
Frequently.It wears electronic equipment 100 and virtual image is generated according to the real-time image and video of capture, and virtual image is passed through into display
Screen 1100 is shown.
The still image or video image that processor 110 can be captured according to camera 180, in conjunction with sensor module 130
The data (such as the data such as brightness, sound) of acquisition, to determine the virtual image shown on display screen 1100, Lai Shixian is in reality
Virtual image is superimposed on world object.
Wherein, digital signal processor, in addition to can handle data image signal, can also be located for handling digital signal
Manage other digital signals.For example, digital signal processor is used for frequency point energy when wearing electronic equipment 100 in frequency point selection
Amount carries out Fourier transformation etc..
Video Codec is used for compression of digital video or decompression.Wear electronic equipment 100 can support it is a kind of or
Various video codec.In this way, the video of a variety of coded formats can be played or record by wearing electronic equipment 100, and such as: it is dynamic
State motion picture expert group version (moving picture experts group, MPEG) 1, MPEG2, mpeg 3, MPEG4 etc..
NPU is neural network (neural-network, NN) computation processor, by using for reference biological neural network structure,
Such as transfer mode between human brain neuron is used for reference, it, can also continuous self study to input information fast processing.Pass through NPU
It may be implemented to wear the application such as intelligent cognition of electronic equipment 100, such as: image recognition, recognition of face, speech recognition, text
Understand etc..
Memory 120 can be used for storing computer executable program code, and the executable program code includes instruction.
Processor 110 is stored in the instruction of memory 120 by operation, thereby executing the various function application for wearing electronic equipment 100
And data processing.For example, processor 110 can call the instruction execution stored in memory 120 such as figure 2 above the method to implement
All or part of implementation steps in example.Memory 120 may include storing program area and storage data area.Wherein, journey is stored
Sequence area can storage program area, application program needed at least one function (such as sound-playing function, image player function
Deng) etc..Storage data area can store data (such as audio data, the phone worn and created in 100 use process of electronic equipment
This etc.) etc..In addition, memory 120 may include high-speed random access memory, it can also include nonvolatile memory, example
Such as at least one disk memory, flush memory device, generic flash memory (universal flash storage, UFS)
Deng.
Wearing electronic equipment 100 can be by audio-frequency module, loudspeaker, microphone 140, at earphone interface, and application
It manages device etc. and realizes audio-frequency function.Such as music, recording etc..
Audio-frequency module is used to for digitized audio message to be converted into analog audio signal output, is also used for analog audio input
Be converted to digital audio and video signals.Audio-frequency module can be also used for audio-frequency signal coding and decoding.In some embodiments, audio
Module can be set in processor 110, or the partial function module of audio-frequency module is set in processor 110.
Loudspeaker, also referred to as " loudspeaker ", for audio electrical signal to be converted to voice signal.Wearing electronic equipment 100 can be with
Music is listened to by loudspeaker, or listens to hand-free call.
Microphone 140, also referred to as " microphone ", " microphone ", for voice signal to be converted to electric signal.Wear electronic equipment
100 can be set at least one microphone 140.In further embodiments, wearing electronic equipment 100 can be set two Mikes
Wind 140 can also realize decrease of noise functions in addition to collected sound signal.In further embodiments, electronic equipment 100 is worn also
It can be set three, four or more microphone 140 realizes that collected sound signal, noise reduction can also identify sound source, real
Existing directional recording function etc..
In some embodiments, wearing electronic equipment 100 may include sound-detection gear 132, which can
To detect and handle the voice signal for controlling mancarried electronic aid.For example, sound-detection gear may include microphone 140.
Just it wears electronic equipment 100 microphone 140 can be used and convert sound into electric signal.Sound-detection gear 132 can then be located
Electric signal is managed, and is the order of wear-type display system 1300 by signal identification.Processor 110 can be configured as from microphone
140 receive voice signal.After receiving voice signal, processor 110 can run sound-detection gear 132 to identify that voice is ordered
It enables.For example, wearing the connection in the user contact lists of the available storage of electronic equipment 110 when receiving phonetic order
It is people, just wearing electronic equipment 100 can be with the automatic dialing contact phone number.
Earphone interface is for connecting wired earphone.Earphone interface can be USB interface, is also possible to the open of 3.5mm and moves
It is dynamic to wear electronic equipment platform (open mobile terminal platform, OMTP) standard interface, American cellular telecommunications work
Industry association (cellular telecommunications industry association of the USA, CTIA) standard
Interface.
In some embodiments, wearing electronic equipment 100 may include one or more keys 150, these keys can be with
Electronic equipment is worn in control, provides the function in access head wearing electronic equipment 100 for user.The form of key 150 can be by
Button, switch, dial and touch or nearly touch sensing device (such as touch sensor).Specifically, for example, user 20 can pass through
The display screen 1100 for wearing electronic equipment 100 is opened by lower button.Key 190 includes power button, volume key etc..Key 190
It can be mechanical key.It is also possible to touch-key.Wearing electronic equipment 100 can receive key-press input, generates and wears
The related key signals input of the user setting and function control of electronic equipment 100.
In some embodiments, wearing electronic equipment 100 may include input/output interface 160, input/output interface 160
Other devices can be connected to by suitable component and wear electronic equipment 100.Component for example may include that audio/video is inserted
Hole, data connector etc..
Sound-detection gear can detecte and handle the voice signal for controlling mancarried electronic aid.
In some embodiments, wearing electronic equipment 100 may be implemented eyeball tracking (eye tracking).Specifically,
Infrared equipment (such as infrared transmitter) and image capture device (such as camera) be can use to detect eyeball fixes direction.
It may include such as light emitting diode (LED) and photodetector, such as photodiode close to optical sensor.It shines
Diode can be infrared light-emitting diode.It wears electronic equipment 100 and infrared light is launched outward by light emitting diode.Wear electricity
Sub- equipment 100 detects the infrared external reflection light from neighbouring object using photodiode.It, can when detecting sufficient reflected light
To determine that wearing electronic equipment 100 nearby has object.When detecting insufficient reflected light, wearing electronic equipment 100 can be with
It determines and wears electronic equipment 100 nearby without object.Wear electronic equipment 100 can use close to optical sensor detection wear electricity
The gesture operation of sub- 100 specific position of equipment, to realize gesture operation purpose associated with operational order.
Range sensor, for measuring distance.Wearing electronic equipment 100 can be by infrared or laser distance measuring.?
In some embodiments, wearing electronic equipment 100 can use range sensor ranging to realize rapid focus.
Ambient light sensor is for perceiving environmental light brightness.Wearing electronic equipment 100 can be according to the environment bright of perception
Spend 1100 brightness of automatic adjusument display screen.Automatic white balance adjustment when ambient light sensor can also be used for taking pictures.
Acceleration transducer is detectable to wear the big of (the generally three axis) acceleration in all directions of electronic equipment 100
It is small.When wear electronic equipment 100 it is static when can detect that size and the direction of gravity.It can be also used for identification and wear electronic equipment
Posture is applied to the application such as pedometer.
Temperature sensor is for detecting temperature.In some embodiments, electronic equipment 100 is worn to examine using temperature sensor
The temperature of survey executes Temperature Treatment strategy.For example, wearing electronic equipment 100 when the temperature that temperature sensor reports is more than threshold value
The performance for reducing the processor being located near temperature sensor is executed, implements Thermal protection to reduce power consumption.In other implementations
In example, when temperature is lower than another threshold value, wears electronic equipment 100 and battery 190 is heated, cause to wear electronics to avoid low temperature
The abnormal shutdown of equipment 100.In some other embodiment, when temperature is lower than another threshold value, electronic equipment 100 is worn to battery
190 output voltage executes boosting, to avoid shutting down extremely caused by low temperature.
Focal length detection optical sensor is used to detect the focal length of the eyeball of user 200.In some embodiments, such as Fig. 7 institute
Show, wearing electronic equipment 100 can also include infrared light sources 1200.Focal length detection optical sensor can be and infrared ray
The focal length of the eyeball of the cooperation of source 1200 detection user 200.
As shown in fig. 7, focal length detection optical sensor 132 and infrared light sources 1200 can be set in display screen close to eye
The side of ball.The quantity that focal length detects optical sensor 132 and infrared light sources 1200 can be two, and each eyeball can be with
Corresponding focal length detection optical sensor 132 and infrared light sources 1200, for detecting the focal length of ocular focusing.
It should be understood that the detection of focal length shown in Fig. 7 optical sensor 132, infrared light sources 1200 and camera 180
Wearing position on electronic equipment 100 and quantity is only used for explaining the embodiment of the present application, restriction should not be constituted.Focal length detection
The quantity of optical sensor 132 and infrared light sources 1200 can also be one.One focal length detects optical sensor 132 and one
The quantity of a infrared light sources 1200 can be used for detecting the focal length of an ocular focusing, or detect two ocular focusings simultaneously
Focal length.
Motion sensor 131 is determined for wearing the athletic posture of electronic equipment 100.In some embodiments, may be used
To determine the angular speed worn electronic equipment 100 and surround three axis (that is, x, y and z-axis) by motion sensor.Motion sensor
It can be also used for navigating, somatic sensation television game scene.The motion sensor includes but is not limited to gyro sensor, angular speed sensing
Device or velocity sensor etc..
Display screen 1100 is for showing image, video etc..Display screen 1100 includes display panel.Display panel can use
Liquid crystal display (liquid crystal display, LCD), Organic Light Emitting Diode (organic light-emitting
Diode, OLED), active matrix organic light-emitting diode or active-matrix organic light emitting diode (active-matrix
Organic light emitting diode's, AMOLED), Flexible light-emitting diodes (flex light-emitting
Diode, FLED), Miniled, MicroLed, Micro-oLed, light emitting diode with quantum dots (quantum dot light
Emitting diodes, QLED) etc..
The embodiment of the present invention also provides a kind of computer non-transitory storage media, in the computer non-transitory storage media
It is stored with instruction, when it runs on a processor, method flow shown in Fig. 2 is achieved.
The embodiment of the present invention also provides a kind of computer program product, when the computer program product is transported on a processor
When row, method flow shown in Fig. 2 is achieved.
The step of method in conjunction with described in disclosure of the embodiment of the present invention or algorithm, can be come real in a manner of hardware
It is existing, it is also possible to execute the mode of software instruction by processor to realize.Software instruction can be made of corresponding software module,
Software module can be stored on random access memory (English: Random Access Memory, RAM), flash memory, read-only deposit
Reservoir (English: Read Only Memory, ROM), Erasable Programmable Read Only Memory EPROM (English: Erasable
Programmable ROM, EPROM), Electrically Erasable Programmable Read-Only Memory (English: Electrically EPROM,
EEPROM), register, hard disk, mobile hard disk, CD-ROM (CD-ROM) or any other form well known in the art are deposited
In storage media.A kind of illustrative storage medium is coupled to processor, believes to enable a processor to read from the storage medium
Breath, and information can be written to the storage medium.Certainly, storage medium is also possible to the component part of processor.It processor and deposits
Storage media can be located in ASIC.In addition, the ASIC can be located in wear-type electronic equipment.Certainly, processor and storage are situated between
Matter can also be used as discrete assembly and be present in wear-type electronic equipment.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the program can be stored in computer-readable storage medium
In, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.And storage medium above-mentioned include: ROM,
The various media that can store program code such as RAM, magnetic or disk.
Claims (15)
1. a kind of image processing method, which is characterized in that the described method includes:
The true picture that camera acquires at the first moment is obtained, the true picture is the camera in real-world coordinates
The image of the lower required display acquired of system；
Posture information according to wear-type electronic equipment at first moment determines re-projection matrix, the re-projection matrix
Image for display needed for reflecting the wear-type electronic equipment is closed from first moment to the mapping the second moment
System, second moment are later than first moment；
According to the re-projection matrix, the true picture is handled, to obtain the wear-type electronic equipment described
The image to be displayed at the second moment.
2. the method according to claim 1, wherein it is described according to the wear-type electronic equipment described first
The posture information at moment determines that re-projection matrix includes:
Posture information according to the wear-type electronic equipment at first moment predicts the posture letter at second moment
Breath；
According to the posture information of the posture information at first moment and second moment, re-projection matrix is calculated.
3. method according to claim 1 or 2, which is characterized in that the posture information includes that the wear-type electronics is set
Standby movement velocity and angular velocity of satellite motion, the posture information according to the wear-type electronic equipment at first moment,
Determine that re-projection matrix includes:
Movement velocity and angular velocity of satellite motion according to the wear-type electronic equipment at first moment, when calculating described second
The relative rotation angle and relative translation vector of the wear-type electronic equipment when carving relative to first moment；
According to the relative rotation angle and relative translation vector, calculates and obtain the re-projection matrix.
4. according to the method described in claim 3, it is characterized in that, it is described according to the relative rotation angle and relative translation to
Amount, calculating the acquisition re-projection matrix includes:
According to the direction of rotation of the relative rotation angle and the relative rotation angle, calculates and obtain spin matrix, the rotation
The relative rotation shape of torque battle array wear-type electronic equipment when being used to indicate second moment relative to first moment
Condition；
According to spin matrix described in the relative translation vector sum, calculates and obtain the re-projection matrix.
5. method according to any of claims 1-4, which is characterized in that it is described according to the re-projection matrix, it is right
The true picture is handled, and includes: to obtain image to be displayed of the wear-type electronic equipment at second moment
Multiple threads are carried out to the true picture, to obtain processing image；
According to the re-projection matrix, image mapping is carried out to the processing image, is existed with obtaining the wear-type electronic equipment
The image to be displayed at second moment.
6. according to the method described in claim 5, it is characterized in that, it is described to the true picture carry out multiple threads, with
Obtaining processing image includes:
The true picture is pre-processed in main thread, to obtain pretreated true picture；
It is each described from the corresponding void of thread to obtain at least one respectively to intermediate image progress dummy synthesis from thread
Quasi- image, the intermediate image are based on the pretreated true picture and obtain；
By the pretreated true picture and it is described at least one synthesize from the corresponding virtual image of thread, with
Obtain the processing image, wherein the main thread and it is described at least one from thread all support multi-threading parallel process.
7. according to the method described in claim 6, it is characterized in that, the intermediate image is to the pretreated true figure
Image as carrying out dimension-reduction treatment acquisition, the dimension-reduction treatment include the processing of drop image channel and/or drop image resolution ratio processing.
8. method according to any one of claims 1-7, which is characterized in that second moment and first moment
Between the when a length of true picture be processed into the duration of image shown by the wear-type electronic equipment.
9. a kind of wear-type electronic equipment, which is characterized in that including communication unit and processing unit, wherein
The communication unit, the true picture acquired for obtaining wear-type electronic equipment at the first moment, the true picture
For the image for the required display that the wear-type electronic equipment acquires under real-world coordinates system；
The processing unit, for the posture information according to the wear-type electronic equipment at first moment, determination is thrown again
Shadow matrix, the re-projection matrix be used to reflect the image that shows needed for the wear-type electronic equipment from first moment to
Mapping relations between second moment, second moment are later than first moment；
The processing unit is also used to handle the true picture according to the re-projection matrix, to obtain the head
Formula electronic equipment is worn in the image to be displayed at second moment.
10. equipment according to claim 9, which is characterized in that the processing unit is specifically used for:
Posture information according to the wear-type electronic equipment at first moment predicts the posture letter at second moment
Breath；
According to the posture information of the posture information at first moment and second moment, re-projection matrix is calculated.
11. equipment according to claim 9 or 10, which is characterized in that the posture information includes the wear-type electronics
The movement velocity and angular velocity of satellite motion of equipment, the processing unit are specifically used for:
Movement velocity and angular velocity of satellite motion according to the wear-type electronic equipment at first moment, when calculating described second
The relative rotation angle and relative translation vector of the wear-type electronic equipment when carving relative to first moment；
According to the relative rotation angle and relative translation vector, calculates and obtain the re-projection matrix.
12. according to equipment described in claim 9-11, which is characterized in that the processing unit is specifically used for:
Multiple threads are carried out to the true picture, to obtain processing image；
According to the re-projection matrix, image mapping is carried out to the processing image, is existed with obtaining the wear-type electronic equipment
The image to be displayed at second moment.
13. equipment according to claim 12, which is characterized in that the processing unit also particularly useful for:
The true picture is pre-processed in main thread, to obtain pretreated true picture；
It is each described from the corresponding void of thread to obtain at least one respectively to the second image progress dummy synthesis from thread
Quasi- image, second image are based on the pretreated true picture and obtain；
By the pretreated true picture and it is described at least one synthesize from the corresponding virtual image of thread, with
Obtain the processing image, wherein the main thread and it is described at least one from thread all support multi-threading parallel process.
14. a kind of wear-type electronic equipment, which is characterized in that including memory, communication interface and with the memory and communication
The processor of interface coupling；For storing instruction, for executing described instruction, the communication connects the processor memory
Mouth under the control of the processor with other equipment for being communicated；Wherein, it is held when the processor executes described instruction
Row method of any of claims 1-8 as above.
15. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In any one of such as claim 1 to 8 of realization the method when the computer program is executed by processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910443434.1A CN110244840A (en) | 2019-05-24 | 2019-05-24 | Image processing method, relevant device and computer storage medium |
| PCT/CN2020/091503 WO2020238741A1 (en) | 2019-05-24 | 2020-05-21 | Image processing method, related device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910443434.1A CN110244840A (en) | 2019-05-24 | 2019-05-24 | Image processing method, relevant device and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110244840A true CN110244840A (en) | 2019-09-17 |
Family
ID=67885069
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910443434.1A Pending CN110244840A (en) | 2019-05-24 | 2019-05-24 | Image processing method, relevant device and computer storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110244840A (en) |
| WO (1) | WO2020238741A1 (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027374A (en) * | 2019-10-28 | 2020-04-17 | 华为终端有限公司 | Image identification method and electronic equipment |
| CN111243027A (en) * | 2020-02-28 | 2020-06-05 | 京东方科技集团股份有限公司 | Delay measurement method, device and system |
| CN111736692A (en) * | 2020-06-01 | 2020-10-02 | Oppo广东移动通信有限公司 | Display method, display device, storage medium and head-mounted device |
| WO2020238741A1 (en) * | 2019-05-24 | 2020-12-03 | 华为技术有限公司 | Image processing method, related device and computer storage medium |
| CN112132108A (en) * | 2020-10-09 | 2020-12-25 | 安徽江淮汽车集团股份有限公司 | Ground point cloud data extraction method, device, equipment and storage medium |
| CN112486318A (en) * | 2020-11-26 | 2021-03-12 | 北京字跳网络技术有限公司 | Image display method, image display device, readable medium and electronic equipment |
| CN112752119A (en) * | 2019-10-31 | 2021-05-04 | 中兴通讯股份有限公司 | Time delay error correction method, terminal equipment, server and storage medium |
| WO2021218683A1 (en) * | 2020-04-30 | 2021-11-04 | 华为技术有限公司 | Image processing method and apparatus |
| CN114071197A (en) * | 2020-07-30 | 2022-02-18 | 华为技术有限公司 | Screen projection data processing method and device |
| CN114449251A (en) * | 2020-10-31 | 2022-05-06 | 华为技术有限公司 | Video perspective method, device, system, electronic equipment and storage medium |
| WO2022099787A1 (en) * | 2020-11-13 | 2022-05-19 | 歌尔股份有限公司 | Head-mounted display device and data acquisition method, apparatus, and host computer thereof |
| CN114640838A (en) * | 2022-03-15 | 2022-06-17 | 北京奇艺世纪科技有限公司 | Picture synthesis method and device, electronic equipment and readable storage medium |
| WO2022241701A1 (en) * | 2021-05-20 | 2022-11-24 | 华为技术有限公司 | Image processing method and device |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4254139B1 (en) * | 2022-03-30 | 2024-05-08 | Holo-Light GmbH | Reprojection method for generating reprojected image data, xr projection system, and machine-learning module |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104202547A (en) * | 2014-08-27 | 2014-12-10 | 广东威创视讯科技股份有限公司 | Method for extracting target object in projection picture, projection interaction method and system thereof |
| CN105474273A (en) * | 2013-07-25 | 2016-04-06 | 微软技术许可有限责任公司 | Late stage reprojection |
| CN105892658A (en) * | 2016-03-30 | 2016-08-24 | 华为技术有限公司 | Method for predicting head posture based on head-mounted display device and head-mounted display device |
| CN108351691A (en) * | 2015-10-26 | 2018-07-31 | 微软技术许可有限责任公司 | remote rendering for virtual image |
| CN108431739A (en) * | 2016-01-25 | 2018-08-21 | 微软技术许可有限责任公司 | Frame for augmented reality environment projects |
| CN109656367A (en) * | 2018-12-24 | 2019-04-19 | 深圳超多维科技有限公司 | Image processing method, device and electronic equipment under a kind of scene applied to VR |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10284816B2 (en) * | 2015-03-23 | 2019-05-07 | Intel Corporation | Facilitating true three-dimensional virtual representation of real objects using dynamic three-dimensional shapes |
| CN105847785A (en) * | 2016-05-09 | 2016-08-10 | 上海乐相科技有限公司 | Image processing method, device and system |
| CN106502427B (en) * | 2016-12-15 | 2023-12-01 | 北京国承万通信息科技有限公司 | Virtual reality system and scene presenting method thereof |
| CN106998409B (en) * | 2017-03-21 | 2020-11-27 | 华为技术有限公司 | Image processing method, head-mounted display and rendering equipment |
| CN110244840A (en) * | 2019-05-24 | 2019-09-17 | 华为技术有限公司 | Image processing method, relevant device and computer storage medium |
-
2019
- 2019-05-24 CN CN201910443434.1A patent/CN110244840A/en active Pending
-
2020
- 2020-05-21 WO PCT/CN2020/091503 patent/WO2020238741A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105474273A (en) * | 2013-07-25 | 2016-04-06 | 微软技术许可有限责任公司 | Late stage reprojection |
| CN104202547A (en) * | 2014-08-27 | 2014-12-10 | 广东威创视讯科技股份有限公司 | Method for extracting target object in projection picture, projection interaction method and system thereof |
| CN108351691A (en) * | 2015-10-26 | 2018-07-31 | 微软技术许可有限责任公司 | remote rendering for virtual image |
| CN108431739A (en) * | 2016-01-25 | 2018-08-21 | 微软技术许可有限责任公司 | Frame for augmented reality environment projects |
| CN105892658A (en) * | 2016-03-30 | 2016-08-24 | 华为技术有限公司 | Method for predicting head posture based on head-mounted display device and head-mounted display device |
| CN109656367A (en) * | 2018-12-24 | 2019-04-19 | 深圳超多维科技有限公司 | Image processing method, device and electronic equipment under a kind of scene applied to VR |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020238741A1 (en) * | 2019-05-24 | 2020-12-03 | 华为技术有限公司 | Image processing method, related device and computer storage medium |
| CN111027374A (en) * | 2019-10-28 | 2020-04-17 | 华为终端有限公司 | Image identification method and electronic equipment |
| CN111027374B (en) * | 2019-10-28 | 2023-06-30 | 华为终端有限公司 | Image recognition method and electronic equipment |
| CN112752119A (en) * | 2019-10-31 | 2021-05-04 | 中兴通讯股份有限公司 | Time delay error correction method, terminal equipment, server and storage medium |
| CN112752119B (en) * | 2019-10-31 | 2023-12-01 | 中兴通讯股份有限公司 | Delay error correction method, terminal equipment, server and storage medium |
| CN111243027A (en) * | 2020-02-28 | 2020-06-05 | 京东方科技集团股份有限公司 | Delay measurement method, device and system |
| CN111243027B (en) * | 2020-02-28 | 2023-06-23 | 京东方科技集团股份有限公司 | Delay measurement method, device and system |
| WO2021218683A1 (en) * | 2020-04-30 | 2021-11-04 | 华为技术有限公司 | Image processing method and apparatus |
| CN111736692A (en) * | 2020-06-01 | 2020-10-02 | Oppo广东移动通信有限公司 | Display method, display device, storage medium and head-mounted device |
| CN111736692B (en) * | 2020-06-01 | 2023-01-31 | Oppo广东移动通信有限公司 | Display method, display device, storage medium and head-mounted device |
| CN114071197B (en) * | 2020-07-30 | 2024-04-12 | 华为技术有限公司 | Screen projection data processing method and device |
| CN114071197A (en) * | 2020-07-30 | 2022-02-18 | 华为技术有限公司 | Screen projection data processing method and device |
| CN112132108A (en) * | 2020-10-09 | 2020-12-25 | 安徽江淮汽车集团股份有限公司 | Ground point cloud data extraction method, device, equipment and storage medium |
| CN114449251A (en) * | 2020-10-31 | 2022-05-06 | 华为技术有限公司 | Video perspective method, device, system, electronic equipment and storage medium |
| CN114449251B (en) * | 2020-10-31 | 2024-01-16 | 华为技术有限公司 | Video perspective method, device, system, electronic equipment and storage medium |
| WO2022099787A1 (en) * | 2020-11-13 | 2022-05-19 | 歌尔股份有限公司 | Head-mounted display device and data acquisition method, apparatus, and host computer thereof |
| US11836286B2 (en) | 2020-11-13 | 2023-12-05 | Goertek Inc. | Head-mounted display device and data acquisition method, apparatus, and host computer thereof |
| CN112486318A (en) * | 2020-11-26 | 2021-03-12 | 北京字跳网络技术有限公司 | Image display method, image display device, readable medium and electronic equipment |
| WO2022241701A1 (en) * | 2021-05-20 | 2022-11-24 | 华为技术有限公司 | Image processing method and device |
| CN114640838B (en) * | 2022-03-15 | 2023-08-25 | 北京奇艺世纪科技有限公司 | Picture synthesis method and device, electronic equipment and readable storage medium |
| CN114640838A (en) * | 2022-03-15 | 2022-06-17 | 北京奇艺世纪科技有限公司 | Picture synthesis method and device, electronic equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020238741A1 (en) | 2020-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110244840A (en) | Image processing method, relevant device and computer storage medium | |
| CN109582141B (en) | Method for controlling display screen according to eyeball focus and head-mounted electronic equipment | |
| CN110139028B (en) | Image processing method and head-mounted display device | |
| CN113810601B (en) | Terminal image processing method and device and terminal equipment | |
| WO2022007862A1 (en) | Image processing method, system, electronic device and computer readable storage medium | |
| CN113542580B (en) | Method and device for removing light spots of glasses and electronic equipment | |
| CN113596321B (en) | Method, device and storage medium for generating transition dynamic effect | |
| US20220245778A1 (en) | Image bloom processing method and apparatus, and storage medium | |
| CN114257920B (en) | Audio playing method and system and electronic equipment | |
| WO2022062884A1 (en) | Text input method, electronic device, and computer-readable storage medium | |
| US20240013432A1 (en) | Image processing method and related device | |
| CN115641867B (en) | Voice processing method and terminal equipment | |
| CN114189631B (en) | Shooting method and electronic equipment | |
| CN115150542B (en) | Video anti-shake method and related equipment | |
| CN112565735B (en) | Virtual reality measuring and displaying method, device and system | |
| CN114812381A (en) | Electronic equipment positioning method and electronic equipment | |
| CN116684726B (en) | Moon detection method and electronic equipment | |
| CN115880198B (en) | Image processing method and device | |
| WO2022179412A1 (en) | Recognition method and electronic device | |
| WO2022206783A1 (en) | Photography method and apparatus, and electronic device and readable storage medium | |
| RU2782312C1 (en) | Image processing method and display device mounted on the head | |
| CN117714894A (en) | Target identification method, electronic equipment and storage medium | |
| CN116847204A (en) | Target identification method, electronic equipment and storage medium | |
| CN117130472A (en) | Virtual space operation guide display method, mobile device and system | |
| CN117130471A (en) | Man-machine interaction method, electronic equipment and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190917 |