CN108235132A - Panoramic video visual angle regulating method and device based on human eye positioning - Google Patents
Panoramic video visual angle regulating method and device based on human eye positioning Download PDFInfo
- Publication number
- CN108235132A CN108235132A CN201810205657.XA CN201810205657A CN108235132A CN 108235132 A CN108235132 A CN 108235132A CN 201810205657 A CN201810205657 A CN 201810205657A CN 108235132 A CN108235132 A CN 108235132A
- Authority
- CN
- China
- Prior art keywords
- human
- pupil
- coordinate
- visual angle
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
- H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
- H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
Abstract
The present invention provides a kind of panoramic video visual angle regulating methods and device based on human eye positioning.Panoramic video visual angle regulating method based on human eye positioning includes：Obtain the offset of pupil of human coordinate；According to the offset of the pupil of human coordinate, the broadcasting visual angle of current panorama video is determined, to play current panorama video according to the broadcasting visual angle.Above-mentioned panoramic video viewing angle adjustment device can be used for performing above-mentioned panoramic video visual angle regulating method.Above-mentioned the panoramic video visual angle regulating method and device based on human eye positioning of the present invention, the accurate variable quantity of pupil of human coordinate is obtained by using average curvature principle, obtain the accurate variable quantity of display device screen centre coordinate, visual angle is played so as to adjust panoramic video, visual angle when adjust automatically panoramic video plays is realized, use is simple and convenient.
Description
Technical field
The present invention relates to image processing techniques more particularly to a kind of panoramic video visual angle regulating methods based on human eye positioning
With device.
Background technology
In the broadcasting of existing panorama, when user is when watching panoramic video, need to manually adjust viewing visual angle, it would be desirable to
The content of viewing is adjusted into display picture.However, this operation in the broadcasting visual angle for manually adjusting panoramic video is excessively cumbersome, effect
Rate is relatively low.
Invention content
The brief overview about the present invention is given below, in order to provide about the basic of certain aspects of the invention
Understand.It should be appreciated that this general introduction is not the exhaustive general introduction about the present invention.It is not intended to determine the pass of the present invention
Key or pith, nor is it intended to limit the scope of the present invention.Its purpose only provides certain concepts in simplified form,
In this, as the preamble in greater detail discussed later.
In consideration of it, the present invention provides a kind of panoramic video visual angle regulating method and device based on human eye positioning, so that
It is few to solve the problem of to need to manually adjust in the prior art in panoramic video playing process to play visual angle.
According to an aspect of the invention, there is provided a kind of panoramic video visual angle regulating method based on human eye positioning, institute
Panoramic video visual angle regulating method is stated to include：Obtain the offset of pupil of human coordinate；According to the inclined of the pupil of human coordinate
Shifting amount determines the broadcasting visual angle of current panorama video, to play current panorama video according to the broadcasting visual angle.
Further, the step of offset of the acquisition pupil of human coordinate includes：Including in acquisition presumptive area
The image sequence of face；Image procossing is carried out to the described image sequence of acquisition to obtain two field pictures pair in described image sequence
The offset of pupil of human centre coordinate answered is used as the offset of the pupil of human coordinate.
Further, the described image sequence of described pair of acquisition carries out image procossing to obtain two frames in described image sequence
The step of offset of the corresponding pupil of human centre coordinate of image, includes：For each frame image in described image sequence,
The human face region in the frame image is positioned, and takes upper three/part of the human face region as ocular image, using flat
Equal curvature principle obtains pupil of human centre coordinate in the ocular image；It is corresponded to according to two field pictures in described image sequence
Pupil of human centre coordinate variation, to obtain the offset (X1- of the corresponding pupil of human centre coordinate of the two field pictures
X0, Y1-Y0), wherein, X0 and Y0 represent the pupil of human center in the two field pictures before the corresponding movement of previous frame image respectively
The abscissa and ordinate of coordinate, X1 and Y1 represent the human eye pupil after the corresponding movement of a later frame image in the two field pictures respectively
The abscissa and ordinate of hole centre coordinate.
Further, the step that pupil of human centre coordinate in the ocular image is obtained using average curvature principle
Suddenly include：Gaussian smoothing denoising is carried out to the ocular image, uses standard deviation as θ=((n/2) -1) × 0.3+0.8 and choosing
N=11 template sizes is taken to be handled；Single order and second order derivation are carried out to the ocular image, led according to its single order and second order
Numerical value calculates corresponding formation curve and all motion vectors for being directed toward isophote center respectively, is created and added according to formation curve
Power ballot array, weighted want motion vector, to generate centralizing mapping region；Gaussian kernel and accumulator are subjected to convolution, made every
A ballot group forms respectively single estimation center, and maximum isocenter point is sat as pupil of human center in the ocular image
Mark.
Further, the offset according to the pupil of human coordinate determines the broadcasting visual angle of current panorama video,
To be included the step of current panorama video to play according to the broadcasting visual angle：Mapping table is pre-set, wherein, a left side for the mapping table
The diverse vector (X1-X0, Y1-Y0) for pupil of human coordinate is held, the right end of the mapping table is the display equipment screen for playing panorama
The diverse vector (Xd, Yd) of curtain centre coordinate, wherein, Xd and Yd represent the display device screen center for playing panorama respectively
Variable quantity of the coordinate on abscissa and ordinate；According in the corresponding display device screen of the diverse vector of pupil of human coordinate
The diverse vector (Xd, Yd) and display device pixel size (Ws, Hs) of heart coordinate determine that panorama spherical shape picture attitude angle changes
It measures (α, β)；Wherein, Ze represents to show that coordinate value, R represent the radius of panorama spherical shape image on Z axis under equipment view frustums coordinate system,
fovvIt is angle component of the field angle in X-axis, fovhIt is angle component of the field angle in Y-axis, wherein：
Panoramic video is adjusted according to attitude angle variable quantity and plays visual angle to play out, attitude angle variable quantity is relative to people
The variable quantity of posture before and after the movement of eye pupil hole.
In addition, the embodiment of the present invention additionally provides a kind of panoramic video viewing angle adjustment device positioned based on human eye, institute
Panoramic video viewing angle adjustment device is stated to include：Offset acquiring unit, suitable for obtaining the offset of pupil of human coordinate；Visual angle tune
Whole unit suitable for the offset according to the pupil of human coordinate, determines the broadcasting visual angle of current panorama video, to be broadcast according to this
Visual angle is put to play current panorama video.
Further, the offset acquiring unit includes：Image capture module, suitable for including in acquisition presumptive area
The image sequence of face；Image processing module, suitable for obtaining the figure to the described image sequence of acquisition progress image procossing
As the offset of the corresponding pupil of human centre coordinate of two field pictures in sequence, it is used as the offset of the pupil of human coordinate
Amount.
Further, described image processing module is suitable for carrying out figure by handling as follows come the described image sequence to acquisition
As handling the offset to obtain the corresponding pupil of human centre coordinate of two field pictures in described image sequence：For described image
Each frame image in sequence, positions the human face region in the frame image, and takes upper three/part of the human face region to be
Ocular image obtains pupil of human centre coordinate in the ocular image using average curvature principle；According to the figure
As the variation of the corresponding pupil of human centre coordinate of two field pictures in sequence, to obtain in the corresponding pupil of human of the two field pictures
The offset (X1-X0, Y1-Y0) of heart coordinate, wherein, X0 and Y0 represent the corresponding shifting of previous frame image in the two field pictures respectively
The abscissa and ordinate of pupil of human centre coordinate before dynamic, X1 and Y1 represent a later frame image pair in the two field pictures respectively
The abscissa and ordinate of pupil of human centre coordinate after the movement answered.
Further, described image processing module is suitable for obtaining human eye pupil in the ocular image by handling as follows
Hole centre coordinate：Gaussian smoothing denoising is carried out to the ocular image, uses standard deviation as θ=((n/2) -1) × 0.3+0.8
And it chooses n=11 template sizes and is handled；Single order and second order derivation are carried out to the ocular image, according to its single order and two
Order derivative value calculates corresponding formation curve and all motion vectors for being directed toward isophote center respectively, is created according to formation curve
Build Nearest Neighbor with Weighted Voting array, weighted wants motion vector, to generate centralizing mapping region；Gaussian kernel and accumulator are subjected to convolution,
Each ballot group is made to form respectively single estimation center, maximum isocenter point is as in pupil of human in the ocular image
Heart coordinate.
Further, the visual angle adjustment unit is suitable for：Mapping table is pre-set, wherein, the left end of the mapping table is behaved
The diverse vector (X1-X0, Y1-Y0) of eye pupil coordinate, the right end of the mapping table are the display device screen centers for playing panorama
The diverse vector (Xd, Yd) of coordinate, wherein, Xd and Yd represent that the display device screen centre coordinate for playing panorama exists respectively
Variable quantity on abscissa and ordinate；According to the corresponding display device screen centre coordinate of the diverse vector of pupil of human coordinate
Diverse vector (Xd, Yd) and display device pixel size (Ws, Hs) determine panorama spherical shape picture attitude angle variable quantity (α,
β)；Wherein, Ze represents to show that coordinate value, R represent the radius of panorama spherical shape image, fov on Z axis under equipment view frustums coordinate systemv
It is angle component of the field angle in X-axis, fovhIt is angle component of the field angle in Y-axis, wherein：
Panoramic video is adjusted according to attitude angle variable quantity and plays visual angle to play out, attitude angle variable quantity is relative to people
The variable quantity of posture before and after the movement of eye pupil hole.
The panoramic video visual angle regulating method and device based on human eye positioning of the present invention, application image treatment technology are true
Determine the position of viewer's pupil of human, panorama is controlled to show the change of content by the movement of the position of pupil of human, realized
Basis be pupil of human and play panorama equipment can ensure to relatively move.Compared with the prior art, it is of the invention based on
The panoramic video visual angle regulating method of human eye positioning obtains the essence of pupil of human coordinate with device by using average curvature principle
So as to obtain the accurate variable quantity of display device screen centre coordinate, visual angle is played so as to adjust panoramic video for true variable quantity.It is real
Visual angle when adjust automatically panoramic video plays is showed, use is simple and convenient.
By the way that below in conjunction with detailed description of the attached drawing to highly preferred embodiment of the present invention, these and other of the invention is excellent
Point will be apparent from.
Description of the drawings
The present invention can be by reference to being better understood, wherein in institute below in association with the description given by attached drawing
Have and the same or similar reference numeral has been used in attached drawing to represent same or similar component.The attached drawing is together with following
The part for including in the present specification and being formed this specification together is described in detail, and for this is further illustrated
The preferred embodiment and explanation the principle of the present invention and advantage of invention.In the accompanying drawings：
Fig. 1 is positioned based on human eye panoramic video visual angle regulating method example for schematically showing the present invention
Property processing flow chart；
Fig. 2 is the stream for a realization method for schematically showing the step of carrying out image procossing to the image sequence of acquisition
Cheng Tu；
Fig. 3 is to schematically show to obtain pupil of human centre coordinate in the ocular image using average curvature principle
The step of a realization method flow chart；
Fig. 4 is the offset schematically shown according to pupil of human coordinate, determines the broadcasting visual angle of current panorama video
The step of a realization method flow chart；
Fig. 5 is an exemplary flow chart of application for schematically showing the adjustment of panoramic video visual angle；
Fig. 6 is to schematically show panoramic video viewing angle adjustment device the exemplary structure positioned based on human eye
Figure；
Fig. 7 is a kind of block diagram for the possible structure for schematically showing offset acquiring unit.
It will be appreciated by those skilled in the art that element in attached drawing is just for the sake of showing for the sake of simple and clear,
And be not necessarily drawn to scale.For example, the size of certain elements may be exaggerated relative to other elements in attached drawing, with
Just the understanding to the embodiment of the present invention is helped to improve.
Specific embodiment
The exemplary embodiment of the present invention is described hereinafter in connection with attached drawing.For clarity and conciseness,
All features of actual implementation mode are not described in the description.It should be understood, however, that developing any this actual implementation
It must be made during example much specific to the decision of embodiment, to realize the objectives of developer, for example, symbol
Conjunction and system and those relevant restrictive conditions of business, and these restrictive conditions may have with the difference of embodiment
Changed.In addition, it will also be appreciated that although development is likely to be extremely complex and time-consuming, to having benefited from the disclosure
For those skilled in the art of content, this development is only routine task.
Herein, it is also necessary to which explanation is a bit, in order to avoid because having obscured the present invention during unnecessary details, in the accompanying drawings
The apparatus structure closely related with scheme according to the present invention and/or processing step are illustrate only, and is omitted and the present invention
The little other details of relationship.
The embodiment provides a kind of panoramic video visual angle regulating method based on human eye positioning, the aphoramas
Frequency visual angle regulating method includes：Obtain the offset of pupil of human coordinate；According to the offset of the pupil of human coordinate, determine
The broadcasting visual angle of current panorama video, to play current panorama video according to the broadcasting visual angle.
Fig. 1 gives positioned based on human eye panoramic video visual angle regulating method exemplary process of the present invention
Flow chart.Wherein, the panoramic video visual angle regulating method based on human eye positioning is such as can be in the smart mobile phone of such as user
It is performed in electronic equipment.
As shown in Figure 1, in this example, step S110 is first carried out, in step s 110, pupil of human coordinate is obtained
Then offset, performs step S120.
In the step s 120, according to the offset of pupil of human coordinate, the broadcasting visual angle of current panorama video is determined, with root
Current panorama video is played according to the broadcasting visual angle.
According to a realization method, can for example be wrapped the step of the offset of acquisition pupil of human coordinate in step S110
It includes：Acquire the image sequence for including face in presumptive area；Image procossing is carried out to the image sequence of acquisition to obtain image
The offset of the corresponding pupil of human centre coordinate of two field pictures in sequence is used as the offset of pupil of human coordinate.
Wherein, collecting device is (as described below in the presumptive area e.g. electronic equipments such as smart mobile phone
Image capture module 710) corresponding pickup area.
In addition, above-mentioned " it is corresponding to obtain two field pictures in image sequence to carry out image procossing to the image sequence of acquisition
" two field pictures " in the offset of pupil of human centre coordinate " can for example choose first frame image and last frame in image sequence
Image, for example, can using first frame image as the corresponding image of pupil of human before mobile, and using end-frame image as moving after
The corresponding image of pupil of human.
Wherein, image procossing is carried out to the image sequence of acquisition to obtain the corresponding human eye pupil of two field pictures in image sequence
The step of offset of hole centre coordinate, can for example include step as shown in Figure 2.
As shown in Fig. 2, it first, in step S210, for each frame image in image sequence, positions in the frame image
Human face region, and upper three/part of the human face region is taken average curvature principle to be utilized to obtain for ocular image
Pupil of human centre coordinate in the ocular image.
Then, in step S220, according to the variation of the corresponding pupil of human centre coordinate of two field pictures in image sequence,
Obtain the offset (X1-X0, Y1-Y0) of the corresponding pupil of human centre coordinate of the two field pictures, wherein, X0 and Y0 difference tables
Show the abscissa and ordinate of the pupil of human centre coordinate in the two field pictures before the corresponding movement of previous frame image, X1 and Y1
The abscissa and ordinate of the pupil of human centre coordinate in the two field pictures after the corresponding movement of a later frame image are represented respectively.
According to a realization method, the utilization average curvature principle described in step S210 is obtained in the ocular image
The step of pupil of human centre coordinate, can for example include：Gaussian smoothing denoising is carried out to the ocular image, using standard
Difference is handled for θ=((n/2) -1) × 0.3+0.8 and selection n=11 template sizes；Single order is carried out to the ocular image
With second order derivation, corresponding formation curve and all direction isophote centers are calculated respectively according to its single order and second derivative values
Motion vector, Nearest Neighbor with Weighted Voting array is created according to formation curve, weighted wants motion vector, to generate centralizing mapping region；
Gaussian kernel and accumulator are subjected to convolution, each ballot group is made to form respectively single estimation center, maximum isocenter point is (i.e. most
Big estimation center) as pupil of human centre coordinate in the ocular image.Step as shown in Figure 3 may be used in the process
Suddenly.
Wherein, θ is standard deviation, and n is Gauss denoising coefficients, here, first to image denoising, with Gauss denoising, mould
Plate coefficient is bigger, and denoising effect is better, selects the Gauss denoising coefficients of n=11, can obtain preferable denoising effect.
In addition, centralizing mapping region, that is, accumulator, above-mentioned Gaussian kernel and accumulator carry out convolution be to accumulator into
Row gaussian filtering.Ballot group is the motion vector after weighting, i.e. centralizing mapping region.
According to a realization method, the offset according to pupil of human coordinate described in step S120 determines current complete
The broadcasting visual angle of scape video, can for example be included the step of playing current panorama video such as Fig. 4 institutes according to the broadcasting visual angle
The step of showing.
As shown in figure 4, first, in step S410, mapping table is pre-set, wherein, the left end of the mapping table is human eye
The diverse vector (X1-X0, Y1-Y0) of pupil coordinate, the right end of the mapping table are the display device screen center seats for playing panorama
Target diverse vector (Xd, Yd), wherein, the display device screen centre coordinate that Xd and Yd represent to play panorama respectively is in abscissa
With the variable quantity on ordinate.
Then, in the step s 420, sat according to the corresponding display device screen center of the diverse vector of pupil of human coordinate
Target diverse vector (Xd, Yd) and display device pixel size (Ws, Hs) determine panorama spherical shape picture attitude angle variable quantity
(α, β), wherein, α is the angle component of spherical surface angle in the direction of the x axis between 2 points on spherical surface in spheroidal coordinate system, and β is
The angle component of spherical surface angle in the y-axis direction between 2 points on spherical surface in spheroidal coordinate system.In addition, Ze represents to show equipment
Coordinate value on Z axis under view frustums coordinate system, R represent the radius of panorama spherical shape image, fovvIt is the field angle angle in X-axis point
Amount, fovhIt is angle component of the field angle in Y-axis.
Panorama spherical shape picture attitude angle variable quantity (α, β) can obtain according to the following formula：
Then, in step S430, panoramic video is adjusted according to attitude angle variable quantity and plays visual angle to play out, posture
Angle variable quantity is the variable quantity that front and rear posture is moved relative to pupil of human.
Wherein, Fig. 5 also shows in the above-mentioned panoramic video visual angle regulating method positioned based on human eye how to carry out panorama
The application example of video visual angle adjustment.Wherein, human eye center, that is, above-described pupil of human center described in Fig. 5.
In addition, the embodiment of the present invention additionally provides a kind of panoramic video viewing angle adjustment device positioned based on human eye, entirely
Scape video viewing angle adjustment device includes：Offset acquiring unit, suitable for obtaining the offset of pupil of human coordinate；Visual angle adjustment is single
Member suitable for the offset according to pupil of human coordinate, determines the broadcasting visual angle of current panorama video, with according to the broadcasting visual angle come
Play current panorama video.
Fig. 6 shows panoramic video viewing angle adjustment device the exemplary structure chart positioned based on human eye.Such as Fig. 6 institutes
Show, panoramic video viewing angle adjustment device includes offset acquiring unit 610 and visual angle adjustment unit 620.
Wherein, offset acquiring unit 610 is suitable for obtaining the offset of pupil of human coordinate, and visual angle adjustment unit 620
Suitable for the offset according to pupil of human coordinate, the broadcasting visual angle of current panorama video is determined, to be broadcast according to the broadcasting visual angle
Put current panorama video.
According to a realization method, offset acquiring unit 610 is for example including image capture module 710 as shown in Figure 7
With image processing module 720.Wherein, image capture module 710 (such as high-definition camera) for example may be adapted to acquire fate
The image sequence for including face in domain.Wherein, image capture module is for example arranged on equipment (such as user mobile phone for playing panorama
Deng) on.
Image processing module 720 for example can carry out image procossing to obtain two in image sequence to the image sequence of acquisition
The offset of the corresponding pupil of human centre coordinate of frame image is used as the offset of pupil of human coordinate.
According to a realization method, image processing module 720 for example can be by handling as follows come the image sequence to acquisition
Row carry out image procossing to obtain the offset of the corresponding pupil of human centre coordinate of two field pictures in image sequence：For image
Each frame image in sequence, positions the human face region in the frame image, and takes upper three/part of the human face region to be
Ocular image obtains pupil of human centre coordinate in the ocular image using average curvature principle；According to image sequence
The variation of the corresponding pupil of human centre coordinate of two field pictures in row is sat to obtain the corresponding pupil of human center of the two field pictures
Target offset (X1-X0, Y1-Y0), wherein, X0 and Y0 are represented in the two field pictures respectively before the corresponding movement of previous frame image
Pupil of human centre coordinate abscissa and ordinate, X1 and Y1 represent that a later frame image is corresponding in the two field pictures respectively
The abscissa and ordinate of pupil of human centre coordinate after movement.
According to a realization method, image processing module 720 for example can obtain the ocular by handling as follows
Pupil of human centre coordinate in image：Gaussian smoothing denoising is carried out to the ocular image, uses standard deviation as θ=((n/
2) -1) × 0.3+0.8 and selection n=11 template sizes are handled；Single order and second order derivation are carried out to the ocular image,
Corresponding formation curve and all motion vectors for being directed toward isophote center are calculated respectively according to its single order and second derivative values,
Motion vector is wanted according to formation curve establishment Nearest Neighbor with Weighted Voting array, weighted, to generate centralizing mapping region；By Gaussian kernel and tire out
Device is added to carry out convolution, each ballot group is made to form respectively single estimation center, maximum isocenter point is as the ocular figure
The pupil of human centre coordinate as in.
According to a realization method, visual angle adjustment unit 620 can for example perform following processing：Pre-set mapping table,
Wherein, the left end of the mapping table is the diverse vector (X1-X0, Y1-Y0) of pupil of human coordinate, and the right end of the mapping table is to play
The diverse vector (Xd, Yd) of the display device screen centre coordinate of panorama, wherein, Xd and Yd represent the broadcasting panorama respectively
Variable quantity of the display device screen centre coordinate on abscissa and ordinate；It is corresponded to according to the diverse vector of pupil of human coordinate
Display device screen centre coordinate diverse vector (Xd, Yd) and display device pixel size (Ws, Hs) determine panorama ball
Shape picture attitude angle variable quantity (α, β)；Wherein, Ze represents to show that coordinate value, R represent complete on Z axis under equipment view frustums coordinate system
The radius of scape spherical shape image, fovvIt is angle component of the field angle in X-axis, fovhIt is angle component of the field angle in Y-axis,
Wherein：
Panoramic video is adjusted according to attitude angle variable quantity and plays visual angle to play out, attitude angle variable quantity is relative to people
The variable quantity of posture before and after the movement of eye pupil hole.
Although the embodiment according to limited quantity describes the present invention, above description, the art are benefited from
It is interior it is clear for the skilled person that in the scope of the present invention thus described, it can be envisaged that other embodiments.Additionally, it should be noted that
The language that is used in this specification primarily to readable and introduction purpose and select rather than in order to explain or limit
Determine subject of the present invention and select.Therefore, in the case of without departing from the scope and spirit of the appended claims, for this
Many modifications and changes will be apparent from for the those of ordinary skill of technical field.For the scope of the present invention, to this
The done disclosure of invention is illustrative and not restrictive, and it is intended that the scope of the present invention be defined by the claims appended hereto.
Claims (10)
1. the panoramic video visual angle regulating method based on human eye positioning, which is characterized in that the panoramic video visual angle regulating method
Including：
Obtain the offset of pupil of human coordinate；
According to the offset of the pupil of human coordinate, the broadcasting visual angle of current panorama video is determined, with according to the broadcasting visual angle
To play current panorama video.
2. the panoramic video visual angle regulating method according to claim 1 based on human eye positioning, which is characterized in that described to obtain
The step of offset for taking pupil of human coordinate, includes：
Acquire the image sequence for including face in presumptive area；
Image procossing is carried out to the described image sequence of acquisition to obtain the corresponding human eye pupil of two field pictures in described image sequence
The offset of hole centre coordinate is used as the offset of the pupil of human coordinate.
3. the panoramic video visual angle regulating method according to claim 2 based on human eye positioning, which is characterized in that described right
The described image sequence of acquisition carries out image procossing to obtain the corresponding pupil of human center of two field pictures in described image sequence
The step of offset of coordinate, includes：
For each frame image in described image sequence, the human face region in the frame image is positioned, and take the human face region
Upper three/part is ocular image, and pupil of human center in the ocular image is obtained using average curvature principle
Coordinate；
According to the variation of the corresponding pupil of human centre coordinate of two field pictures in described image sequence, to obtain the two field pictures pair
The offset (X1-X0, Y1-Y0) for the pupil of human centre coordinate answered, wherein, X0 and Y0 represent previous in the two field pictures respectively
The abscissa and ordinate of pupil of human centre coordinate before the corresponding movement of frame image, X1 and Y1 represent the two field pictures respectively
The abscissa and ordinate of pupil of human centre coordinate after the corresponding movement of middle a later frame image.
4. the panoramic video visual angle regulating method according to claim 3 based on human eye positioning, which is characterized in that the profit
The step of obtaining pupil of human centre coordinate in the ocular image with average curvature principle includes：
Gaussian smoothing denoising is carried out to the ocular image, use standard deviation as θ=((n/2) -1) × 0.3+0.8 and chooses n
=11 template sizes are handled；
Single order and second order derivation are carried out to the ocular image, corresponding life is calculated according to its single order and second derivative values respectively
Into curve and all motion vectors for being directed toward isophote center, creating Nearest Neighbor with Weighted Voting array, weighted according to formation curve will
Motion vector, to generate centralizing mapping region；
Gaussian kernel and accumulator are subjected to convolution, each ballot group is made to form respectively single estimation center, maximum isocenter point
As pupil of human centre coordinate in the ocular image.
5. the panoramic video visual angle regulating method based on human eye positioning according to any one of claim 1-4, feature
It is, the offset according to the pupil of human coordinate, the broadcasting visual angle of current panorama video is determined, with according to the broadcasting
Visual angle includes to play the step of current panorama video：
Mapping table is pre-set, wherein, the left end of the mapping table is the diverse vector (X1-X0, Y1-Y0) of pupil of human coordinate,
The right end of the mapping table is the diverse vector (Xd, Yd) for the display device screen centre coordinate for playing panorama, wherein, Xd and Yd divide
Variable quantity of the display device screen centre coordinate for playing panorama on abscissa and ordinate is not represented；
According to the diverse vector (Xd, Yd) of the corresponding display device screen centre coordinate of the diverse vector of pupil of human coordinate, with
And display device pixel size (Ws, Hs) determines panorama spherical shape picture attitude angle variable quantity (α, β)；Wherein, Ze represents that display is set
Coordinate value, R represent the radius of panorama spherical shape image, fov on Z axis under standby view frustums coordinate systemvIt is angle of the field angle in X-axis
Component, fovhIt is angle component of the field angle in Y-axis, wherein：
Panoramic video broadcasting visual angle is adjusted according to attitude angle variable quantity to play out.
6. the panoramic video viewing angle adjustment device based on human eye positioning, which is characterized in that the panoramic video viewing angle adjustment device
Including：
Offset acquiring unit, suitable for obtaining the offset of pupil of human coordinate；
Visual angle adjustment unit suitable for the offset according to the pupil of human coordinate, determines the broadcasting visual angle of current panorama video,
To play current panorama video according to the broadcasting visual angle.
7. the panoramic video viewing angle adjustment device according to claim 6 based on human eye positioning, which is characterized in that described inclined
Shifting amount acquiring unit includes：
Image capture module, suitable for the image sequence for including face in acquisition presumptive area；
Image processing module, suitable for obtaining two frames in described image sequence to the described image sequence of acquisition progress image procossing
The offset of the corresponding pupil of human centre coordinate of image is used as the offset of the pupil of human coordinate.
8. the panoramic video visual angle regulating method according to claim 7 based on human eye positioning, which is characterized in that the figure
As processing module is suitable for obtaining described image sequence to the described image sequence of acquisition progress image procossing by handling as follows
The offset of the corresponding pupil of human centre coordinate of two field pictures in row：
For each frame image in described image sequence, the human face region in the frame image is positioned, and take the human face region
Upper three/part is ocular image, and pupil of human center in the ocular image is obtained using average curvature principle
Coordinate；
According to the variation of the corresponding pupil of human centre coordinate of two field pictures in described image sequence, to obtain the two field pictures pair
The offset (X1-X0, Y1-Y0) for the pupil of human centre coordinate answered, wherein, X0 and Y0 represent previous in the two field pictures respectively
The abscissa and ordinate of pupil of human centre coordinate before the corresponding movement of frame image, X1 and Y1 represent the two field pictures respectively
The abscissa and ordinate of pupil of human centre coordinate after the corresponding movement of middle a later frame image.
9. the panoramic video viewing angle adjustment device according to claim 4 based on human eye positioning, which is characterized in that the figure
As processing module is suitable for obtaining pupil of human centre coordinate in the ocular image by handling as follows：
Gaussian smoothing denoising is carried out to the ocular image, use standard deviation as θ=((n/2) -1) × 0.3+0.8 and chooses n
=11 template sizes are handled；
Single order and second order derivation are carried out to the ocular image, corresponding life is calculated according to its single order and second derivative values respectively
Into curve and all motion vectors for being directed toward isophote center, creating Nearest Neighbor with Weighted Voting array, weighted according to formation curve will
Motion vector, to generate centralizing mapping region；
Gaussian kernel and accumulator are subjected to convolution, each ballot group is made to form respectively single estimation center, maximum isocenter point
As pupil of human centre coordinate in the ocular image.
10. the panoramic video viewing angle adjustment device based on human eye positioning according to any one of claim 6-9, feature
It is, the visual angle adjustment unit is suitable for：
Mapping table is pre-set, wherein, the left end of the mapping table is the diverse vector (X1-X0, Y1-Y0) of pupil of human coordinate,
The right end of the mapping table is the diverse vector (Xd, Yd) for the display device screen centre coordinate for playing panorama, wherein, Xd and Yd divide
Variable quantity of the display device screen centre coordinate for playing panorama on abscissa and ordinate is not represented；
According to the diverse vector (Xd, Yd) of the corresponding display device screen centre coordinate of the diverse vector of pupil of human coordinate, with
And display device pixel size (Ws, Hs) determines panorama spherical shape picture attitude angle variable quantity (α, β)；Wherein, Ze represents that display is set
Coordinate value, R represent the radius of panorama spherical shape image, fov on Z axis under standby view frustums coordinate systemvIt is angle of the field angle in X-axis
Component, fovhIt is angle component of the field angle in Y-axis, wherein：
Panoramic video broadcasting visual angle is adjusted according to attitude angle variable quantity to play out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810205657.XA CN108235132A (en) | 2018-03-13 | 2018-03-13 | Panoramic video visual angle regulating method and device based on human eye positioning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810205657.XA CN108235132A (en) | 2018-03-13 | 2018-03-13 | Panoramic video visual angle regulating method and device based on human eye positioning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108235132A true CN108235132A (en) | 2018-06-29 |
Family
ID=62658477
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810205657.XA Withdrawn CN108235132A (en) | 2018-03-13 | 2018-03-13 | Panoramic video visual angle regulating method and device based on human eye positioning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108235132A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108921097A (en) * | 2018-07-03 | 2018-11-30 | 深圳市未来感知科技有限公司 | Human eye visual angle detection method, device and computer readable storage medium |
| CN110930340A (en) * | 2019-10-11 | 2020-03-27 | 成都华为技术有限公司 | Image processing method and device |
| WO2021218473A1 (en) * | 2020-04-27 | 2021-11-04 | 海信视像科技股份有限公司 | Display method and display device |
| CN114071392A (en) * | 2021-12-28 | 2022-02-18 | 智小途(上海)数字科技有限公司 | UWB indoor high-precision three-dimensional live-action data construction method and system |
| CN114449162A (en) * | 2021-12-22 | 2022-05-06 | 天翼云科技有限公司 | Method and device for playing panoramic video, computer equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070255098A1 (en) * | 2006-01-19 | 2007-11-01 | Capso Vision, Inc. | System and method for in vivo imager with stabilizer |
| CN102323829A (en) * | 2011-07-29 | 2012-01-18 | 青岛海信电器股份有限公司 | Display screen visual angle regulating method and display device |
| CN102799277A (en) * | 2012-07-26 | 2012-11-28 | 深圳先进技术研究院 | Wink action-based man-machine interaction method and system |
| CN102830797A (en) * | 2012-07-26 | 2012-12-19 | 深圳先进技术研究院 | Man-machine interaction method and system based on sight judgment |
| CN105323552A (en) * | 2015-10-26 | 2016-02-10 | 北京时代拓灵科技有限公司 | Method and system for playing panoramic video |
| CN105809701A (en) * | 2016-03-25 | 2016-07-27 | 成都易瞳科技有限公司 | Panorama video posture calibrating method |
| CN106020461A (en) * | 2016-05-13 | 2016-10-12 | 陈盛胜 | Video interaction method based on eyeball tracking technology |
| CN106534827A (en) * | 2016-12-19 | 2017-03-22 | 暴风集团股份有限公司 | Method and system for playing panoramic video based on user perspective |
| CN106817625A (en) * | 2016-12-19 | 2017-06-09 | 广州视源电子科技股份有限公司 | Adjust method, device, system and the control terminal of panoramic video viewing visual angle |
-
2018
- 2018-03-13 CN CN201810205657.XA patent/CN108235132A/en not_active Withdrawn
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070255098A1 (en) * | 2006-01-19 | 2007-11-01 | Capso Vision, Inc. | System and method for in vivo imager with stabilizer |
| CN102323829A (en) * | 2011-07-29 | 2012-01-18 | 青岛海信电器股份有限公司 | Display screen visual angle regulating method and display device |
| CN102799277A (en) * | 2012-07-26 | 2012-11-28 | 深圳先进技术研究院 | Wink action-based man-machine interaction method and system |
| CN102830797A (en) * | 2012-07-26 | 2012-12-19 | 深圳先进技术研究院 | Man-machine interaction method and system based on sight judgment |
| CN105323552A (en) * | 2015-10-26 | 2016-02-10 | 北京时代拓灵科技有限公司 | Method and system for playing panoramic video |
| CN105809701A (en) * | 2016-03-25 | 2016-07-27 | 成都易瞳科技有限公司 | Panorama video posture calibrating method |
| CN106020461A (en) * | 2016-05-13 | 2016-10-12 | 陈盛胜 | Video interaction method based on eyeball tracking technology |
| CN106534827A (en) * | 2016-12-19 | 2017-03-22 | 暴风集团股份有限公司 | Method and system for playing panoramic video based on user perspective |
| CN106817625A (en) * | 2016-12-19 | 2017-06-09 | 广州视源电子科技股份有限公司 | Adjust method, device, system and the control terminal of panoramic video viewing visual angle |
Non-Patent Citations (2)
| Title |
|---|
| 白瑞广: ""全景视频系统若干关键技术研究"", 《中国优秀硕士学位论文全文数据库》 * |
| 罗兴贤: ""基于视频图像处理的视线方向跟踪研究"", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108921097A (en) * | 2018-07-03 | 2018-11-30 | 深圳市未来感知科技有限公司 | Human eye visual angle detection method, device and computer readable storage medium |
| CN110930340A (en) * | 2019-10-11 | 2020-03-27 | 成都华为技术有限公司 | Image processing method and device |
| CN110930340B (en) * | 2019-10-11 | 2023-09-29 | 成都华为技术有限公司 | Image processing method and device |
| WO2021218473A1 (en) * | 2020-04-27 | 2021-11-04 | 海信视像科技股份有限公司 | Display method and display device |
| CN114449162A (en) * | 2021-12-22 | 2022-05-06 | 天翼云科技有限公司 | Method and device for playing panoramic video, computer equipment and storage medium |
| CN114071392A (en) * | 2021-12-28 | 2022-02-18 | 智小途(上海)数字科技有限公司 | UWB indoor high-precision three-dimensional live-action data construction method and system |
| CN114071392B (en) * | 2021-12-28 | 2023-07-25 | 智小途(上海)数字科技有限公司 | UWB indoor high-precision three-dimensional live-action data construction method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108235132A (en) | Panoramic video visual angle regulating method and device based on human eye positioning | |
| US10855909B2 (en) | Method and apparatus for obtaining binocular panoramic image, and storage medium | |
| Chen et al. | Study of 3D virtual reality picture quality | |
| CN107079141B (en) | Image mosaic for 3 D video | |
| EP3398163A1 (en) | Stitching frames into a panoramic frame | |
| CN102959945B (en) | The method and system of virtual output image is produced according to the data obtained by the array of image-capturing apparatus | |
| CN105894567B (en) | Scaling pixel depth values of user-controlled virtual objects in a three-dimensional scene | |
| US11880939B2 (en) | Embedding complex 3D objects into an augmented reality scene using image segmentation | |
| JP7033865B2 (en) | Image generator, image generation method, and program | |
| CN107079100A (en) | The method and system that Lens shift for camera array is corrected | |
| CN105141841B (en) | Picture pick-up device and its method | |
| EP1903498B1 (en) | Creating a panoramic image by stitching a plurality of images | |
| CN110035223A (en) | Semiconductor equipment and electronic equipment | |
| CN112492223A (en) | Camera lens smoothing method and device and portable terminal | |
| CN108282694A (en) | Panoramic video rendering intent, device and electronic equipment | |
| CN115272570A (en) | Virtual expression generation method and device, electronic equipment and storage medium | |
| US11640058B2 (en) | Image processing method, image processing apparatus, and head-mounted display | |
| CN103248910B (en) | Three-dimensional imaging system and image reproducing method thereof | |
| CN117255184B (en) | Correction method for side projection blur of projector | |
| US9071832B2 (en) | Image processing device, image processing method, and image processing program | |
| CN108124108B (en) | Method for balancing pixels in spherical panoramic image/video | |
| CN111292234B (en) | Panoramic image generation method and device | |
| CN107124590A (en) | Panoramic video compression method, storage medium and video camera based on sound source direction | |
| CN106791366A (en) | A kind of microshot method and terminal | |
| CN109923854B (en) | Image processing apparatus, image processing method, and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20180629 |
|
| WW01 | Invention patent application withdrawn after publication |