CN104980732B - The system and method for measuring the potential eye fatigue of stereoscopic motion picture - Google Patents
The system and method for measuring the potential eye fatigue of stereoscopic motion picture Download PDFInfo
- Publication number
- CN104980732B CN104980732B CN201510315583.1A CN201510315583A CN104980732B CN 104980732 B CN104980732 B CN 104980732B CN 201510315583 A CN201510315583 A CN 201510315583A CN 104980732 B CN104980732 B CN 104980732B
- Authority
- CN
- China
- Prior art keywords
- parallax
- eye fatigue
- image
- fragment
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2213/00—Details of stereoscopic systems
- H04N2213/002—Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Eye Examination Apparatus (AREA)
Abstract
There is provided it is a kind of be used to measure when watching 3D and (for example, stereoscopic motion picture) being presented by the audience's feeling to potential eye fatigue system and method.The measurement of parallax (or depth) and parallax conversion of the eye fatigue measuring system and method for the disclosure based on the 3D stereo-pictures presented.The system and method regulation of the disclosure：The first image and the second image (202) are obtained from the first fragment, estimate the parallax (204) of at least one corresponding points at least one point and the second image in the first image, estimate that the parallax conversion (206) of the sequence of the first and second images, and the parallax conversion of the sequence based on parallax and the first and second images determine the potential eye fatigue (208) experienced when watching 3D and presenting.
Description
The cross reference of related application
The present invention is the applying date on May 12nd, 2008, Application No. 200880129158.1, entitled " is used to survey
The divisional application of the application for a patent for invention of the system and method for measuring the potential eye fatigue of stereoscopic motion picture ".
Technical field
Present disclose relates generally to computer graphical processing and display system, and relate more specifically to a kind of for measuring
By viewing three-dimensional (3D) present the audience's feeling of (for example, stereoscopic motion picture) to potential eye fatigue system and method.
Background technology
Three-dimensional imaging be visually combine at least two images obtained from slightly different viewpoint of scene with
Just the processing of the illusion of three dimensional depth is produced.The technology is dependent on following true：Human eye separates certain distance, and does not therefore have
Watch identical scene.By providing the image from different points of observation (perspective) for each eye, deception is seen
The eyes for the person of seeing feel depth.Typically, in the case where providing two distinguishing points of observation, composition image is claimed respectively
For " left side " and " right side " image, benchmark image and supplemental image are also known as respectively.However, those skilled in the art will recognize that
Arrive, more than two viewpoint can be combined to form stereo-picture.
In application is presented in 3D post-production, VFX workflows and 3D, an important processing is from by left-eye view figure
Depth map is inferred in the stereo-picture of picture and right eye view image composition, to create stereoscopic motion picture.For example, recently commercially
The Autostereoscopic 3D display of change requires image plus depth figure (image-plus-depth-map) pattern of the input so that the display
Device can generate different 3D views to support multiple visual angles.
The processing for inferring depth map from stereo pairs is referred to as Stereo matching in the field of computer vision research, because
To find the corresponding points in left eye and right eye view image using pixel or Block- matching.Identical point from scene is corresponding
Relative distance between two pixels in each image infers depth value.
Many computer visions application (such as, for example, for computer-assisted mapping (CAD) rapid object modeling and
Shaping, Object Segmentation and detection, video compress and vision monitoring (visual for man-machine interaction (HCI)
Surveillance the Stereo matching of digital picture is widely used in)), to provide three-dimensional (3D) depth information.Stereo matching
Two or more cameras positioned at diverse location and direction from scene obtain the image of scene.In the about the same time from every
Individual camera obtains these digital pictures, and corresponding to the point in each image of 3D Point matchings in space.Generally, search is passed through
A part of image and the point from different images is matched using constraint (such as epipolar line restriction), to make in an image
Point is associated with the point in another image.Then three-dimensional 3D motion can be created using the image after matching and depth map to draw
Face.
One of subject matter of current three-dimensional 3D motion picture is that spectators may be after motion picture certain time be watched
Experience eye fatigue.Therefore, work as director make 3D films when, they have to take into account that as how can make by the audience's feeling to
Eye fatigue minimize mode photographed scene or edit a film,.This is to make 3D motion picture than making traditional 2D fortune
The reason in part for animation face is more complicated and time-consuming.
Make 3D motion picture challenge be to direct and editor be very difficult to visually to estimate by the audience's feeling to
Potential eye fatigue.In the presence of facilitating the difficult a number of factors.Firstly, because eye fatigue is with viewing motion picture
The accumulative effect of process, so director or editor have to watch 3D motion picture long enough to experience eye fatigue.Cause
Do not cause eye fatigue generally for a small amount of fragment.Second, eye fatigue is also possible to by the unexpected change in depth between two fragments
Cause.When editor's junction fragment during editing, it is tired that they are difficult to measurement potential eyes as caused by unexpected change in depth
Labor.They will need to use time-consuming repetition test (trial-and-error) process to connect different fragments, and " sense
By " the potential eye fatigue as caused by being changed depth.
Accordingly, there exist for can watch such as stereoscopic motion picture 3D present while measurement experience it is potential
The needs of the technology of eye fatigue.Additionally, there are for that can measure potential eyes during the process of editing 3D motion picture
The automatic system of fatigue and the needs of method.
The content of the invention
There is provided it is a kind of be used to measure when watching 3D and (for example, stereoscopic motion picture) being presented by the audience's feeling to potential eye
The system and method for eyeball fatigue.The system and method for the disclosure consider：Watch when 3D is presented between the convergent point of eyes and focus
The depth of object of the distance with being focused on be closely related, the depth of the object focused on the also parallax phase with the pixel of object
Close.Parallax (or depth) and parallax conversion of the eye fatigue measuring system and method for the disclosure based on the 3D stereo-pictures presented
Measurement.It is useful that the technology of the disclosure effectively makes comfortable 3D films for director and editor.
According to an aspect of this disclosure, there is provided the potential eyes that one kind measurement is experienced when viewing three-dimensional (3D) is presented
The method of fatigue, methods described include obtaining the first image and the second image from the first fragment；Estimate in the first image at least
The parallax of at least one corresponding points in one point and the second image；Estimate the parallax conversion of the sequence of the first and second images；
And the sequence based on parallax and the first and second images parallax conversion determine watch 3D present when experience it is potential
Eye fatigue.
In another aspect, estimating disparity switch process includes：The parallax of the last frame of fragment before estimation；Estimation first
The parallax of first frame of fragment；And before determining between the parallax of the parallax of the last frame of fragment and the first frame of the first fragment
Difference.
In an aspect, estimating disparity switch process includes：Estimate regarding in each frame of multiple frames of the first fragment
Difference；And the difference between the parallax of each frame of the first fragment of determination.
In another aspect, determine that potential eye fatigue step also includes determining the sequence for the first and second images
The instant eye fatigue function of each frame.
In another aspect, determine that potential eye fatigue step also includes decay factor being applied to for first and second
The instant eye fatigue function of each frame of the sequence of image.
In another aspect, determine that potential eye fatigue step also includes the use after accumulation decay in predetermined time period
In the instant eye fatigue function of each frame of the sequence of the first and second images.
In another aspect, determine that potential eye fatigue step also includes making to accumulate in the sequence of the first and second images
Eye fatigue function saturation.
In another aspect, this method also includes：Determine whether potential eye fatigue is subjected to；And if potential eyes
Fatigue is unacceptable, then corrects the parallax of the first and second images.
According to another aspect of the present disclosure, there is provided one kind be used to measure experienced when viewing three-dimensional (3D) is presented it is potential
The system of eye fatigue.The system includes：For obtaining the part of the first image and the second image from the first fragment；Parallax is estimated
Gauge, for estimating the parallax of at least one corresponding points at least one point and the second image in the first image；Parallax turns
Estimator is changed, the parallax conversion of the sequence for estimating the first and second images；And eye fatigue estimator, for based on regarding
The conversion of the parallax of difference and the sequence of the first and second images determines the potential eye fatigue experienced when watching 3D and presenting.
According to another aspect of the present disclosure, there is provided a kind of program storage device that can be read by machine, described program storage
That equipment positively embodies is being performed by machine, tired for performing to measure potential eyes when viewing three-dimensional (3D) is presented
The program of the instruction of the method and step of labor, methods described include：The first image and the second image are obtained from the first fragment；Estimation the
The parallax of at least one corresponding points at least one point and the second image in one image；Estimate the sequence of the first and second images
The parallax conversion of row；And the parallax conversion of the sequence based on parallax and the first and second images is determined when watching 3D presentations
The potential eye fatigue experienced.
Brief description of the drawings
These and other aspects, the feature and advantage of the disclosure, or the these and other aspects of the disclosure, spy will be described
Advantage of seeking peace will become obvious from the described in detail below of preferred embodiment, and preferred embodiment is read in conjunction with the figure in detailed below
Description.
In the accompanying drawings, wherein representing similar element through each accompanying drawing identical reference number：
Fig. 1 is shown in convergent point and Jiao while viewing three-dimensional (3D) is presented by the member of beholder or spectators experience
Point；
Fig. 2 diagram when convergent point for show 3D present screen before when, focusing distance, the depth perceived,
The relation of parallax and convergence angle；
Fig. 3 is illustrated when convergent point is behind screen, focusing distance, the depth perceived, the pass of parallax and convergence angle
System；
Fig. 4 is according to an aspect of this disclosure, for measuring the eyes experienced when viewing three-dimensional (3D) is presented
The graphical representation of exemplary of the system of fatigue；
Fig. 5 is according to an aspect of this disclosure, for measuring the eyes experienced when viewing three-dimensional (3D) is presented
The flow chart of the illustrative methods of fatigue；
Fig. 6 illustrates the sigmoid function of the saturation modeling for feeling eye fatigue；
Fig. 7 illustrates measures equation according to the eye fatigue over time of an aspect of this disclosure；And
Fig. 8 diagrams measure according to the instant eye fatigue function of an aspect of this disclosure and overall eye fatigue.
It should be understood that：(each) accompanying drawing is for illustrating the purpose of the design of the disclosure, and is needed not be for illustrating
The only possible configuration of the disclosure.
Embodiment
It should be understood that：Each element shown in accompanying drawing can be realized with the various forms of hardware, software and its combination.
Preferably, these elements are with hardware and appropriate in the one or more that can include processor, memory and input/output interface
The combination of software on the common apparatus of ground programming is realized.
This description illustrates the principle of the disclosure.Accordingly, it will be recognized that：Those skilled in the art can design implementation originally
Principle disclosed is simultaneously included in the various arrangements in the spirit and scope of present principles, although herein without clearly description or
The arrangement is shown.
All examples and conditional language described herein are intended for the purpose of teaching to help reader to understand this public affairs
The principle opened and contributed by the present inventor to promote the design of prior art, and should be interpreted to be not restricted to this tool
The example and condition of body narration.
In addition, all statements for describing principle, aspect and the embodiment and its particular example of the disclosure herein are intended to wrap
Include its 26S Proteasome Structure and Function equivalent.In addition, it is intended that：Such equivalent includes currently known equivalent and developed in the future
Equivalent both, that is, developed execution identical function any element, regardless of whether its structure how.
Thus, for example, it would be recognized by those skilled in the art that：The block diagram presented herein represents to implement the illustrative of present principles
The conceptual view of circuit.Similarly, it will be recognized that：Any flow diagram (flow chart), flow chart (flow
Diagram), the expression such as state transition graph, false code can be substantially represented in computer-readable medium and therefore by calculating
The various processing of machine or computing device, irrespective of whether clearly showing such computer or processor.
It can be provided by using specialized hardware and the hardware that is able to carry out software associated with appropriate software
The function of the various elements shown in figure.When utilizing a processor to provide the function, can utilize single application specific processor,
The function is provided using single shared processor or the multiple independent processors that can be shared using some of them.Separately
Outside, term " processor " or " controller " are clearly able to carry out the hard of software using being not construed as exclusively referring to
Part, but impliedly can unrestrictedly include digital signal processor (" DSP ") hardware, the read-only storage for storing software
Device (" ROM "), random access memory (" RAM ") and nonvolatile memory.
The hardware of other conventional and/or customs can also be included.Similarly, any switch shown in figure is concept
Property.Their function can by the operation of programmed logic, pass through special logic, the friendship by programme-control and special logic
Even perform mutually or manually, as more specifically understood from context, specific skill can be selected by implementer
Art.
In its claim, be represented as the part for performing specified function any element be intended to comprising perform that
Any mode of individual function, such as including：A) perform the circuit element of that function combination or b) with proper circuit phase group
Any type of software closed, therefore including firmware or microcode etc., the proper circuit is used to perform the software software
To perform the function.The disclosure limited by this claim resides in the fact that, i.e. with the side required by claim
The function of being provided by various described parts is combined and brought together by formula.It is taken as that appointing for those functions can be provided
What part and those parts being shown in which are equivalent.
Generally, stereoscopic motion picture is made up of left eye and right eye picture sequence.Applied for cinema, spectators generally must not
3D glasses (for example, polarization or shutter glasses) viewing 3D is not worn to present.For the display without glasses, although 3D display
Mechanism is different, but the principle that 3D is perceived is identical.That is, 3D systems will make left eye watch left-eye image, and right eye
Only watch eye image.The two images can be grouped together by human brain, correctly to perceive 3D scenes.However, the system
The fatigue and eye fatigue of eyes muscle may be potentially resulted in, because following two reasons：
1. the convergent point and focus of eyes are different.When we watch true 3D scenes, the focus of our eyes with
Convergent point is roughly the same.However, when we watch 3D films, the focus 10 of our eyes is had to always on the screen 12,
And the convergent point 14 of our eyes is had in the above or below of screen 12, to perceive correct 3D scenes, such as Fig. 1 institutes
Show.The difference is the principal element for promoting the fatigue of eyes muscle therefore to cause eye fatigue.
The depth of object changes in 2.3D motion pictures.When the depth of object changes, our eyes have to adjust
Convergent point keeps focus on screen to perceive correct 3D effect.If depth change is frequent and unexpected, I
Eyes must not infrequently change convergent point, cause the tired out of eyes muscle.
So in brief, eye fatigue is mainly caused by two reasons：1) between the convergent point of eyes and focus away from
From (that is, focusing distance 16 as shown in Figure 1)；And 2) the change of convergent point 14.Therefore, measuring eye fatigue needs in consideration
Two, face factor.
Present disclose provides one kind be used for measure watch 3D present (for example, stereoscopic motion picture) when by the audience's feeling to
Potential eye fatigue system and method.The eye fatigue measuring system and method for the disclosure are based on parallax (or depth) and regarded
The measurement of difference conversion.This solution is useful for directing 3D films effectively made with editor and comfortable.
The system and method for the disclosure consider：During the distance between the convergent point of the eyes of beholder and focus are presented with 3D
The depth of the object focused on is closely related, and parallax of the depth of object that 3D is focused in presenting also with the pixel of object has
Close.Fig. 2 shows the relation for the variable that viewing 3D is related in presenting.It can be seen that：Given focusing distance (Cd) 16, perceived
Depth (Dp) 18, spectators' distance (Ad) 20, convergence angle (Ca) 22, eye distance (Ed) 24 and parallax (Ds) 26, are present following
Relation.
1.Cd, Dp and Ad relation：Ad=Cd+Dp
2.Cd, Ds, Ad and Ed relation：Cd (1/Ds+1/Ed)=Ad/Ed
3.Ca, Ed, Ad and Cd relation：Cd=2atan (Ed/ (2 (Ad-Cd)))
When convergent point as shown in Figure 3 behind screen when, these relations keep correct, as long as allowing negative parallax and negative convergence
Distance.For niche audience, eye distance (Ed) 24 and spectators' distance (Ad) 20 are constant during presentation, and are assembled
Distance (Cd) 16, the depth (Dp) 18 perceived, convergence angle (Ca) 22 and parallax (Ds) 26 change during motion picture.Base
In these relations, the calculating of convergent point can be reduced to the estimation of depth or parallax.This causes simpler algorithm for estimating.Because
Many Disparity estimations be present, so traditional parallax measurement can be used for eye fatigue estimation.
Referring now to Fig. 4, exemplary system components in accordance with an embodiment of the present disclosure are shown.Scanning device can be provided
103, for being numeral by film printed document (film print) 104 (for example, camera original negative film (film negative)) scanning
Form (such as Cineon forms or SMPTE DPX files).Scanning device 103 can include for example will be defeated from film generation video
The telecine gone out or any equipment (Arri LocPro such as, such as with video frequency outputTM).Alternately, can be direct
Use the file 106 (for example, being in the file of computer-reader form) from post production process or digital movie.Meter
The potential source of calculation machine readable documents is AVIDTMEditor, DPX file, D5 tapes.
The film printed document scanned is input into postprocessing device 102 (for example, computer).With following hardware
The computer, such as one or more centres of the hardware are realized on any computer platform of various known calculations machine platforms
Manage unit (CPU), the memory 110 and such as key of such as random access memory (RAM) and/or read-only storage (ROM)
(multiple) input/output (I/O) user interface of disk and pointer control device (for example, mouse or control stick) and display device
112.Computer platform also includes operating system and micro-instruction code.Various processing described here and function can be via behaviour
Make a part for micro-instruction code or a part of (or combinations thereof) of software application for system execution.In one embodiment
In, software application is positively embodied on program storage device, and it can upload to any suitable machine (such as later stage
Processing equipment 102) and be performed.In addition, various other ancillary equipment can be by various interfaces and bus structures (such as simultaneously
Row port, serial port or USB (USB)) it is connected to computer platform.Other ancillary equipment can include extra
Storage device 124 and printer (printer) 128.Printer 128 can be used for the (example of revision 126 for developing and printing film
Such as, the stereoscopic version of film), wherein the result as technology described below, may be changed using the object of 3D modeling
Or alternative scenario or multiple scenes.
Alternately, in computer-reader form (for example, can be stored on external fixed disk drive 124
Digital movie) file/film printed document 106 can be directly inputted in computer 102.It is noted that term " film as used herein
(film) film printed document or digital movie can " be referred to.
Software program includes the eye fatigue measurement stored in the memory 110 and reduces module 114, is being seen for measuring
See the potential eye fatigue experienced when 3D is presented by spectators or beholder.
Eye fatigue, which measures and reduced module 114, includes disparity estimator 116, is arranged to estimate in the first image
The parallax (the first and second images produce stereo pairs) of at least one corresponding points at least one point and the second image, and
And for each at least one point in the first image and at least one corresponding points in the second image from estimated
Parallax generation disparity map.Disparity estimator 116 includes being configured as the pixel for matching the pixel in the first and second images
With cost function 132, and for smoothness constraint to be applied to the smooth cost function 134 of disparity estimation.Disparity estimator 116
Also include being used for the belief propagation algorithm or function 136 and DYNAMIC PROGRAMMING algorithm or letter for minimizing estimated parallax
Number 138, the DYNAMIC PROGRAMMING algorithm or function 138 utilize the result of the determination adaptation function applied to the first and second images
To initialize belief propagation function 136 to accelerate belief propagation function 136.It is to be understood that belief propagation and dynamic routine
Design is only for determining two illustrative methods of parallax, and can use other disparity estimations by disparity estimator 116
Method and algorithm.
Parallax conversion estimator 118 is provided, for the conversion or change of the parallax for determining stereo-picture.Eye fatigue is provided
Estimator 120, for the associated figure based on the disparity map from disparity estimator 116 and from parallax conversion estimator 118
The parallax conversion of picture, estimates potential eye fatigue.
Eye fatigue measure and reduce module 114 also include parallax offset mechanism 122, for adjustment the first and second images it
Between parallax to reduce potential eye fatigue.Depth map generator 123 is also provided, for by inverting (invert) disparity map
Parallax value and disparity map is converted into depth map.In one embodiment, depth map generator 123 reduces potential eyes with being used for
The parallax offset mechanism 122 of fatigue cooperates, as will be described below.
Fig. 5 is according to the flow chart of the illustrative methods of an aspect of this disclosure, and the illustrative methods are used to measure extremely
The potential eye fatigue of few two two dimension (2D) images.Initially, postprocessing device 102 obtains at least two two-dimentional (2D) figures
Picture, such as the stereo pairs (step 202) with left eye and right-eye view.Postprocessing device 102 can be by being located
At least two 2D are obtained in the digital standard image file (digital master image file) of computer-readable format
Image.Digital video file can be obtained by using the time series of digital camera capture movement image.Alternately, can be with
Video sequence is captured by traditional film type cameras.In this scenario, film is scanned via scanning device 103.
It is to be understood that either scanning film or film are in number format, the digital document of film will wrap
Include instruction or the information of the position on each frame, for example, frame number, since the film time etc..Digital image file it is every
Individual frame will include an image, for example, I1、I2、…In。
Stereo-picture can be shot by two cameras with identical setting.Any one of camera is demarcated as with phase
Same focal length, focal spot height and parallel focal plains；Or known camera parameter is must be based on to deflect (warp) image, seem
They are shot by the camera with parallel focal plains.The deflection includes camera calibration (camera calibration) and phase
Machine adjustment (camera rectification).Demarcation and the EP point of calibration procedures adjustment stereo-picture so that EP point essence
It is really the horizontal scanning line of image.Because corresponding point discovery occurs along EP point, calibration procedures are simple by corresponding search
Turn to only along scanning line search, this greatly reduces calculating cost.Corresponding points correspond to the picture of same scene point in image
Element.
Next, in step 204, via disparity estimator 116 for each point estimation parallax in scene (for example, frame)
Figure.Parallax for each scene point is calculated as the relative distance of the match point in left eye and eye image, i.e. find it is left and
The pixel corresponding to same scene point in right image.For example, if the horizontal coordinate of a point is x in left-eye image, and
The horizontal coordinate of its corresponding points is x ' in eye image, then parallax d=x '-x.
In one embodiment of estimating disparity, initially, stereo pairs are obtained.Calculate parallax cost function, including meter
Calculate pixel matching cost function 132 and calculate smooth cost function 134.Inexpensive Stereo matching optimization is performed (for example, dynamic journey
Sequence design function 138), to obtain the originally determined result of two images of Stereo matching.Then the result of low cost optimization is used
In initialization belief propagation function 136, to accelerate belief propagation function to minimize parallax cost function.It is to be understood that
Other methods for disparity estimation are known in the art, and can be by the system and method for the disclosure using described other
Method.
In step 206, determine that parallax is changed.The basic parallax conversion that two types be present：Filmstrip (for example, scene,
Camera lens etc.) between be probably unexpected parallax conversion, it is and generally contiguous in filmstrip (for example, scene, camera lens etc.)
Parallax conversion.As discussed below, fragment means the sequence of the frame comprising identical content.
Parallax conversion between fragment is typically discontinuous and unexpected.So in order to measure parallax conversion, estimate first
The disparity map of the start frame of the last frame of fragment and current clip before.In other words, it is assumed that regarding at the end of the i-th fragment
Poor figure is Di, and the disparity map at the beginning of (i+1) fragment is Di+1, then parallax difference is
δ D=Di+1-Di (1)
It is poor using absolute parallax for the amplitude for the change that fathoms
| δ D |=| Di+1-Di| (2)
Change to obtain overall parallax, changed using maximum disparity, it is
Or changed using mean parallax, it is
Wherein W and H is the width and height of disparity map.
For the picture in fragment, parallax conversion is typically continuous, but the big parallax conversion in a small amount of time also will
Promote the eye fatigue of spectators.Change similar to the parallax between fragment, parallax difference can be used as measurement, i.e. δ D=Di+1-
Di.However, not only the measurement will be performed in piece section boundary in each frame.Here, determine in the pixel inside a frame
Mean parallax value in mean parallax value rather than the pixel of each frame of leap, then calculates the difference between each successive frame.
Being changed by big parallax and/or parallax causes eye fatigue.Therefore, in step 208, changed based on disparity map and parallax
Estimate eye fatigue.The system and method for the disclosure utilize very rough eye fatigue model.The system and method are assumed to deposit
By the eye fatigue function related to the parallax at every frame and parallax change, and eye fatigue sense is tired out across each frame
Product, but exponential damping over time.
Firstly it is assumed that there exists " instant eye fatigue function "It will the eye fatigue as caused by parallax
With mean parallaxChanged with parallaxIt is related.Then, if parallax remains zero after the i-th frame, eyes are tired
Labor measurement can be expressed as attenuation model, as follows
WhereinIt is to model function of the parallax to the immediate impact of eye fatigue.λ is attenuation coefficient.Should
If model hypothesis no longer have parallax (that is, convergent point is changed into focus) on screen, then eye fatigue is by fast fade.Pay attention to
Arrive：Because the suddenly change of parallax is there may be between each fragment, therefore functionFor the frame in fragment
Frame between each fragment is probably different, and this is the reason for they are treated differently for printing above.FunctionSimple examples can beWithBetween linear combination, it is as follows：
Wherein a and b is weight coefficient, and it should be not for the parallax conversion between the parallax conversion in fragment and fragment
With.A and b value can be with empirically determined.
When parallax keeps changing across each frame, then, eye fatigue senses this and accumulated over time.However, eye
Eyeball fatigue can not rise forever, it is possible to model the diminuendo effect of eye fatigue sense over time using function.S function is used
It is as follows in the saturation modeling to eye fatigue sense：
S (ES)=1/ (1+exp (- ES)) (7)
Figure 6 illustrates the shape of the function.
These key elements are given, overall eye fatigue is measured and can recursively defined at frame i, as follows：
Wherein ESi(ti) it is eye fatigue measurement in the i-th frame, and tiIt it is the time of the i-th frame, and λ is to be used to control
The constant of the rate of decay.The calculating of the measurement can be realized by simulation over time.
Fig. 7 is the diagram of eye fatigue measurement equation over time.It is determined that in the eye fatigue of (i-t) frame, and
And exp (- λ t) decay factor is applied to eye fatigue measurement.As a result the instant eye fatigue group of functions and then with the i-th frame
Close.Sigmoid function is applied to combined result to determine the eye fatigue in the i-th frame.
Reference picture 8, it is illustrated that instant eye fatigue function and the measurement of overall eye fatigue.In each frame (for example, frame 1,2
Deng) on curve be the frame instant eye fatigue function, and the curve in measurement point is instant eye fatigue function
Accumulation.Referring again to Fig. 5, in step 210, it is determined that whether the overall eye fatigue undergone by beholder is subjected to.In a reality
Apply in example, the measurement of overall eye fatigue is visualized as curve map to operator (for example, director and editor), so as to according to song
Line chart determines whether eye fatigue is too high.In another embodiment, overall eye fatigue can be measured compared with predetermined threshold,
To determine the need for correcting.For example, generation represents the curve of overall eye fatigue measurement, then by each point on curve
Value compared with predetermined threshold.In this embodiment, for different types of scene and film, predetermined threshold will be different.
If it is determined that eye fatigue is too high, then parallax correction or mediation will be performed to stereo-picture in step 212
(grading) to reduce eye fatigue.Mediation is the process for the parallax value for smoothly crossing over each frame.Pass through smoothly regarding across each frame
Difference, it is possible to reduce the suddenly change of parallax, and therefore can reduce eye fatigue.For reducing an exemplary side of parallax
Method is referred to as assembling adjustment, and it adjusts convergent point to realize by offseting eye image to the left or to the right.By to the left or to
Right avertence moves eye image, can manually decrease or increase the parallax of pixel, cause smaller or greater overall depth respectively.
The problem of image shift, is the depth increase equal amount of all pixels, and this is unrelated with 3D scene geometries.So
And if accurate depth map is available, the new view of the scene with new virtual camera positions may be synthesized, so as to
Reduce parallax.In this embodiment, using below equation z=Bf/d, (wherein B is the distance between two cameras, also referred to as base
Line, and f is the focal length of camera), the parallax value d for each scene point being determined above is changed by depth map generator 123
For depth value z, (distance i.e. from scene point to camera).Each depth of at least one image (for example, left-eye view image)
Angle value is stored in depth map.The depth value of depth map accordingly changes, to synthesize new view while to reduce the parallax of image.
For example, in order to create new view, change depth value first, then repaint new left or right (or both) image.Again
Drawing process creates the new right side (or left) image using left (or right) image and depth map.Depth value is the 3D letters of pixel
Breath, therefore, it is possible to use the technology of such as ray trace, to be the 2D pixels in new view by 3D point-renderings.New regards
Figure will be tired by the eyes for causing reduction for beholder or spectators with smaller parallax or the conversion of smaller parallax, and therefore
Labor.
Referring now to Fig. 4, such as correspondence image and associated depth map are stored in storage device 124, and can be with
It is retrieved and is played back for 3D.In addition, all correction chart pictures of motion picture or video clipping can be deposited with associated depth map
Store up in the individual digit file 130 of stereoscopic version of motion picture or editing is represented.Digital document 130 can be stored in storage
Retrieved after being used in equipment 124, for example, the stereoscopic version for developing and printing original film.
Although the embodiment for the teaching for being incorporated to the disclosure, people in the art have been shown specifically and described herein
Member can easily design the embodiment for any other change for being still incorporated to these teachings.Have been described and watched for measuring
(it is intended to illustrative rather than limit to the preferred embodiment of the system and method for the potential eye fatigue that 3D is experienced when presenting
Property processed), it is noted that it can modify and change according to teaching those skilled in the art above.It will consequently be understood that
It can be carried out in the specific embodiment of the disclosed disclosure, in the scope of the present disclosure such as by appended claims general introduction
Change.
Claims (22)
1. the method for potential eye fatigue of one kind measurement when viewing three-dimensional (3D) is presented, methods described include：
The first image and the second image (202) are obtained from the first fragment；
Estimate the parallax (204) of at least one corresponding points at least one point and the second image in the first image；
Estimate the parallax conversion (206) of the sequence of the first and second images；And
The parallax conversion of sequence based on parallax and the first and second images determines that the potential eyes when watching 3D and presenting are tired
Labor (208)；
Wherein the first image and the second image produce the three-dimensional stereo pairs presented.
2. the method as described in claim 1, wherein estimating disparity switch process (206) also include：
The parallax of the last frame of fragment before estimation；
Estimate the parallax of the first frame of the first fragment；And
It is determined that the difference between the parallax of the parallax of the last frame of fragment and the first frame of the first fragment before.
3. the method as described in claim 1, wherein estimating disparity switch process (206) also include：
Estimate the parallax in each frame of multiple frames of the first fragment；And
Determine the difference between the parallax of each frame of the first fragment.
4. the method as described in claim 1, wherein determining that potential eye fatigue step (208) also includes determining to be used for the first He
The instant eye fatigue function of each frame of the sequence of second image.
5. method as claimed in claim 4, wherein determining that potential eye fatigue step (208) is also included decay factor application
To the instant eye fatigue function of each frame of the sequence for the first and second images.
6. method as claimed in claim 5, wherein determining that potential eye fatigue step (208) is additionally included in predetermined time period
The instant eye fatigue function of each frame of the sequence for the first and second images after upper accumulation decay.
7. method as claimed in claim 6, wherein determining that potential eye fatigue step (208) also includes making first and second
The eye fatigue function saturation accumulated in the sequence of image.
8. the method as described in claim 1, in addition to：
Determine whether potential eye fatigue is acceptable (210)；And
If potential eye fatigue is unacceptable, the parallax (212) of the first and second images is corrected.
9. method as claimed in claim 8, wherein aligning step (212) also include at least the one of the first and second images of skew
It is individual, to adjust convergent point of the eyes of beholder relative to the first and second images.
10. method as claimed in claim 8, wherein aligning step (212) also have what is reduced including the first fragment of synthesis
The new view of parallax.
11. one kind is used for the system (100) for measuring the potential eye fatigue when viewing three-dimensional (3D) is presented, the system bag
Include：
For obtaining the part of the first image and the second image from the first fragment；
Disparity estimator (116), for estimate the first image at least one point with it is at least one corresponding in the second image
The parallax of point；
Parallax conversion estimator (118), the parallax conversion of the sequence for estimating the first and second images；And
Eye fatigue estimator (120), the parallax conversion for the sequence based on parallax and the first and second images determine
Watch potential eye fatigue when 3D is presented；
Wherein the first image and the second image produce the three-dimensional stereo pairs presented.
12. system (100) as claimed in claim 11, wherein parallax conversion estimator (118) operation are come fragment before estimating
Last frame parallax, estimate the parallax of the first frame of the first fragment, and before determining the last frame of fragment parallax and the
Difference between the parallax of first frame of one fragment.
13. system (100) as claimed in claim 11, wherein parallax conversion estimator (118) are operated to estimate the first fragment
Multiple frames each frame in parallax, and determine the first fragment each frame parallax between difference.
14. system (100) as claimed in claim 11, wherein eye fatigue estimator (120) are operated to determine to be used for first
With the instant eye fatigue function of each frame of the sequence of the second image.
15. the operation of system (100) as claimed in claim 14, wherein eye fatigue estimator (120) should by decay factor
Use the instant eye fatigue function of each frame of the sequence for the first and second images.
16. the operation of system (100) as claimed in claim 15, wherein eye fatigue estimator (120) came in the predetermined time
The instant eye fatigue function of each frame of the sequence for the first and second images in section after accumulation decay.
17. system (100) as claimed in claim 16, wherein eye fatigue estimator (120) operation makes first and the
The eye fatigue function saturation accumulated in the sequence of two images.
18. system (100) as claimed in claim 11, in addition to parallax offset mechanism (122), if eye fatigue estimator
(120) determine that potential eye fatigue is unacceptable, then parallax offset mechanism (122) is operated to correct regarding for the first and second images
Difference.
19. system (100) as claimed in claim 18, wherein parallax offset mechanism (122) are operated to offset the first and second figures
Picture it is at least one, to adjust convergent point of the eyes of beholder relative to the first and second images.
20. system (100) as claimed in claim 18, wherein parallax offset mechanism (122) are operated to synthesize the tool of the first fragment
There is the new view of the parallax of reduction.
21. a kind of program storage device that can be read by machine, described program storage device, which positively embodies, to be performed by machine
, the program of the instruction of method and step for performing to measure the potential eye fatigue when viewing three-dimensional (3D) is presented,
Methods described includes：
The first image and the second image (202) are obtained from the first fragment；
Estimate the parallax (204) of at least one corresponding points at least one point and the second image in the first image；
Estimate the parallax conversion (206) of the sequence of the first and second images；And
The parallax conversion of sequence based on parallax and the first and second images determines that the potential eyes when watching 3D and presenting are tired
Labor (208)；
Wherein the first image and the second image produce the three-dimensional stereo pairs presented.
22. program storage device as claimed in claim 21, wherein methods described also include：
Determine whether potential eye fatigue is acceptable (210)；And
If potential eye fatigue is unacceptable, the parallax (212) of the first and second images is corrected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510315583.1A CN104980732B (en) | 2008-05-12 | 2008-05-12 | The system and method for measuring the potential eye fatigue of stereoscopic motion picture |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510315583.1A CN104980732B (en) | 2008-05-12 | 2008-05-12 | The system and method for measuring the potential eye fatigue of stereoscopic motion picture |
| CN200880129158.1A CN102027752B (en) | 2008-05-12 | 2008-05-12 | For measuring the system and method for the potential eye fatigue of stereoscopic motion picture |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200880129158.1A Division CN102027752B (en) | 2008-05-12 | 2008-05-12 | For measuring the system and method for the potential eye fatigue of stereoscopic motion picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104980732A CN104980732A (en) | 2015-10-14 |
| CN104980732B true CN104980732B (en) | 2018-01-16 |
Family
ID=54288312
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510315583.1A Expired - Fee Related CN104980732B (en) | 2008-05-12 | 2008-05-12 | The system and method for measuring the potential eye fatigue of stereoscopic motion picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104980732B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005177076A (en) * | 2003-12-18 | 2005-07-07 | Sophia Co Ltd | Game machine |
| CN1838778A (en) * | 2005-03-24 | 2006-09-27 | 精工爱普生株式会社 | Stereo image display device and method |
| JP4121881B2 (en) * | 2003-03-28 | 2008-07-23 | 株式会社トプコン | Three-dimensional observation state measuring apparatus and method |
-
2008
- 2008-05-12 CN CN201510315583.1A patent/CN104980732B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4121881B2 (en) * | 2003-03-28 | 2008-07-23 | 株式会社トプコン | Three-dimensional observation state measuring apparatus and method |
| JP2005177076A (en) * | 2003-12-18 | 2005-07-07 | Sophia Co Ltd | Game machine |
| CN1838778A (en) * | 2005-03-24 | 2006-09-27 | 精工爱普生株式会社 | Stereo image display device and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104980732A (en) | 2015-10-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102027752B (en) | For measuring the system and method for the potential eye fatigue of stereoscopic motion picture | |
| CN102047288B (en) | System and method for depth extraction of images with forward and backward depth prediction | |
| JP5153940B2 (en) | System and method for image depth extraction using motion compensation | |
| US7557824B2 (en) | Method and apparatus for generating a stereoscopic image | |
| EP2618584B1 (en) | Stereoscopic video creation device and stereoscopic video creation method | |
| US8798160B2 (en) | Method and apparatus for adjusting parallax in three-dimensional video | |
| US20140009462A1 (en) | Systems and methods for improving overall quality of three-dimensional content by altering parallax budget or compensating for moving objects | |
| US6388666B1 (en) | System and method for generating stereoscopic image data | |
| CN101542536A (en) | System and method for compositing 3D images | |
| KR20080076628A (en) | Image display device for improving three-dimensional effect of stereo-scopic image and method thereof | |
| JP2013517677A (en) | System and method for compositing 3D text with 3D content | |
| US20140125778A1 (en) | System for producing stereoscopic images with a hole filling algorithm and method thereof | |
| CN104980732B (en) | The system and method for measuring the potential eye fatigue of stereoscopic motion picture | |
| KR20110025083A (en) | Apparatus and method for displaying 3d image in 3d image system | |
| KR101046580B1 (en) | Image processing apparatus and control method | |
| CA2252063C (en) | System and method for generating stereoscopic image data | |
| CN117616760A (en) | Image generation | |
| JP6391629B2 (en) | System and method for compositing 3D text with 3D content | |
| KR20170035885A (en) | Stereoscopic video zooming and foreground and background detection in a video | |
| BR112021007522A2 (en) | image generator apparatus, image generation method and computer program product | |
| JP2007194694A (en) | Three-dimensional video photographing apparatus and program thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190111 Address after: Paris France Patentee after: Interactive Digital Madison Patent Holdings Address before: I Si Eli Murli Nor, France Patentee before: THOMSON LICENSING Effective date of registration: 20190111 Address after: I Si Eli Murli Nor, France Patentee after: THOMSON LICENSING Address before: I Si Eli Murli Nor, France Patentee before: THOMSON LICENSING |
|
| TR01 | Transfer of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180116 Termination date: 20190512 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |