CN107071392A

CN107071392A - Image processing method and device

Info

Publication number: CN107071392A
Application number: CN201611213537.1A
Authority: CN
Inventors: 赵超亚
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2016-12-23
Filing date: 2016-12-23
Publication date: 2017-08-18

Abstract

The application is related to image processing method and device, and wherein the image processing method includes：Obtain digital hologram；According to predetermined condition, it is determined whether carry out Three-dimensional Display；And when it is determined that carrying out Three-dimensional Display, the image for Three-dimensional Display is generated based on the digital hologram；Or when it is determined that during without Three-dimensional Display, the image for two dimension display is generated based on the digital hologram.Switching between Three-dimensional Display and two dimension display can neatly be realized by the image processing method and device of the application, so as to enhance the effect that Consumer's Experience, the continuity for improving display image and various applications show image.

Description

Image processing method and device

Technical field

The application is related to a kind of image processing method and device.It is preferred that, the application is related to a kind of be used in two dimension display The image procossing treating method and apparatus switched between pattern and three-dimensional display mode.

Background technology

Virtual reality technology is one of a kind of technology for having feeling of immersion, interactivity and imagination, its interactive mode realized It is exactly three-dimensional stereo display technique.3 D stereo, which is shown, can comprehensively show position, level and the depth of image, make observation Person more intuitively understands the real distribution situation of image, so as to more comprehensively understand image or stereoprojection.

At present, what is applied in virtual reality is binocular solid technology：The right and left eyes of people are respectively from diverse location and angle The small horizontal image position occurred when watching same object in retina of both eyes knot picture is poor, i.e. horizontal parallax.3 D stereo is shown General principle seek to manually reappear parallax, will have horizontal parallax left-eye image and eye image difference It is conveyed to the left eye and right eye of observer so that observer can perceive three-dimensional depth, forms third dimension.On this basis Miscellaneous three-dimensional stereo display technique is developed, current relative maturity is that the 3 D stereo based on glasses is shown.

Realize that the method flow of one of the prior art of virtual reality is as shown in figure 26, it is projected to by virtual video camera On screen, and the human eye ken is modeled by virtual video camera view frustums.Then the three-dimensional body in view frustums is passed through Perspective projection change is projected on viewport, and Viewport element finally is rendered into indicator screen.Space any point is being obtained on a left side Under conditions of respective coordinates and two camera parameters matrixes in right image, you can carry out the reconstruction of spatial point, that is, realize and be based on The virtual reality of human eye vision principle is shown.

Above-mentioned prior art is the mainstream technology of current virtual reality Stereo display.Its core is obtained in three-dimensional scenic Any point meets the left images of human eye demand, and left images are inputted into left eye and right eye respectively, so as to produce stereoscopic vision Effect.

And as image processing techniques is developed rapidly in today's society, augmented reality (Augmented Reality, Abbreviation AR) technology is also widely used for image processing field.Augmented reality (AR) technology refers to generate by computer The technology that virtual objects increase user perceives to real world, can be by virtual Information application to real world, and will calculate Dummy object, scene or the system prompt information superposition of machine generation are into real scene, so as to realize the enhancing to reality.Enhancing The maximum feature of real (AR) technology is, by three-dimensional registration method, virtual scene to be overlapped with real scene, and is allowed Object and the object of virtual environment that user is watched in real scene simultaneously, obtain the visual experience and information for surmounting true scenery Service.The key technology of augmented reality (AR) includes：Actual situation object matches (being properly termed as three-dimensional registration process again) and three-dimensional are aobvious Show technology.Existing three-dimensional registration technology mainly includes：Based on the cognitive, three-dimensional ring based on digital image analysis and view-based access control model Border register method

Actual situation object matches (i.e. three-dimensional registration process) refer to the virtual objects to real scene image and computer generation Nothing is carried out to be stitched into.The detailed process that above-mentioned nothing is stitched into is：Real object first in real scene determines virtual right As the relation between observer, then, virtual objects are projected to the FOV of observer by correctly projecting.It is based on The Registration method of vision uses single camera, and information gathering is carried out to real scene and is turned using image-forming principle is imaged Change, three-dimensional body image is converted into two dimensional image, realize the coordinate that the coordinate representation of actual environment is virtual environment；So Afterwards, by recognizing and determining marker data model, i.e., by actual environment coordinate system with passing through the virtual coordinates obtained by shooting System is compared, and just can calculate position, direction and angle of the camera relative to real scene；There is camera After Position and orientation parameters, by coordinate system transformation, coordinate information that just can be in actual environment calculates virtual objects and existed Relevant information in real scene, that is, correct position of the virtual three-dimensional object of addition in real world required for determining.

In short, the Registration method of view-based access control model is to carry out information gathering and conversion by real scene, And coordinate system is contrasted and changed, virtual objects positional information in real scene is calculated, the mesh of augmented reality is reached 's.

Either virtual reality technology or augmented reality, can all provide a user Three-dimensional Display (or three-dimensional aobvious Show).Therefore, dimension display technologies are all referred to, stereoscopic vision has been formed.

However, in actual applications, however it remains carry out the demand of two-dimentional display (i.e. plane is shown).

For example, for the angle of Consumer's Experience, having many consumers and preferring the picture of two dimension display.Even like vertical The user of body vision effect, can also produce sense of fatigue, it is therefore desirable to watch after the image of long-time viewing stereoscopic visual effect The picture of two dimension display is adjusted.

Additionally, it is also well known that for two dimension display, Three-dimensional Display may require that bigger operand, therefore also need Resource distribution that will be higher.If resource distribution is (for example, the speed of the capacity of the arithmetic speed of processor, memory, data transfer Rate etc.) when not reaching the requirement of Three-dimensional Display, the fluency of picture will be affected, or even at all can not display picture.This In the case of kind, then the problem of needing to solve how to provide the user the smooth picture that can be watched.

In addition, in game design and video display product, for the story of a play or opera or bandwagon effect the need for, may be specific Plot or scene under carry out two-dimensional picture display, to reach special effect, if only carry out Three-dimensional Display, then obviously It is not enough.

As can be seen here, the demand for showing and switching over to two and three dimensions is there is in the prior art, to solve user The design such as experience, hardware configuration and game and video display needs proposed above mentioned problem.

And shown on 3 D stereo, from technological layer, virtual Display Technique still has problems with：

First, there is gap in 3-D effect of the binocular solid technology still with observation of nature environment.Reason of searching to the bottom is We are only showed the main information of needs by the form of left images, and the full detail of scene is not presented In face of human eye.So, conventional method can only be produced vertical because scene information is lacked and there are technology barriers under critical conditions Body-sensing, it is impossible to show true and comfortable 3-D effect.

Secondly, from algorithm and cost, conventional method generally carries out data using more than two polyphaser arrays and adopted Collection, optimizes gatherer process using polyphaser array, and the camera quantity that the method needs is big, and cost is too high, and a large amount of cameras Staking-out work it is complicated.

In addition, from real-time, conventional method necessarily uses the letter of two or more passages of left images Breath, thus contain much information, it is necessary to memory space it is big, efficiency of transmission is low, so as to cause arithmetic speed slow, system real time difference Problem.Therefore, a kind of memory space is found big and be easy to the data mode of transmission and improve efficiency of transmission and arithmetic speed, enhancing The practicality of system is particularly important.

And in augmented reality system, the three-dimensional registration of virtual objects must is fulfilled for correctness and robustness.Therefore, exist Under different research environments, the three-dimensional registration technology for meeting performance requirement is the bottleneck of current augmented reality system.

However, from its technological layer, the single camera image conversion process that such scheme is used can only obtain plane Information, is that can not obtain the three-dimensional information of real scene, wherein, three-dimensional information refers mainly to phase information.Because shooting imaging is only The change of subject surface light intensity is have recorded, the spatial distribution of subject luminous intensity, i.e. amplitude information can only be stored. The phase information of light wave can not be recorded, i.e., can not obtain depth information.So result in virtual objects and can only appear on true field On one two dimensional surface of scape.Therefore, from the Performance evaluation criterion of three-dimensional registration technology, the three-dimensional environment of view-based access control model Register method does not account for robustness, it is impossible to the attack in terms of resisting cropping, compression, filtering and noise.

As can be seen here, as the technology for providing a user three dimensional rendered images, either virtual reality technology still strengthens , there is the problem of needing to improve in reality technology.

Also, when proposing technology that new 3 D stereo show, there is that 3 D stereo is shown and two dimensional surface is shown again The related question of both patterns, that is, find less than good mode 3 D stereo can be shown with two dimensional surface show both Pattern is integrated.

As can be seen here, there is following technical problem in prior art：1) excessive 3 D stereo is shown or excessive two dimension is stood Body shows caused user-experience degradation；2) by configuring during relatively low device progress 3 D stereo display it is possible that can not Display or the inferior situation of display quality；3) content applied and shown needs the switching of Three-dimensional Display and two dimension display；4) Current three-dimensional stereo display technique cost is too high, the complicated, efficiency of transmission that works is low and arithmetic speed is slow；5) 3 D stereo shows Show and show the problem of implementation of both patterns can not associate integration well with two dimensional surface；6) computer hologram enters Row has zero-order terms and twin-image and conjugate image when reproducing causes diffraction efficiency not high, the problem of signal to noise ratio is very low.

Therefore, in the urgent need to proposition technology to solve technical problem listed above.

The content of the invention

For problems of the prior art, the application proposition one kind is used for can be in Three-dimensional Display and two-dimentional display Between the image processing method that switches over and device to solve problems of the prior art.

According to the one side of the application there is provided a kind of image processing method, including：Obtain digital hologram；According to pre- Fixed condition, it is determined whether carry out Three-dimensional Display.And when it is determined that carrying out Three-dimensional Display, generated and used based on the digital hologram In the image of Three-dimensional Display；Or when it is determined that during without Three-dimensional Display, being generated based on the digital hologram for two dimension display Image.

According to further aspect of the application there is provided a kind of image processing apparatus, including：Hologram acquisition module, is used for Obtain digital hologram；Determining module, for according to predetermined condition, it is determined whether carry out Three-dimensional Display；3-D view generates mould Block, for when it is determined that carrying out Three-dimensional Display, the image for Three-dimensional Display to be generated based on the digital hologram；Two dimensional image Generation module, for when it is determined that during without Three-dimensional Display, the image for two dimension display is generated based on the digital hologram.

According to further aspect of the application there is provided a kind of image processing method, including：Obtain data flow, the data Stream includes digital hologram graphic sequence；According to predetermined condition, it is determined whether carry out Three-dimensional Display；And when determination carries out Three-dimensional Display When, the image for Three-dimensional Display is generated based on the digital hologram in the digital hologram graphic sequence；Or when determine without During Three-dimensional Display, the image for two dimension display is generated based on the digital hologram in the digital hologram graphic sequence.

According to further aspect of the application there is provided a kind of image processing apparatus, including：Data flow acquisition module, is used for Data flow is obtained, the data flow includes digital hologram sequence；Determining module, for according to predetermined condition, it is determined whether enter Row Three-dimensional Display；3-D view generation module, for when it is determined that carrying out Three-dimensional Display, based in the digital hologram graphic sequence Digital hologram generate image for Three-dimensional Display；And two dimensional image generation module, for when determination is without three-dimensional During display, the image for two dimension display is generated based on the digital hologram in the digital hologram graphic sequence.

By the image processing method and device of the application, problems of the prior art are can solve the problem that, so as to Consumer's Experience is improved, is flexibly applied to various hardware configurations and strengthens the expressive force of the application such as game and displaying content.

Brief description of the drawings

Accompanying drawing described herein is used for providing further understanding of the present application, constitutes the part of the application, this Shen Schematic description and description please is used to explain the application, does not constitute the improper restriction to the application.In the accompanying drawings：

Fig. 1 is the flow chart for showing the image processing method according to the embodiment of the present application 1；

Fig. 2 is the schematic diagram for showing the image processing apparatus according to the embodiment of the present application 2；

Fig. 3 is the flow chart for showing the image processing method according to the embodiment of the present application 3；

Fig. 4 is the schematic diagram for showing the image processing apparatus according to the embodiment of the present application 4；

Fig. 5 is the schematic diagram for showing the image processing system according to the embodiment of the present application 5；

Fig. 6 is flow of the image processing apparatus after user instruction is received shown in the embodiment of the present application 5 Figure；

Fig. 7 is to show image processing apparatus in the embodiment of the present application 5 after digital hologram is received Flow chart；

Fig. 8 is the flow chart for the method for showing the formation stereoscopic vision according to the embodiment of the present application 5；

Fig. 9 is to make the schematic flow sheet further illustrated to the step in Fig. 5.

Figure 10 (a) to Figure 10 (d) is the design sketch for illustrating the method according to embodiment 5；

Figure 11 (a) to Figure 11 (d) is the design sketch for illustrating to carry out robust detection；

Figure 12 is the schematic diagram according to hologram reconstruction left image and right image according to the application；

Figure 13 is the method flow that two-dimentional display is carried out according to digital hologram according to the embodiment of the present application 5；

Figure 14 (a) is the schematic diagram for simulating Fourier's frequency domain based on the Object light wave from virtual objects；

Figure 14 (b) is the schematic diagram for the Fourier spectrum distribution for showing the digital hologram；

Figure 15 (a) and Figure 15 (b) are to respectively illustrate actual digital hologram and the distribution of corresponding Fourier spectrum The schematic diagram of figure；

Figure 16 (a) is the schematic diagram for showing the frequency spectrum after being filtered to the Fourier spectrum of digital hologram；

Figure 16 (b) is to show the schematic diagram that the spectrum distribution after frequency displacement is carried out to the Fourier spectrum of digital hologram；

Figure 17 is to show that being used for according to the variation of embodiment 5 generates the flow of the method for digital hologram Figure；

Figure 18 is the schematic diagram for showing the image processing system according to the embodiment of the present application 6；

Figure 19 is that the image processing apparatus shown in the embodiment of the present application 6 is receiving resource distribution test parameter Flow chart afterwards；

Figure 20 is the schematic diagram for showing the image processing system according to the embodiment of the present application 7；

Figure 21 is to show received using the flow after indicating according to the image processing apparatus of the embodiment of the present application 7 Figure；

Figure 22 is the schematic diagram for showing the image processing system according to the embodiment of the present application 8；

Figure 23 is that the image processing apparatus shown in the embodiment of the present application 8 is receiving resource distribution test parameter Flow chart afterwards；

Figure 24 is flow of the image processing apparatus after user instruction is received shown in the embodiment of the present application 8 Figure；

Figure 25 is that the image processing apparatus shown in the embodiment of the present application 8 is being received using the flow after indicating Figure；

Figure 26 is the flow chart for being used to form the method for stereoscopic vision according to prior art；

Figure 27 is the flow chart of the method that two-dimentional display is carried out based on digital hologram according to prior art.

Embodiment

In order that those skilled in the art more fully understand application scheme, below in conjunction with the embodiment of the present application Accompanying drawing, the technical scheme in the embodiment of the present application is clearly and completely described, it is clear that described embodiment is only The embodiment of the application part, rather than whole embodiments.Based on the embodiment in the application, ordinary skill people The every other embodiment that member is obtained under the premise of creative work is not made, should all belong to the model of the application protection Enclose.

It should be noted that term " first " in the description and claims of this application and above-mentioned accompanying drawing, " Two " etc. be for distinguishing similar object, without for describing specific order or precedence.It should be appreciated that so using Data can exchange in the appropriate case, so as to embodiments herein described herein can with except illustrating herein or Order beyond those of description is implemented.In addition, term " comprising " and " having " and their any deformation, it is intended that cover Lid is non-exclusive to be included, for example, containing series of steps or module or process, method, system, product or the equipment of unit Be not necessarily limited to those steps or module or the unit clearly listed, but may include not list clearly or for these The intrinsic other steps of process, method, product or equipment or module or unit.

Embodiment 1

Accompanying drawing 1 shows the flow chart of the image processing method according to embodiment 1, as shown in Figure 1, and this method includes：

S102：Obtain digital hologram；And

S104：According to predetermined condition, it is determined whether carry out Three-dimensional Display；And

S106：When it is determined that carrying out Three-dimensional Display, the image for Three-dimensional Display is generated based on the digital hologram；Or

S108：When it is determined that during without Three-dimensional Display, the image for two dimension display is generated based on the digital hologram.

In the technical scheme of embodiment 1, generated according to predetermined condition based on the digital hologram for Three-dimensional Display Image or for two dimension display image, it is achieved thereby that Three-dimensional Display and two dimension display between switching.And especially, For carry out Three-dimensional Display image and for carry out two-dimentional display be all based on digital hologram generation so that entering The implementation of row Three-dimensional Display and both patterns of the two-dimentional display of progress is whole by this key feature of digital hologram It is combined.So as to when being shown, it is only necessary to provide digital hologram, without providing other picture numbers According to so as to improve the efficiency and data transfer of integration, the efficiency of data processing.It should be noted that the technology of embodiment 1 Scheme is not particularly limited for generating the mode of the image for Three-dimensional Display, every three-dimensional aobvious by digital hologram generation The mode of the image shown can be used in the application.

Alternatively, methods described also includes receiving user's instruction from user or receives application from the application being currently running Indicate, and it is described according to predetermined condition, it is determined whether Three-dimensional Display is carried out, including：Indicated to determine whether according to the user Carry out Three-dimensional Display；Or indicated to determine whether to carry out Three-dimensional Display according to the application.So as to according to this programme, Ke Yigen Tangent line is carried out between two dimension display and Three-dimensional Display according to the instruction of user, so as to improve Consumer's Experience.And according to this skill Art scheme, can be real according to the requirement of the application (game being such as currently running and the video demonstrated) being currently running The switching of existing three peacekeepings two dimension display, enhances the display effect and Consumer's Experience of the application.

Alternatively, methods described is also included to carrying out image procossing and/or carrying out the relevant parameter for the device that image is shown Detected.And it is described according to predetermined condition, it is determined whether to carry out Three-dimensional Display, in addition to：Indicated according to the user Or indicated to determine to carry out after Three-dimensional Display according to the application, further determine whether that progress is three-dimensional aobvious according to the relevant parameter Show.So as to according to this programme, when carrying out Three-dimensional Display, this method can also be to carrying out image procossing and the device of display Relevant parameter is detected (for example, the arithmetic speed of processor, the size of memory span, speed of data transfer etc.), from And determine whether to be adapted for Three-dimensional Display, if it is determined that when the device can carry out Three-dimensional Display, then generate Three-dimensional Display Image, otherwise with regard to carrying out two-dimentional display.So can neatly according to resource distribution realize three peacekeepings two dimension display between cut Change, so as to improve the speed and effect of display.

Alternatively methods described is also included to carrying out image procossing and/or carrying out the relevant parameter for the device that image is shown to enter Row detection.And it is described according to predetermined condition, it is determined whether to carry out Three-dimensional Display, including：According to the relevant parameter, it is determined that being No carry out Three-dimensional Display.So as to according to this programme, when carrying out Three-dimensional Display, this method can also be to carrying out image procossing and showing The relevant parameter of the device shown is detected (for example, the arithmetic speed of processor, the size of memory span, data transfer Speed etc.), determine whether to be adapted for Three-dimensional Display, if it is determined that when the device can carry out Three-dimensional Display, then generate The image of Three-dimensional Display, otherwise with regard to carrying out two-dimentional display.So neatly it can realize that three peacekeepings two dimension is aobvious according to resource distribution Switching between showing, so as to improve the speed and effect of display.

Alternatively, methods described also includes receiving user's instruction from user or should from the application reception being currently running With instruction.And it is described according to predetermined condition, it is determined whether to carry out Three-dimensional Display, including：Determined according to the relevant parameter After Three-dimensional Display, directly determine without Three-dimensional Display.That is, the result judged according to relevant parameter has Highest priority, when according to relevant parameter judging that Three-dimensional Display can not be carried out, then need not consider that user indicates or applied Indicate and only carry out two-dimentional display.

Alternatively, methods described also includes receiving user's instruction from user or should from the application reception being currently running With instruction.Also, it is described according to predetermined condition after progress Three-dimensional Display is determined according to the relevant parameter, it is determined whether Three-dimensional Display is carried out, in addition to：Indicated according to the user, it is determined whether carry out Three-dimensional Display；Or referred to according to the application Show and determine whether to carry out Three-dimensional Display.That is, only when according to relevant parameter judging that Three-dimensional Display can be carried out, just meeting Indicated according to user or application indicate to determine whether to carry out Three-dimensional Display.Further illustrate book according to determining that relevant parameter is made It is fixed to be indicated relative to user and there is of a relatively high priority using instruction.

Alternatively, methods described also includes receiving user's instruction from user and should from the application reception being currently running With instruction, and it is described according to predetermined condition, it is determined whether Three-dimensional Display is carried out, including：Indicate that determination is according to the user No carry out Three-dimensional Display；Or indicated according to the user, it is determined that being indicated based on the application three-dimensional aobvious to determine whether to carry out Show.That is, being indicated relative to application, user is indicated again with of a relatively high priority.Therefore need to be indicated by user To determine to be based on to determine whether using instruction to carry out Three-dimensional Display.

Alternatively, the acquisition digital hologram includes：Obtain the three-dimensional information of virtual objects；And according to the three-dimensional Information generates digital hologram.And described generate includes for the image of Three-dimensional Display：According to the digital hologram and First related to the digital hologram generates left image and the right side for corresponding respectively to left view point and right viewpoint with reference to optical information Image.

Due to generating left image and right image corresponding to left view point and right viewpoint according to digital hologram, therefore in reality In operation, it is only necessary to reproduce the left images that generation system needs according to digital hologram, you can inputted as information source virtual Reality system, so as to reduce camera quantity and camera calibration work, reduction algorithm and cost price.And due to digital hologram To record the information of three-dimensional object, therefore it can effectively prevent the missing of scene information.

Alternatively, the three-dimensional information for obtaining virtual objects, including：Obtain the preset recordings of the virtual objects away from From；And wavefront function of the virtual objects at the preset recording distance is calculated, wherein, the wavefront function includes shaking Width information and phase information, and it is described according to three-dimensional information generation digital hologram, including：According to the wavefront function And second calculate the generation digital hologram with reference to optical information.According to this method, believed using wavefront function and reference light Breath calculates generation digital hologram, so that the generating process of whole digital hologram is all realized by calculating simulation, therefore Save the hardware cost of generation digital hologram.And actual light source and imaging device due to that need not introduce, therefore Also so that this method is easier to realize.In addition, in addition to the above method, it is any be used to obtaining the three-dimensional information of virtual objects so as to The mode of generation digital hologram is suitable for the application, can obtain the support of the technical program.

Alternatively, the first reference light information includes left view point corresponding with the left view point and right viewpoint reproduction reference Optical information and right viewpoint, which reproduce, refers to optical information, wherein, the left view point, which reproduces reference light information, to be included：Simulation from left view point The light that corresponding left reproduction reference light source is sent is irradiated to the incident angle during digital hologram, and the right viewpoint reproduces Reference light information includes：Simulate the light sent from right reproduction reference light source corresponding with right viewpoint and be irradiated to the digital hologram Incident angle during figure.And it is described according to the digital hologram and first reference light related to the digital hologram Information generates the left image and right image for corresponding respectively to left view point and right viewpoint, including：Reproduced and referred to based on the left view point Optical information and right viewpoint reproduce generates the left image and right image with reference to optical information and the digital hologram.According to the party Method, it is only necessary to left view point reproduction corresponding with the left view point and right viewpoint is generated based on the reference optical information and refers to optical information Reproduced with right viewpoint and refer to optical information, then based on the left view point reproduce with reference to optical information and right viewpoint reproduce with reference to optical information with And the digital hologram can generate the left image and right image.Due in practical operation, the left view point reference light Information and the right view reference optical information can be produced (such as according to from the left view point and the right side according to the reference optical information The light that viewpoint is sent is irradiated to the incidence angle during digital hologram, based on reference optical information generation and the left view The corresponding with right viewpoint left view point of point, which reproduces to reproduce with reference to optical information and right viewpoint, refers to optical information), therefore can be just with The left image and right image can be generated with reference to optical information and the hologram, therefore reduces the generation left image and the right side Data volume needed for image.For the application, in addition to this method, others are used to produce left image and right figure The method of picture is also applied for the technical scheme of the application.

Alternatively, the preset recording distance at least two.And it is described to calculate the virtual objects described default Wavefront function at recording distance, including：Wavefront of the virtual objects at each described preset recording distance is calculated respectively Function, and the wavefront function is overlapped.And it is described according to the wavefront function and second with reference to optical information calculate The digital hologram is generated, including：Life is calculated with reference to optical information according to the wavefront function after superposition and described second Into the digital hologram.

Because the wavefront function of three-dimensional object has different preset recordings, apart from d, (different preset recordings are apart from d The different layers of correspondence), therefore wavefront function of the three-dimensional object under different preset recordings distance is overlapped, then with ginseng Optical information combination is examined, forming the process of hologram just turns into the process that hierarchical information is recorded.And during the reconstruction of hologram, can only be in correspondence Reproduction under d, can the clearly corresponding reproduction image of reproducing, will not thus produce corresponding interference.

Alternatively, the acquisition digital hologram includes：The three-dimensional coordinate information of real object in true environment is obtained, and Obtain the three-dimensional coordinate information of virtual objects；And three-dimensional coordinate information according to the real object and the virtual objects Three-dimensional coordinate information generates the digital hologram.And described generate includes for the image of Three-dimensional Display：According to the number Word hologram and the first reference optical information generation related to the digital hologram correspond respectively to left view point and right viewpoint Left image and right image.

In embodiments herein, true environment is properly termed as actual environment again, and real object is properly termed as reality again Real world object in environment；Virtual objects are dummy object, scene or the system prompt letter that research staff is generated by computer At least one of breath.

In the optional embodiment of the application, the three-dimensional coordinate information of real object includes：Real object shakes The phase of width and real object；The three-dimensional coordinate information of virtual objects includes：The amplitude of virtual objects and the phase of virtual objects.

In embodiments of the present invention, by by the three of the three-dimensional coordinate information of real object in true environment and virtual objects Dimension coordinate information, which is overlapped, obtains objective holographic figure, then, objective holographic figure reproduce enables virtual objects table Show in true environment, do not only reached the purpose of augmented reality using the embodiment of the present invention, robustness and just can also be met The performance requirement of true property, and then solve the robustness and correctness of existing augmented reality and can not meet target requirement Technical problem.Also, due to generating left image and right image corresponding to left view point and right viewpoint according to digital hologram, therefore In practical operation, it is only necessary to reproduce the left images that generation system needs according to digital hologram, you can defeated as information source Enter virtual reality system, so as to reduce camera quantity and camera calibration work, reduction algorithm and cost price.And due to numeral Hologram records the information of three-dimensional object, therefore can effectively prevent the missing of scene information.

Alternatively, the three-dimensional coordinate information of real object in true environment is obtained, is specially：First, real object is entered Row holographic process, obtains the hologram of real object, then, and time frequency analysis is carried out to the hologram of real object, obtains true The wavefront function of object, wherein, the wavefront function of real object includes the three-dimensional coordinate information of real object.

In embodiments of the present invention, holographic recording can be carried out to the real object in true environment by optical diffraction, To obtain the hologram of real object, wherein, Digital Holography refers to light wave and the superimposed production of reference light wave of object reflection Raw interference fringe, these interference fringes referred to as hologram being recorded, wherein, hologram can reproduce under certain conditions, To reappear the three-dimensional image that the original is true to nature, traditional holographic technique is divided into optical holographic and calculates holographic.

After the hologram of real object is got, you can to carry out time frequency analysis, place to the hologram of real object Reason obtains the wavefront function of real object, the phase letter of the amplitude comprising real object and real object in the wavefront function Breath.

Alternatively, time frequency analysis is carried out to the hologram of real object, obtaining the wavefront function of real object includes：Pass through The hologram of real object is converted into domain space by Fourier transformation, then, in hologram of the domain space to real object High-pass filtering is carried out in domain space, retains the object light wavefront information for obtaining real object, finally, passes through inverse Fourier transform Object light wavefront information is subjected to inverse Fourier transform, conversion obtains the wavefront function of real object, and (wavefront function is properly termed as again Wavefront complex amplitude function), wherein, include the amplitude of real object and the phase of real object in obtained wavefront function.

Alternatively, obtaining the three-dimensional coordinate information of virtual objects includes：Obtain the preset recording distance of the virtual objects； Wavefront function of the virtual objects at the preset recording distance is calculated, wherein, in the wavefront function of the virtual objects Include the three-dimensional coordinate information of the virtual objects.

Alternatively, generated according to the three-dimensional coordinate information of the three-dimensional coordinate information of the real object and the virtual objects Digital hologram includes：The wavefront function of the wavefront function of the real object and the virtual objects is overlapped and obtains ripple Preceding information；And according to resulting wavefront information and second with reference to the optical information calculating generation digital hologram.According to this Method, generation digital hologram is calculated using wavefront function and with reference to optical information, so that whole digital hologram map generalization Journey is realized by calculating, therefore saves the hardware cost of generation digital hologram.In addition, in addition to the above method, It is any to be suitable for the application for obtaining the three-dimensional information of virtual objects to generate the mode of digital hologram, it can obtain The support of the technical program

Alternatively, the image generated for two dimension display, is by only being protected in the frequency spectrum of the digital hologram The frequency spectrum of the wavefront function associated with the generation image for being used for two dimension display is stayed, and filters out the realization of other frequency spectrums.From And, when carrying out two dimension display, it is used for two-dimentional show by the way that only reservation and generation are described in the frequency spectrum of the digital hologram The associated wavefront function of image frequency spectrum, and filter out the realization of other frequency spectrums, on the one hand reduce the complexity of calculating, separately One side rate improves the quality of display except the influence of Zero-order diffractive and conjugate image.

Alternatively, the image generated for two dimension display, is by retaining ﹢ in the frequency spectrum of the digital hologram 1 grade of frequency spectrum, and filter out 1 grade of frequency spectrum of 0 grade of frequency spectrum and ﹣ and realize.Still optionally further, it is described to generate for two dimension display Image, includes the position of 0 grade of frequency spectrum by described 1 grade of frequency spectrum frequency displacement of ﹢ into the frequency spectrum of the digital hologram.Pass through Which, carry out two dimension display when, by the frequency spectrum of the digital hologram retain with it is described be used for two dimension display The frequency spectrum of the wavefront functional dependence of image, and filter out 1 grade of frequency spectrum of 0 grade of frequency spectrum and ﹣ and generate the image for two dimension display, so that On the one hand the complexity of calculating is reduced, has on the other hand filtered out the influence of Zero-order diffractive and conjugate image, improved the matter of display Amount.

Alternatively, the image generated for two dimension display, including：Fourier's change is carried out to the digital hologram Change, generate the frequency spectrum of the digital hologram；0 grade of frequency spectrum is filtered out in the frequency spectrum of the digital hologram and described -1 grade Frequency spectrum, and retain 1 grade of frequency spectrum of ﹢；By the 0 grade frequency spectrum of the described 1 grade of frequency spectrum frequency displacement of ﹢ into the frequency spectrum of the digital hologram Position；And inverse Fourier transform is carried out to the frequency spectrum of the digital hologram after the frequency displacement, generate aobvious for two dimension The image shown.Conversion of the wavefront function between frequency domain and spatial domain is so realized by Fourier transform, and by institute The frequency spectrum only retained in the frequency spectrum of digital hologram with the wavefront functional dependence of the image for being used for two dimension display is stated, and is filtered out Other frequency spectrums and generate for two dimension display image so that on the one hand reduce the complexity of calculating, on the other hand filter out The influence of Zero-order diffractive and conjugate image, improves the quality of display.

Embodiment 2

Accompanying drawing 2 shows the schematic diagram of the image processing apparatus 200 according to the embodiment of the present application 2.As shown in Fig. 2 the figure As processing unit 200 includes：Hologram acquisition module 201, for obtaining digital hologram；Determining module 202, for according to pre- Fixed condition, it is determined whether carry out Three-dimensional Display；3-D view generation module 203, for when it is determined that carrying out Three-dimensional Display, being based on The digital hologram generates the image for Three-dimensional Display；Two dimensional image generation module 204, for when determination is without three-dimensional During display, the image for two dimension display is generated based on the digital hologram.

Alternatively, the determining module 202 includes：Indicate to receive single for receiving the client that the user from user indicates Member indicates receiving unit for receiving the application that application is indicated from the application being currently running；And identifying unit, for basis The user indicates or the application indicates to determine whether to carry out Three-dimensional Display.

Alternatively, the determining module 202 includes：Parameter detecting unit, for carrying out image procossing and/or scheming The relevant parameter of device as shown in is detected；And identifying unit, indicated or according to the application according to the user Indicate to determine to carry out after Three-dimensional Display, further determined whether to carry out Three-dimensional Display according to the relevant parameter.

Alternatively, the determining module 202 includes：Parameter detecting unit, for carrying out image procossing and/or scheming The relevant parameter of device as shown in is detected；And identifying unit, for being determined whether to carry out according to the relevant parameter Three-dimensional Display.

Alternatively, the determining module 202 also includes indicating to receive for receiving the client that the user from user indicates Unit indicates receiving unit for receiving the application that application is indicated from the application being currently running.And the identifying unit configuration After determining without Three-dimensional Display according to the relevant parameter, directly to determine without Three-dimensional Display.

Alternatively, the determining module 202 also includes indicating to receive for receiving the client that the user from user indicates Unit indicates receiving unit for receiving the application that application is indicated from the application being currently running.And the identifying unit is matched somebody with somebody It is set to after progress Three-dimensional Display is determined according to the relevant parameter, is indicated according to the user, it is determined whether is carried out three-dimensional Display；Or indicated to determine whether to carry out Three-dimensional Display according to the application.

Alternatively, the hologram acquisition module 201 includes：Three-dimensional information acquiring unit, for obtaining virtual objects Three-dimensional information；And digital hologram generation unit, for generating the digital hologram according to the three-dimensional information.And institute Stating 3-D view generation module 203 includes：Left images generation unit, for according to the digital hologram and with the number The first related reference optical information of word hologram generates the left image and right image for corresponding respectively to left view point and right viewpoint.

Alternatively, the three-dimensional information acquiring unit includes：Preset recording is described virtual for obtaining apart from obtaining widget The preset recording distance of object；And wavefront Function Computation Component, for calculate the virtual objects the preset recording away from From the wavefront function at place.Wherein, the wavefront function includes amplitude information and phase information.And the hologram generation unit Including：Hologram calculating unit is complete for calculating the generation numeral with reference to optical information according to the wavefront function and second Breath figure.Generation digital hologram is calculated using wavefront function and with reference to optical information due to image processing apparatus, so as to entirely count The generating process of word hologram is all realized by calculating simulation, therefore saves the hardware cost of generation digital hologram. And due to light source and imaging device need not be introduced, therefore also cause this method to be easier to realize.

Alternatively, the first reference light information includes left view point corresponding with the left view point and right viewpoint reproduction reference Optical information and right viewpoint, which reproduce, refers to optical information, wherein, the left view point, which reproduces reference light information, to be included：Simulation from left view point The light that corresponding left reproduction reference light source is sent is irradiated to the incident angle during digital hologram, and the right viewpoint reproduces Reference light information includes：Simulate the light sent from right reproduction reference light source corresponding with right viewpoint and be irradiated to the digital hologram Incident angle during figure.And the left images generation unit includes：Left images generating unit, for based on the left view Point reproduces to reproduce with reference to optical information and right viewpoint generates the left image and right figure with reference to optical information and the digital hologram Picture.Based on the image processing apparatus, it is only necessary to generated based on the reference optical information corresponding with the left view point and right viewpoint Left view point, which reproduces to reproduce with reference to optical information and right viewpoint, refers to optical information, then is reproduced based on the left view point with reference to optical information and the right side Viewpoint reproduces can generate the left image and right image with reference to optical information and the digital hologram.Due in practical operation In, the left view point can be produced (for example with reference to optical information and the right view reference optical information according to the reference optical information Light according to being sent from the left view point and right viewpoint is irradiated to the incidence angle during digital hologram, based on the reference Optical information generates left view point corresponding with the left view point and right viewpoint and reproduced reproduces reference light letter with reference to optical information and right viewpoint Breath), therefore the left image and right image can be generated just with reference to optical information and the hologram, therefore reduce Data volume needed for the generation left image and right image.

Alternatively, the preset recording distance at least two.And the wavefront Function Computation Component includes：Wavefront letter Number superposition element, calculates wavefront function of the virtual objects at each described preset recording distance respectively, and to the ripple Preceding function is overlapped, and the hologram calculating unit includes：Digital hologram producing element, after according to superposition The wavefront function and described second is with reference to the optical information calculating generation digital hologram.Because the ripple of three-dimensional object Preceding function has different preset recordings apart from d (different preset recordings is apart from the different layers of d correspondences), therefore by three-dimensional Wavefront function of the object under different preset recordings distance is overlapped, then is combined with reference to optical information, forms the mistake of hologram Journey just turns into the process that hierarchical information is recorded.And during the reconstruction of hologram, can only be in corresponding reproduction under d, can be clearly The corresponding reproduction image of reproducing, will not thus produce corresponding interference.

Alternatively, the hologram acquisition module 201 includes：Three-dimensional information acquiring unit, for obtaining in true environment The three-dimensional coordinate information of real object, and obtain the three-dimensional coordinate information of virtual objects；And digital hologram generation unit, use The digital hologram is generated in the three-dimensional coordinate information of the three-dimensional coordinate information according to the real object and the virtual objects Figure.And the 3-D view generation module 203 includes：Left images generation unit, for according to the digital hologram with And related to the digital hologram first generated with reference to optical information the left image that corresponds respectively to left view point and right viewpoint and Right image.

Alternatively, the three-dimensional information acquiring unit includes：Real object holographic process component, for entering to real object Row holographic process, obtains the hologram of real object；And real object function component, enter for the hologram to real object Row time frequency analysis, obtains the wavefront function of real object, wherein, the wavefront function of real object includes the three-dimensional seat of real object Mark information.

Alternatively, the real object function component includes：Time frequency analysis element, enters for the hologram to real object Row time frequency analysis；Wavefront functional unit, for the hologram of real object to be converted into domain space by Fourier transformation, so Afterwards, high-pass filtering is carried out in domain space to the hologram of real object in domain space, retains the thing for obtaining real object Light wavefront information, finally, carries out inverse Fourier transform, conversion obtains true right by inverse Fourier transform by object light wavefront information The wavefront function of elephant (wavefront function is properly termed as wavefront complex amplitude function again).Wherein, include very in obtained wavefront function The amplitude of real object and the phase of real object.

Alternatively, the three-dimensional information acquiring unit includes：Virtual objects recording distance securing component, it is described for obtaining The preset recording distance of virtual objects；Virtual objects wavefront function computation module, for calculating the virtual objects described pre- If the wavefront function at recording distance, wherein, the wavefront function of the virtual objects includes the three-dimensional seat of the virtual objects Mark information.

Alternatively, the digital hologram generation unit includes：Wavefront function superposition component, for by the real object Wavefront function and the wavefront functions of the virtual objects be overlapped and obtain wavefront information；And digital hologram generation group Part, for calculating the generation digital hologram with reference to optical information according to resulting wavefront information and second.

Alternatively, the image generated for two dimension display, is by only being protected in the frequency spectrum of the digital hologram The frequency spectrum of the wavefront function associated with the generation image for being used for two dimension display is stayed, and filters out the realization of other frequency spectrums.

Alternatively, the two dimensional image generation module 204 is included in filter unit, the frequency spectrum for the digital hologram Retain 1 grade of frequency spectrum of ﹢, and filter out 1 grade of frequency spectrum of 0 grade of frequency spectrum and ﹣.

Alternatively, the two dimensional image generation module 204 also include frequency shift unit, for by described 1 grade of frequency spectrum frequency displacement of ﹢ extremely The position of 0 grade of frequency spectrum in the frequency spectrum of the digital hologram.

Alternatively, described two dimensional image generation module 204 includes：Fourier transform unit, for complete to the numeral Breath figure carries out Fourier transformation, generates the frequency spectrum of the digital hologram；Filter unit, for the frequency in the digital hologram 0 grade of frequency spectrum and -1 grade of frequency spectrum are filtered out in spectrum, and retains 1 grade of frequency spectrum of ﹢；Frequency shift unit, for by described 1 grade of frequency spectrum of ﹢ The position of the 0 grade frequency spectrum of the frequency displacement into the frequency spectrum of the digital hologram；And inverse Fourier transform unit, for institute The frequency spectrum for stating the digital hologram after frequency displacement carries out inverse Fourier transform, generates the image for two dimension display.

Embodiment 3

Accompanying drawing 3 shows the flow chart of the image processing method according to embodiments herein 3.As shown in figure 3, the figure As processing method includes：

S302：Data flow is obtained, the data flow includes digital hologram sequence；

S304：According to predetermined condition, it is determined whether carry out Three-dimensional Display；And

S306：When it is determined that carrying out Three-dimensional Display, generated and used based on the digital hologram in the digital hologram graphic sequence In the image of Three-dimensional Display；Or

S308：When it is determined that during without Three-dimensional Display, based on the digital hologram generation in the digital hologram graphic sequence Image for two dimension display.

With the development of network technology, the video flowing that 3 D stereo shows is provided by way of remote service to be turned into One trend, because dimension display technologies need to provide the video of left image and the video of right image, therefore certainly will cause to pass Defeated data volume is excessive.According to embodiment 5, digital hologram graphic sequence is obtained on the basis of embodiment 1, so as to form a left side The video flowing of right image, considerably reduces the quantity for needing to transmit, is conducive to provide more quality services.

In addition, acquisition and processing on the digital hologram in holographic graphic sequence, each in embodiment 1 may be referred to Method, is repeated no more here.

Embodiment 4

Accompanying drawing 4 shows the schematic diagram of the image processing apparatus 400 according to embodiments herein 4, as shown in figure 4, institute Stating image processing apparatus includes：

Data flow acquisition module 401, for obtaining data flow, the data flow includes digital hologram sequence；

Determining module 402, for according to predetermined condition, it is determined whether carry out Three-dimensional Display；

3-D view generation module 403, for when it is determined that carrying out Three-dimensional Display, based in the digital hologram graphic sequence Digital hologram generate image for Three-dimensional Display；And

Two dimensional image generation module 404, for when it is determined that during without Three-dimensional Display, based on the digital hologram graphic sequence In digital hologram generate for two dimension display image.

In addition, acquisition and processing on the digital hologram in holographic graphic sequence, each in embodiment 2 may be referred to Device, is repeated no more here.

Embodiment 5

Below, in embodiment 5, each details of technical scheme is will be explained in detail, for the ease of understanding this reality The technical scheme of example is applied, this example is illustrated by following components：

(1) it is used for the image processing system that display is switched over based on digital hologram according to embodiment 5；

(2) method flow of Three-dimensional Display formation stereoscopic vision is carried out based on digital hologram；

(3) effect of Three-dimensional Display formation stereoscopic vision is carried out based on digital hologram；

(4) obtain digital hologram and the specific algorithm and principle of left images are generated based on digital hologram；

(5) method flow of two-dimentional display (reproducing hologram) is carried out based on digital hologram；

(6) specific algorithm and principle of two-dimentional display are carried out based on digital hologram；

(7) variation of embodiment 5.

It should be noted that the present embodiment be used only for that the technical scheme of the application is explained, and be not construed as to this Apply for the restriction of protection domain.And as long as no opposite, it is impossible to reference to enlightenment, the technical scheme of embodiment 5 can be with Each embodiment above is combined.

In order to make it easy to understand, showing to generate the technical side of digital hologram by using virtual reality technology in embodiment 5 Case, and then show to generate the technical scheme of digital hologram using enhancing Display Technique in variation ((seven) part).But Be no matter the digital hologram that any technical scheme is generated, be used equally for three-dimensional display mode and two dimensional mode it Between switch over.

(1) it is used for the image processing system that display is switched over based on digital hologram

Fig. 5 shown according to embodiment 5, the image processing system 500 for switching over display based on digital hologram Schematic diagram.As shown in figure 5, described image processing system includes image processing apparatus 510 and display 520.It is wherein described Image processing apparatus 510 includes user interface 511, hologram input interface 512, stereoscopic display processing module 513, hologram again Existing module 514, driver module 515, identifying unit 516 and hologram allocation unit 517.

Wherein, the user interface 511 is used to receive the selection on Three-dimensional Display or two dimension display from user and referred to Order or switching command, and user instruction is transmitted to identifying unit 516.

Hologram input interface 512, which is received, carrys out digital hologram, and the digital hologram received is sent to hologram point With unit 517.

Identifying unit 516 after the instruction of user is received from user interface 511, according to the instruction of user judge whether into Row Three-dimensional Display.If it is determined that carrying out Three-dimensional Display, (user instruction for example received is the instruction that selection carries out Three-dimensional Display Either switch to the instruction of Three-dimensional Display), just the instruction for carrying out Three-dimensional Display is sent to hologram allocation unit 517；It is no Then (user instruction for example received is that selection carries out the instruction of two-dimentional display or switches to the instruction of two dimension display) just The instruction for carrying out two-dimentional display is sent to hologram allocation unit 517.

Status register (not shown) is provided with hologram allocation unit 517, the status register is used to preserve current State is to be used to carry out two-dimentional display or carry out Three-dimensional Display, and the status register is received from identifying unit 516 and is used for Carry out Three-dimensional Display or carry out the instruction of two-dimentional display, and the state of preservation is updated according to the instruction.The hologram Allocation unit 517 is preserved after the digital hologram is received from hologram input interface 512 according in the status register State, digital hologram is sent to Three-dimensional Display processing module 513 or two-dimentional Display processing module 514, to carry out three Dimension display carries out two-dimentional display.

Three-dimensional Display processing module 513 is used for the digital hologram based on reception, according to the present embodiment (two) to the (4) flow, algorithm and the principle (step in step S1006, accompanying drawing 11 in Primary Reference and accompanying drawing 10 that part is recorded Content related S1108, and formula (8) and formula (9) related content), generate the left image for Three-dimensional Display and the right side Image, and transmit it to driver module 515.

Two-dimentional Display processing module 514 is used for the digital hologram based on reception, according to the present embodiment (five) to the (6) flow, algorithm and principle (the Primary Reference content related to the step S1206 to S1210 in accompanying drawing 12 that part is recorded And the content with (five) part), the plane picture for two dimension display is generated, and transmit it to driver module 515。

The driver module 515 is according to being received for the left image and right image of Three-dimensional Display or according to reception The plane picture for being used for two dimension display send display drive signals to display 520, show or enter for carrying out 3 D stereo Row two dimensional surface is shown.

Accompanying drawing 6 and Fig. 7 show the specific operational flowchart of the image processing apparatus 510 of embodiment 5, and wherein Fig. 6 shows Flow chart of the image processing apparatus 510 after the instruction of user is received is gone out, Fig. 7 shows that image processing apparatus 510 is connecing Receive the flow chart after digital hologram.

As shown in fig. 6, in step S602, described image processing unit 510 receives user on entering by user interface 511 Row Three-dimensional Display or the selection instruction or switching command of two dimension display.

In step S604, described image processing unit 510 is determined whether by identifying unit 516 according to the instruction of user Carry out Three-dimensional Display.

If it is determined that unit 516 determine carry out Three-dimensional Display, then state from step S606 to hologram allocation unit 517 Register sends the instruction for carrying out Three-dimensional Display.

If it is determined that unit 516 is determined without Three-dimensional Display, then shape from step S610 to hologram allocation unit 517 State register sends the instruction for carrying out two-dimentional display.

In step S608, the status register of the hologram allocation unit 517 is according to the instruction received, to being stored in State in status register is updated, so that the hologram allocation unit 517 is received from hologram input interface 512 After the digital hologram, according to the state preserved in the status register, Three-dimensional Display or two dimension display are carried out.

As shown in fig. 7, in step S702, described image processing unit 510 receives numeral by hologram input interface 512 Hologram.

In step S704, protected in the hologram allocation unit 517 of described image processing unit 510 according in status register Whether the condition adjudgement deposited carries out Three-dimensional Display.If it is judged that being "Yes", then digital hologram is sent to Three-dimensional Display Processing module 513, and enter step S706；If it is judged that being "No", then digital hologram is sent to two-dimentional display Module 514 is managed, and enters step S710.

In step S708, by Three-dimensional Display processing module 513, based on the digital hologram received, and according to this reality Flow, algorithm and principle that example (two) to (four) part is recorded are applied, the left image and right image for Three-dimensional Display is generated, And transmit to driver module 515.In step S710, by two-dimentional Display processing module, the digital hologram based on reception, Flow, algorithm and the principle recorded according to the present embodiment (five) to (six) part, generate the plan for two dimension display Picture, and transmit to driver module 515.

In step S708, by driver module 515 according to the left image and right image received or according to reception Plane picture sends display drive signals to display 520, and showing or carry out two dimensional surface for carrying out 3 D stereo shows.

More than, describe the image processing system for being used to switch over display based on digital hologram according to embodiment 5 And the operating process of included image processing apparatus.Although it should be noted that realizing each interface and mould of image processing flow Block is included in image processing apparatus 510, but it is also possible to is distributed in different devices and is separated from each other.And distinct interface and The function of module can also carry out appropriate adjustment and combination.Also, each interface in described image processing apparatus 510 and Module is also not necessarily limited to hardware circuit or each program module or software module for being realized by processor or controller.

, can be according to the instruction of user in Three-dimensional Display or the mould of two dimension display by the image processing system of the embodiment Selected or switched between formula.So as to there is provided good Consumer's Experience.

Be described in detail below according to the present embodiment, based on digital hologram generate for Three-dimensional Display left images or Method for the image of two dimension display.

(2) method flow based on digital hologram formation stereoscopic vision

Fig. 8 is shown to be used to form stereoscopic vision so as to carry out the method stream of Three-dimensional Display according to the embodiment of the present application 5 Cheng Tu.For the ease of understanding and illustrating, digital hologram is generated by using virtual reality technology in embodiment 5.

As shown in figure 8, methods described includes：

S802, obtains three-dimensional object A.It is used for the three-dimensional object that user is shown to during virtual reality.Wherein, Virtual objects in the present embodiment refer to the three-dimensional object (rather than object of necessary being) produced by calculating simulation, 3 D stereo object such as the human body, object that are produced by calculating simulation.

S804, holographic recording.Holographic recording is carried out for the three-dimensional object A produce digital hologram.For example By Fresnel Diffraction Integral computational methods, obtaining three-dimensional object A wavefront complex amplitude function, (function includes object light Amplitude and phase information).Alternatively, the different diffraction distance of the different aspects of the three-dimensional object A (herein, is spread out Distance is penetrated with the preset recording described in preceding embodiment apart from corresponding, is statement of the same distance under different application scene) Under wavefront complex amplitude information be overlapped, the wavefront complex amplitude function of superposition is recorded in same digital hologram, from And the digital holographic recording of layering is carried out to three-dimensional body, complete three-dimensional information and obtain.

S806, reproduces generation left images.I.e. according to the viewpoint demand of virtual reality system, according to simulation from the left view The light that point and right viewpoint are sent is irradiated to the incidence angle during digital hologram, based on the ginseng for generating the digital hologram Optical information is examined, left images corresponding with all viewpoints are generated, inputted as information source in virtual reality system.

For ease of understanding, with reference to the brief explanation of accompanying drawing 9 on the step S802 in Fig. 8 to step S806.It is as described above, Three-dimensional object A is one by simulating the step S902 in the three-dimensional object that calculating is produced, accompanying drawing 9 corresponding to above-mentioned Step S904 and S906 in step S802, Fig. 9 correspond to above-mentioned step S804, and the step S908 in Fig. 9 corresponds to Above-mentioned steps S806.As shown in figure 9, in step S902, being simulated by computer C and calculating the three-dimensional produced on car Object A, including data such as the three-dimensional object A coordinates (x, y, z) of each point.In order to be in by three-dimensional object A Now to user to form the scene of virtual reality and carry out Three-dimensional Display, it is necessary to generate display three-dimensional object A left image With right image to be shown to the left eye and right eye of user respectively.In order to obtain above-mentioned left image and right image, in step S904, meter Calculation machine C calculates its wavefront at diffraction distance by Fresnel Diffraction Integral computational methods for three-dimensional object A and shaken again Width function, i.e. wavefront complex amplitude function of object light of the simulation from three-dimensional object A at diffraction distance, the wherein wavefront Amplitude of the complex amplitude function comprising the object light and phase information.Optionally, because three-dimensional object A is three-dimensional, therefore right In its different aspect, wavefront complex amplitude function can be calculated respectively according to the diffraction distance of different aspects and be overlapped.So Afterwards in step S906, computer C is calculated according to the three-dimensional object A obtained wavefront complex amplitude function and is obtained numeral entirely Breath figure.And in step S908, computer C is calculated according to the digital hologram obtains the left image and right figure that carry out virtual reality Picture.Wherein, the generation and calculating on digital hologram and left image and right image, enter further in connection with accompanying drawing 10-12 Row is described in detail.Although it should be noted that for the purpose of simplifying the description, said process is completed by identical computer C, It is, according to specific actual conditions, each above-mentioned step can be respectively completed by different computing devices.Continue with Jie The step continued in accompanying drawing 8.

S808, left and right input data is updated using newly-generated left images.

S810, the split screen display available of left and right picture is carried out using the left and right input data after renewal.

S812, is filtered so that left eye picture can only be seen by left eye respectively using display device to the picture of right and left eyes To and the right eye picture can only be seen by right eye.

S814, user forms the vision of solid after the right and left eyes picture is seen.

Alternatively, embodiment 5 also includes the flow that robust detection is carried out to hologram, specifically, in holographic recording (step Rapid S804) after, robust sexual assault is carried out to the digital hologram of generation in step S816, for example, hologram is cut, The processing such as compression and application noise.And the hologram after carrying out robust sexual assault is reproduced in step S818, to enter Row robustness judges.So as to after by robust detection, in step S806, reproduce the left images of generation.

(2) technique effect based on digital hologram formation stereoscopic vision

With reference to the design sketch in Figure 10, to illustrate to carry out three-dimensional object A holographic recording and ultimately generate The process of left image and right image.

First, obtained in step S802 shown in three-dimensional object A, such as Figure 10 (a).Wherein Figure 10 (a) show on Three-dimensional object A (i.e. teddy bear) image.And wherein, image size is 360 × 300, and observed range (herein, is seen Ranging is from being same distance under different application scene with a distance from foregoing diffraction and preset recording is apart from corresponding Statement) be z.

Then, holographic recording is carried out in step S804, i.e., by fresnel diffraction integration method, calculates three-dimensional pair As A is in the wavefront function O of observed range z (i.e. distance of the teddy bear image to holographic plan)_Z, including amplitude and phase letter Breath.That is, the fresnel diffraction integration computing that row distance is z is entered to teddy bear image, and according to predetermined reference optical information, simulation Interference with reference light and calculate and obtain hologram, so as to complete the record of three-dimensional information.Wherein, Figure 10 (b) shows setting When angle from the virtual objects A object lights reflected and reference light is 30 °, the off-axis hologram of generation is calculated.

Then generation left images, the light sent according to simulation from the left view point and right viewpoint are reproduced in step S806 Be irradiated to the incidence angle during digital hologram, based on the reference optical information for generating the digital hologram, calculate generation with The corresponding left images of all viewpoints.Wherein Figure 10 (c) is that analog references light is relative to hologram according to hologram Figure 10 (b) Incidence angle left image corresponding with left view point for reproducing when being 15 °, reproduce apart from z=500mm (i.e. from left view point to hologram Distance, it is identical with above-mentioned observed range size), size be 360 × 300.Figure 10 (d) is according to hologram Figure 10 (b), simulation Reference light is reproducing with right viewpoint correspondence image when being 45 ° relative to the incidence angle of hologram, reproduce apart from z=500mm (i.e. from Right viewpoint to hologram distance, it is identical with above-mentioned observed range size), size be 360 × 300.

And then in step S808, using left images Figure 10 (c) and Figure 10 (d) of generation, left and right input data is updated, and The split screen display available of left and right picture is carried out using the left and right input data after updating in step S810.And finally formed in step S814 Stereoscopic vision.

As can be seen here, stereoscopic vision is realized by using digital hologram, it is not necessary to need to pass through again as prior art Virtual video camera obtains left-eye image and eye image with horizontal parallax, and needs only to generate digital hologram, then The left-eye image and eye image are generated according to digital hologram.So as to solve to generate left figure present in prior art The cost of picture and right image is too high, the low complicated, efficiency of transmission that works and the problems such as slow arithmetic speed.So as to reduce cost The alternative and the free degree of data source are improved simultaneously.

With reference to the design sketch in Figure 11, illustrate the robust detection carried out to hologram.

Wherein Figure 11 (a) is that the hologram after circular shearing attack is carried out to hologram 11 (b).Figure 11 (b) is to utilize sanction The image of hologram reconstruction after cutting, it is the image reproduced when analog references light is 30 ° relative to the incidence angle of hologram, its In reproduction apart from z=500mm；And Figure 11 (c) is that, according to hologram Figure 11 (a), analog references light enters relative to hologram The left image corresponding with left view point that firing angle reproduces when being 15 °, reproduces apart from z=500mm.Figure 11 (d) is according to hologram figure 11 (a), analog references light is reproducing with right viewpoint correspondence image when being 45 ° relative to the incidence angle of hologram, reproduces apart from z= 500mm。

According to Figure 11 as can be seen that the method by digital hologram generation left images of the present embodiment is with very good Robustness.

(4) specific algorithm and principle of generation hologram and reproduction left images；

Illustrate the present embodiment below generates hologram and reproduction left images progress three-dimensional based on virtual reality technology Show the specific algorithm and principle of (stereoscopic display), and record below be used to generate hologram and reproduce left images The operation of relevant generation hologram and reproduction left images of the method suitable for above any one embodiment.

Algorithm and principle

Specifically, algorithm and principle one are the wavefront complex amplitude letters that three-dimensional object is calculated by Fresnel Diffraction Integral Number, the method being superimposed using wavefront information generates hologram, specific as follows：

Step 1) calculate from complex amplitude before the Object light wave of the object light of three-dimensional object reflection

It is possible, firstly, to which it is complex amplitude function before d Object light wave to calculate diffraction distance by following formula 1：

Wherein, d is diffraction distance, (x₀,y₀) on starting material popin face with coordinate x₀And y₀It is corresponding, O (x₀,y₀) be Complex amplitude function before starting material popin face information function, the Object light wave that it is d by diffraction distance that U (x, y), which is, λ is the ripple of object light Long, j is imaginary unit, and k is wave number, x₀、y₀For original object plane coordinate, x, y are the wavefront function apart from d diffraction by diffraction Plane coordinates.

Then, processing is integrated to formula 1, the item unrelated with integration variable is mentioned before the sign of integration, obtain formula 2：

As can be seen that formula 2 is pair from above-mentioned formula 2A Fourier transformation has been done, Then, by transformation results multiplied by with a quadratic phase factor, i.e.,It can just obtain To through diffraction before the Object light wave after d diffraction complex amplitude., can be with order to calculate above formula using Fast Fourier Transform (FFT) FFT F represents Fourier transformation, and object plane sampling width is L₀, hits is N × N, and sampling interval is △ x₀=△ y₀=L₀/ N, so Afterwards, above-mentioned formula 2 is rewritten as formula 3, i.e.,：

In equation 3, △ x=△ y are corresponding Space domain sampling spacing after discrete Fourier transform, now, according in Fu Leaf transformation and the relation of discrete Fourier transform, can obtain formula 4：Wherein, in formula 4 L is the diffraction pattern sample range apart from d through diffraction.Formula 4 is entered into line translation and obtains formula 5：

On the other hand, it can be seen from nyquist sampling theorem, when only meeting formula 6： During described condition, discrete calculation result could approximately meet the diffractional field of nyquist sampling theorem.Now, thing The sampling width of plane and diffractional field plane is equal, and from formula 5, now object plane and diffractional field width are equal, i.e.,

Therefore, result above shows, if kept, the sampling interval and observation area of object plane are constant, and discrete fourier becomes Change the function that diffraction pattern sampling range L of the result of calculation on plane of vision is not only wavelength X and sampling number N, and can be with Diffraction increases apart from d increase.When diffraction is apart from very little, if sampling number N keeps constant, result of calculation can only be corresponded to Diffraction image in neighbouring optical axis very little region on plane of vision.

Step 2) holographic production is carried out to three-dimensional object A according to principle of holography, that is, simulate by reference to interference of light thing Complex amplitude U (x, y) before light wave, obtains objective holographic figure I_H(x, y), I_H(x, y) contains the three-dimensional information (amplification of virtual objects And phase information).In the embodiment of the present application, it can be calculated by following formula 7 and obtain objective holographic figure, I_H(x, y)=| R (x,y)+U(x,y)|²

=| R (x, y) |²+|U(x,y)|²+R^*(x,y)U(x,y)+R(x,y)U^*(x, y) (formula 7), wherein, R (x, y)

Reference light is represented, U (x, y) represents complex amplitude function (that is, above-mentioned wavefront information), R before the Object light wave by diffraction^* (x, y) represents reference light conjugation, U^*(x, y) represents that complex amplitude function is conjugated before Object light wave.

Step 3) according to the viewpoint demand of virtual reality system, generate left image corresponding with left view point and right viewpoint and the right side Image.Different reference angles of light (i.e. the angle of the normal of incident reference light and hologram) are simulated, hologram is entered Row reproduces.When reference light is R (x, y), Fresnel hologram is I_HWhen (x, y), the optical field distribution of reproduction is：

In (formula 8), F is function Fourier transformation symbol, and d is diffraction distance, and λ is wavelength, and j is imaginary unit, and k is Wave number, (x, y) is the hologram plane coordinates generated by diffraction apart from the wavefront function of d diffraction, and (x ', y ') is by referring to The reproduction image plane coordinates of light irradiation hologram generation, (△ x ', △ y ') correspondence Space domain sampling spacing.Below, refer to the attached drawing 12 It is shown to step 3) make and illustrating.

First, the visual field scope of human eye is 30 degree to 100 degree, during the reconstruction of hologram, is selected according to human eye visual field scope Meet the left view point and right viewpoint of system requirements, and reproduced according to reference to light settings left view point corresponding with left view point and right viewpoint Reference light and right viewpoint reproduce reference light.For example, reproducing reference light incident angle when setting reproduction as α, then reference light is reproduced Expression formula is changed into R (x, y) exp (jkxsin α).According to formula 8, secondary reproduction reference light expression formula is substituted into formula 8 to obtain To following formula 9：

Incident angle has affected only the phase information of reconstruction results it can be seen from formula 9.Therefore, two are selected not Same incident angle α, that is, select the reproduction reference light of two out of phase information, so as in correspondence left and right viewpoint, reproduce The left images that generation system needs.

(5) method flow of two-dimentional display (reproducing hologram) is carried out based on hologram

As described in the background art, in actual applications, however it remains carry out two-dimentional display (i.e. plane picture or video Display) demand, therefore be next necessary to introduce the technology for carrying out two-dimentional display.

Above partly described in detail (one) to (three) and three-dimensional is carried out based on digital hologram formation stereoscopic vision The technical scheme of display.So, if only individually adding a function of carrying out two-dimentional display on this basis, carrying While for digital hologram, in addition it is also necessary to provide one be used for two dimension show image.But in fact, user is not necessarily total It is intended to watch two-dimentional display picture, so if the image for two dimension display is provided when user wants and experiences stereoscopic vision, this It is undoubtedly a kind of waste to resource.But it is due to that provider does not know when user can need the image of two dimension display, Therefore in order to optimize Consumer's Experience, the image of two dimension display can only be also provided at any time so that user switches, even if this When user may and need not.This undoubtedly can further trigger that cost is too high, the complicated, efficiency of transmission that works is low and computing is fast The problems such as spending slow.

If it is possible to reproduce the digital hologram, so as to carry out two-dimentional display, then it can avoid carrying The image for two dimension display to be also provided separately while for digital hologram.Based on this, one of the improvement of the present embodiment exists In, by digital hologram by Three-dimensional Display (that is, virtual reality or stereoscopic vision) and two dimension show (i.e. hologram reconstruction) have Combine machine, so as to only need to provide digital hologram, according to the selection or instruction of user, generate for Three-dimensional Display Image or the image for two dimension display.

The mainstream technology of the current reconstruction of hologram (i.e. two dimension display), its core is the inverse process of holographic recording.At image Manage in angle, holographic recording is to carry out frequency domain modulation to object wave signal by reference to light.And the reconstruction of hologram is then to signal modulation As a result it is demodulated, so as to recover original picture signal.However, from technological layer, using traditional signal modulation and Demodulation mode reproduces to hologram, and resulting reproduction image light field includes：There is zero order diffracted light, reproduction image and conjugation Picture, wherein only reproduction image (+1 order diffraction light) is the signal required for us.Therefore, because zero-order terms and twin-image and conjugate image Presence, have impact on reproduction image quality, make the signal to noise ratio of reproduction image very low.Secondly, from diffraction efficiency, using it is traditional again Existing method can not obtain single reproduction image by hologram, and its diffraction efficiency is not high.

Therefore one of improvement of the present embodiment is to provide the method that digital hologram reproduces, and is realizing the same of the reconstruction of hologram When, interference of the zero order diffracted light and conjugate image in traditional reproduction image light field to reproduction image is reduced, the noise of reproduction image is improved Than and diffraction efficiency.

Figure 13 shows the side that two-dimentional display (reproducing hologram) is carried out according to digital hologram according to the present embodiment 5 Method flow.

Wherein step S1302 and step S1304 are used for the step of obtain digital hologram, itself and the step in Fig. 8 S802 is consistent with S804, repeats no more herein.It is however noted that, although being by virtual objects in embodiment 5 The mode for carrying out simulation trial produces digital hologram, but this is intended merely to carry out exemplary explanation.In fact, others are used for The mode of digital hologram is generated, the application is also applied for, as long as can be obtained by the digital hologram for 3-D view The image of display and the image for two dimension display.For example, in variation below, another will be illustrated according to increasing The method for being applied to the application generation digital hologram of strong reality technology.So as to further illustrate the technical scheme phase of the application For the distinct methods for generating digital hologram, with versatility.

Here key is how digital hologram to be reproduced, so as to generate two dimensional image.

As shown in figure 13, in step S1306, Fourier transformation is carried out to the digital hologram of acquisition, so as to obtain numeral The frequency spectrum of hologram；

In step S1308, the frequency spectrum of digital hologram is filtered and frequency displacement, so as to select and object light wavefront information Related portions of the spectrum, i.e. 1 grade of frequency spectrum of ﹢；

In step S1310, inverse Fourier transform is carried out to the portions of the spectrum after filtering and frequency displacement, so as to realize digital hologram The reproduction of figure, obtains the image for two dimension display.

Content wherein about filtering and the frequency displacement of digital hologram frequency spectrum will be described in detail later.

Optionally, in addition, be additionally included in step S1312, by acquisition be used for two dimension display image with it is described virtually it is right As A is contrasted to examine the effect of digital hologram reconstruction.

Using the method described in Figure 13, due to reducing (the 0 grade of frequency of correspondence of the zero order diffracted light in traditional reproduction image light field Spectrum) and conjugate image (correspondingly -1 grade of frequency spectrum) interference, therefore improve the signal to noise ratio and diffraction efficiency of reproduction image.

(6) specific algorithm and principle of two-dimentional display are carried out based on digital hologram

The specific algorithm and principle that two-dimentional display (i.e. hologram reconstruction) is carried out based on digital hologram is described below, wherein Formation part on digital hologram is identical with what is introduced in (four) part, repeats no more here.According to (four) part In explanation understand, the digital hologram of generation is represented by formula 7：

I_H(x, y)=| R (x, y)+U (x, y) |²

=| R (x, y |²+|U(x,y)|²+R^*(x,y)U(x,y)+R(x,y)U^*(x, y) (formula 7)

Wherein, R (x, y) represents reference light, and U (x, y) represents that complex amplitude function is (that is, above-mentioned before the Object light wave by diffraction Wavefront information), R^*(x, y) represents reference light conjugation, U^*(x, y) represents that complex amplitude function is conjugated before Object light wave.

Then Fourier transformation is carried out to the digital hologram shown in formula 7, so as to obtain the frequency spectrum represented by formula 10：

T (ξ, η)=T₀(ξ,η)+G(ξ-α,η-β)+G^*(ξ-α, η-β) (formula 10)

In formula 10, T₀The auto-correlation of (ξ, η) expression thing Diffraction of light wave wavefront complex amplitude and the Fourier of DC terms Portions of the spectrum (that is, 0 grade frequency spectrum), is low-frequency information；Fourier's frequency of G (ξ-α, η-β) expression thing Diffraction of light wave wavefront complex amplitude Spectrum part (that is, 1 grade of frequency spectrum of ﹢), G^*(ξ-α, η-β) represents the Fourier spectrum (that is, 1 grade of frequency spectrum of ﹣) of conjugation item information, wherein α It is carrier frequency with β.

Wherein, shown in the schematic diagram of the spectrum distribution of digital hologram such as Figure 14 (a) and Figure 14 (b).Wherein Figure 14 (a) shows Go out Fourier spectrum distribution of the simulation based on the Object light wave from virtual objects, and Figure 14 (b) then shows that the numeral is complete Cease the Fourier spectrum distribution of figure.

As shown in Figure 14 (a), the scope in Fourier's frequency domain of the simulation based on the Object light wave from virtual objects is B_x× B_y.The rectangle of the rightmost side represents the frequency information (that is, 1 grade of frequency spectrum of ﹢) of the Object light wave from virtual objects, i.e. G in Figure 14 (b) (ξ-α,η-β)；The rectangle in left side represents to be conjugated the frequency information (that is, 1 grade of frequency spectrum of ﹣) of Object light wave, i.e. G in Figure 14 (b)^*(ξ-α, η-β)；The rectangle at center origin in Figure 14 (b) represents DC terms frequency-portions (that is, 0 grade frequency spectrum), i.e. T₀(ξ,η)。

Wherein, actual digital hologram (Figure 15 a) and corresponding Fourier spectrum point is further illustrated in Figure 15 Butut (Figure 15 b).

According to above-mentioned spectrum distribution, frequency spectrum can directly be operated, so that frequency spectrum is filtered and frequency displacement, from And only retain the frequency information (that is, 1 grade of frequency spectrum of ﹢) of the Object light wave from virtual objects, i.e. G (ξ-α, η-β).

Figure 16 show the frequency spectrum of digital hologram is filtered and frequency displacement schematic diagram.Figure 16 (a) shows filtering Effect afterwards, wherein representing frequency information (that is, 1 grade of frequency spectrum of ﹣, G of conjugation Object light wave^*(ξ-α, η-β)) and expression DC terms Frequency-portions (that is, 0 grade frequency spectrum, i.e. T₀(ξ, η)) it is filtered out, so as to only retain the frequency for representing the Object light wave from virtual objects Rate information (that is, 1 grade of frequency spectrum of ﹢), i.e. G (ξ-α, η-β).Figure 16 (b) shows the spectrum distribution after frequency displacement, wherein representing from void The frequency information for intending the Object light wave of object is frequency shifted to center origin, so as to obtain simulating the Object light wave from virtual objects Fourier spectrum is distributed.

Next inverse Fourier transform is carried out to the portions of the spectrum shown by Figure 16 (b), you can obtain the thing of virtual objects Optical diffraction wavefront complex amplitude function.So as to complete the reconstruction of hologram, the image for two dimension display is obtained.And two reproduced More excellent level can be reached by tieing up the signal to noise ratio and diffraction efficiency of the image of display.

It should be noted that the spectrum component of digital hologram be also include the related high-frequency information of many object waves (for example ± 2 grades, ± 3 grades ... are waited frequency spectrum).But the intensity of these high-frequency informations relative to 0 grade and ± 1 grade of spectrum information intensity and Speech, is extremely faint.Therefore, in embodiment, the processing carried out to this 3 frequency spectrums (0 grade and ± 1 grade of frequency spectrum) is only shown, But the protection domain of the application is not represented to be only limitted to handle this 3 frequency spectrums.

(7) variation

Continue with the variation of explanation embodiments herein 5, the wherein variation and the area of the such scheme of embodiment 5 It is not the generation method of digital hologram, but for how to generate the left and right for Three-dimensional Display according to the digital hologram Image generates the image shown for two dimension, then consistent with the such scheme of embodiment 5.The content of the variation is divided into following Part：(1) method flow of digital hologram is generated；(2) principle of digital hologram is generated；(3) generate for Three-dimensional Display With the image of two dimension display；(4) technique effect of this variation.

(1) method flow of digital hologram is generated

In the variation, digital hologram is generated by using the method for three-dimensional registration in display enhancing technology.Figure 17 The variation according to embodiment 5 is shown, the flow chart of the method for generating digital hologram.

Step S1701, obtains the three-dimensional coordinate information of real object A under true environment

In this variation, holographic recording can be carried out to the real object in true environment by optical diffraction, then, The three-dimensional coordinate information of true environment is recorded in optical hologram.For example, arbitrary image is entered row distance for z1 (wherein, Z1 is above-mentioned diffraction distance) fresnel diffraction integration calculate, and with obtaining a hologram with reference to the interference of light, to simulate The optical holographic process of true environment.Then, this hologram is subjected to two-dimensional Fourier transform, high pass is carried out in domain space Filtering, only retains the related part of information before Object light wave, then carries out inverse Fourier transform to this part, obtains true in true environment The wavefront complex amplitude function (being properly termed as wavefront function again) of real object, wherein, three-dimensional coordinate is included in wavefront complex amplitude function InformationThat is, amplitude and real object phase information including real object.

Step S1702, obtains the three-dimensional coordinate information of virtual objectsThat is, obtaining the wavefront function of virtual objects.

, can be by Fresnel Diffraction Integral computational methods in this deformation, the wavefront for obtaining three-dimensional object A shakes again Width function(function includes the amplitude and phase information of object light).Alternatively, by the different layers of the three-dimensional object A (herein, diffraction distance, apart from corresponding, is the different diffraction distance in face with the foregoing preset recording of above the present embodiment Statement of the same distance under different application scene) under wavefront complex amplitude information be overlapped.

Step S1703, by real object A three-dimensional coordinate informationWith the three-dimensional coordinate information of virtual objectsCarry out Superposition；

Step S1704, the wavefront information obtained based on superposition makes objective holographic figure；

, can be to the three-dimensional coordinate information and the wavefront letter of virtual objects of real object in true environment in this variation Number is overlapped, and the result of superposition is still wavefront complex amplitude function.And according to principle of holography, the result to superposition carries out hologram Make.

Step S1705, robust detection is carried out to objective holographic figure.

As can be seen here, in variation, the performance evaluation of three-dimensional registration method that can also be in augmented reality Standard, is attacked the result that actual situation information interweaves, the robustness to this programme is detected and verified.

(2) principle of digital hologram is generated.

Below in conjunction with the method flow illustrated in this variation (1) part, digital hologram is generated to the present embodiment Principle illustrate：

Three-dimensional coordinate information on the acquisition virtual objects described in step S1702The present embodiment is may be referred to lead to Cross the principle that virtual objects generate wavefront function, i.e. the present embodiment (three) part (" generation hologram and reproduction left images Specific algorithm and principle ") in step 1) (" calculate from complex amplitude before the Object light wave of the object light of three-dimensional object reflection ") In content, repeat no more here.

Three-dimensional coordinate information on real object A under the acquisition true environment described in step S1701It can pass through Optical diffraction carries out holographic recording to the real object in true environment.Then, further through above-mentioned fresnel diffraction integration algorithm Description understand, be come simulated optical diffraction process using the calculating of Fresnel Diffraction Integral algorithm in this variation.Cause This, equally can be using Fresnel Diffraction Integral algorithm come simulated optical diffraction mistake when obtaining the holographic recording of real object Journey, to obtain the holographic recording of real object, also, the calculating of wavefront function of the specific calculating process with calculating virtual objects Cheng Xiangtong, here is omitted., can after the hologram of real object in true environment is calculated by above method simulation So that this hologram is carried out into two-dimensional Fourier transform, and high-pass filtering is carried out in frequency domain, only retain information before Object light wave Part, then inverse Fourier transform is carried out to this part, obtain wavefront complex amplitude function (that is, the ripple of real object in true environment Preceding function).

On the three-dimensional coordinate information by real object A described in step S1703Believe with the three-dimensional coordinate of virtual objects BreathIt is overlapped, the wavefront function of the wavefront function of real object and virtual objects can be overlapped and obtain wavefront letter Breath.It is assumed that obtaining the wavefront function of real object by the calculating of Fresnel Diffraction Integral algorithmWith the wavefront of virtual objects FunctionAfterwards, can be byWithIt is overlapped, obtains wavefront information U₁(x, y), wherein, In the embodiment of the present invention,WithIt can be calculated by the method described in aforementioned formula 2.

Objective holographic figure is made on the wavefront information obtained based on superposition described in step S1704, step can be based on The wavefront information U obtained in S1703₁(x, y), with reference to the present embodiment (three) part (" generation hologram and reproduction left and right figure As specific algorithm and principle ") in step 2) described in method, according to formula 7 obtain objective holographic figure.Herein no longer Repeat.

By the way that digital hologram can be generated with upper type.

(3) image for Three-dimensional Display and two dimension display is generated

It may be referred to the present embodiment (three) part (" generation hologram and the specific algorithm and original for reproducing left images Reason ") in step 3), generation left image and right image, and may be referred to (four) part of the present embodiment (" based on holography Figure carries out the method flow of two-dimentional display (reproducing hologram) ") and (five) part (" carried out based on digital hologram two-dimentional The specific algorithm and principle of display ") generate for Three-dimensional Display and the image for two dimension display.Repeat no more herein.

(4) technique effect of variation

In the image processing method and device that this variation is provided, digital hologram and computer-generated hologram technology are applied to In augmented reality system, the advantage that this method and device band are come includes at least one of：

The present invention obtains the hologram of real object in true environment according to optical holographic principle, and this hologram is carried out into two Fourier transformation is tieed up, and high-pass filtering is carried out in frequency, only retains the related part of information before Object light wave, then this part is carried out Inverse Fourier transform, obtains the wavefront function of real object in true environment.This wavefront function is true three-dimensional information, comprising true The amplitude information and phase information of real object in environment.

The wavefront function of virtual objects, and and true environment are calculated according to fresnel diffraction principle and according to diffraction distance The wavefront function superposition of middle real object, the layering for hologram (hologram herein is above-mentioned objective holographic figure) is remembered Record.Hierarchical information record can be carried out to true environment and virtual objects by hologram.And during the reconstruction of hologram, laminated record Information (true environment and virtual objects) will not be interfered so that hologram can store huge information content.

Anti-tear ruining property is the most important properties of holographic technique.Holographic technique is introduced into three-dimensional registration by the present invention, Neng Gouzeng The repellence that strong system is attacked in terms of cutting, compression, filtering and noise, so as to meet the robustness of performance.

Certainly, the figure switched over based on digital hologram between Three-dimensional Display and two dimension display that the application is proposed Processing method and image processing apparatus are not limited to the digital hologram recorded described in embodiment 5 and its variation as at, only It can be used for obtaining the technical side of the digital hologram of the image for Three-dimensional Display and the image for two dimension display Case is suitable for the application.

Embodiment 6

It is described below according to the embodiment of the present application 6, wherein accompanying drawing 18 is shown according to embodiment 6, for based on digital complete Breath figure realizes the schematic diagram of the image processing system 1800 of two dimension display and the switching of Three-dimensional Display.

As shown in figure 18, described image processing system 1800 includes image processing apparatus 1810 and display 1820.Its Described in display device 1810 include parameter input interface 1811, hologram input interface 1812, Three-dimensional Display processing module 1813rd, two-dimentional Display processing module 1814, driver module 1815, identifying unit 1816 and hologram allocation unit 1817。

Wherein on hologram input interface 1812, hologram allocation unit 1817, Three-dimensional Display processing module 1813, two Tie up the function of Display processing module 1814 and driver module 1815 with it is right in the image processing apparatus described in embodiment 5 The hologram input interface 512 answered, hologram allocation unit 517, Three-dimensional Display processing module 513, two-dimentional Display processing module 514 and driver module 515 function phase it is same.And may be referred in embodiment 5 illustrated content perform it is above-mentioned each The operation of part, is repeated no more here, and the operation of parameter input interface 1811 and identifying unit 1816 is introduced below.

Wherein parameter input interface 1811 is used to receive the resource distribution related to the operation of the image processing system 1800 Test parameter (speed of service of such as processor and load, the remaining space of caching, Current resource occupation rate, network speed and Data transfer rate during reception digital hologram etc.), and the resource distribution test parameter of reception is transmitted to identifying unit 1816.Institute State identifying unit 1816 judges currently whether be adapted for Three-dimensional Display according to resource distribution test parameter.If matched somebody with somebody by resource Test parameter is put, the identifying unit 1816 judges that Three-dimensional Display is supported in the resource distribution currently run enough, then to hologram The status register of allocation unit 1817 sends the instruction for carrying out Three-dimensional Display, otherwise to the state of hologram allocation unit 1817 Register sends the instruction for carrying out two-dimentional display.The status register of hologram allocation unit 1817 is being received from identifying unit After 1816 instruction, stored state is updated, so that the hologram allocation unit 1817 is complete by the numeral of reception Breath figure sends to the Three-dimensional Display processing module 1813 or sent to the two-dimentional Display processing module 1814.

Alternatively, the image processing apparatus 1810 in embodiment 6 is also provided with timing means, so as to pre- The fixed time cycle carries out real-time tracking and detection to resource distribution situation repeatedly, real so as to the change according to resource distribution When realize switching between Three-dimensional Display and two dimensional mode.

Figure 19 shows parameter input interface 1811 in embodiment 6, identifying unit 1816 and hologram allocation unit The workflow diagram of status register in 1817.In step S1902, parameter input interface 1811 is received and the image procossing system The resource distribution test parameter of the operation correlation of system 1800, and identifying unit 1816 is sent it to, so as to proceed to step S1904.In step S1904, the identifying unit 1816 judges whether to three according to the resource distribution test parameter received Dimension display.If it is determined that unit 1816 determines that carrying out Three-dimensional Display then enters step S1906, otherwise enters step S1910. In step S1906, identifying unit 1816 sends the finger for carrying out Three-dimensional Display to the status register of hologram allocation unit 1817 Show, in step S1910, identifying unit 1816 sends to the status register of hologram allocation unit 1817 and carries out two-dimentional display Indicate.In step S1908, the status register of hologram allocation unit 1817 updates status register according to the instruction received Middle stored state so that the hologram allocation unit 1817 correspondingly sends the digital hologram received to described Three-dimensional Display processing module 1813 or the two-dimentional Display processing module 1814.Then, step S1912 by the scheduled time it Afterwards, step S1902 is again returned to, the flow of subsequent cycle is carried out.

By the technical scheme of embodiment 6, image processing apparatus 1810 can be according to current resource distribution situation three Dimension stereoscopic display and two dimensional surface are automatically selected or switched between showing.So that when resource distribution is not enough to support When 3 D stereo is shown, it can also be shown by way of two dimensional surface is shown in the case of low configuration.So as to improve The continuity of display.

Although it should be noted that realizing that each interface and module of image processing flow are included in image processing apparatus 1810 In, but it is also possible to branch is separated from each other in different devices.And the function of distinct interface and module can also be carried out suitably Adjustment and combination.Also, each interface and module in described image processing apparatus 1810 are also not necessarily limited to hardware circuit, Can be each program module or software module realized by processor or controller.

Embodiment 7

Illustrate the image processing system 2000 according to the embodiment of the present application 7 below, wherein accompanying drawing 20 is shown according to implementation The schematic diagram of the image processing system 2000 of example 9.

As shown in figure 20, described image processing system 2000 includes image processing apparatus 2010 and display 2020.Its Described in image processing apparatus 2010 include indication signal input interface 2011, hologram input interface 2012, at Three-dimensional Display Manage module 2013, two-dimentional Display processing module 2014, driver module 2015, identifying unit 2016 and hologram distribution single Member 2017.

Wherein on hologram input interface 2012, hologram allocation unit 2017, Three-dimensional Display processing module 2013, two Tie up the function of Display processing module 2014 and driver module 2015 with the image processing apparatus described in embodiment 5 Corresponding hologram input interface 512, hologram allocation unit 517, Three-dimensional Display processing module 513, two dimension display Processing module 514 and the function phase of driver module 515 are same.Here repeat no more, indication signal input is introduced below The operation of interface 2011 and identifying unit 2016.

Wherein indication signal input interface 2011 be used for from be currently running application (game that is for example currently running, by interior Hold program etc. that provider provides) the application indication signal that receives, and this is transmitted to identifying unit 2016 using indication signal. Wherein, this is used to indicate that the progress 3 D stereo of described image processing system 2000 is shown or two dimensional surface is shown using indication signal Selection or switching.The identifying unit 2016 judges whether to Three-dimensional Display according to the application configured information number.If The identifying unit 2016 judges to carry out Three-dimensional Display according to the application configured information, then to hologram allocation unit 2017 Status register sends the instruction for carrying out Three-dimensional Display, otherwise sends and carries out to the status register of hologram allocation unit 2017 The instruction of two dimension display.The status register of hologram allocation unit 2017 receive the instruction from identifying unit 2016 with Afterwards, stored state is updated, so that the hologram allocation unit 2017 sends the digital hologram of reception to institute State Three-dimensional Display processing module 2013 or send to the two-dimentional Display processing module 2014, so as to carry out Three-dimensional Display or progress Two dimension display.

Accompanying drawing 21 shows the flow chart of the image processing apparatus 2010 according to embodiment 9.At step S2102, image Device 2010 is managed by indication signal input interface 2011 from the application reception application indication signal being currently running, and this is applied Indication signal is sent to identifying unit 2016.In step S2104, the identifying unit 2016 is sentenced according to the application indication signal It is disconnected whether to carry out Three-dimensional Display.If it is determined that unit 2016 determines that carrying out Three-dimensional Display then enters step S2106, otherwise enters To step S2110.In step S2106, identifying unit 2016 sends to the status register of hologram allocation unit 2017 and carried out The instruction of Three-dimensional Display, in step S2110, identifying unit 2016 to the status register of hologram allocation unit 2017 send into The instruction of row two dimension display.In step S2108, the status register of hologram allocation unit 2017 according to the instruction received more The state stored in new state register so that the hologram allocation unit 2017 is corresponding by the digital hologram received Ground is sent to the Three-dimensional Display processing module 2013 or the two-dimentional Display processing module 2014.

By embodiment 7, image processing system 2000 can be according to the game being currently running or the section shown Purpose is asked, and is automatically selected or switched between 3 D stereo is shown and two dimensional surface is shown.So as to enhance game Or the bandwagon effect of program, and provide the user finer experience.

Although it should be noted that realizing that each interface and module of image processing flow are included in image processing apparatus 2010 In, but it is also possible to branch is separated from each other in different devices.And the function of distinct interface and module can also be carried out suitably Adjustment and combination.Also, each interface and module in described image processing apparatus 2010 are also not necessarily limited to hardware circuit, Can be each program module or software module realized by processor or controller.

Embodiment 8

Figure 22 shows the schematic diagram of the display system 2200 according to the embodiment of the present application 8.

As shown in figure 22, described image processing system 2200 includes image processing apparatus 2210 and display 2220.Its Described in image processing apparatus 2210 include parameter input interface 2211A, user interface 2211B, indication signal input interface 2211C, hologram input interface 2212, Three-dimensional Display processing module 2213, two-dimentional Display processing module 2214, display driving mould Block 2215, identifying unit 2216, hologram allocation unit 2217 and by resource distribution register 2218A, user selection deposit Device 2218B and application indicator register 2218C compositions register group.

Wherein on hologram input interface 2212, hologram allocation unit 2217, Three-dimensional Display processing module 2213, two The function of tieing up Display processing module 2214 and driver module 2215 is filled with the described image procossing described in embodiment 5 Corresponding hologram input interface 512 in putting, hologram allocation unit 517, Three-dimensional Display processing module 513, at two-dimentional display Manage module 514 and the function phase of driver module 515 is same.Here repeat no more, other parts content is introduced below.

As shown in figure 22, the image processing apparatus 2210 described in embodiment 10 fills the image procossing in embodiment 7,8 and 9 The function of putting is integrated.Include parameter input interface 2211A, Yong Hujie so as to the image processing apparatus 2210 of embodiment 10 Mouth 2211B and indication signal input interface 2211C, and the resource distribution register corresponding with the information of each interface 2218A, user's mask register 2218B and application indicator register 2218C.So that described image processing system 2200 comprehensive reference resource distribution situation, the selection of user and can make phase situations such as be currently running the instruction of application The operation answered.

Wherein, the parameter input interface 2211A, user interface 2211B and indication signal input interface 2211C can be with Corresponding description in reference implementation example 7-9.The resource distribution register 2218A is used to store current resource distribution situation, For example support Three-dimensional Display or do not support Three-dimensional Display.User's mask register is used to store current user's selection, example Three-dimensional Display is such as carried out, two-dimentional display is carried out or abandons selection (now state is " sky ", represents that user is given tacit consent to by system or operation Application program voluntarily select three-dimensional or two dimension display pattern).The application indicator register is used to store what is be currently running The current instruction of application, for example, carry out Three-dimensional Display, carry out two-dimentional display or do not have that (now state is " sky ", represents not transport Capable application, in this case generally with the Modular Display System user interface of two dimension display).That is, the resource is matched somebody with somebody The status register (not shown) put in register 2218A and hologram allocation unit 2217 may each comprise two states, and User's mask register 2218B and the application indicator register 2218C may include three kinds of states, as shown in the table：

Table 1

According to above table, it can be seen that resource distribution register 2218A, user's mask register 2218B and application refer to Show register 2218C state it is possible that the situation collided with one another.In the present embodiment, resource distribution register 2218A State highest priority.That is, when resource distribution register 2218A state is does not support to carry out Three-dimensional Display, no matter User selects which type of instruction which type of instruction or the application send, and can only all carry out two-dimentional display.User instruction Priority higher than game indicate priority, i.e., no matter the application provide two dimension show or Three-dimensional Display instruction, remove Non-user mask register is designated as " sky ", otherwise all should be shown in the way of user selects.

The resource distribution register 2218A, user's mask register 2218B, using indicator register 2218C and institute Stating status register can be accessed by the identifying unit 2216, so that the information sent according to identifying unit 2216 is carried out more Newly and the judgement flow for identifying unit 2216 is read out by identifying unit 2216.

Wherein, the parameter input interface 2211A of described image processing unit 2210 is received and image processing system 2200 Run related resource distribution test parameter (speed of service of such as processor and load, the remaining space of caching, current money Data transfer rate when source occupation rate, network speed and reception digital hologram etc.), and the resource distribution test parameter of reception is transmitted To identifying unit 2216.The user interface 2211B is used to receive the choosing shown on Three-dimensional Display or two dimension from user Instruction or switching command are selected, and user instruction is transmitted to identifying unit 2216.The indication signal input interface 2211C is used for The application received from the application (such as the game being currently running, the program provided as content supplier) being currently running indicates letter Number, and this is transmitted to identifying unit 2216 using indication signal.

Wherein decision process such as accompanying drawing 23 of the described image processing unit 2210 on parameter input interface 2211A input It is shown.As a rule, described image processing system 2200 is when just starting, begin to carry out the detection of resource configuration parameter with And receive.Then real-time tracking and detection are carried out to resource distribution situation with the predetermined time cycle repeatedly, so as to basis The switching between Three-dimensional Display and two dimensional mode is realized in the change of resource distribution in real time.

As shown in figure 23, in step S2302, described image processing unit 2210 is received by parameter input interface 2211A The resource distribution test parameter detected.

Then in step S2304, described image processing unit 2210 reads the user by the identifying unit 2216 Mask register 2218A, and judge user's mask register 2218A whether as sky.If user's selection deposit Device 2218A is not empty ("No"), and that has been made that choosing with regard to specification user to three-dimensional display mode or two dimensional mode Select or switch.Because the priority of user's selection in the present embodiment is higher than the instruction of application program, therefore only user selects It is sky, i.e. user and non-selected progress two dimension or during Three-dimensional Display to select register, and flow can just enter step S2306 to enter one Step judges the state of application indicator register, and otherwise flow enters step S2312 further to be selected to deposit according to user Device 2218B state is handled.

In step S2306, the identifying unit 2216, which is read, applies indicator register 2218C, to judge that the application refers to Whether the state for showing register 2218C is empty or two dimension display.When identifying unit 2216 judges the application indicator register When 2218C state is empty or two dimension display ("Yes"), flow enters step S2308.Otherwise, i.e., using indicator register 2218C state is Three-dimensional Display, and flow enters step S2314.

In step S2308, due to being all to carry out two dimension to show when application indicator register 2218C is empty or two dimension is shown Show, therefore the requirement for Three-dimensional Display to resource distribution can be than relatively low.In this case need not be further according to The situation of resource distribution is judged and can directly determine to carry out two-dimentional display.Therefore it is described to judge single in step S2308 Member 2216 according to application indicator register 2218C status register from state to the hologram allocation unit 2217 send into The instruction of row two dimension display so that the status register updates stored state, so as to by digital hologram distribute to Two-dimentional Display processing module 2214, carries out two-dimentional display；And then identifying unit 2216 is surveyed according to the resource distribution received Whether the current resource distribution situation of examination parameter decision can support Three-dimensional Display, and update resource distribution register accordingly 2218A state, for follow-up judgement flow.Then after the scheduled time, flow backs within step S2302, Resource distribution test parameter is received again.

Continue with and step S2304 is illustrated.When identifying unit 2216 is in step S2304 judgement user's selection deposits Device is not space-time, then flow enters step S2312.In step S2312 described images processing unit 2210 further by sentencing Order member 2216 judges whether the state of user's mask register 2218B is to carry out Three-dimensional Display.

When identifying unit 2216 in step S2312 judges that the current state of user's mask register 2218B is to carry out During Three-dimensional Display ("Yes"), then flow enters step S2314；Otherwise, i.e. the current state of user's mask register 2218B is When carrying out two dimension display, flow enters step S2310.

In step S2310, due to when user's mask register 2218B state is two dimension display, image processing system 2200 will carry out two-dimentional display, therefore the requirement for Three-dimensional Display to resource distribution can be than relatively low.In such case It need not be judged further according to the situation of resource distribution down and can directly determine to carry out two-dimentional display.Therefore in step In S2310, the identifying unit 2216 is according to user's mask register 2218B state to the hologram allocation unit 2217 Status register send the instruction for carrying out two-dimentional display so that the status register updates stored state, so as to Digital hologram is distributed to two-dimentional Display processing module 2214, two-dimentional display is carried out；And the then basis of identifying unit 2216 The resource distribution test parameter received judges whether current resource distribution situation can support Three-dimensional Display, and updates accordingly Resource distribution register 2218A state, for follow-up judgement flow.Then after the scheduled time, flow is returned again Step S2302 is returned to, resource distribution test parameter is received again.

In step S2314, now at least user's mask register 2218B or the state using indicator register 2218C are Carry out Three-dimensional Display.In this case, it is higher due to carrying out requirement of the Three-dimensional Display to resource distribution, in order to avoid occurring Because resource distribution does not reach display quality deterioration or even the situation that can not be shown caused by the requirement of Three-dimensional Display, judge single Member 2216 can judge whether current resource distribution can support Three-dimensional Display according to the resource distribution test parameter received.Such as Fruit identifying unit 2216 judges that current resource can support Three-dimensional Display ("Yes"), then flow enters step S2315, otherwise flows Journey enters step S2317.

In step S2315, identifying unit enters according to the result of determination of support Three-dimensional Display to resource distribution register 2218A Row updates, and the instruction of the status register transmission Three-dimensional Display to hologram allocation unit 2217.

In step S2317, identifying unit is not according to supporting the result of determination of Three-dimensional Display to resource distribution register 2218A It is updated, and the instruction that two dimension is shown is sent to the status register of hologram allocation unit 2217.In addition, display system 2200 also issue the user with the notice that Current resource does not support Three-dimensional Display by display 2220.

In step S2316, the status register of the hologram allocation unit 2217 is received according to from identifying unit 2216 Instruction, status register is correspondingly updated, so as to which digital hologram is sent to Three-dimensional Display processing module 2213 or two dimension Display processing module 2214 shows to carry out Three-dimensional Display or two dimension.

Pass through the scheduled time after step S2316, the flow backs within step S2302, resource is received again and is matched somebody with somebody Put test parameter.

By such mode, image processing apparatus 2210 can be aobvious in 3 D stereo according to current resource distribution situation Show and two dimensional surface show between automatically selected or switched.So that when resource distribution is not enough to support 3 D stereo During display, it can also be shown by way of two dimensional surface is shown in the case of low configuration.So as to improve display Continuity.

Below, Figure 24 shows the flow chart when image processing apparatus 2210 receives user instruction.

As shown in figure 24, in step S2402, image processing apparatus 2210 is received by user interface 2211B and comes from user The instruction that is selected or switched between two dimension display and Three-dimensional Display.Certainly, user can also send and abandon selection (i.e. user is not selected two dimension display or Three-dimensional Display, and voluntarily determines whether to carry out two dimension by system or application for instruction Display or the switching of Three-dimensional Display).

In step S2404,2216 pairs of user instructions received of identifying unit judge, when judging user instruction to enter During row Three-dimensional Display, flow enters step S2406.During when judging user instruction as two dimension display is carried out, flow enters step Rapid S2412.During when judging user instruction as selection is abandoned, flow enters step S2416.

In step S2416, when user instruction to abandon during selection, identifying unit 2216 selects user according to the instruction of user Select the state that register 2218B stored and be updated to sky, then terminate flow.

In step S2412, when user instruction to carry out during two dimension display, identifying unit 2216 will be used according to the instruction of user The state that family mask register 2218B is stored is updated to carry out two-dimentional display, and to the shape of hologram allocation unit 2217 State register sends the instruction for carrying out two-dimentional display.

In step S2406, when user instruction is carries out Three-dimensional Display, identifying unit 2216 is from resource distribution register 2218A reads the state that resource distribution register 2218A is stored, and when the state of storage is supports Three-dimensional Display, illustrates to work as Preceding resource distribution can support Three-dimensional Display, and flow enters step S2408.Otherwise, when the state stored is not support During Three-dimensional Display, flow enters step S2414.

The shape for being stored user's mask register 2218B according to the instruction of user in step S2408, identifying unit 2216 State is updated to carry out Three-dimensional Display, and the finger for carrying out Three-dimensional Display is sent to the status register of hologram allocation unit 2217 Show.

The shape for being stored user's mask register 2218B according to the instruction of user in step S2414, identifying unit 2216 State is updated to carry out Three-dimensional Display, but the finger for carrying out two-dimentional display is sent to the status register of hologram allocation unit 2217 Show.And show that system 2200 also issues the user with the notice that Current resource does not support Three-dimensional Display by display 2220.

In step S2410, the status register of hologram allocation unit 2217 is according to the instruction of reception, by the shape stored State is updated to carry out two-dimentional display or carries out Three-dimensional Display, so as to which digital hologram is sent to Three-dimensional Display processing module 2213 Or two-dimentional Display processing module 2214.Then flow terminates.

Therefore, by the flow, user can be according to the wish of oneself selection two dimension display or Three-dimensional Display, or selection Voluntarily realized to two dimension display or the selection and switching of Three-dimensional Display by system or application program.Therefore Consumer's Experience is enhanced, So as to achieve good effect.

Figure 25 shows the flow chart when indication signal input interface 2211C is received using indication signal.

As shown in figure 25, in step S2502, image processing apparatus 2210 is by indication signal input interface 2211C from just The instruction for being selected or being switched between two dimension display and Three-dimensional Display is received in the application of operation.

In step S2504, identifying unit 2216 conducts interviews to user's mask register 2218B, judges active user's choosing Whether the state for selecting register 2218B is " sky ".If the state of user's mask register is " sky " ("Yes"), flow is entered Enter to step S2506；Otherwise flow enters step S2518.

In step S2518, because the state of user's mask register is not " sky ", this explanation user is to two dimension display Or Three-dimensional Display is made that selection.Because the priority that user selects is higher than the priority that application is indicated, therefore in such case Under, identifying unit 2216 is only according to the application indication signal received, and the state to application indicator register 2218C is carried out more Newly, flow is just terminated.

In step S2506, because user's mask register state is " sky ", this explanation user is abandoned to two dimension display or three The selection of display is tieed up, and transfers to system or application voluntarily to be selected.Therefore, identifying unit 2216 judges that the application received refers to Show whether be to carry out Three-dimensional Display.When judging that application is designated as carrying out Three-dimensional Display ("Yes"), flow enters step S2508.Otherwise when judging that application is designated as without Three-dimensional Display ("No"), flow enters step S2514.

In step S2514, when application is designated as without Three-dimensional Display (that is, carrying out two-dimentional display), identifying unit 2216 indicate to be updated to carry out two-dimentional display using the indicator register 2218C states stored according to the application received, And the instruction for carrying out two-dimentional display is sent to the status register of hologram allocation unit 2217.

In step S2508, when application is designated as carrying out Three-dimensional Display, identifying unit 2216 is from resource distribution register 2218A reads the state that resource distribution register 2218A is stored, and when the state of storage is supports Three-dimensional Display, illustrates to work as Preceding resource distribution can support Three-dimensional Display, and flow enters step S2510.Otherwise, when the state stored is not support During Three-dimensional Display, flow enters step S2516.

In step S2510, identifying unit 2216 will be deposited according to the application instruction received using indicator register 2218C The state of storage is updated to carry out Three-dimensional Display, and carries out three-dimensional show to the status register transmission of hologram allocation unit 2217 The instruction shown.

In step S2516, identifying unit 2216 will be deposited according to the application instruction received using indicator register 2218C The state of storage is updated to carry out Three-dimensional Display, but is sent to the status register of hologram allocation unit 2217 and carry out two-dimentional show The instruction shown.

In step S2512, the status register of hologram allocation unit 2217 is according to the instruction of reception, by the shape stored State is updated to carry out two-dimentional display or carries out Three-dimensional Display, so as to which digital hologram is sent to Three-dimensional Display processing module 2213 Or two-dimentional Display processing module 2014.Then flow is terminated.

So as to according to the flow, image processing system 2200 can shown according to the game being currently running or The request of program, is automatically selected or is switched between 3 D stereo is shown and two dimensional surface is shown.So as to enhance trip Play or the bandwagon effect of program, and provide the user finer experience

To sum up, according to the image processing system 2200 of the present embodiment, can consider user instruction, resource distribution with And application is indicated, and switched between Three-dimensional Display and two dimension display.So as to comprehensively meet Consumer's Experience, display Continuity and many demands such as quality and the demonstrating effect of application.

Although it should be noted that realizing that each interface and module of image processing flow are included in image processing apparatus 2210 In, but it is also possible to branch is separated from each other in different devices.And the function of distinct interface and module can also be carried out suitably Adjustment and combination.Also, each interface and module in described image processing apparatus 2210 are also not necessarily limited to hardware circuit, Can be each program module or software module realized by processor or controller.

Also, in the present embodiment, the priority of resource distribution result of determination is higher than the priority that user selects, and user The priority of selection is higher than the priority that application is indicated.But, this is only intended to illustrate the scheme of the present embodiment, actual Upper the application can also take other priority rankings.The priority that for example application is indicated can be preferential higher than what user selected Level, and the priority that user's selection and application are indicated can also be higher than priority of resource distribution result of determination, etc..And Indicate that three is aobvious in two dimension display and three-dimensional in combination with user instruction, resource distribution and application although being given in embodiment The technical scheme switched between showing, but it is also possible to be combined in above-mentioned factor, for example, only consider user instruction With resource distribution or only consider that user instruction and application are indicated etc., and according to actual needs difference can also be set to examine Priority ranking between worry factor.

Although and the resource distribution register 2218A described in the present embodiment, user's mask register 2218B, application Status register in indicator register 2218C and the hologram allocation unit is register independent of each other, but It is contemplated that its function is implemented in the different memory areas block of same storage device.

In summary, following technology present in prior art is solved by the image processing apparatus and method of the application Problem：1) excessive 3 D stereo is shown or excessive planar perspective shows caused user-experience degradation；2) by configure compared with It is possible that can not show or situation that display quality is inferior when low device carries out 3 D stereo display；3) apply and open up The content shown needs the switching of Three-dimensional Display and two dimension display；4) current three-dimensional stereo display technique cost is too high, work numerous Again, efficiency of transmission is low and arithmetic speed is slow；5) 3 D stereo is shown shows the related question of both patterns with two dimensional surface； 6) there is zero-order terms and twin-image and conjugate image when computer hologram is reproduced causes diffraction efficiency not high, and signal to noise ratio is very low Problem.So as to improving Consumer's Experience, being flexibly applied to various hardware configurations and strengthening game and displaying content etc. The expressive force of application.

Above-mentioned the embodiment of the present application sequence number is for illustration only, and the quality of embodiment is not represented.

In above-described embodiment of the application, the description to each embodiment all emphasizes particularly on different fields, and does not have in some embodiment The part of detailed description, may refer to the associated description of other embodiment.

In several embodiments provided herein, it should be understood that disclosed technology contents, others can be passed through Mode is realized.Wherein, device embodiment described above is only schematical, such as division of described unit or module, It is only a kind of division of logic function, there can be other dividing mode when actually realizing, such as multiple units or module or group Part can combine or be desirably integrated into another system, or some features can be ignored, or not perform.It is another, it is shown Or the coupling each other discussed or direct-coupling or communication connection can be by some interfaces, module or unit it is indirect Coupling is communicated to connect, and can be electrical or other forms.

The unit illustrated as separating component or module can be or may not be it is physically separate, as The part that unit or module are shown can be or may not be physical location or module, you can with positioned at a place, or Person can also be distributed in multiple NEs or module.Some or all of list therein can be selected according to the actual needs Member or module realize the purpose of this embodiment scheme.

In addition, each functional unit or module in the application each embodiment can be integrated in a processing unit or mould In block or unit or module are individually physically present, can also two or more units or module be integrated in In one unit or module.Above-mentioned integrated unit or module can both be realized in the form of hardware, it would however also be possible to employ software The form of functional unit or module is realized.

If the integrated unit is realized using in the form of SFU software functional unit and as independent production marketing or used When, it can be stored in a computer read/write memory medium.Understood based on such, the technical scheme of the application is substantially The part contributed in other words to prior art or all or part of the technical scheme can be in the form of software products Embody, the computer software product is stored in a storage medium, including some instructions are to cause a computer Equipment (can for personal computer, server or network equipment etc.) perform the application each embodiment methods described whole or Part steps.And foregoing storage medium includes：USB flash disk, read-only storage (ROM, Read-Only Memory), arbitrary access are deposited Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic disc or CD etc. are various can be with store program codes Medium.

Described above is only the preferred embodiment of the application, it is noted that for the ordinary skill people of the art For member, on the premise of the application principle is not departed from, some improvements and modifications can also be made, these improvements and modifications also should It is considered as the protection domain of the application.

Claims

1. a kind of image processing method, it is characterised in that including：

Obtain digital hologram；

According to predetermined condition, it is determined whether carry out Three-dimensional Display；And

When it is determined that carrying out Three-dimensional Display, the image for Three-dimensional Display is generated based on the digital hologram；Or

When it is determined that during without Three-dimensional Display, the image for two dimension display is generated based on the digital hologram.

2. image processing method according to claim 1, it is characterised in that methods described also includes receiving from user's User's instruction receives application instruction from the application being currently running, and

It is described according to predetermined condition, it is determined whether carry out Three-dimensional Display, including：

Indicated to determine whether to carry out Three-dimensional Display according to the user；Or

Indicated to determine whether to carry out Three-dimensional Display according to the application.

3. image processing method according to claim 2, it is characterised in that methods described is also included to carrying out image procossing And/or carry out the relevant parameter of device that image shows and detected, and it is described according to predetermined condition, it is determined whether progress three Dimension display, in addition to：

After being indicated according to the user or indicating to determine to carry out Three-dimensional Display according to the application, further according to the correlation Parameter determines whether to carry out Three-dimensional Display.

4. image processing method according to claim 1, it is characterised in that methods described is also included to carrying out image procossing And/or carry out the relevant parameter of device that image shows and detected, and

According to the relevant parameter, it is determined whether carry out Three-dimensional Display.

5. image processing method according to claim 4, it is characterised in that methods described also includes receiving from user's User's instruction receives application instruction from the application being currently running, and described according to predetermined condition, it is determined whether progress three Dimension display, including：

After determining without Three-dimensional Display according to the relevant parameter, directly determine without Three-dimensional Display.

6. image processing method according to claim 4, it is characterised in that methods described also includes receiving from user's User's instruction receives application instruction from the application being currently running, and

It is described according to predetermined condition after progress Three-dimensional Display is determined according to the relevant parameter, it is determined whether to carry out three-dimensional It has been shown that, in addition to：

Indicated according to the user, it is determined whether carry out Three-dimensional Display；Or

7. image processing method according to claim 1, it is characterised in that methods described also includes receiving from user's User indicate and from be currently running application receive application indicate, and

Indicated according to the user, it is determined that indicating to determine whether to carry out Three-dimensional Display based on the application.

8. image processing method according to claim 1, it is characterised in that

The acquisition digital hologram includes：

Obtain the three-dimensional information of virtual objects；And

The digital hologram is generated according to the three-dimensional information, and

Described generate includes for the image of Three-dimensional Display：

Corresponded respectively to according to the digital hologram and related to the digital hologram first with reference to optical information generation The left image and right image of left view point and right viewpoint.

9. image processing method according to claim 8, it is characterised in that the three-dimensional information of the acquisition virtual objects, Including：

Obtain the preset recording distance of the virtual objects；And

Wavefront function of the virtual objects at the preset recording distance is calculated, wherein, the wavefront function includes amplitude Information and phase information, and

It is described that digital hologram is generated according to the three-dimensional information, including：

Calculated according to the wavefront function and second with reference to optical information and generate the digital hologram.

10. image processing method according to claim 9, it is characterised in that the first reference light information includes and institute State left view point and the corresponding left view point of right viewpoint reproduces to reproduce with reference to optical information and right viewpoint and refers to optical information, wherein, the left side Viewpoint, which reproduces reference light information, to be included：The light that simulation is sent from left reproduction reference light source corresponding with left view point is irradiated to described Incident angle during digital hologram, the right viewpoint, which reproduces reference light information, to be included：Simulate from the right side corresponding with right viewpoint again The light sent referring now to light source is irradiated to the incident angle during digital hologram, and

It is described according to the digital hologram and related to the digital hologram first generated with reference to optical information it is right respectively Should in left view point and the left image and right image of right viewpoint, including：Reproduced based on the left view point and refer to optical information and right viewpoint Reproduce and generate the left image and right image with reference to optical information and the digital hologram.

11. image processing method according to claim 9, it is characterised in that the preset recording distance at least two, And

The wavefront function that the virtual objects are calculated at the preset recording distance, including：

Wavefront function of the virtual objects at each described preset recording distance is calculated respectively, and the wavefront function is entered Row superposition；And

Described calculated according to the wavefront function and second with reference to optical information generates the digital hologram, including：

Calculated according to the wavefront function after superposition and described second with reference to optical information and generate the digital hologram.

12. image processing method according to claim 1, it is characterised in that

The acquisition digital hologram includes：

The three-dimensional coordinate information of real object in true environment is obtained, and obtains the three-dimensional coordinate information of virtual objects；And

It is complete according to the three-dimensional coordinate information of the three-dimensional coordinate information of the real object and the virtual objects generation numeral Breath figure, and

Described generate includes for the image of Three-dimensional Display：

13. image processing method according to claim 12, it is characterised in that obtain three of real object in true environment Dimension coordinate information includes：

Holographic process is carried out to the real object, the hologram of the real object is obtained；And

Time frequency analysis is carried out to the hologram of the real object, the wavefront function of the real object is obtained, wherein, it is described true The wavefront function of real object includes the three-dimensional coordinate information of the real object.

14. method according to claim 13, it is characterised in that frequency division when being carried out to the hologram of the real object Analysis, obtaining the wavefront function of the real object includes：

The hologram of the real object is converted into by domain space by Fourier transformation；

High-pass filtering processing is carried out to the hologram of the real object in domain space, the Object light wave of the real object is obtained Preceding information；And

Inverse Fourier transform is carried out to the object light wavefront information, the wavefront function of the real object is obtained.

15. method according to claim 14, it is characterised in that obtaining the three-dimensional coordinate information of virtual objects includes：

Obtain the preset recording distance of the virtual objects；And

Wavefront function of the virtual objects at the preset recording distance is calculated, wherein, the wavefront letter of the virtual objects Number includes the three-dimensional coordinate information of the virtual objects.

16. method according to claim 15, it is characterised in that according to the three-dimensional coordinate information of the real object and institute Stating the three-dimensional coordinate information generation digital hologram of virtual objects includes：

The wavefront function of the wavefront function of the real object and the virtual objects is overlapped and obtains wavefront information；And

Calculated according to resulting wavefront information and second with reference to optical information and generate the digital hologram.

17. image processing method according to claim 1, it is characterised in that

The image generated for two dimension display, is by only retaining described with generation in the frequency spectrum of the digital hologram The frequency spectrum for the wavefront function that image for two dimension display is associated, and filter out the realization of other frequency spectrums.

18. image processing method according to claim 17, it is characterised in that the generation is used for the figure of two dimension display Picture, is realized by retaining 1 grade of frequency spectrum of ﹢ in the frequency spectrum of the digital hologram and filtering out 1 grade of frequency spectrum of 0 grade of frequency spectrum and ﹣ 's.

19. image processing method according to claim 18, it is characterised in that the generation is used for the figure of two dimension display Picture, includes the position of 0 grade of frequency spectrum by described 1 grade of frequency spectrum frequency displacement of ﹢ into the frequency spectrum of the digital hologram.

20. image processing method according to claim 17, it is characterised in that the generation is used for the figure of two dimension display Picture, including：

Fourier transformation is carried out to the digital hologram, the frequency spectrum of the digital hologram is generated；

0 grade of frequency spectrum and -1 grade of frequency spectrum are filtered out in the frequency spectrum of the digital hologram, and retains 1 grade of frequency spectrum of ﹢；

By the position of the 0 grade frequency spectrum of the described 1 grade of frequency spectrum frequency displacement of ﹢ into the frequency spectrum of the digital hologram；And

Inverse Fourier transform is carried out to the frequency spectrum of the digital hologram after the frequency displacement, the figure for two dimension display is generated Picture.

21. a kind of image processing apparatus, it is characterised in that including：

Hologram acquisition module, for obtaining digital hologram；

Determining module, for according to predetermined condition, it is determined whether carry out Three-dimensional Display；

3-D view generation module, for when it is determined that carrying out Three-dimensional Display, being generated based on the digital hologram for three-dimensional The image of display；And

Two dimensional image generation module, for when it is determined that during without Three-dimensional Display, being generated based on the digital hologram for two Tie up the image of display.

22. image processing apparatus according to claim 21, it is characterised in that

23. a kind of image processing method, it is characterised in that including：

Data flow is obtained, the data flow includes digital hologram sequence；

When it is determined that carrying out Three-dimensional Display, generated based on the digital hologram in the digital hologram graphic sequence for Three-dimensional Display Image；Or

When it is determined that during without Three-dimensional Display, being generated based on the digital hologram in the digital hologram graphic sequence aobvious for two dimension The image shown.

24. image processing method according to claim 23, it is characterised in that methods described also includes receiving coming from user User indicate or from be currently running application receive application indicate, and

25. image processing method according to claim 24, it is characterised in that methods described is also included to carrying out at image Reason and/or carry out the relevant parameter of device that image shows and detected, and it is described according to predetermined condition, it is determined whether progress Three-dimensional Display, in addition to：

26. image processing method according to claim 23, it is characterised in that methods described is also included to carrying out at image Reason and/or carry out the relevant parameter of device that image shows and detected, and

27. image processing method according to claim 26, it is characterised in that methods described also includes receiving coming from user User indicate or receive application from the application being currently running to indicate, it is and described according to predetermined condition, it is determined whether to carry out Three-dimensional Display, including：

28. image processing method according to claim 26, it is characterised in that methods described also includes receiving coming from user User indicate or from be currently running application receive application indicate, and

29. image processing method according to claim 23, it is characterised in that methods described also includes receiving coming from user User indicate and from be currently running application receive application indicate, and

30. image processing method according to claim 23, it is characterised in that

The acquisition digital hologram includes：

Obtain the three-dimensional information of virtual objects；And

Described generate includes for the image of Three-dimensional Display：

31. image processing method according to claim 30, it is characterised in that the three-dimensional letter of the acquisition virtual objects Breath, including：

Obtain the preset recording distance of the virtual objects；And

32. image processing method according to claim 31, it is characterised in that the first reference light information includes and institute State left view point and the corresponding left view point of right viewpoint reproduces to reproduce with reference to optical information and right viewpoint and refers to optical information, wherein, the left side Viewpoint, which reproduces reference light information, to be included：The light that simulation is sent from left reproduction reference light source corresponding with left view point is irradiated to described Incident angle during digital hologram, the right viewpoint, which reproduces reference light information, to be included：Simulate from the right side corresponding with right viewpoint again The light sent referring now to light source is irradiated to the incident angle during digital hologram, and

33. image processing method according to claim 31, it is characterised in that the preset recording distance at least two It is individual, and

34. image processing method according to claim 23, it is characterised in that

The acquisition digital hologram includes：

Described generate includes for the image of Three-dimensional Display：

35. image processing method according to claim 34, it is characterised in that obtain three of real object in true environment Dimension coordinate information includes：

36. method according to claim 35, it is characterised in that frequency division when being carried out to the hologram of the real object Analysis, obtaining the wavefront function of the real object includes：

37. method according to claim 36, it is characterised in that obtaining the three-dimensional coordinate information of virtual objects includes：

Obtain the preset recording distance of the virtual objects；And

38. the method according to claim 37, it is characterised in that according to the three-dimensional coordinate information of the real object and institute Stating the three-dimensional coordinate information generation digital hologram of virtual objects includes：

39. image processing method according to claim 23, it is characterised in that

40. the image processing method according to claim 39, it is characterised in that the generation is used for the figure of two dimension display Picture, is realized by retaining 1 grade of frequency spectrum of ﹢ in the frequency spectrum of the digital hologram and filtering out 1 grade of frequency spectrum of 0 grade of frequency spectrum and ﹣ 's.

41. image processing method according to claim 40, it is characterised in that the generation is used for the figure of two dimension display Picture, includes the position of 0 grade of frequency spectrum by described 1 grade of frequency spectrum frequency displacement of ﹢ into the frequency spectrum of the digital hologram.

42. the image processing method according to claim 39, it is characterised in that the generation is used for the figure of two dimension display Picture, including：

43. a kind of image processing apparatus, it is characterised in that including：

Data flow acquisition module, for obtaining data flow, the data flow includes digital hologram sequence；

3-D view generation module, for when it is determined that carrying out Three-dimensional Display, based on the numeral in the digital hologram graphic sequence Hologram generates the image for Three-dimensional Display；And

Two dimensional image generation module, for when it is determined that during without Three-dimensional Display, based on the number in the digital hologram graphic sequence Word hologram generates the image for two dimension display.

44. image processing apparatus according to claim 43, it is characterised in that