CN104700382A - Multi-focus image file handling method - Google Patents

Abstract

The invention relates to a multi-focus image generating device and a multi-focus image file handling method. The device is characterized in that an image file is composed of at least two images with different distance; the image distance or the image spacing is recorded in the image file. The device comprises a group of camera lenses and at least two groups of single-focus imaging assemblies; each single-focus imaging assembly comprises a beam splitter mirror and an imaging unit, wherein the light splitter mirror is used for distributing some light of the camera lens to the imaging unit to image. The device further comprises an imaging lens, a variable focus device, an imaging device, a group of imaging lenses, and at least two groups of liner array light sensing units which are arranged at the front and back under different image distance planes; the liner array light sensing unit is rotatably scanned to form the image. The device has the beneficial effects that all focus surface images of different objects in a long or short distance can be shot and a multi-focus image file format can be generated, so that the image file can be handled in later period.

Description

A kind of multiple focussing image document handling method

Patented claim of the present invention is divisional application.The application number of original bill is: 2012105432727; The applying date is: 20121216; Denomination of invention is: " a kind of multiple focussing image file layout and generating apparatus and image file processing method ".

Technical field

The present invention relates to a kind of multiple focussing image file layout and generating apparatus and image file processing method.

Background technology

Image file format is the form of record and stored images information, digital picture is stored, process, propagate, certain picture format must be adopted, namely the pixel of image is carried out organizing and storing according to certain mode, view data is stored as file and just obtains image file.Image file format determines the information depositing which kind of type hereof, and file is how compatible with various application software, file how with other exchange files data.

Current picture form has: BMP (Bitmap) form, TIFF (TagImage File Format) form (tiff format can add author, copyright, remarks and self-defined information), GIF (Graphic Interehange Format) form (a kind of LZw compressed format), jpeg format (Joint Photographic Experts Group, joint photographic experts group), PDF (Portable Document Format) form, PNG form etc.

Current video image format has: Microsoft's video: wmv, asf, asx; Real Player: rm, rmvb; MPEG video: mpg, mpeg, mpe; Mobile video: 3gp; Apple video: mov; Sony video: mp4, m4v; Other common video: avi, dat, mkv, flv, vob etc.Video file is a kind of container encapsulation format, and namely a receptacle is wrapped in different tracks, and the form of the container of use is related to the extensibility of video shelves.

Above-mentioned picture and video image file are the set of two dimensional image pixel, i.e. all image pixel set of a certain focussing plane, due to object far and near distance difference (object distance is different), so image distance is also different, part (focus point) is clearly had by the plane of delineation after optical focus, have fuzzy part (non-focusing point), the imaging and focusing of all objects different for distance point can not be recorded by two dimensional image recording mode.

Current 3-D view recording mode has STL, DXF, IGS, PLY, obj, the graphic file of the forms such as 3ds is as data exchange interface, interactive stereo display is carried out to figure, if IGS is that a kind of general three-dimensional drawing exchanges form, for solidedge, the digital control processings such as Pro/E, the pixel of this kind of image file format is three-dimensional coordinate form, and each pixel is accurate object dimensional coordinate, namely with three-dimensional coordinate, object is described, the picture format that the present invention proposes describes all image pixels of optical focus plane with three-dimensional coordinate, these image slices have focus point clearly, also fuzzy non-focusing point is had, have any different with existing three-dimensional modeling picture format.

Summary of the invention

Image of the present invention comprises picture and video image.

A kind of multiple focussing image file layout of the present invention and document No. method or file coding format.

The general include file header structure (FILEHEADER) of picture file, information header structure (INFOHEADER), bitmap color table (RGBQUAD), bitmap pixels data, the wherein attributive character of file header structure, information header structure definition file and item of information, bitmap color table, bitmap pixels data are logical places of store images.

Video image file is made up of file header, index block and data block, wherein data block comprises actual data stream, i.e. image and sound sequence data, index block comprises data block list and their positions hereof, to provide data random access ability in file, file header comprises the general information of file, definition data layout, the parameters such as compression algorithm used, by the two field picture of data stream generating video image file.

A kind of multiple focussing image file layout, is characterized in that: image file is by the image construction of at least two different image distances.

A kind of multiple focussing image file layout, is characterized in that: image file is by the image construction of at least two different image distances, and described image slices distance value record is in described image file.

A kind of multiple focussing image file layout, is characterized in that: image file by the image construction of at least two different image distances, being recorded in described image file as distance values between described image.

A kind of multiple focussing image file layout, is characterized in that: image file by the image construction of at least two different image distances, described image slices apart from or be recorded in described image file as distance values.

Described a kind of multiple focussing image file layout, is characterized in that: comprise focal length numerical value in image file.

Described a kind of multiple focussing image file layout, is characterized in that: comprise definition values summary or index in image file.

Described a kind of multiple focussing image file layout, is characterized in that: the index comprising subregion coordinate and image distance composition in image file; Or the index of subregion coordinate and image distance label composition.

A kind of multiple focussing image file layout, it is characterized in that: image file is by the image construction of at least two different image distances, the image pixel point coordinate of described image is by three-dimensional coordinate description and x, y, z, wherein the pixel of x, y Description Image is in the position of image distance focusing surface, and z describes the distance of normal direction apart from reference plane of the plane that pixel determines at x, y.

A kind of multiple focussing image video file coded format, packing forms comprises video track, audio track, it is characterized in that: comprise multiple video track in packing forms, and each video track has its image distance label or picture spacing label.

Described multiple focussing image video file coded format, is characterized in that: also comprise image clarity values summary or index in packing forms.

A kind of multiple focussing image generating apparatus, it is characterized in that: device is made up of one group of imaging len, at least two group monofocal image-forming assemblies, described monofocal image-forming assembly comprises spectroscope, image-generating unit, and the some light of imaging len is distributed to described image-generating unit imaging by spectroscope.

A kind of multiple focussing image generating apparatus, it is characterized in that: the transparent image unit that device is superposed in tandem by one group of imaging len, at least two groups is formed, described transparent image unit is responsible for the image imaging of different image distance.

A kind of multiple focussing image generating apparatus, it is characterized in that: device is made up of imaging len, zoom lens control device, image-generating unit, imaging zoom is responsible for by described zoom lens control device, zoom lens control device is responsible for providing corresponding image distance numerical value simultaneously, described zoom lens control device is in to-and-fro movement state when making a video recording, and one group of image of a reciprocation period acquisition is a multiple focussing image file.

Described a kind of multiple focussing image generating apparatus, is characterized in that: when making a video recording, the motion of zoom lens control device is at the uniform velocity.

Described a kind of multiple focussing image generating apparatus, is characterized in that: when making a video recording, the motion of zoom lens control device is the sub-mode of motion of shake and sinus wave patters.

Described a kind of multiple focussing image generating apparatus, is characterized in that: when making a video recording, the motion of zoom lens control device is step-by-step system.

A kind of multiple focussing image generating apparatus, is characterized in that: device is arranged by not shading before and after one group of imaging len, at least two groups and do not form at the linear array photosensitive unit of same image distance plane, and described linear array photosensitive unit is by moving sweep synthetic image.

Described a kind of multiple focussing image generating apparatus, is characterized in that: the moving sweep mode of linear array photosensitive unit is single-arm rotating scanning or both arms rotation sweep or cylindric rotation sweep or parallel sweep.

Described a kind of multiple focussing image generating apparatus, is characterized in that: arrange column distortion correcting lens in the light path of linear array photosensitive unit and imaging len.

Described a kind of multiple focussing image generating apparatus, is characterized in that: the distance between linear array photosensitive unit and imaging len is adjustable.

Described image-generating unit or linear array photosensitive unit are CCD or COMS elements.

Described a kind of multiple focussing image generating apparatus is applied to camera, video camera, microscope.

A kind of multiple focussing image document handling method, the steps include: A, obtains multiple focussing image file; B, determine prime focus face image; C, extraction adjacent focus face image; D, adjacent focus face image and prime focus face image carried out resolution chart and form fused images 1 as fusion treatment; E, extract another adjacent focus face image; F, another adjacent focus face image and fused images 1 carried out resolution chart and form fused images 2 as fusion treatment; G, extraction time adjacent focus face image; H, secondary adjacent focus face image and fused images 2 carried out resolution chart and form fused images 3 as fusion treatment; I, extract another adjacent focus face image; J, another adjacent focus face image and fused images 3 carried out resolution chart and form fused images 4 as fusion treatment; K, repeat above-mentioned image co-registration process until form final fused images.

Described a kind of multiple focussing image document handling method, it is characterized in that step D ~ K also comprises following sub-step: a, two width images have the step of adjustment picture size before carrying out fusion treatment, fit like a glove with another width picture size after namely adjusting the size of wherein piece image; When b, two width images carry out image co-registration process, the pixel of two width images selects one to choose, and namely the pixel of fused images is that selecting of the pixel of two width images one chooses result.

Or, consider the clear display of prime focus face image, it is characterized in that step D ~ K also comprises following sub-step: when prime focus face image and adjacent focus face image co-registration, select the fused images 1 of image pixel formation clearly in a selection prime focus face image; The fused images 2 of image pixel formation clearly in a selection image 1 is selected when image 1 and another adjacent focus face image co-registration; Successively by merge after image and other focusing surface image merge time select one selection merge after image in clearly image pixel form final fused images.

Described a kind of multiple focussing image document handling method, it is characterized in that step D ~ K also comprises following sub-step: a, two width images have the step of adjustment image slices vegetarian refreshments x, y coordinate before carrying out fusion treatment, image slices vegetarian refreshments x, y coordinate namely adjusting image slices vegetarian refreshments x, y coordinate of wherein piece image and another piece image is completely the same; When b, two width images carry out image co-registration process, the pixel of two width images selects one to choose, and namely the pixel of fused images is that selecting of the pixel of two width images one chooses result.

A kind of multiple focussing image document handling method, it is characterized in that: be that the image extracting a certain image distance carries out current display when carrying out image display, the triggering mode of display and control is at least as one of under type: control knob triggering mode, control knob mouse drag triggering mode, mouse coordinates triggering mode, mouse coordinates and button triggering mode, touch screen triggering mode, touch screen front contact coordinate and touch screen triggering mode, eyeball focus triggering mode.

The step of the image of a certain image distance of described extraction also comprises: the step judged image definition; Or retrieval and inquisition definition values summary or index determine the step that definition values is the highest, or determine the step of image distance in the index that is made up of subregion coordinate and image distance; Or determine the step of image distance label in the index that is made up of subregion coordinate and image distance label.

A kind of multiple focussing image document handling method, be the steps include: S1, obtains eyeball image by camera, judged the focus of eyeball, and determine focus coordinate by recognition and tracking software; S2, to extract in multiple focussing image file according to the coordinate of focus focus region clearly image carry out current display.

Further, also comprise in step S2 and in multiple focussing image sharpness index or summary, search focus region correspondence image clearly according to eyeball focus coordinate, the i.e. image of a certain image distance value, and the image of described correspondence is shown as present image at display interface.

Current image fusion technology has: Chinese patent bulletin 200910104632.1 discloses a kind of multi-focus image fusing method utilizing Fisher classification and wavelet transformation; Chinese patent announces 200910147247.5 image processing equipments, imaging device, image processing method and program; Chinese patent announces 201010527509.3 1 kinds of multi-focus image fusing methods based on wavelet transformation and domain features; Chinese patent announces 200710064739.9 1 kinds of image combining methods and device; Chinese patent bulletin 200810017326.X is based on imaging mechanism and nonsubsampled contourlet transform multi-focus image fusing method.

The invention has the beneficial effects as follows: all focusing surface images that far and near different objects can be taken by multiple focussing image generating apparatus, and generate a kind of multiple focussing image file layout, can process image file in the later stage like this, such as take out the focusedimage of jobbie, or fusion treatment is carried out to image, generate the picture rich in detail that a kind of far and near object all focuses on, or the relative distance of the Distance geometry object according to the focal length of camera head, the image distance of object, the picture distance computation object of focusing surface, or the stereo-picture generating object.Adopt the embodiment of spectroscope and linear array multi-focus imaging heap can solve a difficult problem for shooting activity multiple focussing image.Browse multiple focussing image based on eye tracking device, can realize the simulation of human eye focus to real world, namely multi-focus display device realizes zoom display, forms a kind of brand-new image-watching mode, comprises picture and video.

Accompanying drawing explanation

The graph of a relation that when Fig. 1 is lens imaging focusing, image distance changes with object distance.

Fig. 2 is that multiple focussing image generating apparatus of the present invention adopts spectroscopical embodiment.

Fig. 3 is the embodiment that multiple focussing image generating apparatus of the present invention adopts transparent image unit.

Fig. 4 is image pixel coordinates describing mode in transparent image unit in Fig. 3.

Fig. 5 is that multiple focussing image generating apparatus of the present invention adopts the reciprocating embodiment of image-generating unit.

Fig. 6 is that multiple focussing image generating apparatus of the present invention adopts linear array photosensitive unit to be arranged on wedge inclined-plane and follows the embodiment of inclined-plane at x-axis direction to-and-fro movement scanning imagery.

Fig. 7 is the embodiment that multiple focussing image generating apparatus of the present invention adopts both arms linear array photosensitive unit rotating scan imaging.

Fig. 8 is the view in both arms linear array photosensitive unit z-axis direction in Fig. 7.

Fig. 9 is both arms linear array photosensitive unit z-axis direction isometric plan in Fig. 7.

Figure 10 is the embodiment that multiple focussing image generating apparatus of the present invention adopts single armed linear array photosensitive unit rotating scan imaging.

Figure 11 is the view in single armed linear array photosensitive unit z-axis direction in Figure 10.

Figure 12 is single armed linear array photosensitive unit z-axis direction isometric plan in Figure 10.

Figure 13 is that multiple focussing image generating apparatus of the present invention adopts linear array photosensitive unit to be arranged in the embodiment of rotating scan imaging on the different column structure of diameter.

Figure 14 is image file format attribute list of the present invention.

Figure 15 is file format structure figure of the present invention.

Figure 16 is step scanning signal voltage u, time t, z-axis displacement relation figure.

Figure 17 is the block diagram of image-generating unit of the present invention synthesis multiple focussing image file.

Figure 18 is the block diagram that linear array photosensitive unit of the present invention generates multiple focussing image file.

Figure 19 is a kind of method of image co-registration of the present invention.

Figure 20 is a kind of image viewer provided by the invention or video player surface chart.

Figure 21 is the step of multiple focussing image file generated 3-D view of the present invention.

Figure 22 is linear array motion encoder Image Coding principle.

Figure 23 is space three-dimensional image image-forming principle.

Figure 24 is the multiple focussing image viewing mode schematic diagram based on human eye focus.

Figure 25 is the schematic diagram of multiple focussing image file set up sharpness summary or index.

Figure 26 adopts image distance label to be a kind of clear picture region summary or concordance list that identify.

Figure 27 is the step that Figure 26 sets up image distance label k value in clear picture region summary or concordance list.

Embodiment

Below in conjunction with drawings and Examples, the present invention is further described.

The graph of a relation that when Fig. 1 is lens imaging focusing, image distance changes with object distance.Lens imaging formula (also known as Gaussian imaging equation) form is: 1/f=1/u+1/v

Wherein f is focal length, convex just recessed negative; U is object distance; V is image distance, real weakened body resistance negative.

Fortran is: v=f*u/ (u-f), and the areas imaging of real image is image distance f ~ 2f, and corresponding object distance is ∞ ~ 2f

V to u differentiate into: v'=-f^2* (u-f) ^ (-2), can draw: the derivative absolute value the closer to 2f is larger, and namely slope is larger, and namely the rate of change of image distance is larger; Derivative absolute value the closer to f is less, and namely slope is less, and namely the rate of change of image distance is less.

Illustrate, obtain the picture rich in detail of different object distances object, the image-generating unit near 2f wants intensive layout, and the image-generating unit near f can sparsely be arranged; Or the intensive layout in crucial image distance position; Or image-generating unit is arranged in a certain section of formation multi-focus imaging heap between f ~ 2f, the distance between image-generating unit is adjustable, and the distance between multi-focus imaging heap and imaging len is adjustable.

Illustrate simultaneously and also illustrate, as required, as long as arrange that the image-generating unit of limited quantity just can obtain the picture rich in detail of different object distances object.

If arrange image-generating unit in f ~ 2f universe, form total focus image.

If employing full shot, as fish eye lens, panorama total focus image can be formed.

For imaging lens group, as zoom system, pancreatic system, areas imaging f ~ 2f of the present invention is the imaging region of zoom system, pancreatic system.

Optical imaging concept of the present invention is positioned at ∞ ~ 2f with object, the simple lens that areas imaging is positioned at f ~ 2f is example explanation, 2f ~ f is positioned at for object, areas imaging is then 2f ~ ∞, as the screen of microscope, motion picture projection, so areas imaging f ~ 2f of the present invention is no longer applicable, can state with imaging region, so areas imaging f ~ 2f conceptual understanding of the present invention is imaging region, multi-focus imaging heap is positioned at imaging region.

Described multi-focus imaging heap is applied to camera, video camera, microscope.

In fact, the object of camera, shot by camera is generally in the position much larger than 2f, so imaging region is generally positioned at the position close to f, microscopical imaging is generally in the position much larger than object lens 2f, so be the intensive layout image-generating unit in crucial image distance position in practical application, preferably embodiment adopts the multi-focus imaging heap of certain image distance direction thickness as image-generating unit, is applied to required imaging region.

Fig. 2 is that multiple focussing image generating apparatus of the present invention adopts spectroscopical embodiment.201,202,203,204,205 is objects of different distance, 206 is imaging len, 207,210,211,214,215 for being arranged in the image-generating unit of different image distance, 208,209,212,213 is spectroscope, spectroscopical effect is transmission part light, antireflection part light, can be divided into two projecting directions light like this.Here illustrate for the object choosing 5 different distance, by regulating the image distance of image-generating unit, image-generating unit 207 is made to be positioned at the focusing surface of object 201, image-generating unit 210 is positioned at the focusing surface of object 202, image-generating unit 211 is positioned at the focusing surface of object 203, image-generating unit 214 is positioned at the focusing surface of object 204, image-generating unit 215 is positioned at the focusing surface of object 205, spectroscope 208 is responsible for a part of light to distribute to image-generating unit 207, spectroscope 209 is responsible for a part of light to distribute to image-generating unit 210, spectroscope 212 is responsible for a part of light to distribute to image-generating unit 211, spectroscope 213 is responsible for a part of light to distribute to image-generating unit 214, each like this image-generating unit all can obtain the picture rich in detail of respective focusing surface object.

Fig. 3 is the embodiment that multiple focussing image generating apparatus of the present invention adopts transparent image unit.301, 302, 303 is objects of different distance, 304 is imaging lens, 305, 306, 307 for arranging the transparent image unit be superimposed, the transparent image unit composition three-dimensional imaging unit that at least 2 arrangements are superimposed, for the three-dimensional imaging unit of 3 transparent image unit compositions in figure, by regulating the image distance of three-dimensional imaging unit, transparent image unit 305 is positioned at the focusing surface of object 301, transparent image unit 306 is positioned at the focusing surface of object 302, transparent image unit 307 is positioned at the focusing surface of object 303, the picture rich in detail of different distance object is formed like this in three-dimensional imaging unit.

Further, the intensive layout of described transparent image unit, distance such as between transparent image unit equals the distance of transparent image unit pixel, so just form three-dimensional imaging heap or three-dimensional imaging unit, in three-dimensional imaging heap or three-dimensional imaging unit, arbitrary pixel can pass through x, y, z coordinate describes to be determined, pixel that so just can be all in writing lens imaging space, the image that can intercept arbitrary image distance plane when image repetition carries out plane display, and the image appearance form of described intercepting arbitrary image distance plane is the image of a certain image distance.In addition, also all pixels can be carried out space diagram display.

Fig. 4 is image pixel coordinates describing mode in transparent image unit in Fig. 3.P1 is a certain pixel in transparent image unit 305, the coordinate of pixel is (x1, y1, z1), wherein x1, y1 is the two-dimensional coordinate of image slices vegetarian refreshments at the plane of delineation, described two-dimensional coordinate is the expression packaged type of image pixel positions in conventional images file layout, z1 is the location presentation of pixel in z-axis direction, can be spend distance absolutely, as the distance of p1 pixel Range Imaging lens center, or the distance of distance focal point, also can be relative distance, as the relative distance between transparent image unit.In like manner p2 is a certain pixel in transparent image unit 306, and the coordinate of pixel is (x2, y2, z2), and p3 is a certain pixel in transparent image unit 307, and the coordinate of pixel is (x3, y3, z3).

It can be equidistant or Unequal distance that transparent image unit arranges the distance be superimposed, as intensive in the location arrangements near 2f, sparse near f location arrangements; Or the intensive layout in crucial image distance position; Or transparent image cell layout a certain section of formation multi-focus imaging heap between f ~ 2f, the distance between multi-focus imaging heap and imaging len is adjustable.

Fig. 5 is that multiple focussing image generating apparatus of the present invention adopts the reciprocating embodiment of image-generating unit.401 is objects, and 402 is imaging lens, and 403 is telefocus positions, i.e. the focal position of farthest object, and 404 is near focus positions, i.e. the focal position of most near objects, and 405 is kinematic driving unit, servomechanism installation of namely making a video recording.Device is in shooting process, and drive image-generating unit to-and-fro movement along the z-axis direction by kinematic driving unit, a period of motion forms a complete image form bag, and described picture format bag at least comprises 2 focusing surface picture frames.

The mode of motion of kinematic driving unit has: 1, rectilinear motion mode, and namely image-generating unit is in z-axis direction, at the uniform velocity to move back and forth between f ~ 2f; 2, movement of pendulum mode, namely image-generating unit is in z-axis direction, move back and forth with sinusoidal wave vibration mode between f ~ 2f, amplitude is focal distance f, the vibrations cycle is T, then image distance v can be expressed as: v=0.5*f*sin (2 ∏ t/T), and v to time differentiate is: v'=f* (∏/T) * cos (2 ∏ t/T), parameter v and v' is used for picture format coding, and ∏ is circular constant; 3, step motion mode, namely there is moment static mode of motion midway, and namely image-generating unit is in z-axis direction, move back and forth between f ~ 2f in process have at least 2 rest time section, rest time Duan Weiyi picture frame period.

The mode of motion of kinematic driving unit is controlled by moving sweep signal, and moving sweep signal is correspondingly divided into according to mode of motion: 1, linear sweep signal, namely can be driven into and move back and forth with even speed along the z-axis direction as unit; 2, sinusoidal scanning signal, namely can be driven into and move back and forth with sinusoidal wave vibration mode along the z-axis direction as unit; 3, step scanning signal, namely can be driven into and move back and forth along the z-axis direction as unit, have at least in motion process 2 rest time section, rest time Duan Weiyi picture frame period FT, Figure 16 is step scanning signal voltage u, time t, z-axis displacement relation figure, z1, z2, z3 are respectively 3 rest points of image-generating unit in z-axis, rest time, section was FT, an i.e. picture frame period, 3 rest point corresponding voltage value u1, u2, u3 respectively, namely magnitude of voltage u1, u2, u3 remains unchanged in FT section rest time.U-z figure can find out that image-generating unit moves with scanning voltage value, and the displacement namely occurred is controlled by magnitude of voltage.

According to the persistence of vision of human eye, generally minimum is 24Hz, if the sweep frequency choosing kinematic driving unit is 24Hz, i.e. about 0.04 second cycle, image-generating unit picture frame period must be less than 0.04 second, as chosen 1/500 second, then can hold the image of the different image distance of 20 frames in the period of motion, so just can form multi-focus live image.

For movement of pendulum mode, consider the natural frequency of vibrations object, by regulating the additional mass of image-generating unit and the elasticity of stationary installation to make natural frequency consistent with driving sweep signal frequency during enforcement.

Described mode of motion and moving sweep mode, namely image-generating unit presses mode of motion shuttle-scanning motion back and forth in lens imaging region.

For the camera apparatus of shooting one group of image, image-generating unit in shooting shooting process can be adopted only to complete a moving sweep cycle, the coordinate (x, y, z) of all image slices vegetarian refreshments of shooting process record, form multiple focussing image file.

Fig. 6 is that multiple focussing image generating apparatus of the present invention adopts linear array photosensitive unit to be arranged on wedge inclined-plane and follows the embodiment of inclined-plane at x-axis direction to-and-fro movement scanning imagery.601, 602, 603 is the object of different distance, 604 is imaging len, 605 is wedge inclined-plane, wedge inclined-plane 605 is furnished with linear array photosensitive unit, 606 is one of linear array photosensitive unit, linear array photosensitive unit is made up of a photoelectric chip, this linear array photosensitive unit scanning imagery that current scanline instrument adopts, the vertical projection distance of two adjacent linear array photosensitive units in z-axis direction is h, h value can as image distance value parameter, linear array photosensitive unit is arranged between z-axis projector distance f ~ 2f, can be that equidistant placement or Unequal distance are arranged, as intensive in the location arrangements near 2f, sparse near f location arrangements.Device is when making a video recording, and wedge inclined-plane 605 moves back and forth at x projecting direction, and the linear array photosensitive unit of so different image distance, by x scanning direction, obtains the focusing surface image of different image distance.Wedge inclined-plane 605 is at x projecting direction in the mode at the uniform velocity moved back and forth, or movement of pendulum mode, namely wedge inclined-plane 605 moves back and forth with sinusoidal wave vibration mode at x projecting direction.

Described mode of motion and moving sweep mode, namely linear array photosensitive unit presses mode of motion shuttle-scanning motion back and forth in lens imaging region.

The scanning motion of described linear array photosensitive unit is parallel sweep, and namely linear array photosensitive unit is parallel to each other at the to-and-fro movement track in x-axis direction.

The structure of linear array photosensitive unit is by the arrangement of sensitivity speck photoelectric chip in a straight line, photoelectric chip is generally CCD or COMS, simultaneously, the three primary colours that each sensitivity speck is also responsible for pixel are separated, and prior art has: color filter isolation technics, three CCD look isolation technics, single CCD look are separated scanning technique.The pixel signal that the sensitivity speck of linear array photosensitive unit obtains is simulating signal, so also have process simulating signal being converted into digital signal, i.e. and A/D conversion.Linear array photosensitive unit is a kind of one dimensional image sensor, and existing one dimensional image sensor is used for traditional scanner.

Fig. 7 is the embodiment that multiple focussing image generating apparatus of the present invention adopts both arms linear array photosensitive unit rotating scan imaging.701,702,703 is the object of different distance, and 704 is imaging len, and 705,706,707,708,709,710 is both arms linear array photosensitive unit, and 711 is rotating shaft, and the central point of both arms linear array photosensitive unit is connected in rotating shaft 711.During device shooting, both arms linear array photosensitive unit 705,706,707,708,709,710, with the focusing surface of the different image distance of rotating shaft 711 rotary scanning, generates the image of different image distance focusing surface.

The mode of motion of the present embodiment is that both arms linear array photosensitive unit rotates the moving sweep mode forming focusing surface in lens imaging region.

Fig. 8 is the view in both arms linear array photosensitive unit z-axis direction in Fig. 7.Both arms linear array photosensitive unit can be evenly arranged also can unevenly be arranged.Or the intensive layout in crucial image distance position.Or both arms linear array photosensitive unit is arranged in a certain section of f ~ 2f, the distance between both arms linear array photosensitive unit is adjustable, and the distance between both arms linear array photosensitive unit and imaging len is adjustable, and namely both arms linear array photosensitive unit can move around in z-axis direction and regulates.

Fig. 9 is both arms linear array photosensitive unit z-axis direction isometric plan in Fig. 7.Both arms linear array photosensitive unit is arranged between f ~ 2f, can be that equidistant placement or Unequal distance are arranged, as intensive in the location arrangements near 2f, sparse near f location arrangements.Or the intensive layout in crucial image distance position.Or both arms linear array photosensitive unit is arranged in a certain section of f ~ 2f formation linear array multi-focus imaging heap, distance between both arms linear array photosensitive unit is adjustable, distance between both arms linear array photosensitive unit and imaging len is adjustable, and namely the imaging of linear array multi-focus is piled to move around in z-axis direction and regulated.

Figure 10 is the embodiment that multiple focussing image generating apparatus of the present invention adopts single armed linear array photosensitive unit rotating scan imaging.1001,1002,1003 is the object of different distance, and 1004 is imaging len, and 1005,1006,1007,1008,1009,1010 is single armed linear array photosensitive unit, and 1011 is rotating shaft, and one end of single armed linear array photosensitive unit is connected in rotating shaft 1011.During device shooting, single armed linear array photosensitive unit 1005,1006,1007,1008,1009,1010, with the focusing surface of the different image distance of rotating shaft 1011 rotary scanning, generates the image of different image distance focusing surface.

The mode of motion of the present embodiment is that single armed linear array photosensitive unit rotates the moving sweep mode forming focusing surface in lens imaging region.

Figure 11 is the view in single armed linear array photosensitive unit z-axis direction in Figure 10.Single armed linear array photosensitive unit can be evenly arranged also can unevenly be arranged; Or the intensive layout in crucial image distance position; Or single armed linear array photosensitive unit is arranged in a certain section of f ~ 2f formation linear array multi-focus imaging heap, distance between single armed linear array photosensitive unit is adjustable, distance between single armed linear array photosensitive unit and imaging len is adjustable, and namely the imaging of linear array multi-focus is piled to move around in z-axis direction and regulated.

Figure 12 is single armed linear array photosensitive unit z-axis direction isometric plan in Figure 10.Single armed linear array photosensitive unit is arranged between f ~ 2f, can be that equidistant placement or Unequal distance are arranged, as intensive in the location arrangements near 2f, sparse near f location arrangements; Or the intensive layout in crucial image distance position; Or single armed linear array photosensitive unit is arranged in a certain section of f ~ 2f formation linear array multi-focus imaging heap, distance between single armed linear array photosensitive unit is adjustable, distance between single armed linear array photosensitive unit and imaging len is adjustable, and namely the imaging of linear array multi-focus is piled to move around in z-axis direction and regulated.

Figure 13 is that multiple focussing image generating apparatus of the present invention adopts linear array photosensitive unit to be arranged in the embodiment of rotating scan imaging on the different column structure of diameter.1301,1302,1303 is linear array photosensitive unit, 1304 is rotating shaft, 1305 is column distortion correcting lens, linear array photosensitive unit be arranged in different radii circumferentially and front and back not shading, the radial distance arranged is between f ~ 2f, is generated the image of different image distance focusing surface by linear array photosensitive unit in the circular-rotation scanning of different radii.

The mode of motion of the present embodiment is that linear array photosensitive unit rotates the moving sweep mode forming focusing cylindrical in lens imaging region column.

Linear array photosensitive unit can be radial equidistantly layout or radial unequal-interval layout in the mode of radial arrangement, as intensive in the location arrangements near 2f, sparse near f location arrangements.Or linear array photosensitive unit is in the intensive layout in crucial image distance position.Or linear array photosensitive unit a certain section of being arranged between f ~ 2f forms linear array multi-focus imaging heap, linear array photosensitive unit radial spacing is adjustable, distance between rotating shaft and imaging len is adjustable, namely rotating shaft 1304 can move around in z-axis direction adjustment, and the heap of linear array multi-focus imaging in other words can move around in z-axis direction adjustment.

In fact, the object of camera, shot by camera is generally in the position much larger than 2f, so imaging region is generally positioned at the position close to f, microscopical imaging is generally in the position much larger than object lens 2f, so be the intensive layout image-generating unit in crucial image distance position in practical application, preferably embodiment adopts the linear array multi-focus imaging heap of certain image distance direction thickness as image-generating unit, is applied to required imaging region.

The present embodiment and cylindric scanning embodiment, consider that cylindric scanning can produce histogram image distortion, column distortion correcting lens 1305 can be set in the light path of imaging len to correct histogram image distortion, specifically can be arranged in the light path between imaging len and linear array photosensitive unit.

Figure 14 is image file format attribute list of the present invention.Have in traditional images file format properties table: width, highly, horizontal resolution, vertical resolution, bit depth, frame number, equipment, focal length, color represent, ISO speed, the time, address etc., the image of several different focusing surface is comprised in the image file format of Figure 14 example, formed by different focusing surface package images, except traditional images attribute, increase focusing surface spacing, image distance parameter.

Focusing surface spacing i.e. picture spacing.

A certain cross-sectional image in the image of described different focusing surface and the imaging region space of imaging len.

In Figure 14 example, focusing surface spacing is equidistant, i.e. 2mm, and focusing surface spacing can be Unequal distance, as intensive in the location arrangements near 2f, sparse near f location arrangements.Or the intensive layout in crucial image distance position.

In addition, focusing surface spacing and image distance optional in multiple focussing image file layout of the present invention, the image of different image distance can arrange and be bundled in multiple focussing image file, only distinguishes the image of different image distance by arrangement sequence number.

In addition, image for image distance different in multiple focussing image also can utilize existing software to carry out packing process, form a file bag be made up of different image distance image, described different image distance image carries out sequencing numbers according to image distance value size, and described numbering can as the filename of different image distance image.

Figure 15 is file format structure figure of the present invention.File is made up of file header, information header, image distance value, bitmap color table, bitmap pixels data, encapsulate the image of several different focusing surface in file, be respectively 1 ~ n, image distance value can be absolute image distance value, also can be the spacing in adjacent focus face, in information header, can also focal length value be comprised.

In addition, image distance value and picture spacing optional in multiple focussing image file layout of the present invention, the image of different image distance can arrange and be bundled in multiple focussing image file, only distinguishes the image of different image distance by arrangement sequence number.

Further, image clarity values summary or index in multiple focussing image is increased in described information header.

Or in described information header, comprise the index of subregion coordinate and image distance composition; Or the index of subregion coordinate and image distance label composition.

In addition, image for image distance different in multiple focussing image also can utilize existing software to carry out packing process, form a file bag be made up of different image distance image, described different image distance image carries out sequencing numbers according to image distance value size, the file header of each self-corresponding image file of described numbering write or information header.

Figure 17 is the block diagram of image-generating unit of the present invention synthesis multiple focussing image file.The image file that image-generating unit obtains carries out coded treatment by image distance scrambler, and generate multiple focussing image file, the coding parameter of image distance scrambler is from image distance data, and image distance data can be absolute image distances, or relative image distance is as the relative distance between image-generating unit.In addition, image distance numerical value is optional in multiple focussing image file layout of the present invention, and the image of different image distance can sort and be bundled in multiple focussing image file, only distinguishes the image of different image distance by arrangement sequence number.Certainly, do not get rid of with existing software to the image of different image distance carry out sequence packing.

Spectroscopical embodiment and Fig. 3 multiple focussing image generating apparatus of the present invention is adopted to adopt the embodiment of transparent image unit for Fig. 2 multiple focussing image of the present invention generating apparatus.The image-generating unit spacing distance of these two embodiments is predetermined or determines, namely image distance data are determined values, extracts image distance data determined value and forms multiple focussing image file layout by image distance scrambler write package file.

The reciprocating embodiment of image-generating unit is adopted for Fig. 5 multiple focussing image of the present invention generating apparatus.The image distance numerical value of this scheme is determined by kinematic driving unit, the mode of motion of kinematic driving unit is controlled by moving sweep signal, moving sweep signal is correspondingly divided into according to mode of motion: 1, linear sweep signal, namely can be driven into and move back and forth with even speed along the z-axis direction as unit, the position of image distance can be expressed as: v=β * (U/T) * t, and v is instant image distance, β is fixed coefficient, U is scanning voltage peak value, and T is the scan period, and t is a certain time in moment, 2, sinusoidal scanning signal, namely can be driven into and move back and forth with sinusoidal wave vibration mode along the z-axis direction as unit, the position of image distance can be expressed as: v=0.5*f*sin (2 ∏ t/T), v is instant image distance, amplitude is focal distance f, and the vibrations cycle is T, t is a certain time in moment, 3, step scanning signal, namely can be driven into and move back and forth along the z-axis direction as unit, have at least in motion process 2 rest time section, rest time Duan Weiyi picture frame period FT, Figure 16 is step scanning signal voltage u, time t, z-axis displacement relation figure, z1, z2, z3 is respectively 3 rest points of image-generating unit in z-axis, rest time, section was FT, an i.e. picture frame period, 3 rest point corresponding voltage value u1 respectively, u2, u3, i.e. magnitude of voltage u1, u2, u3 remains unchanged in FT section rest time, u-z figure can find out that image-generating unit moves with scanning voltage value, namely the displacement occurred is controlled by magnitude of voltage.Described linear sweep signal or sinusoidal scanning signal or step scanning signal generate image distance value by modulus calculation (A/D conversion), and image distance value or picture distance values are by image distance scrambler write package file formation multiple focussing image file layout.

Multiple focussing image file generated step is: the image file 1, obtaining at least two image-generating units; 2, image distance value or picture distance values is obtained; 3, image distance value or picture distance values are write package file and multiple focussing image file, or multiple focussing image coded format file.

Multi-focus video image file generation step is: the video image file 1, obtaining at least two image-generating units; 2, image distance or the picture spacing of described video image file is obtained; 3, by image distance value or picture distance values write package file, image distance value or be the label of video image file as distance values.

Figure 18 is the block diagram that linear array photosensitive unit of the present invention generates multiple focussing image file.1, linear array photosensitive unit moving sweep generates respective image pixel, image pixel and brightness value and color value; 2, demarcate by linear array motion encoder the coordinate that linear array photosensitive unit scans the image slices vegetarian refreshments generated; 3, by image slices vegetarian refreshments and Coordinate generation linear array two field picture thereof; 4, by image distance scrambler, the distance values of linear array photosensitive unit or image distance value write package file are formed multiple focussing image file layout.In addition, image distance numerical value is optional in multiple focussing image file layout of the present invention, and the image of different image distance can arrange and be bundled in multiple focussing image file, only distinguishes the image of different image distance by arrangement sequence number.Certainly, do not get rid of with existing software to the image of different image distance carry out sequence packing.

The function of linear array motion encoder carries out demarcation sequence to the coordinate of the pixel that linear array photosensitive unit obtains, form view data, namely each point of linear array photosensitive unit is in the pixel value of starting point to the image slices vegetarian refreshments of terminating point scanning acquisition and the data of respective coordinates mapping.

Adopt linear array photosensitive unit to be arranged on wedge inclined-plane for Fig. 6 multiple focussing image generating apparatus of the present invention and follow the embodiment of inclined-plane at x-axis direction to-and-fro movement scanning imagery.Linear array motion encoder is rectilinear scanning motion scrambler, is scanned the image pixel coordinates generated by straight line linear array motion encoder calibration line array element.Such as, a certain pixel p on a certain linear array photosensitive unit, coordinate is expressed as (x, y, z), y is determined by the physical location of p point, z is determined by the position of linear array photosensitive unit in z-axis direction, x can be described as x=A*(t/T), A is the largest motion displacement (have positive and negative values, represent direction of motion) of wedge inclined-plane in x-axis direction, T is reciprocation period, and t is a certain time in moment.

The embodiment of both arms linear array photosensitive unit rotating scan imaging and Figure 10 multiple focussing image generating apparatus of the present invention is adopted to adopt the embodiment of single armed linear array photosensitive unit rotating scan imaging for Fig. 7 multiple focussing image of the present invention generating apparatus.Linear array motion encoder is rotary scanning motion scrambler, scans by rotating linear array motion encoder calibration line array element the image pixel coordinates generated.Such as, a certain pixel p on a certain linear array photosensitive unit, polar coordinates are expressed as (r, ω t, z), and r is the radial distance of p point to spindle central, and ω is angular velocity of rotation, and z is determined by the position of linear array photosensitive unit in z-axis direction, and t is a certain time in moment.

Linear array photosensitive unit is adopted to be arranged in the embodiment of rotating scan imaging on the different cylinder of diameter for Figure 13 multiple focussing image of the present invention generating apparatus.Linear array motion encoder is garden column rotary scanning motion scrambler, is scanned the image pixel coordinates generated by garden column rotary scanning motion scrambler calibration line array element.Such as, a certain pixel p on a certain linear array photosensitive unit, polar coordinates are expressed as (r, ω t, y), and r is the radius of turn of linear array photosensitive unit, and ω is angular velocity of rotation, and y is the y-axis coordinate of p point, and t is a certain time in moment.

Figure 19 is a kind of method of image co-registration of the present invention.Here for the image of 2 focusing surfaces, 2 images are encapsulated in multiple focussing image formatted file, 1901 is an image, in image, A is clear area, b is fuzzy region, 1902 is another image, a is fuzzy region, B is clear area, 1903 is adjustment picture size step, be normalized by image 1901 and image 1902, make the pixel value of two width images consistent, 1904 images are the normalized image of 1902 images, 1904 image pixel values are consistent with 1901 image pixel values, 1905 is image co-registration treatment step, namely select one to the image pixel of 1901 and the image pixel of 1904 to choose, here select one and choose picture rich in detail pixel, finally obtain image 1906, the pixel composition of image 1906 is A and B, i.e. full width image clearly.

Comprise focal length value in multiple focussing image file layout of the present invention, image distance value, the object distance of different focusing surface (or point) object can be calculated by imaging formula, i.e. the distance of different objects.Imaging formula is: 1/f=1/u+1/v, if known f and v just can calculate u=f*v/ (v-f), the steps include: 1, obtains multiple focussing image file; 2, carry out sharpness identification, obtain the image distance of clear area pixel; 3, object distance u=f*v/ (v-f) is calculated according to image distance v and focal distance f; 4, object distance and the distance of each object or object parts in multiple focussing image file is obtained.

Figure 20 is a kind of image viewer provided by the invention or video player surface chart.Image viewer or video player comprise menu bar, picture browsing or video playback window, image distance adjusting knob, normalization button, image co-registration button, three-dimensional button, normalization button can be normalized picture size, to make the picture size of different image distance consistent, image co-registration button can carry out fusion treatment to shown image, obtain image clearly everywhere, three-dimensional button can carry out process to multiple focussing image file and generate stereo-picture, and image distance regulates button that the image of different image distance can be selected to carry out current display.

Because the graphical arrangement of multiple focussing image file by different image distance forms, namely divide image distance layer to form, certain one deck during display, can only be selected to carry out plane display.

Further, multiple focussing image is browsed based on eye tracking device, the simulation of human eye focus to real world can be realized, system is by eye tracking device, computer installation (image processing apparatus) forms, multiple focussing image software for display installed by computer installation, described software provides browses window, browser image distance regulates button to be controlled by the signal of eye tracking device, eye tracking device is responsible for the focus of resolving eyeball, multiple focussing image software for display has and judges focus area image sharpness step, multiple focussing image software for display by choose focus area image clearly image as current display.

As another embodiment, mouse coordinates guides as eyeball focus, the steps include: 1, extracts mouse coordinates; 2, according to the focus coordinate of mouse coordinates determination image display interfaces; 3, using focus region, image is as current display clearly, and namely the image of corresponding image distance is as current display.Mouse coordinates guides as eyeball focus two kinds of triggering modes: mouse coordinates directly triggers, or is triggered by mousebutton according to mouse coordinates, as pressed left button.

An embodiment again, i.e. touch-screen image display and control triggering mode, touch-screen and touch screen, comprise single-point touch screen and multipoint touch screen, and its image Graphics Processing step is: 1, extract touch screen front contact coordinate; 2, according to the focus coordinate of touch screen front contact coordinate determination image display interfaces; 3, using focus region, image is as current display clearly, and namely the image of corresponding image distance is as current display.Certainly, the display and control triggering mode directly touching touch screen is not got rid of.

A kind of multiple focussing image document handling method, it is characterized in that: be that the image extracting a certain image distance carries out current display when carrying out image display, the triggering mode of display and control is at least as one of under type: control knob triggering mode, control knob mouse drag triggering mode, mouse coordinates triggering mode, mouse coordinates and button triggering mode, touch screen triggering mode, touch screen front contact coordinate and touch screen triggering mode, eyeball focus triggering mode.

The step of the image of a certain image distance of described extraction also comprises: the step judged image definition; Or retrieval and inquisition definition values summary or index determine the step that definition values is the highest, or determine the step of image distance in the index that is made up of subregion coordinate and image distance; Or determine the step of image distance label in the index that is made up of subregion coordinate and image distance label.

A kind of method of image Graphics Processing step and multiple focussing image file processing.

Figure 21 is the step of multiple focussing image file generated 3-D view of the present invention.1, multiple focussing image file is obtained; 2, pixel resolution of image judges, namely carries out definition judgment to the image of image distance different in multiple focussing image file; 3, obtain sharpness image pixel and three-dimensional coordinate, pixel coordinate x, y for the image of same image distance are known, and the numerical value of coordinate z is identical, is also known; 4, span 3-D view, space three-dimensional image is the space real image formed between f ~ 2f by imaging len in kind, and space real image becomes geometric center symmetrical with in kind with lens center point, and material object is positioned between 2f ~ ∞; 5, space three-dimensional image normalization process, the image of all different image distances is unified for same size, as: normalization coefficient β, the image distance of image 1 is v1, image 2 image distance is v2, then β=(v2-f)/(v1-f), by unified for the size of image 1 to image 2, image 1 size * β=image 2 size; 5, generate solid threedimensional image, after normalized, space three-dimensional image coordinate is converted into solid threedimensional coordinate, i.e. the image mapped coordinate of true coordinate in kind.

Consider that image-generating unit or linear array photosensitive unit can not unlimited arrangements in imaging len imaging region f ~ 2f, the quantity of that is arranging is limited, and the step generating solid threedimensional image is: A, obtain multiple focussing image file; B, extract image 1 data line definition of going forward side by side and judge, obtain pixel and the three-dimensional coordinate of clear part simultaneously; C, extract image 2 data line definition of going forward side by side and judge, obtain pixel and the three-dimensional coordinate of clear part simultaneously, calculate the normalization coefficient β 1=(image distance 1-f to Fig. 1 simultaneously)/(image distance 2-f); D, extract image n data line definition of going forward side by side and judge, obtain pixel and the three-dimensional coordinate of clear part simultaneously, calculate normalization coefficient β n=(image distance 1-f)/(the image distance n-f) to Fig. 1 simultaneously; The pixel of the clear part obtained in E, obtaining step B ~ C and three-dimensional coordinate span 3-D view; F, image 2 coordinate is multiplied by factor beta 1, image n coordinate is multiplied by factor beta n, calculates and generates solid threedimensional image.

Figure 22 is linear array motion encoder Image Coding principle.2201 is a certain sensitivity speck of linear array photosensitive unit, and in its movement locus, the position of moment is p, can describe, or (r, ω t, y) polar coordinates describes by (x, x, z) Cartesian coordinates; 2203 is the row data of image, i.e. the data of a scan line; 2202 is a pixel number certificate in 2203 row data, and 2202 pixel numbers are the pixel values of p point according to correspondence, and the brightness of pixel value and pixel and colourity, generally represent with the brightness of three primary colours; 2204 is the read cycle of a pixel Data Data; 2205 is line-scanning period of image; 2206 is the movement locus of a certain sensitivity speck 2201 of linear array photosensitive unit.Like this, all sensitivity specks of linear array photosensitive unit by motion by the image pixel value that obtains and virtual borderlines to video data block, formation linear array two field picture.Certainly, the pixel signal that the sensitivity speck of linear array photosensitive unit obtains is simulating signal, so also have process simulating signal being converted into digital signal, i.e. and A/D conversion.

Consider that linear array motion is for regular motion, namely uniform motion vibrations or uniform rotation, in order to reduce system overhead, pixel read cycle can not be used, only use the line-scanning period, line-scanning period signal rising edge starts pixel read operation, and line-scanning period signal negative edge closes pixel read operation, thus completes an image line scanning reading.If linear array motion is uniform motion or uniform rotation, then pixel number is according to being evenly arrangement, if linear array motion is vibration mode, then pixel number is according to being sinusoidal arrangement, by function algorithm, pixel can be reduced to linear array.

Figure 23 is space three-dimensional image image-forming principle.ABCD is object, A1B1C1D1 space three-dimensional image is formed by imaging len, f is imaging len focal length, F is imaging len focus, v1 is A1D1 image distance, and v2 is B1C1 image distance, and A2B2C2D2 is the solid threedimensional image that process generates, ignore flat focus coordinate in figure, only study longitudinal depth coordinate.Can draw the following conclusions according to imaging formula 1/f=1/u+1/v: the optical imagery being positioned at the real world objects of ∞ ~ 2f is imaged lens " non-linear compression " in the space of f ~ 2f, the physical size of real world objects and coordinate are linear, " non-linear compression " algorithm obeys imaging formula, can the flat focus coordinate of computational entity 3-D view and longitudinal coordinate by imaging formula algorithm, as A1B1C1D1 space three-dimensional image is calculated A2B2C2D2 solid threedimensional image, in fact the solid threedimensional image of D1C1 section any point size can be converted into, we are transformed into A1D1 size at hypothesis, i.e. A1D1=A2D2, by trigonometric function relation: B1C1/A1D1=(v2-f)/(v1-f), be the coordinate of B2C2 by the coordinate transformation of B1C1 like this.

Mathematically can be understood as: the linear three-dimensional coordinate of real world by the non-linear three-dimensional matrix of 1/f=1/u+1/v Algorithm mapping to f ~ 2f space, the field of definition of image distance v be (f, 2f], i.e. { v ︳ f < v≤2f }.

Figure 24 is the multiple focussing image viewing mode schematic diagram based on human eye focus.2401 is camera, 2402 is display interface, 2403 is eyeball, p point is eyeball focus, coordinate is (x, y), by camera 2401 and recognition and tracking software sharing eye tracking device, display interface 2402 is the display interface of multiple focussing image software for display, multiple focussing image software for display can show arbitrarily the image of different image distance in multiple focussing image file as required, and required image is shown in display interface as present image, its course of work is: 1, eyeball image is obtained by camera, the focus p of eyeball is judged by recognition and tracking software, and determine its coordinate (x, y), 2, the image that p (x, y) puts region is positioned to the image of different image distances all in multiple focussing image file and carries out definition judgment, and determine wherein one at the highest corresponding image distance image of p point region sharpness, be i.e. p point region parent image clearly, 3, using p point region clearly parent image as display interface present image.

Further, in order to improve image switch speed, the virtual grid of multiple focussing image is carried out sharpness demarcation, and calibration value is formed index or summary record in multiple focussing image file, after determining focus like this, required image can be shown immediately by index or summary.Like this, the course of work is: 1, obtain eyeball image by camera, judged the focus p of eyeball, and determine its coordinate (x, y) by recognition and tracking software; 2, image corresponding in index or summary is determined according to the coordinate (x, y) of p point, i.e. the image of a certain image distance value; 3, the image of described correspondence is shown as present image at display interface.

For video image, multiple focussing image video file coded format comprises video track, the audio track of encapsulation, video track has its image distance label or picture spacing label, further interpolation clear picture calibrations index or summary, its course of work is: 1, obtain eyeball image by camera, judged the focus p of eyeball by recognition and tracking software, and determine its coordinate (x, y); 2, video track corresponding in index or summary is determined according to the coordinate (x, y) of p point, i.e. the video track of a certain image distance label; 3, the video track of described correspondence is shown as current video at display interface.

Further, before the focus p carrying out eyeball shows, there is the step extracted and show other adjacent image distance image, its objective is that solving image switches the sudden change sense produced human eye, also can allow human eye perceives arrive zoom visual impression simultaneously.

Existing eye tracking technology has: Chinese patent announces 03133764.3 intelligent control method for visual tracking, Chinese patent bulletin 200710090883.X is based on the visual tracking method of particle filter and system, Chinese patent bulletin 200810121650.6 is based on the Web page text individuation search method of eye tracking, and Chinese patent bulletin 200910096060.7 is based on the English text automatic abstracting method of eye tracking.

A kind of multiple focussing image display device, it is characterized in that: device is made up of eye tracking device and image display device, eye tracking device is by camera and recognition and tracking software sharing, camera is responsible for obtaining eyeball image, the eyeball image that recognition and tracking software is responsible for camera obtains carries out recognition and tracking to determine eyeball focus coordinate, image display device comprises multi-focus software for display, and multi-focus software for display is paid close attention to coordinate extraction respective image according to the eyeball determined and carried out current display.

Figure 25 is the schematic diagram of multiple focussing image file set up sharpness summary or index.Frequently insert image definition judgement due to system and can increase system overhead, in order to improve image switch speed, sharpness calibration value summary or index is increased in multiple focussing image file, 2501 is a certain image distance image in multiple focussing image file, 2502 is region clearly in 2501 images, in like manner, 2504 is region clearly in 2503 images, and 2506 is region clearly in 2505 images.Concrete processing procedure is: 1, carry out subregion to the virtual grid of images all in multiple focussing image; 2, definition judgment is carried out to each subregion and form definition values; 3, definition values is formed summary or index; 3, summary or index are added in multiple focussing image file, maybe summary or index are encapsulated in video file.

Namely definition values judges the sharpness of image, and forms numerical expression.Definition values is the numerical expression of image definition in other words.

The step of image display is: 1, the selected image zonule that will show, and determines area coordinate; 2, retrieval and inquisition definition values summary or index determine the corresponding image distance image that described zonule definition values is the highest; 3, extract described corresponding image distance image and carry out current display.

Figure 26 adopts image distance label to be a kind of clear picture region summary or concordance list that identify.First virtual grid subregion is carried out to the image of different image distances all in multiple focussing image, x, y, k array in summary or concordance list is formed by sharpness comparison, wherein x, y is the grid coordinates in clear picture region, and k is the image distance tag identifier of a certain image distance image, determines that this grid (i.e. the grid of clear picture) is the image of which corresponding image distance by k value, as x3, y3, ki array representation grid coordinates x3, y3 area image clearly image be ki mark corresponding image distance image.

Certainly, image distance label can directly replace with image distance value, namely identifies the image of different image distance by image distance value.

The step of image display is: 1, the selected image zonule that will show, and determines grid coordinates x, y; 2, in the index be made up of grid coordinates and image distance label, image distance label k is determined uniquely; 3, determine that the image of corresponding image distance carries out current display by image distance label k.

Figure 27 is the step that Figure 26 sets up image distance label k value in clear picture region summary or concordance list.1, multiple focussing image file is obtained; 2, assignment image distance label k value, namely carries out identifier number to the image of different image distances all in multiple focussing image file; 3, carry out grid subregion to image, with x, y coordinate determination grid in the position of display interface, the image panes subregion of different image distance is at the position coordinates x of display interface, and y is consistent; 4, grid subregion sharpness comparison, namely carry out sharpness comparison to the grid subregion of identical x, the y coordinate of different image distance image, choose the image k value that the highest grid of sharpness is corresponding, the array of this grid is x, y, k

In addition, in grid subregion, sharpness is not necessarily uniform, has the situation of transition, can abandon process for there being the grid of transition.

Certainly, image distance label k value can directly replace with image distance value, namely identifies the image of different image distance by image distance value.

Consider that subregion not necessarily adopts grid subregion, a kind of multiple focussing image file layout, is characterized in that: the index comprising subregion coordinate and image distance composition in image file; Or the index of subregion coordinate and image distance label composition.

Described image distance label namely according to image slices apart from the numbering of carrying out image or mark.

Claims (6)

1. a multiple focussing image document handling method, the steps include: 1, carries out subregion to the virtual grid of images all in multiple focussing image; 2, definition judgment is carried out to each subregion and form definition values; 3, definition values is formed summary or index; 3, summary or index are added in multiple focussing image file, maybe summary or index are encapsulated in video file.

2. a kind of multiple focussing image document handling method according to claim 1, its step is: 1, the selected image zonule that will show, and determines area coordinate; 2, retrieval and inquisition definition values summary or index determine the corresponding image distance image that described zonule definition values is the highest; 3, extract described corresponding image distance image and carry out current display.

3. a kind of multiple focussing image document handling method according to claim 2, it is characterized in that: be that the image extracting a certain image distance carries out current display when carrying out image display, the triggering mode of display and control is at least as one of under type: control knob triggering mode, control knob mouse drag triggering mode, mouse coordinates triggering mode, mouse coordinates and button triggering mode, touch screen triggering mode, touch screen front contact coordinate and touch screen triggering mode, eyeball focus triggering mode.

4. a kind of multiple focussing image document handling method according to claim 3, be the steps include: S1, obtains eyeball image by camera, judged the focus of eyeball, and determine focus coordinate by recognition and tracking software; S2, to extract in multiple focussing image file according to the coordinate of focus focus region clearly image carry out current display.

5. a kind of multiple focussing image document handling method according to claim 1, the steps include: A, obtains multiple focussing image file; B, determine prime focus face image; C, extraction adjacent focus face image; D, adjacent focus face image and prime focus face image carried out resolution chart and form fused images 1 as fusion treatment; E, extract another adjacent focus face image; F, another adjacent focus face image and fused images 1 carried out resolution chart and form fused images 2 as fusion treatment; G, extraction time adjacent focus face image; H, secondary adjacent focus face image and fused images 2 carried out resolution chart and form fused images 3 as fusion treatment; I, extract another adjacent focus face image; J, another adjacent focus face image and fused images 3 carried out resolution chart and form fused images 4 as fusion treatment; K, repeat above-mentioned image co-registration process until form final fused images.

6. a kind of multiple focussing image document handling method according to claim 5, it is characterized in that step D ~ K also comprises following sub-step: a, two width images have the step of adjustment picture size before carrying out fusion treatment, fit like a glove with another width picture size after namely adjusting the size of wherein piece image; When b, two width images carry out image co-registration process, the pixel of two width images selects one to choose, and namely the pixel of fused images is that selecting of the pixel of two width images one chooses result.