CN204377058U - Based on the three-dimensional display system of hard mirror - Google Patents
Based on the three-dimensional display system of hard mirror Download PDFInfo
- Publication number
- CN204377058U CN204377058U CN201420865673.9U CN201420865673U CN204377058U CN 204377058 U CN204377058 U CN 204377058U CN 201420865673 U CN201420865673 U CN 201420865673U CN 204377058 U CN204377058 U CN 204377058U
- Authority
- CN
- China
- Prior art keywords
- unit
- image
- gauge point
- hard mirror
- display system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The utility model belongs to field of medical technology, a kind of three-dimensional display system based on hard mirror is provided, comprise: display unit, spectrophotometric unit, tracking equipment and image capturing unit, this spectrophotometric unit is divided into left view and right view on the image space shown by this display unit, this tracking equipment is for obtaining the positional information of first object object, this image capturing unit is for taking the second destination object, image player treatment facility should be also comprised based on the three-dimensional display system of hard mirror, according to the positional information of this first object object, the display parameters of the grating parameter of this spectrophotometric unit and this display unit, the stereo-picture that this image capturing unit received photographs is processed in real time, send this display unit after process to show in real time.Image player speed of the present utility model improves greatly compared to prior art, can meet the requirement of real-time volume display, has the advantage facilitating surgical operation and assist physician to improve success rate of operation.
Description
Technical field
The utility model relates to technical field of medical equipment, in particular to a kind of three-dimensional display system based on hard mirror be applied on clinical medicine.
Background technology
Endoscope is a pipe being equipped with light, and it can enter in stomach or enter in body through other ducts by direct oral cavity.Utilize endoscope can see the pathology that X ray can not show, therefore it is very useful to doctor.Such as, ulcer in stomach or tumour can be observed by endoscope doctor, make best therapeutic scheme accordingly.Endoscope is a kind of optical instrument, and be made up of cold light source camera lens, fiber optic wire, image delivering system, on-screen display system etc., it can expand surgical field of view.The outstanding feature using endoscope is that operative incision is little, and incisional scar is not obvious, and after-operation response is light, and hemorrhage, the livid purple and swelling time can greatly reduce, and recovers also comparatively traditional operation fast, meets very much the requirement of the beautiful not trace of aesthetic surgery.
Endoscope is divided into hard material and soft material two kinds, is called medical soft lens and hard mirror.Initial endoscope stereoplasm tube is made, and invents before more than 100 years, and this is the blank of modern hard mirror.Along with the development of electronic technology and imaging technique, hard mirror has increased more function newly, as image taking imaging etc., still in application in medical domain.
The existing medical endoscope of minority has the function of display 3D picture on the market at present.But the assist type 3D being only limitted to wear 3D spectacle shows.Its technology adopted makes right and left eyes inject different polarization light, produces parallax, form three-dimensional sense organ.The shortcoming of this technology needs to wear polarized filter glasses, the use of polarized filter glasses on the one hand, make the light entered in doctor's eyes be reduced to below original half, waste itself most valuable light information in cavity, reduce in cavity compared with the information discrimination under dark situation; On the other hand, for the doctor of not wearing spectacles in life on ordinary days, in operation, wear polarized filter glasses, very easily produce sense of discomfort, and easily produce fog when breathing on lens surface because wearing mouth mask simultaneously, impact is greatly produced on operation safety.
Therefore, how to overcome the problems referred to above, become the large technical barrier that medical circle faces at present.
Utility model content
The purpose of this utility model is to provide a kind of three-dimensional display system based on hard mirror, is intended to solve the above-mentioned one or more technical problem caused by limitation and the shortcoming of prior art.
A kind of three-dimensional display system based on hard mirror that the utility model provides, comprise: display unit, spectrophotometric unit, tracking equipment and image capturing unit, described spectrophotometric unit is positioned at the display side of described display unit, left view and right view is divided on the image space shown by described display unit, described tracking equipment is for obtaining the positional information of first object object, described image capturing unit is for taking the second destination object, it is characterized in that, the described three-dimensional display system based on hard mirror also comprises: for the positional information according to described first object object, the grating parameter of described spectrophotometric unit and the display parameters of described display unit, the stereo-picture that the described image capturing unit received photographs is processed in real time, the image player treatment facility that described display unit shows in real time is sent after process, wherein, described image player treatment facility is connected with described tracking equipment, described display unit and described image capturing unit respectively, and described image capturing unit is located at the front end of described hard mirror.
The three-dimensional display system based on hard mirror that the utility model provides, image player speed improves greatly compared to prior art, can meet the requirement of real-time volume display, has the advantage facilitating surgical operation and assist physician to improve success rate of operation.
Accompanying drawing explanation
Fig. 1 shows the structural representation of the three-dimensional display system based on hard mirror of the utility model execution mode one.
Fig. 2 is the structural representation of a specific embodiment of the three-dimensional display system based on hard mirror of the utility model execution mode one.
Fig. 3 shows the structural representation of the image player processing unit in Fig. 2.
Fig. 4 is the structural representation of fitting based on spectrophotometric unit and display unit in the three-dimensional display system of hard mirror of the utility model execution mode one.
Fig. 5 shows the preferred embodiment structural representation based on tracking equipment in the three-dimensional display system of hard mirror of the utility model execution mode one.
Fig. 6 shows the concrete structure schematic diagram of the acquiring unit in Fig. 4.
Fig. 7 shows the concrete structure schematic diagram rebuilding unit first variation in Fig. 4.
Fig. 8 shows the concrete structure schematic diagram rebuilding unit second variation in Fig. 4.
Fig. 9 shows the concrete structure schematic diagram rebuilding unit the 3rd variation in Fig. 4.
Figure 10 shows corresponding first object object in the tracking equipment of Fig. 4 and arranges the structural representation of the locating support of gauge point.
Embodiment
In order to more clearly understand above-mentioned purpose of the present utility model, feature and advantage, below in conjunction with the drawings and specific embodiments, the utility model is further described in detail.It should be noted that, when not conflicting, the feature in the execution mode of the application and execution mode can combine mutually.
Set forth a lot of detail in the following description so that fully understand the utility model; but; the utility model can also adopt other to be different from other modes described here and implement, and therefore, protection range of the present utility model is not by the restriction of following public embodiment.
Execution mode one
Refer to Fig. 1, Fig. 1 is the structural representation of the utility model based on the three-dimensional display system of hard mirror.As shown in Figure 1, the three-dimensional display system based on hard mirror of the present utility model comprises: image capturing unit 10, tracking equipment 30, spectrophotometric unit 50 and display unit 40.The image of this second destination object photographed for taking the second destination object, and is sent to this image play unit by this image capturing unit 10 in real time.This tracking equipment 30 is for obtaining the positional information of first object object, and this spectrophotometric unit 50 is positioned at the display side of described display unit 40, on the image space shown by this display unit 40, be divided into left view and right view.Image player treatment facility 20 should be also comprised based on the three-dimensional display system of hard mirror, be connected with this tracking equipment 30 and this display unit 40 respectively, this image player treatment facility 20 processes image to be played in real time according to the display parameters of the positional information of this first object object, the grating parameter of this spectrophotometric unit 50 and display unit 40, sends this display unit 40 and show after process.In addition, this image capturing unit 10 is located on hard mirror, and when hard mirror enters in human or animal body, the image of shooting human or animal inside, facilitates doctor's Real Time Observation and operation in time.Specifically, hard mirror is a kind of optical instrument, is made up of cold light source camera lens, fiber optic wire, image delivering system, on-screen display system etc.Image capturing unit of the present utility model only need replace original camera can not taking stereo-picture in hard mirror, is arranged at the front end of hard mirror, and this front end refers to that soft lens directly enters into the end in human body or animal body, realizes the function of stereoscopic shooting.
Because tracking equipment 30 and display unit 40 are directly connected to image player treatment facility 20, image player treatment facility 20 gets the positional information of first object object, grating parameter and display parameters in time, and carry out image procossing accordingly, eliminate in prior art the processing procedure needed through central processing unit, thus the speed of image player improves greatly compared to prior art, the requirement of real-time volume display can be met, facilitate surgical operation and assist physician to improve the advantage of success rate of operation.This is because doctor is when performing the operation, stereo-picture accurately can be obtained in real time, and performing the operation in time, there will not be the problem mentioned in background technology.The pitch (pitch) that above-mentioned grating parameter mainly comprises grating with grating relative to parameters such as the angle of inclination of display floater, the placement distances of the relative display floater of grating.These grating parameters can be directly be stored in the memory in image player treatment facility, may also be other checkout equipment and detect in real time and the grating parameter obtaining spectrophotometric unit, grating parameter value is sent to image player treatment facility 20.Above-mentioned display unit parameter comprises the size of display unit, the screen resolution of display unit, the putting in order and the parameter such as arrangement architecture of the pixel cell sub-pixel of display unit.Arrangement of subpixels order and sub-pixel are according to RGB arrangement or RBG arrangement, or become BGR arrangement, still become other order arrangements; Arrangement of subpixels structure and sub-pixel are vertically arrangement or transversely arranged, as being the mode cycle arrangement according to RGB from top to bottom, or are the mode cycle arrangement etc. according to RGB from left to right successively.
The image of this second destination object photographed for taking the second destination object, and is sent to this image play unit by this image capturing unit 10 in real time.Here the second destination object mainly refers to the various scenes of being taken record by video camera, as the scene of operation, and image etc. in the body of patient.Stereo-picture is taken in real time by image capturing unit 10, and the stereo-picture photographed is shown on the display unit in real time, without the need to through extra image procossing, also show the various scenes photographed truly in time, meet the demand of user to display in real time, improve Consumer's Experience.Image capturing unit 10 can comprise at least one in monocular-camera, binocular camera or multi-lens camera.
When this image capturing unit 10 comprises monocular-camera, take according to this monocular-camera and obtain the stereo-picture of the second destination object.Preferably, this monocular-camera can adopt liquid crystal lens imaging device or liquid crystal microlens array imaging device.In a specific embodiment, this monocular-camera is not obtaining two width digital pictures of measured object in the same time from different perspectives, and recovers the three-dimensional geometric information of object based on principle of parallax, rebuilds object three-dimensional contour outline and position.
When this image capturing unit 10 comprises binocular camera, comprise two video cameras or a video camera has two cameras, by binocular camera, shooting second destination object is carried out to the second destination object and form stereo-picture.Particularly, binocular camera is mainly based on principle of parallax and by multiple image acquisition object dimensional geological information.Binocular Stereo Vision System is generally obtained two width digital pictures of measured object (the second destination object) from different perspectives simultaneously by twin camera, and recovers the three-dimensional geometric information of object based on principle of parallax, rebuilds object three-dimensional contour outline and position.
When this image capturing unit 10 comprises multi-lens camera, the video camera of namely more than three (comprising three), these video cameras are matrix arrangement, for obtaining stereo-picture.Obtained several digital pictures of the second destination object by the video cameras of above-mentioned more than three from different perspectives simultaneously, recover the three-dimensional geometric information of object based on principle of parallax, rebuild object three-dimensional contour outline and position.
This image capturing unit 10 also comprises collecting unit, and this collecting unit for gathering the stereo-picture of this second destination object, and extracts left view information and right view information from this stereo-picture.This collecting unit one end is connected with above-mentioned monocular-camera, binocular camera or above-mentioned multi-lens camera, and the other end is connected on image player treatment facility 20.Extract the left view information of stereo-picture and right view information by collecting unit limit when limit shooting stereo-picture, improve the speed of image procossing, ensure that the display effect of carrying out stereo display in real time.
Above-mentioned tracking equipment 30 can be camera and/or infrared sensor, is mainly used in the position of following the trail of first object object, the position of the position of the face of the eyes of such as people or the head of people or people or the upper part of the body of people.The quantity of camera or infrared sensor does not limit, and can be one, also can be multiple.Camera or infrared sensor can be arranged on the frame of display unit, or are placed on separately the position being easy to track first object object.In addition, if adopt infrared sensor as tracking equipment, also infrared transmitter can be set in the position of corresponding first object object, by receiving the infrared framing signal that infrared transmitter sends, utilize the relative position relation of infrared transmitter and first object object, calculate the positional information of first object object.
Particularly, this tracking equipment 30 comprises video camera, and this first object object taken by this video camera.The quantity of video camera can be one or more, can arrange on the display unit, also can arrange separately.
This tracking equipment 30 comprises infrared remote receiver, correspondingly, corresponding first object object is provided with infrared transmitter, this infrared transmitter can be arranged on the relevant position of first object object, also can be arranged on other and the relatively-stationary object of first object object's position, this infrared remote receiver receives the infrared signal sent from the infrared transmitter set by corresponding first object object.The location to first object object is realized by common infrared positioning method.
In addition, above-mentioned tracking equipment 30 can also adopt GPS locating module, sends locating information to image player treatment facility 20 by GPS locating module.
Above-mentioned spectrophotometric unit 50 is located at the light emission side of display unit 40, the left view with parallax shown by display unit 40 and right view are sent to left eye and the right eye of people respectively, according to left eye and the right eye synthetic stereo image of people, people is made to watch the effect of stereo display.Preferably, above-mentioned spectrophotometric unit can be disparity barrier or lenticulation.This disparity barrier can be liquid crystal slit or solid slit grating sheet or electrochromism slit grating sheet etc., and this lenticulation can be liquid crystal lens or liquid crystal lens grating.Liquid crystal is mainly cured on thin slice by ultraviolet light by liquid crystal lens grating, forms solid lens, shines left eye and the right eye of people to light after carrying out light splitting.Preferably, using above-mentioned display unit 40 and spectrophotometric unit 50 as an integrated display device 60, this display device 60 is display sections of the whole three-dimensional display system based on hard mirror, can fit together as playback process equipment and tracking equipment with earlier figures, also can be an independent sector individualism.Such as, can need according to viewing, separately display device 60 is placed on the position being convenient to watch, and image player treatment facility 20 and tracking equipment 30 can be the equipment separately with standalone feature, during use, these device assembles are got up to realize real-time volume Presentation Function of the present utility model.Such as, this image player treatment facility 20 can be VMR 3D playback equipment, itself has 3D playback process function, and use is assembled in the three-dimensional display system based on hard mirror of the present utility model, connects with miscellaneous equipment.
Refer to Fig. 2, Fig. 2 is the structural representation of a specific embodiment of the three-dimensional display system based on hard mirror of the utility model execution mode one.As shown in Figure 2, in the three-dimensional display system based on hard mirror of the utility model embodiment 1, image player treatment facility 20 comprises further: image player processing unit 22 and memory cell 23, image player processing unit 22 is mainly according to the positional information of described first object object, the grating parameter of described spectrophotometric unit and the display parameters of display unit, the stereo-picture received is carried out real-time row and scheme process, send described display unit after process and show in real time.The image that memory cell 23 transmits for memory image shooting unit 10.Needs play stereo-picture time, image player processing unit 22 call in memory cell 23 store stereo-picture carry out row scheme process.
Further, this image player treatment facility 20 also comprises further: signal processing unit 21, this signal processing unit 21 is connected with memory cell 23 and image player processing unit 22 respectively, wherein signal processing unit 21 mainly processes the signal of the stereo-picture that the image capturing unit 10 received is taken, and comprises the process such as image format conversion, image compression.Image after compression is stored in memory cell 23.Signal after signal processing unit 21 processes can correspond to left view, right view information output image signal respectively, or outputs in the lump in image player processing unit 22.The positional information of image player processing unit 22 according to described first object object and the grating parameter of described spectrophotometric unit, stereo-picture after being processed by the signal processing unit 21 received carries out real-time row and schemes process, sends described display unit and show in real time after process.Here image player processing unit 22 can directly call the stereo-picture in memory cell 23 and decompress, also can directly Received signal strength processing unit 21 process after stereo-picture, then carry out row scheme process.
Refer to Fig. 3, image player processing unit 22 comprises further:
Row's graph parameter determination module 201, the display parameters of the positional information of described first object object that foundation gets and the grating parameter of described spectrophotometric unit and display list calculate row's graph parameter on the display unit;
Parallax image arrangement module 202, for showing the anaglyph on unit according to described row's graph parameter; This anaglyph is by spatially dividing left-eye image and eye image generates.
Anaglyph playing module 203, plays described anaglyph.After anaglyph after receiving arrangement, play, beholder sees the stereo-picture of display in real time at display unit.
Further, this image player processing unit 22 also comprises: stereo-picture acquisition module 204, obtains the stereo image information of described image capturing unit 10 shooting, i.e. the left view of stereo-picture and right view information.Stereo-picture comprises left view and right view, therefore, to stereo-picture to be played, needs the image information first obtaining left view and right view, just can carry out row and scheme process.
Please continue see Fig. 2, tracking equipment 30 in the present invention comprises track and localization processing unit 31 and tracking cell 32 further.Tracking cell 32 is for following the tracks of the realtime graphic of first object object, and it mainly refers to that camera, infrared remote receiver etc. can catch a kind equipment of human eye or head vision signal exactly.The realtime graphic of the first object object that track and localization processing unit 31 traces into according to tracking cell 32, by extracting the characteristic point of first object object, calculates the space coordinates of first object object.Particularly, such as, carry out real time record face by camera, carry out human face characteristic point extraction through track and localization processing unit 31, calculate the space coordinates of human eye; Can also by increasing the mode of characteristic point, as the head people increases characteristic point, as infrared launcher, then caught the realtime graphic of this characteristic point by camera, track and localization processing unit 31 finally calculates the space coordinates of human eye.
In addition, track and localization processing unit 31 can follow the tracks of position of human eye fast in real time when position of human eye is moved, and provides the space coordinates of human eye, and this coordinate information is supplied to image player processing unit 22.
Also have above-mentioned tracking cell 32 can comprise video camera or infrared remote receiver, when tracking cell 32 comprises video camera, by the change in location of first object object characteristic of correspondence point described in Camera location.When tracking cell 32 comprises infrared remote receiver, described infrared remote receiver receives and comes from corresponding described first object object and the infrared framing signal sent as the infrared transmitter of described characteristic point.
Improved the viewing effect of stereo display by above-mentioned track and localization processing unit 31 and tracking cell 32, stereo display is adjusted automatically with the movement of people, provide optimum stereo display effect in real time.
Embodiment 1
In the utility model embodiment 1, obtain preferably real-time volume display effect, need to carry out optical design according to the grating parameter of spectrophotometric unit and the display parameters of display unit to spectrophotometric unit and display unit, this optical design is according to following formula:
(3)m*t＝p-pitch
In above-mentioned formula, F is the distance (the placement distance of the relative display floater of grating namely in above-mentioned grating parameter) between spectrophotometric unit with display unit, L is the distance of beholder and display unit, IPD is coupling interpupillary distance, distance between common people's double vision, such as, general value is 62.5mm, l-pitch is the pitch (pitch) of spectrophotometric unit, p-pitch is the row figure pitch of the pixel on display unit, n is three-dimensional view quantity, the pixel quantity that m covers for spectrophotometric unit, p is the some distance of display unit, here point is apart from the size (belonging to the one of display parameters) mainly referring to a pixel cell, this pixel cell generally includes R, G, B tri-sub-pixels.In order to eliminate moire fringes, spectrophotometric unit generally can rotate a certain angle (namely spectrophotometric unit has certain angle of inclination compared to display unit) when laminating, and therefore, the pitch of actual spectrophotometric unit is given by the following formula:
(4)W
lens＝l-pitch*sinθ
Wherein, W
lensfor the actual pitch of spectrophotometric unit, θ is the angle of inclination (i.e. above-mentioned grating parameter one of) of spectrophotometric unit relative to display floater.
As mentioned above, for the distance F between spectrophotometric unit and display unit, when the medium between display unit and spectrophotometric unit is air, F just equals the actual range between spectrophotometric unit and display unit; Medium between display unit and spectrophotometric unit is refractive index when being the transparent medium of n (n is greater than 1), and F equals actual range between spectrophotometric unit and display unit divided by this refractive index n; When there is different media between display unit from spectrophotometric unit, and the refractive index of medium is respectively n1, n2, n3 (refractive index is all more than or equal to 1), F=s1/n1+s2/n2+s3/n3, and wherein s1, s2, s3 are the thickness of respective media.
By above-mentioned optical computing formula, spectrophotometric unit and display unit are arranged, can moire fringes be reduced, improve the stereo display effect of viewing in real time.
In addition, in a variant embodiment, arrange laminating unit, refer to Fig. 4 between spectrophotometric unit and display unit, Fig. 4 is the bonding structure schematic diagram based on spectrophotometric unit and display unit in the three-dimensional display system of hard mirror of the utility model execution mode one.As shown in Figure 4, be provided with laminating unit between spectrophotometric unit 50 and display unit 40, three is similar to " sandwich structure ", and laminating unit comprises first substrate 42 and second substrate 43, and the air layer 41 between first substrate 42 and second substrate 43.This air layer 41 is in sealing state between first substrate 42 and second substrate 43, prevents air from overflowing.First substrate 42 and display floater are fitted, and can be that transparent glass material is formed, also can be the formations such as transparent resin material.Second substrate 43 and first substrate 42 are oppositely arranged, and its side deviating from first substrate 42 is for spectrophotometric unit 50 of fitting.Owing to arranging laminating unit between spectrophotometric unit 50 and display unit 40, and laminating unit adopts said structure, for the 3 d display device of large-screen, both ensure that the evenness that grating is fitted, alleviate again the weight of whole 3 d display device, when preventing from adopting pure glass, cause because of overweight screen to fall the risk split.
Embodiment 2
In the present embodiment 2, this tracking equipment 30 comprises video camera, and this first object object taken by this video camera.The quantity of video camera can be one or more, can arrange on the display unit, also can arrange separately.Further, video camera can be monocular-camera, binocular camera or multi-lens camera.
In addition, this tracking equipment 30 can also be comprise infrared remote receiver, correspondingly, corresponding first object object is provided with infrared transmitter, this infrared transmitter can be arranged on the relevant position of first object object, also can be arranged on other and the relatively-stationary object of first object object's position, this infrared remote receiver receives the infrared signal sent from the infrared transmitter set by corresponding first object object.The location to first object object is realized by common infrared positioning method.
In addition, above-mentioned tracking equipment 30 can also adopt GPS locating module, sends locating information to image player processing unit 20 by GPS locating module.
Embodiment 3
Refer to Fig. 5, Fig. 5 shows the preferred embodiment structural representation based on tracking equipment in the three-dimensional display system of hard mirror of the utility model execution mode one.As shown in Figure 5, the utility model embodiment 3 also proposes another kind of tracking equipment 30, and this tracking equipment 30 comprises:
Gauge point setting unit 1, the locus for corresponding first object object arranges gauge point; Here gauge point can arrange on first object object, also can not be arranged on first object object, but is arranged on and has relative position relation with first object object, also can with on the object of first object object synchronous motion.Such as, first object to liking human eye, then can arrange gauge point around the eye socket of human eye; Or around human eye, configure glasses, gauge point is located on the picture frame of glasses, or gauge point is located at on the ear of the relatively-stationary people of position of human eye relation.This gauge point can be send the infrared emission transducer of signal, LED, GPS transducer, the various parts such as laser positioning sensor, and also can be that other can the physical label that catches of camera being shot, such as, be the object with shape facility and/or color characteristic.Preferably, for avoiding the interference of extraneous veiling glare, improve the robustness that gauge point is followed the trail of, preferably use the infrared LED lamp that frequency spectrum is comparatively narrow as gauge point, and use can only by infrared LED use the corresponding thermal camera of frequency spectrum to catch gauge point.Consider that extraneous veiling glare mostly is irregular shape and Luminance Distribution is uneven, gauge point can be arranged to the hot spot that can send regular shape, luminous intensity is higher, brightness uniformity.Multiple gauge point can also be set in addition, the corresponding hot spot of each gauge point, the geometry of each gauge point composition rule, as triangle, quadrangle etc., thus be easy to trace into gauge point, obtain the spatial positional information of gauge point, and improve the accuracy of hot spot extraction.
Acquiring unit 2, for obtaining the positional information of this gauge point; This can be the signal sent by receiving gauge point, determines the positional information of gauge point, also can be the image that employing video camera takes containing gauge point, extract the gauge point in image.The positional information of gauge point is obtained by image processing algorithm.
Rebuild unit 3, for the positional information according to this gauge point, rebuild the locus of this first object object.When after the positional information acquiring this gauge point, rebuild out the locus of gauge point, then the relative position relation of foundation gauge point and first object object, is transformed into the locus (locus of two, the left and right of such as people) of first object object by the locus of gauge point.
The tracking equipment 30 of the utility model embodiment by obtaining the positional information of the gauge point of corresponding first object object, and according to this positional information, rebuilds out the locus of first object object.Need to carry out signature analysis to two dimensional image as human eye capture device with using video camera in prior art thus obtain position of human eye or use other to utilize the human eye capture device of human eye iris reflex effect to obtain compared with position of human eye, there is good stability, high, the with low cost and distance between tracking equipment and first object object is not the required advantage of accuracy.
Refer to Fig. 6, Fig. 6 shows the concrete structure schematic diagram of the acquiring unit in Fig. 5.Aforementioned acquiring unit comprises further:
Presetting module 21, for presetting a standard picture, being provided with reference marker point, and obtaining space coordinates and the plane coordinates of described reference marker point in described standard picture; Standard picture can be such as the standard picture gathered by image capture device, obtain the image coordinate of reference marker point, and using other accurate measurement in space equipment as laser scanner, the equipment such as structured light scanner (as Kinect etc.) obtain space coordinates and the plane coordinates of reference marker point in standard picture.
Acquisition module 22, for obtaining the present image comprising described first object object and described gauge point, and described gauge point is at the plane coordinates of described present image;
Matching module 23, for mating the gauge point in described present image with the described reference marker point of described standard picture.Here first gauge point to be set up corresponding relation at the plane coordinates of described present image and reference marker point between the plane coordinates of standard picture, then gauge point be mated with reference marker point.
Being be convenient to can there be an object of reference when obtaining the locus of present image by arranging standard picture and reference marker point, it further ensures stability and the accuracy of the target tracker of the utility model execution mode.
Further, this tracking equipment 30 also comprises:
Collecting unit, for gathering described gauge point;
Screening unit, screens target label point from described gauge point.
Specifically, when the quantity of gauge point is multiple, adopt all gauge points of the corresponding first object object of camera acquisition, gauge point maximally related with first object object is chosen from all gauge points, then use corresponding image processing algorithm to extract the gauge point on image, this extraction needs to carry out according to the feature of gauge point.Generally, the method extracted the feature of this gauge point extracts function H to image I use characteristic, obtains the feature scores of each point in image, and filter out the sufficiently high gauge point of characteristic value.Here can conclude with following formula and represent:
S(x，y)＝H(l(x，y))
F＝{arg
(x，y)(S(x，y)＞s0)}
In above-mentioned formula, H is feature extraction function, I (x, y) be each pixel (x, y) image value corresponding to can be gray value or three-channel color energy value etc., S (x, y) be each pixel (x, y) feature scores after feature extraction, s0 is a feature scores threshold value, is greater than the S (x of s0, y) can be considered to gauge point, F is gauge point set.Preferably, the utility model embodiment use infrared markers point and thermal camera become the energy feature of image comparatively obvious.Owing to using narrow-band LED infrared lamp, and corresponding thermal camera, video camera become most of pixel energy of image very low, only have pixel corresponding to gauge point to have high-energy.Therefore corresponding function H (x, y) carries out region growing to the image B (x, y) after using Threshold segmentation operator to obtain some subgraphs, and carries out center of gravity extraction to the subgraph acquired.Simultaneously, environmentally can the veiling glare of imaging in thermal camera in light, we can add such as facula area that gauge point becomes in infrared markers point leaching process, and the constraintss such as the position relationship of gauge point in two dimensional image are screened the gauge point extracted.
When video camera number is greater than 1, needs to carry out reference points matching to different cameras at synchronization or close to the image that synchronization obtains, thus provide condition for follow-up gauge point three-dimensional reconstruction.The method of reference points matching needs to determine according to feature extraction function H.The feature point extraction operator based on gradient of image and gray scale figure that we can use some classics and matching process such as the methods such as Harris, SIFT, FAST matched with it obtain and matched indicia point.Also can retrain by operating limit, the modes such as the priori conditions of gauge point carry out reference points matching.Here the method that operating limit constraint carries out mating screening is: be all in this principle of same plane according to projection on two different cameras images in same o'clock, for the some gauge point p0 in some video camera c0, we can calculate a polar curve equation in other video cameras c1, and the gauge point p1 that gauge point p0 corresponds on this other video camera c1 meets following relation:
[p1；1]
TF[p0；1]＝0
In above-mentioned formula, F is the basis matrix of video camera c0 to video camera c1.By using above-mentioned relation, we can reduce candidate's number of gauge point p1 greatly, improve matching accuracy.
In addition, we can the priori conditions of usage flag point be the spatial order of gauge point, the size etc. of gauge point.Such as make two pixels of the every a pair corresponding the same space point on the image captured by it equal in some dimensions such as y-axis according to the mutual alignment relation of two video cameras, this process is also referred to as image calibration (rectification).Then now the coupling of gauge point also just can perform according to the x-axis order of gauge point, i.e. the corresponding minimum x of minimum x, the like, the x that maximum x is corresponding maximum.
Following foundation, for the number of video camera number of following the tracks of, introduces target tracker of the present utility model in detail.
Refer to Fig. 7, Fig. 7 shows the concrete structure schematic diagram rebuilding unit in Fig. 5.As shown in Figure 7, in the present embodiment, the gauge point that the first object object of this tracking equipment 30 tracking is corresponding is no more than four, and when adopting monocular-camera to obtain the positional information of gauge point, rebuilds unit and comprise further:
First computing module 31, for calculating the homograph relation between described present image and described standard picture according to the plane coordinates of the plane coordinates of the gauge point in described present image and the described reference marker point of described standard picture and the assumed conditions of described first object object place scene; The gauge point of present image is mated with the reference marker point in standard picture, and calculates the homograph relation between present image and standard picture according to the two respective plane coordinates.So-called homograph is the homography in corresponding geometry, is a kind of transform method of normal application in computer vision field.
First reconstructed module 32, for calculating the rigid transformation of described gauge point in the locus in shooting described standard picture moment to the locus of current time according to described homograph relation, then calculate the locus of described gauge point at current time, and calculate the current locus of described first object object according to described gauge point in the locus of current time.
Specifically, for the assumed conditions of scene, the numerical value of a certain dimension when we can suppose the rigid transformation of the gauge point in scene is constant, in such as three dimensional spatial scene, space coordinates is x, y, z, x with y is parallel with y-axis with x-axis in the image coordinate (plane coordinates) of camera respectively, and z-axis is perpendicular to the image of camera, assumed conditions can be that the coordinate of gauge point in z-axis is constant, also can be that the coordinate of gauge point in x-axis and/or y-axis is constant.Different suppositive scenario conditions, the estimation method used also is not quite similar.Again such as, under another kind of assumed conditions, suppose first object object towards camera towards between the anglec of rotation in use remain constant, then can according to the gauge point in present image mutually between distance and gauge point on standard picture mutually between distance between ratio infer the locus that first object object is current.
By above computational methods, can realize the locus rebuilding described first object object when the quantity of monocular-camera to gauge point is no more than four, it is simple to operate, and tracking results is also more accurate, owing to adopting monocular, reduce the cost of first object Object tracking.
Above-mentioned use single camera gathers image and recovers in object dimensional seat calibration method, because the image information obtained is less, therefore needs the number increasing gauge point to provide more image information thus the three-dimensional coordinate of calculating object.Theoretical according to machine vision, the steric information of scene to be extrapolated from single image, need five calibration points at least determined in image.Therefore, monocular scheme adds gauge point quantity, too increases the complexity of design, but simultaneously, only needs a video camera thus reduces the complexity of IMAQ, reducing cost.
Refer to Fig. 8, Fig. 8 shows the concrete structure schematic diagram rebuilding the second variant embodiment of unit in Fig. 5.As shown in Figure 7, in the present embodiment, when the quantity of described gauge point is more than five, and adopt monocular-camera when obtaining the positional information of described gauge point, described in rebuild unit and comprise further:
Second computing module 33, for the plane coordinates according to the plane coordinates of the gauge point in described present image and the described reference marker point of described standard picture, calculates the homograph relation between described present image and described standard picture.
Second reconstructed module 34, for calculating the rigid transformation of described gauge point in the locus in shooting described standard picture moment to the locus of current time according to described homograph relation, then calculate the locus of described gauge point at current time, and calculate the current locus of first object object according to the locus of described gauge point current time.
First gather a width standard picture, use the devices such as accurate depth camera or laser scanner to measure the locus of reference marker point, and obtain the two dimensional image coordinate (i.e. plane coordinates) of reference marker point now.
In use, video camera constantly catches the two dimensional image coordinate of all gauge points in the present image containing first object object, and the rigid transformation between gauge point when calculating gauge point under current state and shooting standard picture according to the two-dimensional coordinate of now two dimensional image coordinate and standard picture reference marker point, when supposing that between gauge point, relative position is constant, and then calculate out now gauge point relative to locus conversion during standard picture, thus calculate the locus of current markers point.
Here, use the point of more than five points can calculate current markers point and the locus rigid transformation [R|T] taking standard picture tense marker point, preferably, not in one plane, and the projection matrix P of camera is demarcated this point of more than five in advance.The concrete mode calculating [R|T] is as follows:
Each gauge point is respectively X0 in the homogeneous coordinates of standard picture and present image, Xi.The two meets limit restraint, i.e. X
0p
-1[R|T] P=X
i.All gauge points form the equation group that a unknown parameter is [R|T].When gauge point quantity is greater than 5, can solve [R|T]; When gauge point quantity is greater than 6, can ask optimal solution to [R|T], its method can use singular value decomposition SVD, and/or uses the method for iteration to calculate non-linear optimal solution.After calculating gauge point locus, we can infer according to the mutual alignment relation between the gauge point demarcated in advance and first object object (such as human eye) locus first object object (as human eye).
The present embodiment only uses a video camera, uses the gauge point of more than five or five, just can construct the locus of first object object exactly, not only simple to operate, and with low cost.
Refer to Fig. 9, Fig. 9 shows the concrete structure schematic diagram rebuilding the 3rd variant embodiment of unit in Fig. 5.As shown in Figure 9, the present embodiment uses two or more video camera, one or more gauge point.When adopting binocular camera or multi-lens camera to obtain the positional information of described gauge point, described in rebuild unit and comprise further:
3rd computing module 35, adopts binocular or many orders three-dimensional reconstruction principle, calculates the locus of each gauge point at current time; So-called binocular or three orders rebuild principle can adopt following methods, such as, parallax between the gauge point adopting left and right camera to mate, calculates the locus of each gauge point at current time.Or adopt other existing common methods to realize.
Reconstruction module 36, calculates the current locus of first object object according to the locus of described gauge point current time.
Particularly, the method for multi-lens camera calibration is first used to demarcate the mutual alignment relation between each video camera.Then in use, to the image zooming-out gauge point coordinate that each video camera gets, and each gauge point is mated, namely obtain it at gauge point corresponding to each video camera, then use mutual alignment relation between the gauge point of coupling and video camera to calculate the locus of gauge point.
In a concrete example, use multi-lens camera (namely number of cameras is more than or equal to 2) to carry out shot mark point, realize stereo reconstruction.The coordinate u of a known gauge point on a certain shot by camera image and this camera parameters matrix M, we can calculate a ray, and this gauge point is on this ray in space.
α
ju
j=M
jx j=1 ... n (wherein n be more than or equal to 2 natural number)
In like manner, according to above-mentioned formula, this gauge point also can calculate should the ray of other video camera on other video camera.Theoretically, these two rays converge on a point, i.e. the locus of this gauge point.In fact due to the digitizer error of video camera, error that video camera internal reference and outer ginseng are demarcated etc., these rays can not converge at a bit, therefore need to use the method approximate calculation of triangulation (triangululation) to go out the locus of gauge point.Least square judgment criterion such as can be used to determine, and the point nearest apart from all light is as object point.
After calculating gauge point locus, we can infer according to the mutual alignment relation between the gauge point demarcated in advance and first object object (as human eye) locus first object object (human eye).
Realize in the method for stereo reconstruction at above-mentioned use multi-lens camera, preferably method uses binocular camera to calculate.It is the same that its principle and aforementioned multi-lens camera rebuild principle, is all to calculate gauge point locus according to the mutual alignment relation of two video cameras and gauge point at the two-dimensional coordinate of two video camera imagings.Its minute differences is binocular camera laid parallel, according to after simple demarcation, image calibration as previously described is done to the image of two video cameras, the two-dimensional marker point making two to match each other y (or x) equal on axle, then now gauge point apart from video camera the degree of depth can at x, (or gap y) on axle calculates by the two-dimensional marker point after calibrating.The method can regard as multi-eye stereo be reconstituted in biocular case under specific process, which simplify stereo reconstruction step and be easier on device hardware realize.
Embodiment 4
Refer to Figure 10, Figure 10 shows corresponding first object object in the tracking means of Fig. 5 and arranges the structural representation of the locating support of gauge point.As shown in Figure 10, the utility model provides a kind of locating support, this locating support is positioned at human eye (first object object) front, structure is similar to glasses, it is worn and is similar to glasses, comprise: crossbeam 11, fixed part 12, support portion 13 and control part 14, crossbeam 11 is provided with gauge point 111; Support portion 13 is arranged on crossbeam 11; Fixed part 12 is connected with the end pivot of crossbeam 11.Wherein the position of gauge point 111 setting is corresponding with the position of human eye (first object object), by obtaining the spatial positional information of gauge point 111, then calculates the spatial positional information of human eye accordingly.When the head of people is moved, correspondingly, the gauge point 111 corresponding with human eye is also moved, the movement of Camera location gauge point 111, then the scheme of the destination object tracking of aforementioned embodiments one is adopted to obtain the spatial positional information of gauge point 111, utilize the relative tertiary location relation of gauge point 111 and human eye, rebuild out the locus (namely three-dimensional coordinate) in space of human eye (first object object).
In the present embodiment, crossbeam 11 is a strip, and has certain radian, and the forehead radian of its radian and people is similar to, with easy to use.Crossbeam 11 comprises upper surface 112, lower surface corresponding thereto, is arranged on first surface 114 between upper surface 112 and lower surface and second surface.
In the present embodiment, gauge point 111 is three LED, and it is arranged on the first surface 114 of crossbeam 11 evenly and at intervals.Be understandable that, gauge point 111 also can be one, two or more, and can be any light source, comprise LED, infrared lamp or uviol lamp etc.Further, the arrangement mode of described gauge point 111 and setting position also can adjust as required.
Be understandable that, crossbeam 11 also can be designed to linear or other shapes as required.
In the present embodiment, fixed part 12 has two, be pivotally connected with the two ends of crossbeam 11 respectively, and two fixed parts 12 can be relative interior folding, simultaneously, two fixed parts 12 can respectively to be expanded to outward and crossbeam 11 in the interior angle of about 100 °, concrete, can according to the size of practical operation demand adjustment interior angle.Understandable, fixed part 12 also can be one.
Fixed part 12 away from crossbeam 11 one end along support portion 13 bearing of trend bending arrange, for by the end winding support of fixed part 12 on the ear of people.
In the present embodiment, support portion 13 in strip, be arranged on the lower surface 113 of crossbeam 11 middle part and to downward-extension.Further, support portion 13 is provided with nose support 131 away from the end of crossbeam 11, coordinates the bridge of the nose, and be arranged at above human eye by positioner for by positioner.Understandable, in other embodiments, if do not arrange nose support 131, then " Y " type can be arranged to down in support portion 13, and along crossbeam 11 middle part and to downward-extension, in order to positioner is coordinated the bridge of the nose, and positioner is arranged at above human eye.
Control part 14, in fillet cuboid, is arranged on fixed part 12.Control part 14 is for giving described LED, and infrared lamp or uviol lamp provide power supply and/or person to control described LED, the using state of infrared lamp or uviol lamp, and it comprises mains switch 141, power supply indicator and charging indicator light.Be understandable that, control part 14 does not limit shape, and it can have any shape, and also can be an integrated chip.Further, control part 14 also can be arranged on other positions, as on crossbeam 11.
During use, turn on the power switch 141, power supply indicator display LED is in power supply state, and LED is lit; When electricity is not enough, charging indicator light prompting electricity is not enough; Powered-down switch, power supply indicator extinguishes, and represent that LED is in closed condition, LED is extinguished.
Interpupillary distance scope due to people is 58mm ~ 64mm, can be similar to and think that the interpupillary distance of people is definite value, the locating support that the utility model provides is similar to spectacle frame, and be fixed on above human eye, be similar to spectacle frame, as required, gauge point is arranged on the precalculated position of positioner, thus the position of human eye can be determined simply and easily according to the position of gauge point.Positioning device structure is simple, and designing and employing is convenient.
The foregoing is only preferred implementation of the present utility model, be not limited to the utility model, for a person skilled in the art, the utility model can have various modifications and variations.All within spirit of the present utility model and principle, any amendment done, equivalent replacement, improvement etc., all should be included within protection range of the present utility model.
Claims (10)
1. the three-dimensional display system based on hard mirror, comprise: display unit, spectrophotometric unit, tracking equipment and image capturing unit, described spectrophotometric unit is positioned at the display side of described display unit, left view and right view is divided on the image space shown by described display unit, described tracking equipment is for obtaining the positional information of first object object, described image capturing unit is for taking the second destination object, it is characterized in that, the described three-dimensional display system based on hard mirror also comprises: for the positional information according to described first object object, the grating parameter of described spectrophotometric unit and the display parameters of described display unit, the stereo-picture that the described image capturing unit received photographs is processed in real time, the image player treatment facility that described display unit shows in real time is sent after process, wherein, described image player treatment facility is connected with described tracking equipment, described display unit and described image capturing unit respectively, and described image capturing unit is located at the front end of described hard mirror.
2., as claimed in claim 1 based on the three-dimensional display system of hard mirror, it is characterized in that, described image capturing unit is monocular-camera, binocular camera or multi-lens camera.
3., as claimed in claim 1 based on the three-dimensional display system of hard mirror, it is characterized in that, described tracking equipment also comprises a locating support, and described locating support is provided with at least one gauge point.
4., as claimed in claim 3 based on the three-dimensional display system of hard mirror, it is characterized in that, described locating support comprises: crossbeam, fixed part, support portion, and described crossbeam is provided with described gauge point; Described support portion is arranged on crossbeam, supports described crossbeam; Described fixed part is connected with the end pivot of described crossbeam.
5. as claimed in claim 3 based on the three-dimensional display system of hard mirror, it is characterized in that, described gauge point is light source that can be luminous.
6. the three-dimensional display system based on hard mirror as described in any one of claim 2 to 5, it is characterized in that, described image capturing unit comprises collecting unit, described collecting unit for gathering the stereo-picture of described second destination object, and extracts left view information and right view information from described stereo-picture.
7., as claimed in claim 1 based on the three-dimensional display system of hard mirror, it is characterized in that, described tracking equipment comprises video camera, the change in location of first object object described in described Camera location.
8. as claimed in claim 1 based on the three-dimensional display system of hard mirror, it is characterized in that, described tracking equipment comprises infrared remote receiver, and described infrared remote receiver receives the infrared framing signal coming from the infrared transmitter set by corresponding described first object object and send.
9., as claimed in claim 1 based on the three-dimensional display system of hard mirror, it is characterized in that, between described spectrophotometric unit and described display unit, be provided with laminating unit, by described laminating unit, described spectrophotometric unit is fitted on described display unit.
10., as claimed in claim 9 based on the three-dimensional display system of hard mirror, it is characterized in that, described laminating unit comprises first substrate, second substrate, and the air layer between described first substrate and described second substrate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201420865673.9U CN204377058U (en) | 2014-12-29 | 2014-12-29 | Based on the three-dimensional display system of hard mirror |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201420865673.9U CN204377058U (en) | 2014-12-29 | 2014-12-29 | Based on the three-dimensional display system of hard mirror |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN204377058U true CN204377058U (en) | 2015-06-03 |
Family
ID=53333233
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201420865673.9U Active CN204377058U (en) | 2014-12-29 | 2014-12-29 | Based on the three-dimensional display system of hard mirror |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN204377058U (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106773080A (en) * | 2015-12-25 | 2017-05-31 | 深圳超多维光电子有限公司 | 3 d display device and display methods |
| CN106921857A (en) * | 2015-12-25 | 2017-07-04 | 珠海明医医疗科技有限公司 | Three-dimensional display system and stereo display method |
-
2014
- 2014-12-29 CN CN201420865673.9U patent/CN204377058U/en active Active
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106773080A (en) * | 2015-12-25 | 2017-05-31 | 深圳超多维光电子有限公司 | 3 d display device and display methods |
| CN106921857A (en) * | 2015-12-25 | 2017-07-04 | 珠海明医医疗科技有限公司 | Three-dimensional display system and stereo display method |
| CN106773080B (en) * | 2015-12-25 | 2019-12-10 | 深圳超多维光电子有限公司 | Stereoscopic display device and display method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN205610834U (en) | Stereo display system | |
| US11199706B2 (en) | Head-mounted display for virtual and mixed reality with inside-out positional, user body and environment tracking | |
| CN101072366B (en) | Free stereo display system based on light field and binocular vision technology | |
| CN106101689B (en) | The method that using mobile phone monocular cam virtual reality glasses are carried out with augmented reality | |
| CN105809654A (en) | Target object tracking method and device, and stereo display equipment and method | |
| CN204578692U (en) | Three-dimensional display system | |
| AU2006282764B2 (en) | Capturing and processing facial motion data | |
| CN106773080B (en) | Stereoscopic display device and display method | |
| CN204377059U (en) | Based on the three-dimensional display system of soft lens | |
| CN109491087A (en) | Modularized dismounting formula wearable device for AR/VR/MR | |
| CN105427385A (en) | High-fidelity face three-dimensional reconstruction method based on multilevel deformation model | |
| CN102109677B (en) | Head-mounted stereoscopic visual slide viewer | |
| CN105812774B (en) | Three-dimensional display system and method based on intubation mirror | |
| CN105342561B (en) | The wearable molecular image navigation system of Wireless sound control | |
| CN108616752B (en) | Head-mounted equipment supporting augmented reality interaction and control method | |
| CN204636276U (en) | A kind of wearing die cavity mirror virtual display system | |
| CN108836236A (en) | Endoscopic surgery naked eye 3D rendering display system and display methods | |
| CN105812772A (en) | Stereo display system and method for medical images | |
| CN204377058U (en) | Based on the three-dimensional display system of hard mirror | |
| CN204377057U (en) | Based on the three-dimensional display system of intubate mirror | |
| CN204539353U (en) | Medical image three-dimensional display system | |
| CN104967839A (en) | Intelligent glasses and using method thereof | |
| CN109068035A (en) | A kind of micro- camera array endoscopic imaging system of intelligence | |
| CN104887316A (en) | Virtual three-dimensional endoscope displaying method based on active three-dimensional displaying technology | |
| CN105812776A (en) | Stereoscopic display system based on soft lens and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |