CN105739106B - A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method - Google Patents
A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method Download PDFInfo
- Publication number
- CN105739106B CN105739106B CN201510325032.3A CN201510325032A CN105739106B CN 105739106 B CN105739106 B CN 105739106B CN 201510325032 A CN201510325032 A CN 201510325032A CN 105739106 B CN105739106 B CN 105739106B
- Authority
- CN
- China
- Prior art keywords
- gesture
- projector
- array
- image
- projectors
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/20—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
- G02B30/26—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type
Abstract
The invention discloses a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and methods to be conveyed into computer control system by the image for the known color scale pattern that industrial camera acquired projections instrument is projected;Every projector is connect with a card computer in the card computer array in array of rear-projectors;Card computer access to LAN carries out real time video data with computer control system and mutually passes;Array of rear-projectors is output on the vertical scattering film as display screen;Motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera and identifies, recognition result is inputted computer control system.By establishing projected light field analysis model, propose the projection image generation method based on scene rendering, user can be in multi-angle naked eye to scene Three-dimensional Display high-definition, experience intimate true 3-D effect, and good interaction can be realized by gesture control image rotation, movement, scaling.
Description
Technical field
The present invention relates to multiple views light field dimension display technologies fields, and in particular to a kind of body-sensing multiple views large scale light field
True three-dimensional display apparatus and method.
Background technique
Two-dimentional display technology makes people that can not obtain the three-dimensional information of object from shown image or experiences object
Physical depth, deprived the third dimensional feature of object, there is great limitation, therefore Three-dimensional Display is come into being.It is sending out
There is binocular solid during exhibition and shows (anaglyph spectacles) and naked eye 3D TV etc. " puppet " dimension display technologies, such display skill
Art viewpoint number is few, can only form stereoscopic effect by binocular parallax, be not carried out the mobile parallax of object substantially.
Into after the 1990s, due to the rapid development of the technologies such as laser, electronics, computer, many countries start
It is active in the correlative study work of true three-dimension display technlolgies, such as the U.S., Germany, Japan, South Korea and China, and obtains phase
When big progress, wherein the most significant with the achievement that the U.S. and Germany obtain.Stanford Univ USA (Stanford
University the laboratory 3DTL (3D Technology Laboratories)) propose based on infra-red laser diode
3D solid display device obtains the U.S. " exploration " (Discover) and " Industry Week " (Industry Week) magazine and issues
" excellent technical prize ".In the same year, USN's marine systems center (NOSC) is in U.S. Department of Defense (DARPA) and naval (Navy)
Funding under be proposed body three-dimensional display apparatus of the second generation based on krypton argon ion laser, it is contemplated that be applied in shallow water area
The many aspects such as Submarine Navigation, more torpedo monitoring positions and air traffic control.And United States Air Force and NASA
(NASA) true 3-D technology is also paid much attention to, a large amount of funding research has been put into.Texas Instruments (TI) are in US Air
Under the support of army, research by many years to laser scanning system proposes true tripleplane's technology based on DMD.GENEX
Deng Duojia u s company is also dedicated to study and answers the true three-dimensional rotation surface technology with actual physical depth and the observation of full what comes into a driver's
For the fields such as aviation aerospace and geoscience research.
However, fine definition real three-dimensional display system bulky complex and with high costs at present, is unable to satisfy the prior art
Demand.
Summary of the invention
Goal of the invention: in order to overcome the deficiencies in the prior art, the present invention provides a kind of body-sensing multiple views large scale
The true three-dimensional display apparatus of light field and method propose the perspective view based on scene rendering by establishing projected light field analysis model
As generation method, solve the deficiencies in the prior art.
Technical solution: to achieve the above object, a kind of the technical solution adopted by the present invention are as follows: body-sensing multiple views large scale light
The true three-dimensional display apparatus in field, which is characterized in that the device includes vertical scattering film, array of rear-projectors, computer control system, card
Piece computer array, motion sensing control device and industrial camera;
The image for the known color scale pattern that the industrial camera acquired projections instrument is projected, is conveyed into computer control
System processed;
Every projector passes through in high-definition multimedia interface line and the card computer array in the array of rear-projectors
A card computer connection;The card computer access to LAN and computer control system progress real time video data are mutual
It passes;The array of rear-projectors is output on the vertical scattering film as display screen;
The motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera
It is identified, recognition result is inputted into computer control system.
Further, array of rear-projectors is one group of array of rear-projectors, including 16 projectors, and projector is arc-shaped point
Cloth, the arc radius are 2m;The interval angles of adjacent two projector are 1.36 °.
Further, the maximum resolution of projector is 1920*1080.
Further, sliceable multiple array of rear-projectors, projector number are progressively increased by 16, and projector number is no more than 64
Platform;The circular arc being spliced into along the arc-shaped distribution of projector continues several groups array of rear-projectors of arranging;
A kind of true 3 D displaying method of body-sensing multiple views large scale light field, which is characterized in that method includes the following steps:
1) projected image based on scene rendering generates: using OpenGL to respectively to figure horizontally and vertically
As rendering the corresponding image of projector;
2) projector calibrating and correction;
3) motion sensing control is realized.
Further, vertical direction described in step 1) image rendering the following steps are included:
Define the three-dimensional point P on perpendicular determined by camera and viewpoint1=(Xp, Yp, Zp), point institute on the screen
At picture be the point p that is projected from viewpoint Oc1=(XOc, YOc, ZOc), put the normalized coordinate on the column are as follows:
The position projector optical center Op in horizontal direction are as follows: Op=(XOp, YOp, ZOp)
Horizontal coordinate are as follows: px=(Xp-Xop)/(Zp-Zop)
Point P in three-dimensional scenic1Corresponding normalized coordinate is answered on projection images are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Projector is used to project to obtain point p ' as optical center1It is rendered to obtain:
p′y=(Yp-Yop)/(Zp-Zop)
Enable Yoc=Yop, the abscissa y of the point projected to projector optical center realizes and correctly throws multiplied by a coefficient
Shadow process obtains py=p 'y(Zp-Zop)/(Zp-Zoc)。
Further, step 2) includes proceeding as follows to every projector in array of rear-projectors:
2.1) checkerboard pattern is acquired with industrial camera, which is projected on screen;
2.2) feature detection is carried out to collected checkerboard pattern, extract angle point and calculates chessboard angle point and projector institute
Homography matrix H1 between the pattern of projection extracts the homography matrix H2 between chessboard angle point and target checkerboard pattern;
2.3) it corrects the projector list and answers square H matrix representation are as follows:
Further, step 3) the following steps are included:
3.1) sample database is established: by the way that commonly bending moment algorithm does not calculate the Hu not bending moments, and and hand of several certain gestures
Characteristic parameter vector of the area perimeter of gesture than constituting these template gestures.
3.2) it acquires: being acquired in user gesture input FPGA and handled by camera;
3.3) gesture is divided from background: the pretreatment denoised by image of the median filtering algorithm to input,
Then the Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using threshold value parted pattern, determines that one is cut off from according to statistical result
Value is the colour of skin within the scope of the threshold value, otherwise is the non-colour of skin, completes the binaryzation of image, gesture is split from background;
3.4) denoise: the gesture after segmentation goes noise to obtain complete image by Morphological scale-space;The Morphological scale-space
Including etch and plavini;
3.5) it obtains the image information of prominent gesture: edge detection being carried out to image with Sobel operator, and to gesture area
It is scanned, defines the perimeter that all pixels point number on the boundary line of gesture area is gesture, define in gesture area and own
Pixel number is gesture area, obtains area perimeter and compares parameter;
3.6) obtain similarity highest gesture: the Euclidean distance by calculating target to be identified and template judges target and template
Matching degree represents the point (x of target gesture feature1, x2..., xn) with represent the template gesture feature point (y1,
y2..., yn) Euclidean distance d calculation formula are as follows:
By 7 Hu of images of gestures, bending moment and area perimeter ratio are not extracted as characteristic parameter, and with corresponding to each template
Feature vector calculate Euclidean distance, template corresponding to the shortest distance is most matched gesture.
The utility model has the advantages that
The invention proposes body-sensing multiple views large scale light field real three-dimensional display system, described includes raspberry pie card computer
Array, computer control system, array of rear-projectors and motion sensing control module.Raspberry pie card computer array carries out distributed data
Processing, and will treated video information real-time display with array of rear-projectors;Computer control system and raspberry pie card computer
Array passes through Local network communication;Motion sensing control module is connect with computer control system, by acquiring, identifying that user gesture reaches
Rotation, movement, zoom function to display image.User can be aobvious to scene three-dimensional high-definition in multi-angle naked eye
Show, experiences intimate true 3-D effect, and can realize good friendship by gesture control image rotation, movement, scaling
Mutually.
Detailed description of the invention
Fig. 1 is system construction drawing of the invention
Fig. 2 is multiple views light field dimension display technologies schematic illustration
Fig. 3 is that array of rear-projectors unit arranges schematic diagram
Fig. 4 is the vertical direction Projection Analysis schematic diagram based on scene rendering
Fig. 5 is the gesture identification flow chart based on FPGA
Specific embodiment
The present invention will be further explained with reference to the accompanying drawing.
The true three-dimensional display apparatus of a kind of body-sensing multiple views large scale light field, which is characterized in that the device includes vertical scattering
Film, array of rear-projectors, computer control system, card computer array, motion sensing control device and industrial camera;
The image for the known color scale pattern that the industrial camera acquired projections instrument is projected, is conveyed into computer control
System processed;Array of rear-projectors is one group of array of rear-projectors, including 16 projectors, projector are arc-shaped distribution, the circular arc half
Diameter is 2m;The interval angles of adjacent two projector are 1.36 °.The maximum resolution of projector is 1920*1080.It is sliceable multiple
Array of rear-projectors, projector number are progressively increased by 16, and projector number is no more than 64;It is described to be spliced into along arc-shaped point of projector
The circular arc of cloth continues several groups array of rear-projectors of arranging;
Every projector passes through in high-definition multimedia interface line and the card computer array in the array of rear-projectors
A card computer connection;The card computer access to LAN and computer control system progress real time video data are mutual
It passes;The array of rear-projectors is output on the vertical scattering film as display screen;
The motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera
It is identified, recognition result is inputted into computer control system.
A kind of true 3 D displaying method of body-sensing multiple views large scale light field, which is characterized in that method includes the following steps:
1) projected image based on scene rendering generates: using OpenGL to respectively to figure horizontally and vertically
As rendering the corresponding image of projector;
The image rendering of the vertical direction the following steps are included:
Define the three-dimensional point P on perpendicular determined by camera and viewpoint1=(Xp, Yp, Zp), point institute on the screen
At picture be the point p that is projected from viewpoint Oc1=(XOc, YOc, ZOc), put the normalized coordinate on the column are as follows:
The position projector optical center Op in horizontal direction are as follows: Op=(XOp, YOp, ZOp)
Horizontal coordinate are as follows: px=(Xp-Xop)/(Zp-Zop)
Point P in three-dimensional scenic1Corresponding normalized coordinate is answered on projection images are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Projector is used to project to obtain point p ' as optical center1It is rendered to obtain:
p′y=(Yp-Yop)/(Zp-Zop)
Enable Yoc=Yop, the abscissa y of the point projected to projector optical center realizes and correctly throws multiplied by a coefficient
Shadow process obtains py=p 'y(Zp-Zop)/(Zp-Zoc)。
2) projector calibrating and correction, proceed as follows every projector in array of rear-projectors:
2.1) checkerboard pattern is acquired with industrial camera, which is projected on screen;
2.2) feature detection is carried out to collected checkerboard pattern, extract angle point and calculates chessboard angle point and projector institute
Homography matrix H1 between the pattern of projection extracts the homography matrix H2 between chessboard angle point and target checkerboard pattern;
2.3) it corrects the projector list and answers square H matrix representation are as follows:
3) motion sensing control is realized:
3.1) sample database is established: by the way that commonly bending moment algorithm does not calculate the Hu not bending moments, and and hand of several certain gestures
Characteristic parameter vector of the area perimeter of gesture than constituting these template gestures.
3.2) it acquires: being acquired in user gesture input FPGA and handled by camera;
3.3) gesture is divided from background: the pretreatment denoised by image of the median filtering algorithm to input,
Then the Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using threshold value parted pattern, determines that one is cut off from according to statistical result
Value is the colour of skin within the scope of the threshold value, otherwise is the non-colour of skin, completes the binaryzation of image, gesture is split from background;
3.4) denoise: the gesture after segmentation goes noise to obtain complete image by Morphological scale-space;The Morphological scale-space
Including etch and plavini;
3.5) it obtains the image information of prominent gesture: edge detection being carried out to image with Sobel operator, and to gesture area
It is scanned, defines the perimeter that all pixels point number on the boundary line of gesture area is gesture, define in gesture area and own
Pixel number is gesture area, obtains area perimeter and compares parameter;
3.6) obtain similarity highest gesture: the Euclidean distance by calculating target to be identified and template judges target and template
Matching degree represents the point (x of target gesture feature1, x2..., xn) with represent the template gesture feature point (y1,
y2..., yn) Euclidean distance d calculation formula are as follows:
By 7 Hu of images of gestures, bending moment and area perimeter ratio are not extracted as characteristic parameter, and with corresponding to each template
Feature vector calculate Euclidean distance, template corresponding to the shortest distance is most matched gesture.
Embodiment:
The true three-dimensional display apparatus of body-sensing multiple views large scale light field, described device include vertical scattering film, array of rear-projectors,
The industrial camera of computer control system, raspberry pie card computer array, motion sensing control device and one for calibration.Projector
Every projector in array is connect by HDMI wire with a raspberry pie card computer, and each raspberry pie access to LAN passes through
TCP transmission agreement and PC carry out real time video data transmission.PC and raspberry pie card computer array to the data of required processing into
Row distributed treatment realizes transmission, processing and the simultaneous display of magnanimity light field data.Known throw is acquired by using industrial camera
Shadow image calculates homography matrix required for all projector image alignments, realizes the Geometry rectification to projector;
With the image for the known color scale pattern that industrial camera acquired projections instrument is projected, color is calculated by least square method
Transformation matrix is to realize that colour brightness is demarcated.The motion sensing control device acquires user gesture using camera, passes through FPGA reality
The identification of existing gesture, to carry out the control moved, rotate, scaled to displayed image.
The maximum resolution of the projector is 1920*1080.
It shows that screen size is 39 inches, is made of two layers of vertical scattering film.
For installation and convenient transportation, it is one group of array of rear-projectors unit that the present invention, which devises 16 projectors, when use
As long as multiple array of rear-projectors unit splicings are combined, the Three-dimensional Display of 16,32 or 64 light fields can be built into
System.Projector distribution radius of circle is 2m on the array of rear-projectors unit, and adjacent projectors interval angles are 1.36 degree.
It is corresponding with array of rear-projectors unit, every 16 raspberry pie card computers and its power supply, the network switch together as
One raspberry pie card computer array element, is mounted in plastic mould, is easily installed and transports.
The vertical scattering film is as display screen, since there are special shapes and micro-structure on its surface, to light
There are anisotropy for the reflection or transmission of line, and after laser passes through vertical scattering film, due to vertical dispersion characteristics, line beam laser will
Become fan-beam laser.This system is projected using horizontal homogeneous distribution projection instrument array to vertical scattering screen, and projector is thrown
Light out will form a special light field after vertical scattering film.We devise projected image generating algorithm and make
The special light field can produce the image of object different angle in different points of view, reach very close to the light field of real-world object
Three-dimensional Display effect.
As shown in Figure 1, body-sensing multiple views large scale light field real three-dimensional display system of the present invention, implementation process are as follows:
(1) the true Three-dimensional Display of multiple views large scale
Multiple views light field dimension display technologies are to generate 3-D effect using binocular in horizontal direction and mobile parallax.
True physics light field not only has the parallax in horizontal direction, also with the parallax of vertical direction.Due to two eyes of people
Between baseline be horizontal direction, therefore normal use habit in, light field three-dimensional display apparatus need to only generate horizontal direction
Parallax.As shown in Fig. 2, the present invention is projected using horizontal homogeneous distribution projection instrument array to vertical scattering screen, due to hanging down
Straight scattering film surface has special micro-structure, has optical anisotropy, the light that projector is launched is by vertical scattering
It will form a special light field after film.The present invention has devised a kind of projected image generating algorithm and makes the special light field very
Close to the light field of real-world object, the image of object different angle can produce in different points of view, reach Three-dimensional Display effect.
The usually vertical practical scattering angle range of scattering film is between 0.5 to 60 °, it is assumed that scattering strength is two about angle
Normal distribution is tieed up, horizontal dispersion angle is σφ=1 °, vertical scattering angle is σφ=60 °.So reflection or transmission function can be with
Writing:
Wherein functionCan two mutually independent probability distribution shown in the approximate following figure Joint Distribution letter
Number.
By the corresponding transmission function of unidirectional scattering filmThe available following light field of convolution is carried out with projection light field
Function
When being projected using multiple projected arrays, obtained light field is the linear superposition of the light field of difference.Assuming that making
It is projected with n projector, the corresponding light field of i-th of projector isProjected image is Ii(x, y), then n
The light field of the generation of projectorIt can indicate:
According to above formula, we can derive the multiple views light field Three-dimensional Display that some viewpoint Oc is observed from space
Image:
It can be seen that by the equation being derived by above for any viewpoint, multiple views light field three-dimensional display system produces
Raw image is that multiple projector images and a distribution function modulation are formed by stacking.Therefore multiple views light field Three-dimensional Display is used
Certain visual point image is shown, needs for the image of the viewpoint to be split, and projected using different projectors, i.e.,
It needs to realize using an image conversion process from collected light field to display light field.
The present invention realizes multiple views large-sized true three by the tiled display of vertical scattering film using more projectors
Dimension display.Using the high-resolution projector of small size, it is suitble to compact pose pattern, and projector distance and resolution ratio are full
Sufficient demand.As shown in figure 3, constituting an array of rear-projectors unit according to the disposing way of autonomous Design by 16 projectors, throw
Shadow instrument close-packed arrays, angle is 1.36 degree between every projector, and it is 2 meters that projector, which is distributed radius of circle,.Multiple array of rear-projectors
Unit can be spliced into array of rear-projectors to meet the needs of different display effects.
(2) based on the projection image generation method of scene rendering
The model of scene is constructed by 3DMAX, can obtain the image of any viewpoint by rendering, therefore can be used above-mentioned
Method based on multi-view image, calculates projection image collection.But this method need to render the image of different points of view, and benefit
It is handled with the method for compressed sensing, for the comparatively inefficient defect of this whole process, actually knows scene at oneself
In the case where model, the present invention directly renders the corresponding figure of projector using virtual projectors as projection centre, using OpenGL
Picture.Since the scattering of display screen has directionality, image both horizontally and vertically has heterogeneity, therefore need to be right respectively
Both horizontally and vertically image is rendered.
In vertical direction, as shown in figure 4, the three-dimensional point P on perpendicular determined by camera and viewpoint1=(xp, yp,
zp), imaging should be the point p projected from viewpoint Oc to the point on the screen1, projected by projector optical center Op
The p ' arrived1, point p1Normalized coordinate on the column are as follows:
In horizontal direction, because of the arbitrarily normalization of the corresponding column image of three-dimensional point in the plane that projector and viewpoint determine
Horizontal coordinate is identical, if projector optical center position are as follows:
Op=(XOp, YOp, ZOp)
Then the horizontal coordinate may be expressed as:
px=(Xp-Xop)/(Zp-Zop)
Therefore the point P1 in three-dimensional scenic on projection images answer by corresponding normalized coordinate are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Comprehensive Projection Analysis:
If following result will be obtained by directly using projector to carry out rendering as optical center:
px=(Xp-Xop)/(Zp-Zop)
p′y=(Yp-Yop)/(Zp-Zop)
If enabling
Yoc=Yop,
Then
py=p 'y(Zp-Zop)/(Zp-Zoc)
It knows after calculating projection coordinate, to y multiplied by a coefficient, that is, realizes correct projection process.
(3) calibration and correction of projector
Since projector angle is different, it not can guarantee installation site angle in installation process and design be completely the same,
Meanwhile there are certain parameter differences between projector, therefore after the installation is complete, need to demarcate projector.It removes
Other than geometric calibration, due to light source parameters difference, it is also necessary to carry out colour brightness calibration.
Geometric calibration is realized by being similar to projector calibrating method.Known projection figure is acquired by using industrial camera
Picture calculates homography matrix required for all projector image alignments, realizes the Geometry rectification to projector.It realizes
The correction needs of projector proceed as follows each projector:
1. known checkerboard pattern is projected on screen, the checkerboard pattern come out using industrial camera acquired projections.
2. a pair collected checkerboard pattern carries out feature detection, extracts angle point and calculate chessboard angle point and thrown with projector
Homography matrix H2 between homography matrix H1 between the pattern of shadow and chessboard angle point and target checkerboard pattern.
3. correcting the projector homography matrix can indicate are as follows:
The known coloured silk that projector progress brightness and color calibration are needed to be projected using industrial camera acquired projections instrument
Colour code determines the image of pattern.Since camera usually there will be color distortion, firstly the need of the color correction for calculating camera
Parameter HC, it is assumed that actual color CCThe collected color of camera institute is C 'C, camera can be calculated using least square method
Color correction matrix HC=C 'C+CC.After calculating the color correction matrix of camera, color can be carried out to projector and rectified
Just.Assuming that projected color is CP, by projection after by the collected color of camera be CPC, then the practical projected color of camera is C 'P
=CPCHC, colour switching matrix H is calculated by least square methodP=C 'P+CPThe correction to color can be realized.
(4) realization of motion sensing control
Body-sensing part uses homemade somatosensory device, user gesture is acquired by camera, by image transmitting into FPGA
It is handled, FPGA selects the chip of the Spartan6 series of Xilinx company, and using SP605 development kit, sensor is used
Mt9v032 is connect by FMC-LPC expansion board with FPAG, which has global exposure function, can shoot quick movement
Object, will not there are moving object inclinations as exposed light transducer line by line, in addition, this sensor performance in half-light line is super
General ccd sensor is crossed, design requirement can be met.FPGA is connected into local area network by gigabit Ethernet, and the image of identification is believed
Breath is transferred to computer control system, and control system passes through the gesture information recognized, completes the rotation to display image, moves
Dynamic, scaling.
As shown in Figure 5, the template stored in sample database is the characteristic parameter of the gesture of several fixations to specific implementation process, packet
Include the area perimeter ratio of 7 Hu not bending moment and gesture profile.Camera inputs FPGA after collecting images of gestures information, due to defeated
There is much noise in the image entered, the pretreatment that the present invention is denoised by image of the median filtering algorithm to input, then
The Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using threshold value parted pattern, a threshold value is determined according to statistical result,
It is the colour of skin within the scope of threshold value, otherwise is the non-colour of skin, the binaryzation of image is completed with this.When gesture is divided from background substantially
After out, the noise in image is removed by the Morphological scale-spaces such as corroding and expanding.After handling above, completely
Image has obtained, and in order to protrude the structure feature of gesture, simplify image information, this system is using being easier to realize in FPGA
Sobel operator carries out edge detection to image, and is scanned to gesture area, and all pixels point number is on gesture boundary line
For the perimeter of gesture, all pixels point number is gesture area in gesture area, and area perimeter is more important than as one
Characteristic parameter.Then bending moment algorithm does not calculate 7 Hu of images of gestures not bending moment for application, and with the area perimeter of front than altogether
With characteristic parameter vector is constituted, target and template are judged finally by the Euclidean distance for calculating target to be identified and each template
Matching degree represents the point (x of target gesture feature1, x2..., xn) with represent the point (y of template gesture feature1, y2..., yn)
Euclidean distance calculation formula are as follows:
It regards several characteristic parameters of images of gestures as feature vector, and calculates Europe with feature vector corresponding to each template
Formula distance, template corresponding to the shortest distance are most matched gesture.
The above is only a preferred embodiment of the present invention, it should be pointed out that: for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (6)
1. a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field, which is characterized in that the device include vertical scattering film,
Array of rear-projectors, computer control system, card computer array, motion sensing control device and industrial camera;
The image for the known color scale pattern that the industrial camera acquired projections instrument is projected is conveyed into computer control system
System;
Every projector passes through one in high-definition multimedia interface line and the card computer array in the array of rear-projectors
A card computer connection;The card computer access to LAN carries out real time video data with computer control system and mutually passes;Institute
Array of rear-projectors is stated to be output on the vertical scattering film as display screen;
The motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera and carries out
Identification, inputs computer control system for recognition result;
The array of rear-projectors is one group of array of rear-projectors, including 16 projectors, projector are arc-shaped distribution, the circular arc half
Diameter is 2m;The interval angles of adjacent two projector are 1.36 °.
2. a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field as described in claim 1, the projector is most
Big resolution ratio is 1920*1080.
3. a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field as claimed in claim 2, which is characterized in that can spell
Multiple array of rear-projectors are connect, projector number is progressively increased by 16, and projector number is no more than 64;It is described to be spliced into along projector circle
The circular arc of arc distribution continues several groups array of rear-projectors of arranging.
4. a kind of true 3 D displaying method of body-sensing multiple views large scale light field, which is characterized in that method includes the following steps:
1) projected image based on scene rendering generates: using OpenGL to respectively to image wash with watercolours horizontally and vertically
Dye the corresponding image of projector;
2) projector calibrating and correction;
3) motion sensing control is realized;
The image rendering of vertical direction described in step 1) the following steps are included:
Define the three-dimensional point P on perpendicular determined by camera and viewpoint1=(Xp,Yp,Zp), point imaging on the screen
For the point p projected from viewpoint Oc1=(XOc,YOc,ZOc), normalized coordinate of this on the column where the point are as follows:
The position projector optical center Op in horizontal direction are as follows: Op=(XOp,YOp,ZOp)
Horizontal coordinate are as follows: px=(Xp-Xop)/(Zp-Zop)
Point P in three-dimensional scenic1Corresponding normalized coordinate is answered on projection images are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Projector is used to project to obtain point p' as optical center1It is rendered to obtain:
p'y=(Yp-Yop)/(Zp-Zop)
Enable Yoc=Yop, the abscissa y of the point projected to projector optical center is multiplied by a coefficient (Zp-Zop)/(Zp-Zoc),
Realize that correct projection process obtains py=p'y(Zp-Zop)/(Zp-Zoc)。
5. a kind of true 3 D displaying method of body-sensing multiple views large scale light field as claimed in claim 4, which is characterized in that step 2)
Including being proceeded as follows to every projector in array of rear-projectors:
2.1) checkerboard pattern is acquired with industrial camera, which is projected on screen;
2.2) feature detection is carried out to collected checkerboard pattern, extract angle point and calculates chessboard angle point and is projected with projector
Pattern between homography matrix H1, extract the homography matrix H2 between chessboard angle point and target checkerboard pattern;
2.3) it corrects the projector list and answers square H matrix representation are as follows:
6. a kind of true 3 D displaying method of body-sensing multiple views large scale light field as claimed in claim 4, which is characterized in that step 3)
The following steps are included:
3.1) establish sample database: by the way that commonly bending moment algorithm does not calculate the Hu of several certain gestures not bending moment, and with gesture
Characteristic parameter vector of the area perimeter than constituting these template gestures;
3.2) it acquires: being acquired in user gesture input FPGA and handled by camera;
3.3) gesture is divided from background: the pretreatment denoised by image of the median filtering algorithm to input, then
The Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using Threshold segmentation model, a threshold value is determined according to statistical result,
It is the colour of skin in the threshold range, otherwise is the non-colour of skin, completes the binaryzation of image, gesture is split from background;
3.4) denoise: the gesture after segmentation goes noise to obtain complete image by Morphological scale-space;The Morphological scale-space includes
Etch and plavini;
3.5) it obtains the image information of prominent gesture: edge detection being carried out to image with Sobel operator, and gesture area is carried out
Scanning defines the perimeter that all pixels point number on the boundary line of gesture area is gesture, defines all pixels in gesture area
Point number is gesture area, obtains area perimeter and compares parameter;
3.6) obtain similarity highest gesture: the Euclidean distance by calculating target and template to be identified judges the matching of target and template
Degree represents the point (x of target gesture feature1,x2,...,xn) with represent the template gesture feature point (y1,y2,...,
yn) Euclidean distance d calculation formula are as follows:
By 7 Hu of images of gestures, bending moment and area perimeter ratio are not extracted as characteristic parameter, and with spy corresponding to each template
It levies vector and calculates Euclidean distance, template corresponding to the shortest distance is most matched gesture.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510325032.3A CN105739106B (en) | 2015-06-12 | 2015-06-12 | A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510325032.3A CN105739106B (en) | 2015-06-12 | 2015-06-12 | A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN105739106A CN105739106A (en) | 2016-07-06 |
CN105739106B true CN105739106B (en) | 2019-07-09 |
Family
ID=56295924
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510325032.3A Expired - Fee Related CN105739106B (en) | 2015-06-12 | 2015-06-12 | A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105739106B (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106887005B (en) * | 2017-03-03 | 2019-07-30 | 湖南大学 | A kind of blank pipe composite filter segment length's measurement method of view-based access control model detection |
CN108828893A (en) * | 2018-06-06 | 2018-11-16 | 北京邮电大学 | Three-dimensional display system based on Lenticular screen |
CN109255832A (en) * | 2018-09-21 | 2019-01-22 | 四川长虹电器股份有限公司 | A kind of target light field creation method based on OpenGL |
CN110309806B (en) * | 2019-07-08 | 2020-12-11 | 哈尔滨理工大学 | Gesture recognition system and method based on video image processing |
CN110908510B (en) * | 2019-11-08 | 2022-09-02 | 四川大学 | Application method of oblique photography modeling data in immersive display equipment |
CN111228792B (en) * | 2020-01-14 | 2023-05-05 | 深圳十米网络科技有限公司 | Motion recognition method, device, computer equipment and storage medium for motion recognition game |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102404598A (en) * | 2011-11-22 | 2012-04-04 | 浙江大学 | Image generation system and method for stereoscopic 3D display |
CN103083901A (en) * | 2011-10-31 | 2013-05-08 | 北京德信互动网络技术有限公司 | Holographic projection somatosensory interactive system and holographic projection somatosensory interactive method |
CN103440677A (en) * | 2013-07-30 | 2013-12-11 | 四川大学 | Multi-view free stereoscopic interactive system based on Kinect somatosensory device |
CN104011788A (en) * | 2011-10-28 | 2014-08-27 | 奇跃公司 | System And Method For Augmented And Virtual Reality |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102116938A (en) * | 2011-03-30 | 2011-07-06 | 浙江大学 | Panoramic viewing field three-dimensional display device based on cylindrical convergent directional screen |
KR101922722B1 (en) * | 2012-08-22 | 2018-11-27 | 엘지디스플레이 주식회사 | Stereoscopic image display device |
-
2015
- 2015-06-12 CN CN201510325032.3A patent/CN105739106B/en not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104011788A (en) * | 2011-10-28 | 2014-08-27 | 奇跃公司 | System And Method For Augmented And Virtual Reality |
CN103083901A (en) * | 2011-10-31 | 2013-05-08 | 北京德信互动网络技术有限公司 | Holographic projection somatosensory interactive system and holographic projection somatosensory interactive method |
CN102404598A (en) * | 2011-11-22 | 2012-04-04 | 浙江大学 | Image generation system and method for stereoscopic 3D display |
CN103440677A (en) * | 2013-07-30 | 2013-12-11 | 四川大学 | Multi-view free stereoscopic interactive system based on Kinect somatosensory device |
Non-Patent Citations (1)
Title |
---|
基于三维模型的多视图像采集与显示方法;龙吉昌 等;《计算机与现代化》;20150531(第237期);第57-61页 |
Also Published As
Publication number | Publication date |
---|---|
CN105739106A (en) | 2016-07-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105739106B (en) | A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method | |
CN104933718B (en) | A kind of physical coordinates localization method based on binocular vision | |
CN113412614B (en) | Three-dimensional localization using depth images | |
US10977818B2 (en) | Machine learning based model localization system | |
US20180247456A1 (en) | Method and device for augmented reality display of real physical model | |
CN107016704A (en) | A kind of virtual reality implementation method based on augmented reality | |
CN106462943A (en) | Aligning panoramic imagery and aerial imagery | |
TW201142745A (en) | Information processing apparatus, information processing system, and information processing method | |
CN105469389B (en) | A kind of grid ball target for vision sensor calibration and corresponding scaling method | |
CN101673394A (en) | content-aware wide-angle images | |
CN107111598A (en) | Use the light stream imaging system and method for ultrasonic depth sense | |
CN104599317A (en) | Mobile terminal and method for achieving 3D (three-dimensional) scanning modeling function | |
CN111275731B (en) | Projection type physical interaction desktop system and method for middle school experiments | |
CN110838164A (en) | Monocular image three-dimensional reconstruction method, system and device based on object point depth | |
CN104200476B (en) | The method that camera intrinsic parameter is solved using the circular motion in bimirror device | |
CN110880161A (en) | Depth image splicing and fusing method and system for multi-host multi-depth camera | |
CN111914790B (en) | Real-time human body rotation angle identification method based on double cameras under different scenes | |
CN108564654A (en) | The picture mode of entrance of three-dimensional large scene | |
JP6304815B2 (en) | Image processing apparatus and image feature detection method, program and apparatus thereof | |
Apartment et al. | Kinect sensor based object feature estimation in depth images | |
CN106713741B (en) | Panoramic video quality diagnosis method and device | |
Pramod et al. | Techniques in Virtual Reality | |
Chon et al. | Three-dimensional image mosaicking using multiple projection planes for 3-D visualization of roadside standing buildings | |
Fudono et al. | Interactive 3-D modeling system using a hand-held video camera | |
Tai et al. | A fully automatic approach for fisheye camera calibration |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190709 Termination date: 20210612 |
|
CF01 | Termination of patent right due to non-payment of annual fee |