CN105739106B - A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method - Google Patents

A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method Download PDF

Info

Publication number
CN105739106B
CN105739106B CN201510325032.3A CN201510325032A CN105739106B CN 105739106 B CN105739106 B CN 105739106B CN 201510325032 A CN201510325032 A CN 201510325032A CN 105739106 B CN105739106 B CN 105739106B
Authority
CN
China
Prior art keywords
gesture
projector
array
image
projectors
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201510325032.3A
Other languages
Chinese (zh)
Other versions
CN105739106A (en
Inventor
尹彦卿
沈春林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Aeronautics and Astronautics
Original Assignee
Nanjing University of Aeronautics and Astronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Aeronautics and Astronautics filed Critical Nanjing University of Aeronautics and Astronautics
Priority to CN201510325032.3A priority Critical patent/CN105739106B/en
Publication of CN105739106A publication Critical patent/CN105739106A/en
Application granted granted Critical
Publication of CN105739106B publication Critical patent/CN105739106B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B30/00Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
    • G02B30/20Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
    • G02B30/26Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the autostereoscopic type

Abstract

The invention discloses a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and methods to be conveyed into computer control system by the image for the known color scale pattern that industrial camera acquired projections instrument is projected;Every projector is connect with a card computer in the card computer array in array of rear-projectors;Card computer access to LAN carries out real time video data with computer control system and mutually passes;Array of rear-projectors is output on the vertical scattering film as display screen;Motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera and identifies, recognition result is inputted computer control system.By establishing projected light field analysis model, propose the projection image generation method based on scene rendering, user can be in multi-angle naked eye to scene Three-dimensional Display high-definition, experience intimate true 3-D effect, and good interaction can be realized by gesture control image rotation, movement, scaling.

Description

A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method
Technical field
The present invention relates to multiple views light field dimension display technologies fields, and in particular to a kind of body-sensing multiple views large scale light field True three-dimensional display apparatus and method.
Background technique
Two-dimentional display technology makes people that can not obtain the three-dimensional information of object from shown image or experiences object Physical depth, deprived the third dimensional feature of object, there is great limitation, therefore Three-dimensional Display is come into being.It is sending out There is binocular solid during exhibition and shows (anaglyph spectacles) and naked eye 3D TV etc. " puppet " dimension display technologies, such display skill Art viewpoint number is few, can only form stereoscopic effect by binocular parallax, be not carried out the mobile parallax of object substantially.
Into after the 1990s, due to the rapid development of the technologies such as laser, electronics, computer, many countries start It is active in the correlative study work of true three-dimension display technlolgies, such as the U.S., Germany, Japan, South Korea and China, and obtains phase When big progress, wherein the most significant with the achievement that the U.S. and Germany obtain.Stanford Univ USA (Stanford University the laboratory 3DTL (3D Technology Laboratories)) propose based on infra-red laser diode 3D solid display device obtains the U.S. " exploration " (Discover) and " Industry Week " (Industry Week) magazine and issues " excellent technical prize ".In the same year, USN's marine systems center (NOSC) is in U.S. Department of Defense (DARPA) and naval (Navy) Funding under be proposed body three-dimensional display apparatus of the second generation based on krypton argon ion laser, it is contemplated that be applied in shallow water area The many aspects such as Submarine Navigation, more torpedo monitoring positions and air traffic control.And United States Air Force and NASA (NASA) true 3-D technology is also paid much attention to, a large amount of funding research has been put into.Texas Instruments (TI) are in US Air Under the support of army, research by many years to laser scanning system proposes true tripleplane's technology based on DMD.GENEX Deng Duojia u s company is also dedicated to study and answers the true three-dimensional rotation surface technology with actual physical depth and the observation of full what comes into a driver's For the fields such as aviation aerospace and geoscience research.
However, fine definition real three-dimensional display system bulky complex and with high costs at present, is unable to satisfy the prior art Demand.
Summary of the invention
Goal of the invention: in order to overcome the deficiencies in the prior art, the present invention provides a kind of body-sensing multiple views large scale The true three-dimensional display apparatus of light field and method propose the perspective view based on scene rendering by establishing projected light field analysis model As generation method, solve the deficiencies in the prior art.
Technical solution: to achieve the above object, a kind of the technical solution adopted by the present invention are as follows: body-sensing multiple views large scale light The true three-dimensional display apparatus in field, which is characterized in that the device includes vertical scattering film, array of rear-projectors, computer control system, card Piece computer array, motion sensing control device and industrial camera;
The image for the known color scale pattern that the industrial camera acquired projections instrument is projected, is conveyed into computer control System processed;
Every projector passes through in high-definition multimedia interface line and the card computer array in the array of rear-projectors A card computer connection;The card computer access to LAN and computer control system progress real time video data are mutual It passes;The array of rear-projectors is output on the vertical scattering film as display screen;
The motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera It is identified, recognition result is inputted into computer control system.
Further, array of rear-projectors is one group of array of rear-projectors, including 16 projectors, and projector is arc-shaped point Cloth, the arc radius are 2m;The interval angles of adjacent two projector are 1.36 °.
Further, the maximum resolution of projector is 1920*1080.
Further, sliceable multiple array of rear-projectors, projector number are progressively increased by 16, and projector number is no more than 64 Platform;The circular arc being spliced into along the arc-shaped distribution of projector continues several groups array of rear-projectors of arranging;
A kind of true 3 D displaying method of body-sensing multiple views large scale light field, which is characterized in that method includes the following steps:
1) projected image based on scene rendering generates: using OpenGL to respectively to figure horizontally and vertically As rendering the corresponding image of projector;
2) projector calibrating and correction;
3) motion sensing control is realized.
Further, vertical direction described in step 1) image rendering the following steps are included:
Define the three-dimensional point P on perpendicular determined by camera and viewpoint1=(Xp, Yp, Zp), point institute on the screen At picture be the point p that is projected from viewpoint Oc1=(XOc, YOc, ZOc), put the normalized coordinate on the column are as follows:
The position projector optical center Op in horizontal direction are as follows: Op=(XOp, YOp, ZOp)
Horizontal coordinate are as follows: px=(Xp-Xop)/(Zp-Zop)
Point P in three-dimensional scenic1Corresponding normalized coordinate is answered on projection images are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Projector is used to project to obtain point p ' as optical center1It is rendered to obtain:
p′y=(Yp-Yop)/(Zp-Zop)
Enable Yoc=Yop, the abscissa y of the point projected to projector optical center realizes and correctly throws multiplied by a coefficient Shadow process obtains py=p 'y(Zp-Zop)/(Zp-Zoc)。
Further, step 2) includes proceeding as follows to every projector in array of rear-projectors:
2.1) checkerboard pattern is acquired with industrial camera, which is projected on screen;
2.2) feature detection is carried out to collected checkerboard pattern, extract angle point and calculates chessboard angle point and projector institute Homography matrix H1 between the pattern of projection extracts the homography matrix H2 between chessboard angle point and target checkerboard pattern;
2.3) it corrects the projector list and answers square H matrix representation are as follows:
Further, step 3) the following steps are included:
3.1) sample database is established: by the way that commonly bending moment algorithm does not calculate the Hu not bending moments, and and hand of several certain gestures Characteristic parameter vector of the area perimeter of gesture than constituting these template gestures.
3.2) it acquires: being acquired in user gesture input FPGA and handled by camera;
3.3) gesture is divided from background: the pretreatment denoised by image of the median filtering algorithm to input, Then the Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using threshold value parted pattern, determines that one is cut off from according to statistical result Value is the colour of skin within the scope of the threshold value, otherwise is the non-colour of skin, completes the binaryzation of image, gesture is split from background;
3.4) denoise: the gesture after segmentation goes noise to obtain complete image by Morphological scale-space;The Morphological scale-space Including etch and plavini;
3.5) it obtains the image information of prominent gesture: edge detection being carried out to image with Sobel operator, and to gesture area It is scanned, defines the perimeter that all pixels point number on the boundary line of gesture area is gesture, define in gesture area and own Pixel number is gesture area, obtains area perimeter and compares parameter;
3.6) obtain similarity highest gesture: the Euclidean distance by calculating target to be identified and template judges target and template Matching degree represents the point (x of target gesture feature1, x2..., xn) with represent the template gesture feature point (y1, y2..., yn) Euclidean distance d calculation formula are as follows:
By 7 Hu of images of gestures, bending moment and area perimeter ratio are not extracted as characteristic parameter, and with corresponding to each template Feature vector calculate Euclidean distance, template corresponding to the shortest distance is most matched gesture.
The utility model has the advantages that
The invention proposes body-sensing multiple views large scale light field real three-dimensional display system, described includes raspberry pie card computer Array, computer control system, array of rear-projectors and motion sensing control module.Raspberry pie card computer array carries out distributed data Processing, and will treated video information real-time display with array of rear-projectors;Computer control system and raspberry pie card computer Array passes through Local network communication;Motion sensing control module is connect with computer control system, by acquiring, identifying that user gesture reaches Rotation, movement, zoom function to display image.User can be aobvious to scene three-dimensional high-definition in multi-angle naked eye Show, experiences intimate true 3-D effect, and can realize good friendship by gesture control image rotation, movement, scaling Mutually.
Detailed description of the invention
Fig. 1 is system construction drawing of the invention
Fig. 2 is multiple views light field dimension display technologies schematic illustration
Fig. 3 is that array of rear-projectors unit arranges schematic diagram
Fig. 4 is the vertical direction Projection Analysis schematic diagram based on scene rendering
Fig. 5 is the gesture identification flow chart based on FPGA
Specific embodiment
The present invention will be further explained with reference to the accompanying drawing.
The true three-dimensional display apparatus of a kind of body-sensing multiple views large scale light field, which is characterized in that the device includes vertical scattering Film, array of rear-projectors, computer control system, card computer array, motion sensing control device and industrial camera;
The image for the known color scale pattern that the industrial camera acquired projections instrument is projected, is conveyed into computer control System processed;Array of rear-projectors is one group of array of rear-projectors, including 16 projectors, projector are arc-shaped distribution, the circular arc half Diameter is 2m;The interval angles of adjacent two projector are 1.36 °.The maximum resolution of projector is 1920*1080.It is sliceable multiple Array of rear-projectors, projector number are progressively increased by 16, and projector number is no more than 64;It is described to be spliced into along arc-shaped point of projector The circular arc of cloth continues several groups array of rear-projectors of arranging;
Every projector passes through in high-definition multimedia interface line and the card computer array in the array of rear-projectors A card computer connection;The card computer access to LAN and computer control system progress real time video data are mutual It passes;The array of rear-projectors is output on the vertical scattering film as display screen;
The motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera It is identified, recognition result is inputted into computer control system.
A kind of true 3 D displaying method of body-sensing multiple views large scale light field, which is characterized in that method includes the following steps:
1) projected image based on scene rendering generates: using OpenGL to respectively to figure horizontally and vertically As rendering the corresponding image of projector;
The image rendering of the vertical direction the following steps are included:
Define the three-dimensional point P on perpendicular determined by camera and viewpoint1=(Xp, Yp, Zp), point institute on the screen At picture be the point p that is projected from viewpoint Oc1=(XOc, YOc, ZOc), put the normalized coordinate on the column are as follows:
The position projector optical center Op in horizontal direction are as follows: Op=(XOp, YOp, ZOp)
Horizontal coordinate are as follows: px=(Xp-Xop)/(Zp-Zop)
Point P in three-dimensional scenic1Corresponding normalized coordinate is answered on projection images are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Projector is used to project to obtain point p ' as optical center1It is rendered to obtain:
p′y=(Yp-Yop)/(Zp-Zop)
Enable Yoc=Yop, the abscissa y of the point projected to projector optical center realizes and correctly throws multiplied by a coefficient Shadow process obtains py=p 'y(Zp-Zop)/(Zp-Zoc)。
2) projector calibrating and correction, proceed as follows every projector in array of rear-projectors:
2.1) checkerboard pattern is acquired with industrial camera, which is projected on screen;
2.2) feature detection is carried out to collected checkerboard pattern, extract angle point and calculates chessboard angle point and projector institute Homography matrix H1 between the pattern of projection extracts the homography matrix H2 between chessboard angle point and target checkerboard pattern;
2.3) it corrects the projector list and answers square H matrix representation are as follows:
3) motion sensing control is realized:
3.1) sample database is established: by the way that commonly bending moment algorithm does not calculate the Hu not bending moments, and and hand of several certain gestures Characteristic parameter vector of the area perimeter of gesture than constituting these template gestures.
3.2) it acquires: being acquired in user gesture input FPGA and handled by camera;
3.3) gesture is divided from background: the pretreatment denoised by image of the median filtering algorithm to input, Then the Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using threshold value parted pattern, determines that one is cut off from according to statistical result Value is the colour of skin within the scope of the threshold value, otherwise is the non-colour of skin, completes the binaryzation of image, gesture is split from background;
3.4) denoise: the gesture after segmentation goes noise to obtain complete image by Morphological scale-space;The Morphological scale-space Including etch and plavini;
3.5) it obtains the image information of prominent gesture: edge detection being carried out to image with Sobel operator, and to gesture area It is scanned, defines the perimeter that all pixels point number on the boundary line of gesture area is gesture, define in gesture area and own Pixel number is gesture area, obtains area perimeter and compares parameter;
3.6) obtain similarity highest gesture: the Euclidean distance by calculating target to be identified and template judges target and template Matching degree represents the point (x of target gesture feature1, x2..., xn) with represent the template gesture feature point (y1, y2..., yn) Euclidean distance d calculation formula are as follows:
By 7 Hu of images of gestures, bending moment and area perimeter ratio are not extracted as characteristic parameter, and with corresponding to each template Feature vector calculate Euclidean distance, template corresponding to the shortest distance is most matched gesture.
Embodiment:
The true three-dimensional display apparatus of body-sensing multiple views large scale light field, described device include vertical scattering film, array of rear-projectors, The industrial camera of computer control system, raspberry pie card computer array, motion sensing control device and one for calibration.Projector Every projector in array is connect by HDMI wire with a raspberry pie card computer, and each raspberry pie access to LAN passes through TCP transmission agreement and PC carry out real time video data transmission.PC and raspberry pie card computer array to the data of required processing into Row distributed treatment realizes transmission, processing and the simultaneous display of magnanimity light field data.Known throw is acquired by using industrial camera Shadow image calculates homography matrix required for all projector image alignments, realizes the Geometry rectification to projector; With the image for the known color scale pattern that industrial camera acquired projections instrument is projected, color is calculated by least square method Transformation matrix is to realize that colour brightness is demarcated.The motion sensing control device acquires user gesture using camera, passes through FPGA reality The identification of existing gesture, to carry out the control moved, rotate, scaled to displayed image.
The maximum resolution of the projector is 1920*1080.
It shows that screen size is 39 inches, is made of two layers of vertical scattering film.
For installation and convenient transportation, it is one group of array of rear-projectors unit that the present invention, which devises 16 projectors, when use As long as multiple array of rear-projectors unit splicings are combined, the Three-dimensional Display of 16,32 or 64 light fields can be built into System.Projector distribution radius of circle is 2m on the array of rear-projectors unit, and adjacent projectors interval angles are 1.36 degree.
It is corresponding with array of rear-projectors unit, every 16 raspberry pie card computers and its power supply, the network switch together as One raspberry pie card computer array element, is mounted in plastic mould, is easily installed and transports.
The vertical scattering film is as display screen, since there are special shapes and micro-structure on its surface, to light There are anisotropy for the reflection or transmission of line, and after laser passes through vertical scattering film, due to vertical dispersion characteristics, line beam laser will Become fan-beam laser.This system is projected using horizontal homogeneous distribution projection instrument array to vertical scattering screen, and projector is thrown Light out will form a special light field after vertical scattering film.We devise projected image generating algorithm and make The special light field can produce the image of object different angle in different points of view, reach very close to the light field of real-world object Three-dimensional Display effect.
As shown in Figure 1, body-sensing multiple views large scale light field real three-dimensional display system of the present invention, implementation process are as follows:
(1) the true Three-dimensional Display of multiple views large scale
Multiple views light field dimension display technologies are to generate 3-D effect using binocular in horizontal direction and mobile parallax. True physics light field not only has the parallax in horizontal direction, also with the parallax of vertical direction.Due to two eyes of people Between baseline be horizontal direction, therefore normal use habit in, light field three-dimensional display apparatus need to only generate horizontal direction Parallax.As shown in Fig. 2, the present invention is projected using horizontal homogeneous distribution projection instrument array to vertical scattering screen, due to hanging down Straight scattering film surface has special micro-structure, has optical anisotropy, the light that projector is launched is by vertical scattering It will form a special light field after film.The present invention has devised a kind of projected image generating algorithm and makes the special light field very Close to the light field of real-world object, the image of object different angle can produce in different points of view, reach Three-dimensional Display effect.
The usually vertical practical scattering angle range of scattering film is between 0.5 to 60 °, it is assumed that scattering strength is two about angle Normal distribution is tieed up, horizontal dispersion angle is σφ=1 °, vertical scattering angle is σφ=60 °.So reflection or transmission function can be with Writing:
Wherein functionCan two mutually independent probability distribution shown in the approximate following figure Joint Distribution letter Number.
By the corresponding transmission function of unidirectional scattering filmThe available following light field of convolution is carried out with projection light field Function
When being projected using multiple projected arrays, obtained light field is the linear superposition of the light field of difference.Assuming that making It is projected with n projector, the corresponding light field of i-th of projector isProjected image is Ii(x, y), then n The light field of the generation of projectorIt can indicate:
According to above formula, we can derive the multiple views light field Three-dimensional Display that some viewpoint Oc is observed from space Image:
It can be seen that by the equation being derived by above for any viewpoint, multiple views light field three-dimensional display system produces Raw image is that multiple projector images and a distribution function modulation are formed by stacking.Therefore multiple views light field Three-dimensional Display is used Certain visual point image is shown, needs for the image of the viewpoint to be split, and projected using different projectors, i.e., It needs to realize using an image conversion process from collected light field to display light field.
The present invention realizes multiple views large-sized true three by the tiled display of vertical scattering film using more projectors Dimension display.Using the high-resolution projector of small size, it is suitble to compact pose pattern, and projector distance and resolution ratio are full Sufficient demand.As shown in figure 3, constituting an array of rear-projectors unit according to the disposing way of autonomous Design by 16 projectors, throw Shadow instrument close-packed arrays, angle is 1.36 degree between every projector, and it is 2 meters that projector, which is distributed radius of circle,.Multiple array of rear-projectors Unit can be spliced into array of rear-projectors to meet the needs of different display effects.
(2) based on the projection image generation method of scene rendering
The model of scene is constructed by 3DMAX, can obtain the image of any viewpoint by rendering, therefore can be used above-mentioned Method based on multi-view image, calculates projection image collection.But this method need to render the image of different points of view, and benefit It is handled with the method for compressed sensing, for the comparatively inefficient defect of this whole process, actually knows scene at oneself In the case where model, the present invention directly renders the corresponding figure of projector using virtual projectors as projection centre, using OpenGL Picture.Since the scattering of display screen has directionality, image both horizontally and vertically has heterogeneity, therefore need to be right respectively Both horizontally and vertically image is rendered.
In vertical direction, as shown in figure 4, the three-dimensional point P on perpendicular determined by camera and viewpoint1=(xp, yp, zp), imaging should be the point p projected from viewpoint Oc to the point on the screen1, projected by projector optical center Op The p ' arrived1, point p1Normalized coordinate on the column are as follows:
In horizontal direction, because of the arbitrarily normalization of the corresponding column image of three-dimensional point in the plane that projector and viewpoint determine Horizontal coordinate is identical, if projector optical center position are as follows:
Op=(XOp, YOp, ZOp)
Then the horizontal coordinate may be expressed as:
px=(Xp-Xop)/(Zp-Zop)
Therefore the point P1 in three-dimensional scenic on projection images answer by corresponding normalized coordinate are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Comprehensive Projection Analysis:
If following result will be obtained by directly using projector to carry out rendering as optical center:
px=(Xp-Xop)/(Zp-Zop)
p′y=(Yp-Yop)/(Zp-Zop)
If enabling
Yoc=Yop,
Then
py=p 'y(Zp-Zop)/(Zp-Zoc)
It knows after calculating projection coordinate, to y multiplied by a coefficient, that is, realizes correct projection process.
(3) calibration and correction of projector
Since projector angle is different, it not can guarantee installation site angle in installation process and design be completely the same, Meanwhile there are certain parameter differences between projector, therefore after the installation is complete, need to demarcate projector.It removes Other than geometric calibration, due to light source parameters difference, it is also necessary to carry out colour brightness calibration.
Geometric calibration is realized by being similar to projector calibrating method.Known projection figure is acquired by using industrial camera Picture calculates homography matrix required for all projector image alignments, realizes the Geometry rectification to projector.It realizes The correction needs of projector proceed as follows each projector:
1. known checkerboard pattern is projected on screen, the checkerboard pattern come out using industrial camera acquired projections.
2. a pair collected checkerboard pattern carries out feature detection, extracts angle point and calculate chessboard angle point and thrown with projector Homography matrix H2 between homography matrix H1 between the pattern of shadow and chessboard angle point and target checkerboard pattern.
3. correcting the projector homography matrix can indicate are as follows:
The known coloured silk that projector progress brightness and color calibration are needed to be projected using industrial camera acquired projections instrument Colour code determines the image of pattern.Since camera usually there will be color distortion, firstly the need of the color correction for calculating camera Parameter HC, it is assumed that actual color CCThe collected color of camera institute is C 'C, camera can be calculated using least square method Color correction matrix HC=C 'C+CC.After calculating the color correction matrix of camera, color can be carried out to projector and rectified Just.Assuming that projected color is CP, by projection after by the collected color of camera be CPC, then the practical projected color of camera is C 'P =CPCHC, colour switching matrix H is calculated by least square methodP=C 'P+CPThe correction to color can be realized.
(4) realization of motion sensing control
Body-sensing part uses homemade somatosensory device, user gesture is acquired by camera, by image transmitting into FPGA It is handled, FPGA selects the chip of the Spartan6 series of Xilinx company, and using SP605 development kit, sensor is used Mt9v032 is connect by FMC-LPC expansion board with FPAG, which has global exposure function, can shoot quick movement Object, will not there are moving object inclinations as exposed light transducer line by line, in addition, this sensor performance in half-light line is super General ccd sensor is crossed, design requirement can be met.FPGA is connected into local area network by gigabit Ethernet, and the image of identification is believed Breath is transferred to computer control system, and control system passes through the gesture information recognized, completes the rotation to display image, moves Dynamic, scaling.
As shown in Figure 5, the template stored in sample database is the characteristic parameter of the gesture of several fixations to specific implementation process, packet Include the area perimeter ratio of 7 Hu not bending moment and gesture profile.Camera inputs FPGA after collecting images of gestures information, due to defeated There is much noise in the image entered, the pretreatment that the present invention is denoised by image of the median filtering algorithm to input, then The Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using threshold value parted pattern, a threshold value is determined according to statistical result, It is the colour of skin within the scope of threshold value, otherwise is the non-colour of skin, the binaryzation of image is completed with this.When gesture is divided from background substantially After out, the noise in image is removed by the Morphological scale-spaces such as corroding and expanding.After handling above, completely Image has obtained, and in order to protrude the structure feature of gesture, simplify image information, this system is using being easier to realize in FPGA Sobel operator carries out edge detection to image, and is scanned to gesture area, and all pixels point number is on gesture boundary line For the perimeter of gesture, all pixels point number is gesture area in gesture area, and area perimeter is more important than as one Characteristic parameter.Then bending moment algorithm does not calculate 7 Hu of images of gestures not bending moment for application, and with the area perimeter of front than altogether With characteristic parameter vector is constituted, target and template are judged finally by the Euclidean distance for calculating target to be identified and each template Matching degree represents the point (x of target gesture feature1, x2..., xn) with represent the point (y of template gesture feature1, y2..., yn) Euclidean distance calculation formula are as follows:
It regards several characteristic parameters of images of gestures as feature vector, and calculates Europe with feature vector corresponding to each template Formula distance, template corresponding to the shortest distance are most matched gesture.
The above is only a preferred embodiment of the present invention, it should be pointed out that: for the ordinary skill people of the art For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered It is considered as protection scope of the present invention.

Claims (6)

1. a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field, which is characterized in that the device include vertical scattering film, Array of rear-projectors, computer control system, card computer array, motion sensing control device and industrial camera;
The image for the known color scale pattern that the industrial camera acquired projections instrument is projected is conveyed into computer control system System;
Every projector passes through one in high-definition multimedia interface line and the card computer array in the array of rear-projectors A card computer connection;The card computer access to LAN carries out real time video data with computer control system and mutually passes;Institute Array of rear-projectors is stated to be output on the vertical scattering film as display screen;
The motion sensing control device includes camera and FPGA, identifies that the gesture of user is transferred in FPGA by camera and carries out Identification, inputs computer control system for recognition result;
The array of rear-projectors is one group of array of rear-projectors, including 16 projectors, projector are arc-shaped distribution, the circular arc half Diameter is 2m;The interval angles of adjacent two projector are 1.36 °.
2. a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field as described in claim 1, the projector is most Big resolution ratio is 1920*1080.
3. a kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field as claimed in claim 2, which is characterized in that can spell Multiple array of rear-projectors are connect, projector number is progressively increased by 16, and projector number is no more than 64;It is described to be spliced into along projector circle The circular arc of arc distribution continues several groups array of rear-projectors of arranging.
4. a kind of true 3 D displaying method of body-sensing multiple views large scale light field, which is characterized in that method includes the following steps:
1) projected image based on scene rendering generates: using OpenGL to respectively to image wash with watercolours horizontally and vertically Dye the corresponding image of projector;
2) projector calibrating and correction;
3) motion sensing control is realized;
The image rendering of vertical direction described in step 1) the following steps are included:
Define the three-dimensional point P on perpendicular determined by camera and viewpoint1=(Xp,Yp,Zp), point imaging on the screen For the point p projected from viewpoint Oc1=(XOc,YOc,ZOc), normalized coordinate of this on the column where the point are as follows:
The position projector optical center Op in horizontal direction are as follows: Op=(XOp,YOp,ZOp)
Horizontal coordinate are as follows: px=(Xp-Xop)/(Zp-Zop)
Point P in three-dimensional scenic1Corresponding normalized coordinate is answered on projection images are as follows:
px=(Xp-Xop)/(Zp-Zop)
py=(Yp-Yoc)/(Zp-Zoc)
Projector is used to project to obtain point p' as optical center1It is rendered to obtain:
p'y=(Yp-Yop)/(Zp-Zop)
Enable Yoc=Yop, the abscissa y of the point projected to projector optical center is multiplied by a coefficient (Zp-Zop)/(Zp-Zoc), Realize that correct projection process obtains py=p'y(Zp-Zop)/(Zp-Zoc)。
5. a kind of true 3 D displaying method of body-sensing multiple views large scale light field as claimed in claim 4, which is characterized in that step 2) Including being proceeded as follows to every projector in array of rear-projectors:
2.1) checkerboard pattern is acquired with industrial camera, which is projected on screen;
2.2) feature detection is carried out to collected checkerboard pattern, extract angle point and calculates chessboard angle point and is projected with projector Pattern between homography matrix H1, extract the homography matrix H2 between chessboard angle point and target checkerboard pattern;
2.3) it corrects the projector list and answers square H matrix representation are as follows:
6. a kind of true 3 D displaying method of body-sensing multiple views large scale light field as claimed in claim 4, which is characterized in that step 3) The following steps are included:
3.1) establish sample database: by the way that commonly bending moment algorithm does not calculate the Hu of several certain gestures not bending moment, and with gesture Characteristic parameter vector of the area perimeter than constituting these template gestures;
3.2) it acquires: being acquired in user gesture input FPGA and handled by camera;
3.3) gesture is divided from background: the pretreatment denoised by image of the median filtering algorithm to input, then The Hand Gesture Segmentation based on the colour of skin is carried out in YCbCr space, using Threshold segmentation model, a threshold value is determined according to statistical result, It is the colour of skin in the threshold range, otherwise is the non-colour of skin, completes the binaryzation of image, gesture is split from background;
3.4) denoise: the gesture after segmentation goes noise to obtain complete image by Morphological scale-space;The Morphological scale-space includes Etch and plavini;
3.5) it obtains the image information of prominent gesture: edge detection being carried out to image with Sobel operator, and gesture area is carried out Scanning defines the perimeter that all pixels point number on the boundary line of gesture area is gesture, defines all pixels in gesture area Point number is gesture area, obtains area perimeter and compares parameter;
3.6) obtain similarity highest gesture: the Euclidean distance by calculating target and template to be identified judges the matching of target and template Degree represents the point (x of target gesture feature1,x2,...,xn) with represent the template gesture feature point (y1,y2,..., yn) Euclidean distance d calculation formula are as follows:
By 7 Hu of images of gestures, bending moment and area perimeter ratio are not extracted as characteristic parameter, and with spy corresponding to each template It levies vector and calculates Euclidean distance, template corresponding to the shortest distance is most matched gesture.
CN201510325032.3A 2015-06-12 2015-06-12 A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method Expired - Fee Related CN105739106B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510325032.3A CN105739106B (en) 2015-06-12 2015-06-12 A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510325032.3A CN105739106B (en) 2015-06-12 2015-06-12 A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method

Publications (2)

Publication Number Publication Date
CN105739106A CN105739106A (en) 2016-07-06
CN105739106B true CN105739106B (en) 2019-07-09

Family

ID=56295924

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510325032.3A Expired - Fee Related CN105739106B (en) 2015-06-12 2015-06-12 A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method

Country Status (1)

Country Link
CN (1) CN105739106B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106887005B (en) * 2017-03-03 2019-07-30 湖南大学 A kind of blank pipe composite filter segment length's measurement method of view-based access control model detection
CN108828893A (en) * 2018-06-06 2018-11-16 北京邮电大学 Three-dimensional display system based on Lenticular screen
CN109255832A (en) * 2018-09-21 2019-01-22 四川长虹电器股份有限公司 A kind of target light field creation method based on OpenGL
CN110309806B (en) * 2019-07-08 2020-12-11 哈尔滨理工大学 Gesture recognition system and method based on video image processing
CN110908510B (en) * 2019-11-08 2022-09-02 四川大学 Application method of oblique photography modeling data in immersive display equipment
CN111228792B (en) * 2020-01-14 2023-05-05 深圳十米网络科技有限公司 Motion recognition method, device, computer equipment and storage medium for motion recognition game

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102404598A (en) * 2011-11-22 2012-04-04 浙江大学 Image generation system and method for stereoscopic 3D display
CN103083901A (en) * 2011-10-31 2013-05-08 北京德信互动网络技术有限公司 Holographic projection somatosensory interactive system and holographic projection somatosensory interactive method
CN103440677A (en) * 2013-07-30 2013-12-11 四川大学 Multi-view free stereoscopic interactive system based on Kinect somatosensory device
CN104011788A (en) * 2011-10-28 2014-08-27 奇跃公司 System And Method For Augmented And Virtual Reality

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102116938A (en) * 2011-03-30 2011-07-06 浙江大学 Panoramic viewing field three-dimensional display device based on cylindrical convergent directional screen
KR101922722B1 (en) * 2012-08-22 2018-11-27 엘지디스플레이 주식회사 Stereoscopic image display device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104011788A (en) * 2011-10-28 2014-08-27 奇跃公司 System And Method For Augmented And Virtual Reality
CN103083901A (en) * 2011-10-31 2013-05-08 北京德信互动网络技术有限公司 Holographic projection somatosensory interactive system and holographic projection somatosensory interactive method
CN102404598A (en) * 2011-11-22 2012-04-04 浙江大学 Image generation system and method for stereoscopic 3D display
CN103440677A (en) * 2013-07-30 2013-12-11 四川大学 Multi-view free stereoscopic interactive system based on Kinect somatosensory device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于三维模型的多视图像采集与显示方法;龙吉昌 等;《计算机与现代化》;20150531(第237期);第57-61页

Also Published As

Publication number Publication date
CN105739106A (en) 2016-07-06

Similar Documents

Publication Publication Date Title
CN105739106B (en) A kind of true three-dimensional display apparatus of body-sensing multiple views large scale light field and method
CN104933718B (en) A kind of physical coordinates localization method based on binocular vision
CN113412614B (en) Three-dimensional localization using depth images
US10977818B2 (en) Machine learning based model localization system
US20180247456A1 (en) Method and device for augmented reality display of real physical model
CN107016704A (en) A kind of virtual reality implementation method based on augmented reality
CN106462943A (en) Aligning panoramic imagery and aerial imagery
TW201142745A (en) Information processing apparatus, information processing system, and information processing method
CN105469389B (en) A kind of grid ball target for vision sensor calibration and corresponding scaling method
CN101673394A (en) content-aware wide-angle images
CN107111598A (en) Use the light stream imaging system and method for ultrasonic depth sense
CN104599317A (en) Mobile terminal and method for achieving 3D (three-dimensional) scanning modeling function
CN111275731B (en) Projection type physical interaction desktop system and method for middle school experiments
CN110838164A (en) Monocular image three-dimensional reconstruction method, system and device based on object point depth
CN104200476B (en) The method that camera intrinsic parameter is solved using the circular motion in bimirror device
CN110880161A (en) Depth image splicing and fusing method and system for multi-host multi-depth camera
CN111914790B (en) Real-time human body rotation angle identification method based on double cameras under different scenes
CN108564654A (en) The picture mode of entrance of three-dimensional large scene
JP6304815B2 (en) Image processing apparatus and image feature detection method, program and apparatus thereof
Apartment et al. Kinect sensor based object feature estimation in depth images
CN106713741B (en) Panoramic video quality diagnosis method and device
Pramod et al. Techniques in Virtual Reality
Chon et al. Three-dimensional image mosaicking using multiple projection planes for 3-D visualization of roadside standing buildings
Fudono et al. Interactive 3-D modeling system using a hand-held video camera
Tai et al. A fully automatic approach for fisheye camera calibration

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20190709

Termination date: 20210612

CF01 Termination of patent right due to non-payment of annual fee