CN106060523B - Panoramic stereo image acquisition, display methods and corresponding device - Google Patents

Panoramic stereo image acquisition, display methods and corresponding device Download PDF

Info

Publication number
CN106060523B
CN106060523B CN201610500635.7A CN201610500635A CN106060523B CN 106060523 B CN106060523 B CN 106060523B CN 201610500635 A CN201610500635 A CN 201610500635A CN 106060523 B CN106060523 B CN 106060523B
Authority
CN
China
Prior art keywords
image
eye region
camera unit
region image
flat
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610500635.7A
Other languages
Chinese (zh)
Other versions
CN106060523A (en
Inventor
孙立
候永竞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Hongxiang Technical Service Co Ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Qizhi Software Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd, Qizhi Software Beijing Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201610500635.7A priority Critical patent/CN106060523B/en
Publication of CN106060523A publication Critical patent/CN106060523A/en
Application granted granted Critical
Publication of CN106060523B publication Critical patent/CN106060523B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/296—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Studio Devices (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The present invention relates to a kind of panoramic stereo image acquisition method and device and display methods and devices.The method and step that stereo-picture is generated in panoramic stereo image acquisition device is as follows: receiving the flat image for being no less than the acquisition of three pairs of camera units；It flat image is processed into can be used for three-dimensional imaging left eye region image and right eye region image in conjunction with the attitude information of camera unit；Synchronization left eye region image and right eye region image are merged into same data frame and are sent to display device/server.Display equipment shows that the method and step of stereo-picture is as follows: the data by obtaining sensor determine attitude information, scan received area image, call corresponding area image to show on the screen of a display device according to the label information of area image.The present invention improves the image quality and efficiency of transmission of stereo-picture.Processing and efficiency of transmission are improved in server or display terminal, the power consumption of display terminal is especially saved, improves user experience.

Description

Panoramic stereo image acquisition, display methods and corresponding device
Technical field
The present invention relates to panoramic stereo image field of cameras, specifically, the present invention relates to a kind of panoramic stereo images Acquisition, display methods and its corresponding device.
Background technique
In recent years, virtual reality technology is quickly grown, and has become new development in science and technology trend in the short time.360 degree It is three-dimensional complete to be widely used in the virtual guide to visitors in tourist attractions, urban environment for the one kind of panoramic stereo image as virtual reality technology Scape is virtual, navigation streetscape is virtual etc., has very actual application value, the three-dimensional environment of virtual reality needs Image Acquisition Equipment can be shot with 360 degrees omnidirection, this greatly increases the technical requirements of shooting.
In the prior art, method is to acquire several to cover entire visible space in different direction by single camera Flat image or video, panoramic stereo image is generated after the processing of stereo-picture processing software, in the method, since user claps Shooting direction and moving direction when taking the photograph is irregular, and therefore, the panoramic stereo image being processed into can generate defect or distortion；It is another Method is to carry out image taking to different directions simultaneously in fixed position using multiple video cameras, in the method, due to each camera shooting Machine is independently shot, and captured flat image needs later period otherwise processed, at high cost and can not show and set in virtual reality at any time Standby middle viewing.
Summary of the invention
Primary and foremost purpose of the invention is intended to provide a kind of method that panoramic stereo image acquires and its corresponding original device
Another object of the present invention is to provide a kind of method that panoramic stereo image is shown and its corresponding original devices.
To achieve the goals above, the present invention provides a kind of panoramic stereo image acquisition method, include the following steps:
Receive each camera unit flat image collected at least three pairs of camera units of covering panoramic viewing angle；
It the flat image is processed into is used for the left eye region of three-dimensional imaging in conjunction with the attitude information of camera unit Image and right eye region image；
Left eye region image and right eye region image are associated and are sent to display device/server.
Preferably, at least three pairs of camera units acquire the flat image for surrounding same circumference range simultaneously, and each pair of camera shooting is single Two camera units in member acquire the two width plan views for being suitable for being respectively formed left eye region image and right eye region image respectively Picture.
Preferably, each camera unit flat image collected camera unit adjacent thereto planar image sections collected Overlapping.Generally, camera unit is more, and the flat image lap of acquisition is bigger, and the part cut is smaller, produces The raw stereo-picture distortion factor is lower.
Preferably, the camera unit flat image collected in each pair of camera unit positioned at the left side is used to form left eye area Area image, the camera unit flat image collected positioned at the right are to be used to form right eye region image.
The above-mentioned implementation using multipair binocular camera shooting unit simulation people's eyes acquisition image, due to binocular camera shooting unit Two width flat images are acquired from different angles, therefore are substantially the detections adjusted the distance, and are calculated by binocular real-time deep, it is extensive It appears again three-dimensional geometric information, thus generates stereoscopic vision.
Preferably, flat image is processed into the left eye area for being used for three-dimensional imaging in conjunction with the attitude information of camera unit The step of area image and right eye region image, specifically includes:
Each pair of camera unit two sheet of planar image collected is cut out according to binocular stereo imaging principle, forms phase Corresponding left eye region image and right eye region image, wherein left eye region image and right eye region image at least part scape Object content overlaps；
The corresponding relationship for establishing the attitude information of each area image and its corresponding camera unit makes to show equipment end It can match and the area image of corresponding attitude information is called to be shown in plane two sides.
Preferably, flat image is processed into the left eye area for being used for three-dimensional imaging in conjunction with the attitude information of camera unit The step of area image and right eye region image, specifically includes:
Flat image acquired in the camera unit of the same side of each pair of camera unit is spliced into panoramic plane image；
According to binocular stereo imaging principle, two width panoramic plane images will be formed by and handled, it is respectively become Left eye region image and right eye region image；
Attitude information based on camera unit establishes the specific part of left eye region image and right eye region image Between corresponding relationship, enable show equipment end match and the area image of corresponding attitude information called to be shown in plane two Side.
Preferably, by left eye region image and right eye region image associated the step of being sent to display equipment, pass through The left eye region image and right eye region image that synchronization is obtained and generated, which are merged into same data frame, to be transmitted and realizes It is associated.
Preferably, display equipment is that mobile terminal or virtual reality head show equipment.
Preferably, in the same data frame, include what is be correspondingly arranged according to the occupied spatial position of camera unit The attitude information of left eye region image and right eye region image.
Preferably, attitude information is indicated with label information.
Preferably, camera unit is in photograph mode to obtain the flat image in real time, and this method is according to camera unit Frame rate under the image pickup mode obtains each frame flat image for camera unit and corresponds to circulation and execute once.
Preferably, this method is configured as receiving four pairs of camera unit flat images collected of covering panoramic viewing angle, Wherein, flat image acquired in each camera unit has the space wide-angle greater than 90 degree.
Second aspect, the present invention provides a kind of panoramic stereo image acquisition devices, including at least three pairs or more camera shootings are single Member, for acquiring the image of covering panoramic viewing angle, this device includes:
Receiving module, each camera unit at least three pairs of camera units for receiving covering panoramic viewing angle are acquired Flat image；
The flat image is processed into for the attitude information in conjunction with camera unit and is used to stand by image processing module The left eye region image and right eye region image of body imaging；
Transmission module, for being associated left eye region image and right eye region image and being sent to display equipment/service Device.
Preferably, at least three pairs of camera units are equidistantly surrounded on the outside of device, and acquisition surrounds the plane of same circumference range Image, two camera units in each pair of camera unit acquire respectively suitably forms left eye region image and right eye region image Two width flat images.
Preferably, each camera unit flat image collected camera unit adjacent thereto planar image sections collected Overlapping.
Preferably, the camera unit flat image collected in each pair of camera unit positioned at left side is used to form left eye area Area image, the camera unit flat image collected positioned at right side are to be used to form right eye region image.
Preferably, image processing module includes:
Unit is cut, each pair of camera unit two sheet of planar image collected is cut according to binocular stereo imaging principle It cuts out, forms corresponding left eye region image and right eye region image, wherein left eye region image and right eye region image are at least There is part scene content to overlap；
Unit is indicated, the corresponding relationship of the attitude information of each area image and its corresponding camera unit is established.
Preferably, image processing module includes:
Concatenation unit, for helping the splicing of flat image acquired in the camera unit of the same side of each pair of camera unit Scape flat image；
Processing unit will be formed by two width panoramic plane images and handle, make it according to binocular stereo imaging principle Respectively become left eye region image and right eye region image；
Unit is indicated, the attitude information based on the camera unit establishes left eye region image and right eye region image Specific part between corresponding relationship.
Preferably, transmission module includes:
Combining unit, the left eye region image and right eye region image that synchronization is obtained and generated are merged into together Association is realized in one data frame；
Associated rear left Vitrea eye area image and right eye region image are sent to display device/server by transmission unit.
Preferably, display equipment is that mobile terminal or virtual reality head show equipment.
Preferably, in same data frame, include the left eye area being correspondingly arranged according to the occupied spatial position of camera unit The attitude information of area image and right eye region image.
Preferably, attitude information is indicated with label information.
Preferably, camera unit is in photograph mode to obtain the flat image in real time, takes the photograph according to camera unit at this As the frame rate under mode, the corresponding circulation of method of each frame flat image panoramic stereo image acquisition is obtained for camera unit It executes primary.
Preferably, the present apparatus is configured with four pairs of camera units of covering panoramic viewing angle, wherein acquired in each camera unit Flat image have greater than 90 degree of space wide-angle.
The third aspect, the present invention provides a kind of methods for showing panoramic stereo image, include the following steps:
Sensing data is obtained, determines attitude information according to sensing data；
Received data frame is parsed, the left eye region corresponding with attitude information wherein loaded is obtained Image and right eye region image；
Left eye region image corresponding with attitude information and right eye region image are respectively corresponded into display in screen Two sides.
Specifically, it obtains in sensing data step, by obtaining sensor measurement current spatial azimuth information and the number of people Portion's motion trace data estimates next attitude information, believes next attitude information as the dimensional orientation determined Breath.
Specifically, received data frame is parsed, acquisition wherein loads corresponding with attitude information The step of left eye region image and right eye region image, specifically includes:
The attitude information according to measured by sensor reads the area image in received data frame；
According to the label information of area image, left eye corresponding with attitude information measured by sensor area is determined Area image and right eye region image；
Call the corresponding left eye region image of the attitude information and right eye region image.
Specifically, left eye region image corresponding with attitude information and right eye region image are respectively corresponded into display In the step of screen sides are shown, display device screen is equally divided into left hand display area and right hand display area, the left side Vitrea eye area image is shown in left hand display area, and right eye region image is shown in right hand display area.
Specifically, display equipment receives data frame according to the frame rate under camera unit image pickup mode, for passing through sensing Each attitude information that device data determine, shows one data frame of the every reception of equipment, and the corresponding circulation of this method executes primary.
Fourth aspect, the present invention provides a kind of device for showing panoramic stereo image, this device includes control unit and shows Display screen, the control unit specifically include:
Module is obtained, for obtaining sensing data, determines attitude information according to sensing data；
Parsing module, for parsing to received data frame, what acquisition wherein loaded believes with the dimensional orientation The corresponding left eye region image of manner of breathing and right eye region image；
Display module, for dividing the left eye region image corresponding with attitude information and right eye region image Screen sides Dui Ying be shown in.
Preferably, next by obtaining sensor measurement current spatial azimuth information and the estimation of head part's motion trace data Attitude information, using next attitude information as the attitude information determined.
Preferably, parsing module includes:
Reading unit reads the region in received data frame for the attitude information according to measured by sensor Image；
Matching unit, for the label information according to area image, determination and attitude information measured by sensor Corresponding left eye region image and right eye region image；
Extraction unit, for calling left eye region image corresponding with the attitude information and right eye region image.
Preferably, display device screen is equally divided into left hand display area and right hand display area, the left eye region figure As being shown in left hand display area, right eye region image is shown in right hand display area.
Preferably, display equipment receives data frame according to the frame rate under camera unit image pickup mode, for passing through sensing Each attitude information that device data determine shows one data frame of the every reception of equipment, shows the method pair of panoramic stereo image It is primary that execution should be recycled.
Compared with prior art, the solution of the invention has the following advantages:
The present invention is based on binocular imaging principles, obtain full-view stereo around same circumference by three pairs or more of camera units The basic material of image, is sent to recipient after preliminary treatment, for keeping in or directly displaying.Since it is from hardware view Solve the problems, such as that the picture material of three-dimensional imaging obtains, therefore, the image of output just can be subject to without replication processes It utilizes, improves the image quality and efficiency of transmission of stereo-picture.Further, server or display terminal also can be larger The processing time is saved in degree, improves processing and efficiency of transmission, more particularly to save the power consumption of display terminal, shown by raising Stereo-picture fluency and whole image quality, improve user experience.
The additional aspect of the present invention and advantage will be set forth in part in the description, these will become from the following description Obviously, or practice through the invention is recognized.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached Figure.
Fig. 1 is the flow chart of one embodiment of the invention panoramic stereo image generation method
Fig. 2 is that one embodiment of the invention camera unit covers field of view block diagram
Fig. 3 is one embodiment of the invention panoramic stereo image acquisition device structural block diagram
Fig. 4 is the module map of one embodiment of the invention panoramic stereo image acquisition device central processor CPU
Fig. 5 is the detailed block diagram of the image processing module of one embodiment of the invention
Fig. 6 is the detailed block diagram of the image processing module of another embodiment of the present invention
Fig. 7 is the detailed block diagram of one embodiment of the invention transmission module
Fig. 8 is the flow chart for the method that one embodiment of the invention shows panoramic stereo image
Fig. 9 is the module map for the control unit that one embodiment of the invention shows equipment
Figure 10 is the detailed block diagram of one embodiment of the invention parsing module
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described.
In some processes of the description in description and claims of this specification and above-mentioned attached drawing, contain according to Multiple operations that particular order occurs, but it should be clearly understood that these operations can not be what appears in this article suitable according to its Sequence is executed or is executed parallel, and serial number of operation such as 101,102 etc. is only used for distinguishing each different operation, serial number It itself does not represent and any executes sequence.In addition, these processes may include more or fewer operations, and these operations can To execute or execute parallel in order.It should be noted that the description such as " first " herein, " second ", is for distinguishing not Same message, equipment, module etc., does not represent sequencing, does not also limit " first " and " second " and be different type.
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, those skilled in the art's every other implementation obtained without creative efforts Example, shall fall within the protection scope of the present invention.
The proper noun used in the present invention is explained as follows:
Left eye region image, right eye region image, in the present invention, left eye region image and right eye region image refer to Flat image can be used to generate after treatment the image that stereoscopic vision is shown in display equipment.
Attitude information, in the present invention, attitude information refer to orientation letter when camera unit acquires picture Breath.
Data frame, the protocol Data Unit of data link layer, including three parts: frame head, data portion, postamble.Frame head and Postamble includes some necessary control information, such as synchronizing information, address information, error control information etc.；Data portion then wraps Containing the data that network layer hands down, such as IP data packet.
Frame rate refers to shown frozen frozen mass lattice number per second.
Sensing data, the present invention in refer to the attitude information that sensor detects.
In a first aspect, providing a kind of method stream of panoramic stereo image acquisition in one embodiment in the present invention Journey figure is as shown in Figure 1, comprising:
S310: each camera unit plan view collected at least three pairs of camera units of covering panoramic viewing angle is received Picture.
It will be understood by those skilled in the art that at least three pairs of camera units are in equidistantly to be arranged in acquisition device around array Outside can refer to Fig. 2 by taking four pairs of camera units as an example, the acquired flat image of each camera unit acquisition, which has, to be greater than 90 degree of space wide-angle, the sum of space wide-angle of flat image acquired in 8 camera units are greater than 360 degree, that is, cover panorama Visual angle.
Specifically, at least three pairs of camera units acquire the flat image for surrounding same circumference range simultaneously, and each pair of camera shooting is single Two camera units in member acquire the two width plan views for being suitable for being respectively formed left eye region image and right eye region image respectively Picture.It will be understood by those skilled in the art that the logarithm of camera unit is more within the scope of certain space angle, the plane of acquisition Image lap is more, and 3 D stereo reduction degree is higher.
Specifically, each camera unit flat image collected camera unit adjacent thereto planar image sections collected Overlapping.It will be understood by those skilled in the art that obtaining the three-dimensional geometric information of captured image according to triangle principle, the left side is taken the photograph As any point on unit flat image collected can on the right camera unit acquire flat image same level Match point is found on polar curve, is constructed triangle relation, is extrapolated three-dimensional relationship i.e. solid space relationship.
Specifically, the camera unit flat image collected in each pair of camera unit positioned at the left side is used to form left eye area Area image, the camera unit flat image collected positioned at the right are to be used to form right eye region image.
Specifically, this method is configured as receiving four pairs of camera units plan view collected of covering panoramic viewing angle Picture, wherein flat image acquired in each camera unit has the space wide-angle greater than 90 degree.Those skilled in the art can be with Understand, when there is four pairs of camera units, flat image acquired in each camera unit has the space wide-angle greater than 90 degree, when When having N to camera unit, flat image acquired in each camera unit has the space wide-angle greater than 360/N degree, as a result, respectively Camera unit flat image collected can cover panoramic viewing angle.
The first purpose of this method is capturing panoramic view image, therefore is acquired simultaneously using at least three pairs of camera units around same The flat image of one circumference range, each camera unit flat image collected camera unit adjacent thereto plan view collected Picture partly overlaps and the sum of shooting visual angle of each camera unit is greater than 360 degree, and the flat image of acquisition covers panoramic viewing angle.We The second purpose of method is to generate stereo-picture, two camera units according to binocular stereo imaging principle, in each pair of camera unit Acquisition is suitable for being respectively formed two width flat images of left eye region image and right eye region image respectively.Position in each pair of camera unit Camera unit flat image collected in the left side is used to form left eye region image, and the camera unit positioned at the right is acquired Flat image be used to form right eye region image.Due to the left side camera unit and right side camera shooting list in each pair of camera unit Member acquires two width flat images from different angles, therefore is substantially the detection adjusted the distance, and is calculated by binocular real-time deep Three-dimensional geometric information is recovered, stereoscopic vision is thus generated.
S320: it the flat image is processed into is used for the left eye of three-dimensional imaging in conjunction with the attitude information of camera unit Area image and right eye region image.
In the present invention, multipair camera unit is in equidistantly to be fixed on the outside of device around array, therefore, each camera unit With specific dimensional orientation, the attitude information refers to the space side of corresponding camera unit when every sheet of planar image generates Position.
Specifically, process provides a kind of is processed into flat image on the left eye region image for being used for three-dimensional imaging and the right side The specific embodiment of Vitrea eye area image, specific steps include:
Each pair of camera unit two sheet of planar image collected is cut out according to binocular stereo imaging principle, forms phase Corresponding left eye region image and right eye region image, wherein left eye region image and right eye region image at least part scape Object content overlaps；
Establish the corresponding relationship of the attitude information of each area image and its corresponding camera unit.
It will be understood by those skilled in the art that binocular stereo imaging is the principle based on parallax, respectively with left and right two take the photograph World coordinate system progress camera coordinate system imaging as unit to target object, recycles corresponding in camera unit coordinate system Coordinate and video camera central point distance, focal length carry out mathematics conversion, target point true three-dimension information is found out with this.Acquire mesh It needs that reflection object is finally realized in the detection of the image progress distance of different angle shooting with different angles when punctuate The three-dimensional information of body.Binocular image is usually all to acquire one under different parallax angles by two camera units of the same level The two images of a target object.
Generally, the flat image and right side camera unit with left side camera unit acquisition in a pair of of camera unit acquire Flat image has most of region all to essentially coincide, however due to left side camera unit and right side camera unit position Viewpoint it is inconsistent, left side camera unit acquisition flat image and right side camera unit acquisition flat image have certainly centainly Deviation, i.e. there is the region that is not overlapped certainly in two images.Typically, a left side for left side camera unit flat image collected Edge is that right side camera unit flat image collected is unexistent, and the right side of right side camera unit flat image collected Edge is that camera unit flat image collected in side is unexistent, i.e., above-mentioned two marginal portion is the region not being overlapped, and is needed It punctures, two marginal portions of stereo-picture of not so subsequent synthesis are still two-dimensional flat image.Therefore, implement herein In example, in same a pair of of camera unit left side camera unit and right side camera unit flat image collected carry out image and cut Sanction includes: between the flat image for respectively acquiring left side camera unit and the flat image of right side camera unit acquisition without weight The part of conjunction punctures.What is punctured is usually left edge part and the right side camera shooting of the flat image of left side camera unit acquisition The right hand edge part of the flat image of unit acquisition.Flat image after cutting respectively becomes left eye region image and right eye region Image.The area of visual field of left eye region image and right eye region image is completely coincident, but due to left eye region image and Right eye region image be respectively in a pair of of camera unit left side camera unit and right side camera unit acquire and generate, because And left side camera unit and right side camera unit have certain parallax, left eye region image and will have Vitrea eye domain in display The sight of image is isolated, i.e., left eye only sees left eye region image, has eye only to see right eye region image, and there are parallaxes for two width as a result, Area image pass through brain synthesis, at the eyes of user generate stereo-picture.
Due to needing to be merged into the area image for acquiring and generating in the same time in same data frame simultaneously root in subsequent step The sensing data matching obtained according to display equipment to the left eye region image and right eye region image with attitude information, because This, needs to indicate area image with opposite with acquisition and the attitude information of camera unit of formation zone image It answers.It is indicated in a manner of label information in the present embodiment, i.e., includes mark attitude information in each data frame Field.By taking four pairs of camera units as an example, fixation is corresponding with a dimensional orientation to eight camera units respectively, therefore has eight skies Between azimuth information, it is assumed that the field is there are three byte, and 000 represents the dimensional orientation of first camera unit, and 001 represents second The dimensional orientation of camera unit is pushed away with this example, and the dimensional orientation of eight camera units has corresponding numerical value to be corresponding to it.When aobvious When showing that equipment needs to call the area image to camera unit such as the upper right side Fig. 2, according to receiving and parsing through data frame acceptance of the bid Show that the label information of the attitude information of area image just can find out the camera unit of corresponding direction in space and acquire and generate Area image.In the present invention, to the administrative division map for establishing camera unit attitude information and camera unit is acquired and generated The technology of corresponding relationship is without limitation as between, however not excluded that has using can establish the technology of the corresponding relationship.
Specifically, process provides second flat image is processed into be used for three-dimensional imaging left eye region image and The specific embodiment of right eye region image, specific steps include:
Flat image acquired in the camera unit of the same side of each pair of camera unit is spliced into panoramic plane image；
According to binocular stereo imaging principle, two width panoramic plane images will be formed by and handled, it is respectively become Left eye region image and right eye region image；
Attitude information based on the camera unit establishes the specific portion of left eye region image Yu right eye region image / corresponding relationship.
It will be understood by those skilled in the art that generally, by extracted left area image mosaic at left side panorama sketch Picture, by the right area image mosaic of extraction at right side panoramic picture, Block- matching splicing method is can be used in the method for splicing, first Feature point extraction is carried out to two width adjacent images, the big of formwork is then determined in the first sub-picture according to the position of characteristic point It is small, search range is determined in the second width image, then carries out matching primitives, completes image mosaic.It is single with four pairs of camera shootings shown in Fig. 2 For member, four flat images of camera unit acquisition on the left of synchronization are spliced, first extract piece image and the The characteristic point of two width images, according to the position of characteristic point piece image determining module plate size, it is true in the second width image Determine search range, carry out matching primitives, completes the splicing of the first width flat image and the second width flat image, third width plan view Picture and the 4th width image repeat this process, and the flat image by fourth officer shooting in different direction covering panoramic viewing angle is spliced into one Width panoramic plane image.In the present invention, without limitation to joining method image, however not excluded that using the splicing based on grid, base The flat image of acquisition is spliced in the methods of splicing splicing related to the phase based on FFT in ratio method.
It will be understood by those skilled in the art that binocular stereo imaging is the principle based on parallax, respectively with left and right two take the photograph World coordinate system progress camera coordinate system imaging as unit to target object, recycles corresponding in camera unit coordinate system Coordinate and video camera central point distance, focal length carry out mathematics conversion, target point true three-dimension information is found out with this.Acquire mesh It needs that reflection object is finally realized in the detection of the image progress distance of different angle shooting with different angles when punctuate The three-dimensional information of body.Binocular image is usually all to acquire one under different parallax angles by two camera units of the same level The two images of a target object.
Generally, it after the flat image that the camera unit of the same side acquires being spliced to form panoramic plane image, needs Panoramic plane image is further processed the left eye region image and right eye region image that could be formed for display.Example Such as, image enhancement is carried out, image not can increase original image information generally, but can be convex by weak signal for part image-forming condition It is aobvious, so that image information is easier to differentiate.Image enhancement includes frequency domain and airspace two parts, and spatial domain method is mainly in image Each pixel is operated；Frequency domain method is modified in a transform domain of image if Fourier coefficient, DTC transformation coefficient are to figure As being operated.In the present invention, the processing to panoramic plane image not only includes image enhancement technique, however not excluded that uses other The other technologies for facilitating the area image that panoramic plane image is generated for display three-dimensional imaging, as image stitching, image melt Conjunction technology etc..
Further, due to needing the area image for acquiring and generating in the same time being merged into same data in subsequent step In frame and according to display equipment obtain sensing data matching to attitude information left eye region image and right eye area Specific part in area image, therefore, it is necessary to relationship of the area image specifically between part indicated with acquisition simultaneously The attitude information of the camera unit of formation zone image is corresponding.It is marked in a manner of label information in the present embodiment Show, i.e., includes the field for indicating attitude information in each data frame.By taking four pairs of camera units as an example, eight camera shootings are single Fixation is corresponding with a dimensional orientation to member respectively, therefore has eight attitude informations, it is assumed that byte that there are three the fields, 000 The dimensional orientation of first camera unit is represented, 001 represents the dimensional orientation of second camera unit, pushes away with this example, and eight are taken the photograph As the dimensional orientation of unit has corresponding numerical value to be corresponding to it.When display equipment needs to call if this of the upper right side Fig. 2 is to taking the photograph As unit area image when, according to receive and parse through in data frame indicate area image attitude information label information Just the image that the camera unit of corresponding direction in space is acquired and generated can be found out, which is one of the area image of panorama Point.In the present invention, to the technology for establishing corresponding relationship between specific part in camera unit attitude information and area image Without limitation, however not excluded that have using the technology that can establish the corresponding relationship.
S330: left eye region image and right eye region image are associated and are sent to display device/server.
It will be understood by those skilled in the art that the panoramic stereo image acquisition device in the present invention is in captured in real-time mould Formula, therefore the image of magnanimity will be generated, intermediate transit point of the server as network has the big feature of memory capacity, will Image transmitting to server stores, and is conducive to the management and data sharing of image, and display equipment can be by sending a request to service Device, the area image that server will respond its corresponding dimensional orientation of request transmission are shown into display equipment.
It will be understood by those skilled in the art that the flat image that panoramic stereo image acquisition device acquires in the present invention can be straight It connects to be transmitted in display equipment and show.
Specifically, it is merged by the left eye region image and right eye region image that synchronization is obtained and generated It transmits and realizes associated in same data frame.It will be understood by those skilled in the art that by the left eye region image of synchronization It is merged into same data frame transmit with right eye region image and helps to improve data processing speed.
Specifically, in same data frame, include the left side being correspondingly arranged according to the occupied spatial position of camera unit The attitude information of Vitrea eye area image and right eye region image.
Specifically, the attitude information is indicated with label information.It will be understood by those skilled in the art that each It include the field for indicating attitude information, i.e. label information in data frame.
Specifically, the camera unit is in photograph mode to obtain the flat image in real time, and this method is according to camera shooting Frame rate of the unit under the image pickup mode obtains each frame flat image for camera unit and corresponds to circulation and execute once. It will be understood by those skilled in the art that typical frame rate is that can generate flat under this frame rate 24 frames/second~30 frames/second Sliding and continuous effect.
Second aspect provides a kind of use of panoramic stereo image acquisition device 1000 in one embodiment of the invention In executing panoramic stereo image generation method described in first aspect, refering to what is shown in Fig. 3, including no less than three pairs of camera shooting list Member 100, for acquiring the flat image of covering panoramic viewing angle.
It will be understood by those skilled in the art that at least three pairs of camera units are equidistantly arranged in circular array and acquisition device Outside can refer to Fig. 2 by taking four pairs of camera units as an example, the acquired flat image of each camera unit acquisition, which has, to be greater than 90 degree of space wide-angle, the sum of space wide-angle of flat image acquired in 8 camera units are greater than 360 degree, and referred to as covering is entirely Scape visual angle.
Specifically, at least three pairs of camera units are equidistantly surrounded on the outside of device, and acquisition is around same circumference range Flat image, two camera units in each pair of camera unit acquire respectively suitably forms left eye region image and right eye region figure Two width flat images of picture.It will be understood by those skilled in the art that within the scope of certain space angle camera unit logarithm More, the flat image lap of acquisition is more, and 3 D stereo reduction degree is higher.
Specifically, each camera unit flat image collected camera unit adjacent thereto planar image sections collected Overlapping.It will be understood by those skilled in the art that obtaining the three-dimensional geometric information of captured image according to triangle principle, the left side is taken the photograph As any point on unit flat image collected can on the right camera unit acquire flat image same level Match point is found on polar curve, is constructed triangle relation, is extrapolated three-dimensional relationship i.e. solid space relationship.
Specifically, the camera unit flat image collected in each pair of camera unit positioned at left side is used to form left eye area Area image, the camera unit flat image collected positioned at right side are to be used to form right eye region image.
Specifically, four pairs of camera units configured with the flat image that can acquire covering panoramic viewing angle in the present apparatus, wherein Flat image acquired in each camera unit has the space wide-angle greater than 90 degree.It will be understood by those skilled in the art that working as When having four pairs of camera units, flat image acquired in each camera unit has greater than 90 degree of space wide-angle, when having N to taking the photograph When as unit, flat image acquired in each camera unit has the space wide-angle greater than 360/N degree, and each camera shooting is single as a result, Member flat image collected can cover panoramic viewing angle.
The first purpose of the present apparatus is capturing panoramic view image, therefore is acquired simultaneously using at least three pairs of camera units around same The flat image of one circumference range, each camera unit flat image collected camera unit adjacent thereto plan view collected Picture partly overlaps and the sum of shooting visual angle of each camera unit is greater than 360 degree, and the flat image of acquisition covers panoramic viewing angle.This dress The second purpose set is to generate stereo-picture, two camera units according to binocular stereo imaging principle, in each pair of camera unit Acquisition is suitable for being respectively formed two width flat images of left eye region image and right eye region image respectively.Position in each pair of camera unit Camera unit flat image collected in the left side is used to form left eye region image, and the camera unit positioned at the right is acquired Flat image be used to form right eye region image.Due to the left side camera unit and right side camera shooting list in each pair of camera unit Member acquires two width flat images from different angles, therefore is substantially the detection adjusted the distance, and is calculated by binocular real-time deep Three-dimensional geometric information is recovered, stereoscopic vision is thus generated.
The present apparatus further includes central processor CPU 200.It is mainly used for receiving the flat image of camera unit acquisition, to institute Received flat image is further handled and is incorporated into same data frame transfer to display equipment or server.It is taken the photograph with four Duis As for unit, each camera unit 100 acquires flat image of the space wide-angle greater than 90 degree and is transmitted to central processing unit CPU200, central processing unit 200 handle received flat image, and synchronization is generated left eye region image and the right side Vitrea eye area image is simultaneously merged into same data frame, is transmitted to display equipment end or server.
Specifically, central processor CPU 200 includes:
Receiving module 201, each camera unit institute at least three pairs of camera units for receiving covering panoramic viewing angle The flat image of acquisition；It will be understood by those skilled in the art that at least three pairs of camera units be in equidistantly arrange around array and The outside of acquisition device can refer to Fig. 2, the acquired flat image of each camera unit acquisition by taking four pairs of camera units as an example With the space wide-angle for being greater than 90 degree, the sum of the space wide-angle of flat image acquired in 8 camera units is greater than 360 degree, i.e., Cover panoramic viewing angle.
The flat image is processed into use for the attitude information in conjunction with camera unit by image processing module 202 In the left eye region image and right eye region image of three-dimensional imaging；In the present invention, multipair camera unit is in equidistant around array It is fixed on the outside of device, therefore, each camera unit has specific dimensional orientation, and the attitude information refers to every sheet of planar The dimensional orientation of image corresponding camera unit when generating.
Transmission module 203, for being associated left eye region image and right eye region image and being sent to display equipment/clothes Business device.It will be understood by those skilled in the art that the panoramic stereo image acquisition device in the present invention is in captured in real-time mode, because This will generate the image of magnanimity, and intermediate transit point of the server as network has the big feature of memory capacity, image is passed Server storage is transported to, the management and data sharing of image are conducive to, display equipment can be serviced by sending a request to server Device shows the area image that respond request transmits corresponding dimensional orientation into display equipment.
It will be understood by those skilled in the art that the flat image that panoramic stereo image acquisition device acquires in the present invention can be straight It connects to be transmitted in display equipment and show.
Further, image processing module includes:
Cut unit 2021, by each pair of camera unit two sheet of planar image collected according to binocular stereo imaging principle into Row is cut out, and forms corresponding left eye region image and right eye region image, wherein left eye region image and right eye region image At least part scene content overlaps；
Generally, the flat image and right side camera unit with left side camera unit acquisition in a pair of of camera unit acquire Flat image has most of region all to essentially coincide, however due to left side camera unit and right side camera unit position Viewpoint it is inconsistent, the flat image of left side camera unit acquisition and the flat image of right side camera unit acquisition exist certain Certainly there is the region not being overlapped in deviation, i.e. two images.Typically, the left side of left side camera unit flat image collected Edge is that right side camera unit flat image collected is unexistent, and the right of right side camera unit flat image collected Edge is that camera unit flat image collected in side is unexistent, i.e., above-mentioned two marginal portion is the region not being overlapped, and is needed It punctures, two marginal portions of stereo-picture of not so subsequent synthesis are still two-dimensional flat image.Therefore, in this embodiment In, to the left side camera unit and right side camera unit flat image progress image cutting-out collected in same a pair of of camera unit It include: that will be not overlapped between the flat image that left side camera unit acquires and the flat image of right side camera unit acquisition respectively Part puncture.What is punctured is usually left edge part and the right side camera shooting list of the flat image of left side camera unit acquisition The right hand edge part of the flat image of member acquisition.Flat image after cutting respectively becomes left eye region image and right eye region figure Picture.The area of visual field of left eye region image and right eye region image is completely coincident, but due to left eye region image and the right side Vitrea eye area image be respectively in a pair of of camera unit left side camera unit and right side camera unit acquire and generate, thus Left side camera unit and right side camera unit have certain parallax again, left eye region image and will have an administrative division map in display The sight of picture is isolated, i.e., left eye only sees left eye region image, has eye only to see right eye region image, and there are parallaxes for two width as a result, Area image passes through the synthesis of brain, and stereo-picture is generated at the eyes of user.
Unit 2022 is indicated, the corresponding of the attitude information of each area image and its corresponding camera unit is established and closes System.Due to needing that the area image for acquiring and generating in the same time is merged into same data frame and according to display in subsequent step The sensing data that equipment obtains is matched to the left eye region image and right eye region image with attitude information, therefore, is needed Area image is indicated with corresponding with acquisition and the attitude information of camera unit of formation zone image.This reality It applies in example and is indicated in a manner of label information.It include the field for indicating attitude information in i.e. each data frame. By taking four pairs of camera units as an example, fixation is corresponding with a dimensional orientation to eight camera units respectively, therefore has eight dimensional orientations Information, it is assumed that the field represents the dimensional orientation of first camera unit there are three byte, 000, and 001 represents second camera shooting list The dimensional orientation of member, is pushed away, the dimensional orientation of eight camera units has corresponding numerical value to be corresponding to it with this example.When display equipment When needing to call the area image to camera unit such as the upper right side Fig. 2, region is indicated in data frame according to receiving and parsing through The label information of the attitude information of image just can find out the region that the camera unit of corresponding direction in space is acquired and generated Image.In the present invention, to establishing camera unit attitude information and camera unit acquires and between the area image that generates The technology of corresponding relationship is without limitation, however not excluded that has using the technology that can establish the corresponding relationship.
Further, image processing module includes:
Concatenation unit 2023, for splicing flat image acquired in the camera unit of the same side of each pair of camera unit At panorama flat image；
It will be understood by those skilled in the art that generally, by extracted left area image mosaic at left side panorama sketch Picture, by the right area image mosaic of extraction at right side panoramic picture, Block- matching splicing method is can be used in the method for splicing, first Feature point extraction is carried out to two width adjacent images, the big of formwork is then determined in the first sub-picture according to the position of characteristic point It is small, search range is determined in the second width image, then carries out matching primitives, completes image mosaic.It is single with four pairs of camera shootings shown in Fig. 2 For member, four flat images of camera unit acquisition on the left of synchronization are spliced, first extract piece image and the The characteristic point of two width images, according to the position of characteristic point piece image determining module plate size, it is true in the second width image Determine search range, carry out matching primitives, completes the splicing of the first width flat image and the second width flat image, third width plan view Picture and the 4th width image repeat this process, and the flat image by fourth officer shooting in different direction covering panoramic viewing angle is spliced into one Width panoramic plane image.In the present invention, without limitation to the working method of the concatenation unit of image, however not excluded that using being based on grid Splicing, based on the splicing of ratio method it is related to the phase based on FFT splicing the methods of the flat image of acquisition is spliced.
Processing unit 2024 will be formed by two width panoramic plane images and handle according to binocular stereo imaging principle, It is set to respectively become left eye region image and right eye region image；
It will be understood by those skilled in the art that binocular stereo imaging is the principle based on parallax, respectively with left and right two take the photograph World coordinate system progress camera coordinate system imaging as unit to target object, recycles corresponding in camera unit coordinate system Coordinate and video camera central point distance, focal length carry out mathematics conversion, target point true three-dimension information is found out with this.Acquire mesh It needs that reflection object is finally realized in the detection of the image progress distance of different angle shooting with different angles when punctuate The three-dimensional information of body.Binocular image is usually all to acquire one under different parallax angles by two camera units of the same level The two images of a target object.
Generally, it after the flat image that the camera unit of the same side acquires being spliced to form panoramic plane image, needs Panoramic plane image is further processed the left eye region image and right eye region image that could be formed for display.Example Such as, image enhancement is carried out, image not can increase original image information generally, but can be convex by weak signal for part image-forming condition It is aobvious, so that image information is easier to differentiate.Image enhancement includes frequency domain and airspace two parts, and spatial domain method is mainly in image Each pixel is operated；Frequency domain method is modified in a transform domain of image if Fourier coefficient, DTC transformation coefficient are to figure As being operated.In the present invention, the processing to panoramic plane image not only includes image enhancement technique, however not excluded that uses other The other technologies for facilitating the area image that panoramic plane image is generated for display three-dimensional imaging, as image stitching, image melt Conjunction technology etc..
Unit 2025 is indicated, the attitude information based on the camera unit establishes left eye region image and right eye region Corresponding relationship between the specific part of image.
Further, due to needing the area image for acquiring and generating in the same time being merged into same data in subsequent step In frame and according to display equipment obtain sensing data matching to attitude information left eye region image and right eye area Specific part in area image, therefore, it is necessary to relationship of the area image specifically between part indicated with acquisition simultaneously The attitude information of the camera unit of formation zone image is corresponding.It is marked in a manner of label information in the present embodiment Show, i.e., includes the field for indicating attitude information in each data frame.By taking four pairs of camera units as an example, eight camera shootings are single Fixation is corresponding with a dimensional orientation to member respectively, therefore has eight attitude informations, it is assumed that byte that there are three the fields, 000 The dimensional orientation of first camera unit is represented, 001 represents the dimensional orientation of second camera unit, pushes away with this example, and eight are taken the photograph As the dimensional orientation of unit has corresponding numerical value to be corresponding to it.When display equipment needs to call if this of the upper right side Fig. 2 is to taking the photograph As unit area image when, according to receive and parse through in data frame indicate area image attitude information label information Just the image that the camera unit of corresponding direction in space is acquired and generated can be found out, which is one of the area image of panorama Point.In the present invention, to the technology for establishing corresponding relationship between specific part in camera unit attitude information and area image Without limitation, however not excluded that have using the technology that can establish the corresponding relationship.
Further, transmission module includes:
Combining unit 2031, the left eye region image and right eye region image that synchronization is obtained and generated merge Association is realized into same data frame；It will be understood by those skilled in the art that by the left eye region image of synchronization and the right side Vitrea eye area image, which is merged into same data frame transmit, helps to improve data processing speed.
Associated rear left Vitrea eye area image and right eye region image are sent to display equipment/service by transmission unit 2032 Device.It will be understood by those skilled in the art that the panoramic stereo image acquisition device in the present invention is in captured in real-time mode, therefore The image of magnanimity will be generated, intermediate transit point of the server as network has the big feature of memory capacity, by image transmitting It is stored to server, is conducive to the management and data sharing of image, display equipment can be by sending a request to server, server The area image that respond request transmits corresponding dimensional orientation is shown into display equipment.
Specifically, the display equipment is that mobile terminal or virtual reality head show equipment.
Specifically, in the same data frame, include the institute being correspondingly arranged according to the occupied spatial position of camera unit The attitude information of the left eye region image and right eye region image stated.
Specifically, the attitude information is indicated with label information.It will be understood by those skilled in the art that each It include the field for indicating attitude information, i.e. label information in data frame.
Further, the camera unit 100 is in photograph mode to obtain the flat image in real time, according to camera shooting list Frame rate of the member under the image pickup mode, the method for obtaining each frame flat image panoramic stereo image acquisition for camera unit Corresponding circulation executes primary.It will be understood by those skilled in the art that typical frame rate is 24 frames/second~30 frames/second, in this frame Under rate, effect smoothly and continuously can be generated.
Preferably, the present apparatus is configured with four pairs of camera units of covering panoramic viewing angle, wherein acquired in each camera unit Flat image have greater than 90 degree of space wide-angle.It will be understood by those skilled in the art that when there is N to camera unit, often Flat image acquired in a camera unit has the space wide-angle greater than 360/N degree, and each camera unit is collected flat as a result, Face image can cover panoramic viewing angle.
Further, CPU processor 200 further includes communication module 204 and memory module 205.
Communication module 204 is mainly used for executing the data communication with display device/server, specifically includes:
Landline communication unit, Landline communication unit can be to display equipment and provide the HDMI unit or USB unit of interface；
Wireless communication unit, wireless communication unit include close communication unit and telecommunication unit, are closely led to Letter unit can be at least one of WiFi communication unit, infrared communication unit or bluetooth-communication unit；Telecommunication unit It can be at least one of cellular communication, 3G mobile comm unit or 4G mobile comm unit.
Memory module 205 is mainly used to the left side for storing each camera unit flat image collected and being formed after processing Vitrea eye area image and right eye region image.
The third aspect, the present invention also provides a kind of methods for showing panoramic stereo image, as shown in Figure 8, comprising:
S410: obtaining sensing data, determines attitude information according to sensing data；
Specifically, next by obtaining sensor measurement current spatial azimuth information and the estimation of head part's motion trace data Attitude information, using next attitude information as the attitude information determined.
The motion-sensing data are to show multiple samplings of at least one aspect sensor acquisition in equipment moving process The data of point, head part's motion profile are formed by connecting the data of multiple sampled points.The life of head part's motion profile At method in accordance with the method connected point into line, sampled point is more, then the preliminary track generated is more accurate.In the present embodiment, raw The method of adult head motion profile is to connect point into line the motion-sensing data of multiple sampled points, generates a head movement rail The motion-sensing data of mark are either the data that single sensor acquires, are also possible to the data of multiple sensors acquisitions.Example Such as, being had altogether on the display device there are two inertial sensor and a direction sensor, inertial sensor is gyroscope and acceleration Sensor is spent, by the state of the available equipment of track motion change data of inertial sensor, direction sensor is magnetic force Meter can detecte out absolute geographic direction using magnetometer and change as the course bearing of the determination space operation track Data, then first head movement track can be fitted by the data for multiple points that acceleration transducer acquires generates；Article 2 Head movement track can be fitted life by the data of multiple points of the aggregation of data magnetometer acquisition for multiple points that gyroscope acquires At；Article 3 head movement track can be fitted by the aggregation of data magnetometer for multiple points that acceleration transducer acquires and be generated. After a plurality of head movement track has finally been determined, space operation track and sky can be determined according to the extent of deviation of a plurality of track Between azimuth information.
S420: parsing received data frame, and acquisition wherein loads corresponding with the attitude information Left eye region image and right eye region image.
Specifically, described that received data frame is parsed, it obtains wherein loading with the attitude information The specific steps of corresponding left eye region image and right eye region image include:
The attitude information according to measured by sensor reads the area image in received data frame；
According to the label information of area image, left eye corresponding with attitude information measured by sensor area is determined Area image and right eye region image；
Call the corresponding left eye region image of the attitude information and right eye region image.
Generally, in the data frame of inclusion region image also simultaneously the corresponding attitude information of inclusion region image, such as It is preceding described, the field of the attitude information of inclusion region image, i.e. label information are assigned in data frame, thus, it to obtain The information such as area image and its corresponding attitude information need to parse the information in data frame, generally, packet Include data frame format parsing and the parsing of data frame parameter.For example, the analysis mode that tree structure can be used solves data frame The characteristics of analysis, is first abstracted data handling procedure, condition resolution layer-by-layer according to data handling procedure lists tree structure. Initial data frame obtains frame parameter and data to be resolved, analytic tree will be selected according to frame parameter by parsing root vertex parsing The resolution path of next step；Data to be resolved are given to two node layer of analytic tree to continue to parse；The rest may be inferred, successively parses.? This resolving, the parsing comprising the attitude information to area image each in data frame.Parsing of the present invention to data frame Mode is without limitation, however not excluded that other are suitble to the analysis mode of image data frame parsing, the data frame driven such as formalized description Parsing and chain structure data frame analyzing etc..
It will be understood by those skilled in the art that determining region corresponding with the data space azimuth information of sensor measurement After image, display equipment can load the area image automatically and the area image is called to show on the display screen.
S430: the left eye region image corresponding with attitude information and right eye region image are respectively corresponded aobvious Show in screen sides.
Specifically, the left eye region image corresponding with attitude information and right eye region image are respectively corresponded In the step of being shown in screen sides display, display device screen is equally divided into left hand display area and right hand display area, institute It states left eye region image and is shown in left hand display area, right eye region image is shown in right hand display area.Those skilled in the art Member is it is appreciated that the present invention constructs three-dimensional imaging using parallax method, according to binocular stereo imaging principle, need left eye and The sight of right eye is isolated, and so that left eye and right eye is watched the two width area images with parallax respectively, after brain synthesizes, two Stereoscopic vision is generated at eye.
Specifically, display equipment receives data frame according to the frame rate under camera unit image pickup mode, for passing through sensing Each attitude information that device data determine shows one data frame of the every reception of equipment, shows the method pair of panoramic stereo image It is primary that execution should be recycled.It will be understood by those skilled in the art that frame speed of the display equipment according to camera unit under image pickup mode Rate receives image, shows on a display screen by the rate after treatment, and typical frame rate is 24 frames/second~30 frames/second, Under this frame rate, effect smoothly and continuously could be generated.
Fourth aspect, the present invention also provides a kind of devices for showing panoramic stereo image, vertical for executing display panorama The method of body image, described device include control unit 500 and display screen, and control unit 500 described in reference diagram 9 specifically includes:
Module 510 is obtained, for obtaining sensing data, determines attitude information according to sensing data；
The sensor can be inertial sensor and direction sensor.
Inertial sensor can by the track motion change data of inertial sensor for gyroscope and acceleration transducer To obtain the state of equipment.
Direction sensor can detecte out absolute geographic direction using magnetometer and as the determination space operation rail The course bearing delta data of mark.
Specifically, next by obtaining sensor measurement current spatial azimuth information and the estimation of head part's motion trace data Attitude information, using next attitude information as the attitude information determined.Generally, the motion-sensing number According to the data for multiple sampled points of at least one aspect sensor acquisition in display equipment moving process, head part's movement Track is formed by connecting the data of multiple sampled points.In the present apparatus, sensor generates the mode of head part's motion profile It is to connect point into line the motion-sensing data of multiple sampled points, generating the motion-sensing data of a head motion profile both can be with It is the data of single sensor acquisition, is also possible to the data of multiple sensor acquisitions.For example, one sharing two on the display device A inertial sensor and a direction sensor, then first head movement track can be acquired multiple by acceleration transducer The data of point, which are fitted, to be generated；The aggregation of data magnetometer for multiple points that Article 2 head movement track can be acquired by gyroscope is adopted The data of multiple points of collection, which are fitted, to be generated；The number for multiple points that Article 3 head movement track can be acquired by acceleration transducer It is fitted and generates according to comprehensive magnetic meter.It, can be according to the extent of deviation of a plurality of track after a plurality of head movement track has finally been determined Determine space operation track and attitude information.
It will be understood by those skilled in the art that producing head movement rail by connecting point into line multiple sensing datas Mark, sampled point is more, then head movement track is more accurate, can determine corresponding attitude information by the motion profile.
Parsing module 520 obtains wherein loading with the dimensional orientation for parsing to received data frame The corresponding left eye region image of information and right eye region image；It will be understood by those skilled in the art that inclusion region image In data frame also simultaneously the corresponding attitude information of inclusion region image, to obtain area image and its corresponding space side The information such as position information need to parse the information in data frame.
Specifically, as shown in Figure 10, the parsing module includes:
Reading unit 5201 is read in received data frame for the attitude information according to measured by sensor Area image.It will be understood by those skilled in the art that for indicating area image dimensional orientation in reading unit read data frame The label information of information, for example, by taking four pairs of camera units as an example, three of representation space azimuth information field in read data frame Corresponding binary numeral in byte, such as 000, inquire the attitude information of the corresponding camera unit of the binary numeral.
Matching unit 5202, according to the label information of area image, determination and attitude information measured by sensor Corresponding left eye region image and right eye region image；
Extraction unit 5203 calls left eye region image corresponding with the attitude information and right eye region image.
Display module 530, for the left eye region image corresponding with attitude information by described in and right eye region figure Screen sides are shown in as respectively corresponding.
Generally, data frame analyzing includes data frame format parsing and the parsing of data frame parameter.For example, tree-like knot can be used The analysis mode of structure parses data frame, is abstracted first to data handling procedure, layer-by-layer according to data handling procedure The characteristics of condition resolution, lists tree structure.Initial data frame obtains frame parameter and to be resolved by parsing root vertex parsing Data, analytic tree will select the resolution path of next step according to frame parameter；By data to be resolved give two node layer of analytic tree after Continuous parsing；The rest may be inferred, successively parses.In this resolving, include the attitude information to area image each in data frame Parsing.The present invention to the analysis mode of data frame without limitation, however not excluded that other are suitble to the parsing side of image data frames parsing Formula, such as the data frame analyzing and chain structure data frame analyzing of formalized description driving.
Specifically, display device screen is equally divided into left hand display area and right hand display area, the left eye region figure As being shown in left hand display area, right eye region image is shown in right hand display area.It will be understood by those skilled in the art that this Invention using parallax method construct three-dimensional imaging, according to binocular stereo imaging principle, need by the sight of left eye and right eye every From, so that left eye and right eye is watched the two width area images with parallax respectively, after brain synthesizes, the generation solid at two Vision.
Specifically, display equipment receives data frame according to the frame rate under camera unit image pickup mode, for passing through sensing Each attitude information that device data determine shows one data frame of the every reception of equipment, shows the method pair of panoramic stereo image It is primary that execution should be recycled.It will be understood by those skilled in the art that frame speed of the display equipment according to camera unit under image pickup mode Rate receives image, shows on a display screen by the rate after treatment, and typical frame rate is 24 frames/second~30 frames/second, Under this frame rate, effect smoothly and continuously can be generated.
In conclusion the present invention provides a kind of methods of panoramic stereo image acquisition, by configuring on acquisition device At least three pairs of camera unit, the flat image of acquisition covering panorama；The later periods such as cut, spliced to the flat image of acquisition Flat image, is processed into the area image for being used for three-dimensional imaging by processing；The area image that synchronization is acquired and generated closes And it is extremely transmitted to display equipment after same data frame and directly displays or be transmitted to server with temporary.This method is according to camera unit The rate of captured in real-time is carried out in image pickup mode, for camera unit one frame flat image of every acquisition, corresponding circulation executes primary. The present invention also provides a kind of panoramic stereo image acquisition devices, the method for executing panoramic stereo image acquisition.Further Ground, the present invention provides a kind of methods for showing panoramic stereo image, by obtaining the sensing in the display equipment that user wears Device data, determine display equipment pair dimensional orientation；It will be parsed from the received data frame of acquisition device, and match and obtain Area image corresponding with the attitude information that sensing data determines；According to parallax 3 D stereo construction method, respectively Left eye region image and right eye region image are shown into the left hand display area and right hand display area in display screen, keep user double The independent viewing scene of eye, generates stereoscopic visual effect after brain synthesizes at eyes.This method is being imaged according to camera unit The rate receiving area image that captured in real-time is carried out under mode, for every received frame image, corresponding circulation executes primary.This Invention additionally provides a kind of for showing the device of panoramic stereo image, the method for executing display panoramic stereo image.It is logical Embodiments of the present invention are crossed, it can be achieved that viewing panoramic stereo image, is supplied to user's experience on the spot in person.Since it is from hard Part level solves the problems, such as that the picture material of three-dimensional imaging obtains, and therefore, the image of output can be without replication processes just It can be used, improve the image quality and efficiency of transmission of stereo-picture.Further, also can in server or display terminal The processing time is largely saved, processing and efficiency of transmission are improved, more particularly to save the power consumption of display terminal, is improved The fluency of shown stereo-picture and whole image quality, improve user experience.
The above is only some embodiments of the invention, it is noted that for the ordinary skill people of the art For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered It is considered as protection scope of the present invention.

Claims (20)

1. a kind of panoramic stereo image acquisition method, which comprises the steps of:
Receive each camera unit flat image collected at least three pairs of camera units of covering panoramic viewing angle；It is each pair of to take the photograph As two width that two camera units in unit are acquired respectively suitable for being respectively formed left eye region image and right eye region image are put down Face image；
It the flat image is processed into according to binocular stereo imaging principle is used to stand in conjunction with the attitude information of camera unit The left eye region image and right eye region image of body imaging:
Each pair of camera unit two sheet of planar image collected is cut out according to binocular stereo imaging principle, is formed corresponding Left eye region image and right eye region image, wherein in left eye region image and right eye region image at least part scenery Appearance overlaps；Establish the corresponding relationship of the attitude information of each area image and its corresponding camera unit；
Left eye region image and right eye region image are associated and are sent to display device/server；The display equipment is to move Dynamic terminal or virtual reality head show equipment.
2. panoramic stereo image acquisition method according to claim 1, which is characterized in that at least three pairs of camera units are same When acquisition surround same circumference range flat image.
3. panoramic stereo image acquisition method according to claim 1, which is characterized in that each camera unit plane collected The planar image sections overlapping collected of image camera unit adjacent thereto.
4. panoramic stereo image acquisition method according to claim 1, which is characterized in that be located at the left side in each pair of camera unit Camera unit flat image collected be used to form left eye region image, the plane collected of the camera unit positioned at the right Image is to be used to form right eye region image.
5. panoramic stereo image acquisition method described in any one according to claim 1~4, which is characterized in that described by left eye Area image and right eye region image were associated in the step of being sent to display equipment, passed through what synchronization was obtained and generated The left eye region image and right eye region image, which are merged into same data frame, to be transmitted and realizes associated.
6. panoramic stereo image acquisition method according to claim 5, which is characterized in that in the same data frame In, the sky comprising the left eye region image and right eye region image that are correspondingly arranged according to the occupied spatial position of camera unit Between azimuth information.
7. panoramic stereo image acquisition method according to claim 6, which is characterized in that the attitude information is to mark Label information is indicated.
8. panoramic stereo image acquisition method according to any one of claims 1 to 4, which is characterized in that the camera shooting Unit is in photograph mode to obtain the flat image, frame speed of this method according to camera unit under the image pickup mode in real time Rate obtains each frame flat image for camera unit and corresponds to circulation and execute once.
9. panoramic stereo image acquisition method according to any one of claims 1 to 4, which is characterized in that this method quilt It is configured to receive four pairs of camera units flat image collected of covering panoramic viewing angle, wherein each camera unit institute The flat image of acquisition has the space wide-angle greater than 90 degree.
10. a kind of panoramic stereo image acquisition device, which is characterized in that including at least three pairs or more camera units, for acquiring The image of panoramic viewing angle is covered, described device includes:
Receiving module, each camera unit at least three pairs of camera units for receiving covering panoramic viewing angle are collected flat Face image；
Image processing module, the flat image is processed into for the attitude information in conjunction with camera unit be used for it is three-dimensional at The left eye region image and right eye region image of picture；
Transmission module, for being associated left eye region image and right eye region image and being sent to display device/server；
Unit is cut, each pair of camera unit two sheet of planar image collected is cut out according to binocular stereo imaging principle, Form corresponding left eye region image and right eye region image, wherein left eye region image and right eye region image are at least Part scene content overlaps；
Unit is indicated, the corresponding relationship of the attitude information of each area image and its corresponding camera unit is established.
11. panoramic stereo image acquisition device according to claim 10, which is characterized in that at least three pairs of camera units It is equidistantly surrounded on the outside of device, acquisition surrounds the flat image of same circumference range, and two camera shootings in each pair of camera unit are single Member acquires the two width flat images for suitably forming left eye region image and right eye region image respectively.
12. panoramic stereo image acquisition device according to claim 10, which is characterized in that each camera unit is collected flat The planar image sections overlapping collected of face image camera unit adjacent thereto.
13. panoramic stereo image acquisition device according to claim 10, which is characterized in that be located at a left side in each pair of camera unit The camera unit of side flat image collected is used to form left eye region image, and the camera unit positioned at right side is collected flat Face image is to be used to form right eye region image.
14. panoramic stereo image acquisition device according to claim 10, which is characterized in that described image processing module packet It includes:
Concatenation unit is put down for flat image acquired in the camera unit of the same side of each pair of camera unit to be spliced into panorama Face image；
Processing unit will be formed by two width panoramic plane images and handle, make its difference according to binocular stereo imaging principle As left eye region image and right eye region image；
Unit is indicated, the attitude information based on the camera unit establishes the tool of left eye region image Yu right eye region image Corresponding relationship between body portion.
15. panoramic stereo image acquisition device according to claim 10, which is characterized in that the transmission module includes:
Combining unit, the left eye region image and right eye region image that synchronization is obtained and generated are merged into same number Association is realized according in frame；
Associated rear left Vitrea eye area image and right eye region image are sent to display device/server by transmission unit.
16. panoramic stereo image acquisition device according to claim 10, which is characterized in that the display equipment is movement Terminal or virtual reality head show equipment.
17. panoramic stereo image acquisition device according to claim 15, which is characterized in that in the same data frame In, the sky comprising the left eye region image and right eye region image that are correspondingly arranged according to the occupied spatial position of camera unit Between azimuth information.
18. panoramic stereo image acquisition device according to claim 10, which is characterized in that the attitude information with Label information is indicated.
19. panoramic stereo image acquisition device described in 0 to 14 any one according to claim 1, which is characterized in that described to take the photograph As unit is in photograph mode to obtain the flat image in real time, according to frame rate of the camera unit under the image pickup mode, It is executed for the corresponding circulation of method that camera unit obtains each frame flat image panoramic stereo image acquisition primary.
20. panoramic stereo image acquisition device described in 0 to 14 any one according to claim 1, which is characterized in that the present apparatus It is configured with four pairs of camera units of covering panoramic viewing angle, wherein flat image acquired in each camera unit, which has, is greater than 90 The space wide-angle of degree.
CN201610500635.7A 2016-06-29 2016-06-29 Panoramic stereo image acquisition, display methods and corresponding device Active CN106060523B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610500635.7A CN106060523B (en) 2016-06-29 2016-06-29 Panoramic stereo image acquisition, display methods and corresponding device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610500635.7A CN106060523B (en) 2016-06-29 2016-06-29 Panoramic stereo image acquisition, display methods and corresponding device

Publications (2)

Publication Number Publication Date
CN106060523A CN106060523A (en) 2016-10-26
CN106060523B true CN106060523B (en) 2019-06-04

Family

ID=57200343

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610500635.7A Active CN106060523B (en) 2016-06-29 2016-06-29 Panoramic stereo image acquisition, display methods and corresponding device

Country Status (1)

Country Link
CN (1) CN106060523B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106371281A (en) * 2016-11-02 2017-02-01 辽宁中蓝电子科技有限公司 Multi-module 360-degree space scanning and positioning 3D camera based on structured light
CN107172047A (en) * 2017-04-05 2017-09-15 杭州优海信息系统有限公司 The virtual augmented reality 3D interactive systems of factory and its implementation
CN109961395B (en) * 2017-12-22 2022-10-11 展讯通信（上海）有限公司 Method, device and system for generating and displaying depth image and readable medium
CN108910710B (en) * 2018-07-27 2019-10-01 北京建筑大学 A kind of intelligent derrick crane using stereoscopic imaging technology
CN109146949B (en) * 2018-09-05 2019-10-22 天目爱视（北京）科技有限公司 A kind of 3D measurement and information acquisition device based on video data
CN114422819A (en) * 2022-01-25 2022-04-29 纵深视觉科技(南京)有限责任公司 Video display method, device, equipment, system and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1247994A (en) * 1998-08-28 2000-03-22 朗迅科技公司 Stereoscopic full views watching system
CN101627623A (en) * 2007-08-24 2010-01-13 索尼株式会社 Image processing device, dynamic image reproduction device, and processing method and program in them
CN104506761A (en) * 2014-12-20 2015-04-08 中国地质大学（武汉） 360-degree panoramic stereoscopic camera
CN204855942U (en) * 2015-06-29 2015-12-09 深圳市虚拟现实科技有限公司 Wear -type display device of wireless long -distance transmission
CN105262949A (en) * 2015-10-15 2016-01-20 浙江卓锐科技股份有限公司 Multifunctional panorama video real-time splicing method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8896671B2 (en) * 2010-04-09 2014-11-25 3D-4U, Inc. Apparatus and method for capturing images

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1247994A (en) * 1998-08-28 2000-03-22 朗迅科技公司 Stereoscopic full views watching system
CN101627623A (en) * 2007-08-24 2010-01-13 索尼株式会社 Image processing device, dynamic image reproduction device, and processing method and program in them
CN104506761A (en) * 2014-12-20 2015-04-08 中国地质大学（武汉） 360-degree panoramic stereoscopic camera
CN204855942U (en) * 2015-06-29 2015-12-09 深圳市虚拟现实科技有限公司 Wear -type display device of wireless long -distance transmission
CN105262949A (en) * 2015-10-15 2016-01-20 浙江卓锐科技股份有限公司 Multifunctional panorama video real-time splicing method

Also Published As

Publication number Publication date
CN106060523A (en) 2016-10-26

Similar Documents

Publication Publication Date Title
CN106060523B (en) Panoramic stereo image acquisition, display methods and corresponding device
US11024092B2 (en) System and method for augmented reality content delivery in pre-captured environments
US10893219B2 (en) System and method for acquiring virtual and augmented reality scenes by a user
CN106993181B (en) More VR/AR equipment collaboration systems and Synergistic method
US20200288097A1 (en) Method, apparatus, medium, terminal, and device for multi-angle free-perspective interaction
CN103946890B (en) Follow the tracks of the method and apparatus of three-dimensional body
CN105704479B (en) The method and system and display equipment of the measurement human eye interpupillary distance of 3D display system
CN101180653A (en) Method and device for three-dimensional rendering
CN110495163A (en) Camera system and bearing calibration
US20140192164A1 (en) System and method for determining depth information in augmented reality scene
CN111445583B (en) Augmented reality processing method and device, storage medium and electronic equipment
CN107852487A (en) Method for the electronic equipment of 360 degree of 3-D views of generation and for the electronic equipment
US10924691B2 (en) Control device of movable type imaging device and control method of movable type imaging device
US20150187132A1 (en) System and method for three-dimensional visualization of geographical data
CN107071389A (en) Take photo by plane method, device and unmanned plane
CN109743626A (en) A kind of image display method, image processing method and relevant device
KR20140057610A (en) Using motion parallax to create 3d perception from 2d images
CN105262949A (en) Multifunctional panorama video real-time splicing method
CN108693970A (en) Method and apparatus for the video image for adjusting wearable device
CN110275179A (en) A kind of building merged based on laser radar and vision ground drawing method
CN101872243A (en) System and method for realizing 360-degree panoramic play following real space direction
EP3635517B1 (en) Apparatus and method of matching head mounted display view to head movement that controls articulated camera
CN110160529A (en) A kind of guide system of AR augmented reality
CN113253845A (en) View display method, device, medium and electronic equipment based on eye tracking
CN108769755A (en) High-resolution full view frequency live streaming camera system and method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220728

Address after: 300450 No. 9-3-401, No. 39, Gaoxin 6th Road, Binhai Science Park, Binhai New Area, Tianjin

Patentee after: 3600 Technology Group Co.,Ltd.

Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park)

Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd.

Patentee before: Qizhi software (Beijing) Co.,Ltd.

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230714

Address after: 1765, floor 17, floor 15, building 3, No. 10 Jiuxianqiao Road, Chaoyang District, Beijing 100015

Patentee after: Beijing Hongxiang Technical Service Co.,Ltd.

Address before: 300450 No. 9-3-401, No. 39, Gaoxin 6th Road, Binhai Science Park, Binhai New Area, Tianjin

Patentee before: 3600 Technology Group Co.,Ltd.