CN104079916A - Panoramic three-dimensional visual sensor and using method - Google Patents

Panoramic three-dimensional visual sensor and using method Download PDF

Info

Publication number
CN104079916A
CN104079916A CN201410266573.9A CN201410266573A CN104079916A CN 104079916 A CN104079916 A CN 104079916A CN 201410266573 A CN201410266573 A CN 201410266573A CN 104079916 A CN104079916 A CN 104079916A
Authority
CN
China
Prior art keywords
camera
panorama
sensor
image
module
Prior art date
Application number
CN201410266573.9A
Other languages
Chinese (zh)
Inventor
刘伟恒
刘建
张荃
陈杰
钟晨
Original Assignee
深圳市德赛微电子技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市德赛微电子技术有限公司 filed Critical 深圳市德赛微电子技术有限公司
Priority to CN201410266573.9A priority Critical patent/CN104079916A/en
Publication of CN104079916A publication Critical patent/CN104079916A/en

Links

Abstract

The invention discloses a panoramic three-dimensional visual sensor and a using method. The visual sensor at least comprises three wide angle cameras fixed to a set mold, wherein any two cameras are placed on one side of the mold, and the third camera faces the other side of the visual sensor. Images collected by the two adjacent cameras are utilized for carrying out image three-dimensional vision analysis, images collected by all the cameras installed on the visual sensor mold are utilized for achieving image panoramic splicing, then all-directional visual analysis is carried out, and whether one object needing to be concerned exists or not is analyzed. According to the panoramic three-dimensional visual sensor, the characteristic of visual recognition of multi-view vision of a three-dimensional visual sensor and the characteristic of being capable of simultaneously collecting images in multiple directions of a panoramic visual sensor are combined at the same time, so that the panoramic three-dimensional images of the target object and the surroundings are represented.

Description

A kind of panorama three-dimensional visual sensor and using method

  

Technical field

The present invention relates to vision sensor field, particularly a kind of panorama three-dimensional visual sensor and using method thereof.

Background technology

Along with the development of computer vision and artificial intelligence, the form of vision sensor and function are also in rapid lifting.Existing vision sensor is divided into two large classes conventionally, one class is to pay attention to the stereoscopic vision of 3D, by two or more cameras, catch the picture of object in reality, and by the image collecting being analyzed to the athletic posture of acquisition object and apart from the distance of video camera.An other class is in recent years emerging panoramic picture acquisition scheme, by the camera of fixed alignment different directions, gathers the image around all angles around object, and process calibration and splicing obtain the panoramic picture around object.

But what stereo vision sensor was paid close attention to is only a direction, although and panoramic vision sensor solution can gather the image of multiple directions simultaneously, can not carry out the visual identity of used for multi-vision visual, greatly increased the difficulty of carrying out artificial intelligence.

In addition, the panoramic vision sensor of present stage has three kinds of solutions such as multi-cam splicing, refraction-reflection type, bugeye lens splicing, and multi-cam splicing solution cost is higher, and refraction-reflection type camera use scenes is limited, and can only generate Cylindrical panoramic image.

Summary of the invention

The object of the invention is in order to overcome above shortcomings in prior art, a kind of panorama three-dimensional visual sensor and using method thereof that simultaneously possesses stereoscopic vision and panoramic vision is provided, and this panorama three-dimensional visual sensor is used wide-angle lens splicing; Transducer provided by the invention is the vivo athletic posture of display-object thing and the around state of scene more truly.

To achieve these goals, according to an aspect of the present invention, a kind of panorama three-dimensional visual sensor is provided, described panorama three-dimensional visual sensor at least comprises that three wide-angle cameras are fixed on the mould setting, wherein two cameras are placed in a side of mould, and for stereoscopic vision analysis, the angle of these two camera optical axises is less than 80 degree, and the optical axis of these two cameras is parallel with supporting surface, the 3rd camera is towards the opposite side of vision sensor.

Preferably, the 3rd camera mentioned above that is not used in stereoscopic vision analysis can become with ground certain elevation angle.

Preferably, the 3rd camera mentioned above that is not used in stereoscopic vision analysis can become with ground certain angle of depression, and above mould, increases an auxiliary camera, on the optical axis direction of auxiliary camera.

Preferably, can a range finding servicing unit be installed in the nearer side of the camera spacing of analyzing for stereoscopic vision.

Preferably, in the nearer side of the camera spacing of analyzing for stereoscopic vision, an infrared transmitter based on structured light technique is installed, and is there is for two cameras of stereoscopic vision analysis the function that gathers infrared light.

Preferably, can have camera control module, module is to the attitude of camera, position, motion state and towards controlling

Preferably, can have sensor localization module, module is analyzed this vision sensor initial attitude, position, motion state and orientation information by several transducers such as electronic compass, accelerometer or gyroscopes.

  

According to another aspect of the present invention, provide a kind of using method of panorama three-dimensional visual sensor, said method comprising the steps of:

Step 1): camera collection image;

Step 2): the image that utilizes the camera collection for stereoscopic vision analysis described in step 1 to arrive, carries out the stereoscopic vision analysis of image;

Step 3): utilize be arranged on camera collections all on vision sensor mould to image carry out the panorama splicing of image;

Step 4): carry out omnidirectional visual analysis;

Preferably, can use other range finder module to measure the degree of depth of scene around, and auxiliary stereopsis module is carried out stereoscopic vision identification.

Preferably, can use infrared transmitter on object, to project infrared light towards periphery, particularly project infrared structure light, then adopt the camera that can gather infrared light, utilize infrared light information and original storage to set and measure the degree of depth of scene around.

Preferably, can use the auxiliary panorama splicing of information of stereopsis module, improve the effect of panorama splicing.

Preferably, can use the several sensor localization attitudes such as electronic compass, accelerometer or gyroscope, position, towards step.

Compared with prior art, the present invention has following feature and beneficial effect:

1: two camera close together, towards a side, for stereoscopic vision analysis, camera is towards the vision sensor modelling of an other side, at stereo vision sensor, only pay close attention on unidirectional basis, take into account and consider other directions, reach and can carry out the visual identity of used for multi-vision visual, can gather again the image of multiple directions to obtain the object of panorama 3-D view simultaneously;

2: use stereoscopic vision to add infrared acquisition or other module combined measurements scene depth around, realized the splicing to panorama and be optimized;

3: utilize wide-angle lens splicing, avoided the high and refraction-reflection type camera use scenes of multi-cam splicing cost limited etc. not enough;

4: utilize Multi-sensor Fusion positioning shooting head towards.

Accompanying drawing explanation

In order to be illustrated more clearly in the technical scheme of the embodiment of the present invention, in describing embodiment below, needed accompanying drawing is done simple introduction, apparently, the accompanying drawing the following describes is only some embodiments of the present invention, for those of ordinary skills, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.

Fig. 1 is the vertical view of the panorama three-dimensional visual sensor scheme of the present invention's the simplest a kind of version;

Fig. 2 is the vertical view of a kind of preferred panorama three-dimensional visual sensor scheme of the present invention

Fig. 3 is the front view of a kind of preferred panorama three-dimensional visual sensor scheme of the present invention;

Fig. 4 is the left view of a kind of preferred panorama three-dimensional visual sensor scheme of the present invention;

Fig. 5 is that the present invention uses the supporting hardware composition proposal figure of the simplest version vision sensor shown in Fig. 1;

Fig. 6 is a kind of preferred panorama three-dimensional visual sensor hardware composition proposal figure of the present invention;

Fig. 7 is the method for work flow chart of the panorama three-dimensional visual sensor of the present invention's the simplest a kind of version;

Embodiment

Below by embodiment, by reference to the accompanying drawings technical scheme of the present invention is carried out to clear, complete description; obviously; described embodiment is only a part of embodiment of the present invention; rather than whole embodiment; based on embodiments of the invention; those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.

According to one embodiment of present invention, a kind of panorama three-dimensional visual sensor is provided, shown in Fig. 1, a kind of panorama three-dimensional visual sensor of the simplest version, it is upper that described transducer comprises that three wide-angle camera A, C, D are fixed on the mould E that sets, and wherein two camera A, C are placed in a side of mould E, and the angle of these two camera optical axises is 60 degree, and the optical axis of these two cameras is parallel with supporting surface, the 3rd camera D is towards the opposite side of vision sensor.

Fig. 2 increases range finding servicing unit on the basis of the vision sensor shown in Fig. 1, this range finding servicing unit can be laser ranging module or ultrasonic distance measuring module, or be the infrared transmitter based on structured light technique, if be infrared transmitter, be arranged on two camera A of sensor die E front side, C has the function that receives infrared light, the feature of this design is to utilize infrared transmitter to the different angles emitting structural light in space, utilization is arranged on two camera collection infrared structure reflection of light information of vision sensor front end, thereby calculate transducer the place ahead environment degree of depth.

Fig. 5 has provided the hardware composition proposal of transducer shown in Fig. 1, and described transducer at least comprises with lower module;

The camera 1 that relative distance is nearer and camera 2 for gathering the image of a side both direction, provide image information to stereopsis module and panorama splicing processing module simultaneously;

Opposite side camera, for gathering the image of opposite side, and splices processing module by image transfer to panorama;

Stereopsis module, for being used the image of the nearlyer camera collection of relative distance between described camera to carry out computer vision analysis;

Panorama splicing processing module, for splicing the image information of whole camera collections;

Vision sensor of the present invention, except the hardware composition proposal shown in Fig. 5, also can increase other miscellaneous function modules, and as shown in Figure 6, other miscellaneous function modules mainly comprise:

Range finding supplementary module, the distance near object obtaining with respect to camera, and offer stereopsis module, for auxiliary stereoscopic vision analysis, or offer panorama concatenation module, for optimizing panorama splicing.

Camera control module, the visual information that can collect by analysis, near object situation analyzing, adjust vision sensor front end camera towards, or the visual information collecting by analysis, analyze omnirange ambient light conditions, control the exposure status of camera.

Sensor localization module, analyzes this vision sensor initial attitude, position, motion state and orientation information by several transducers such as electronic compass, accelerometer or gyroscopes.

According to another aspect of the present invention, provide a kind of using method of transducer of panoramic vision, as shown in Figure 7, said method comprising the steps of:

Step 1): camera collection image

Step 2): the image that utilizes the camera collection for stereoscopic vision analysis described in step 1 to arrive, carries out the stereoscopic vision analysis of image;

Step 3): utilize be arranged on camera collections all on vision sensor mould to image carry out the panorama splicing of image;

Step 4): carry out omnidirectional visual analysis;

If transducer increase of the present invention has auxiliary range finder module, corresponding method step is:

Step 1): camera collection image

Step 2): by being arranged on the depth measurement device of the nearlyer side of camera relative distance, measure near object distance;

Step 3): the information that comprehensive step 1, step 2 obtain is carried out stereoscopic vision analysis;

Step 4): utilize be arranged on camera collections all on vision sensor mould to image carry out the panorama splicing of image;

Step 5): carry out omnidirectional visual analysis according to stereoscopic vision analysis result and Panorama Mosaic information;

Panorama three-dimensional visual sensor provided by the present invention has been taken into account the visual identity of stereo vision sensor used for multi-vision visual and the feature that panoramic vision sensor can gather the image of multiple directions simultaneously, can reproduce object and panorama 3-D view around thereof.

The above embodiment has only expressed preferred embodiment of the present invention, and it describes comparatively concrete and detailed, but can not therefore be interpreted as the restriction to the scope of the claims of the present invention.It should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection range of patent of the present invention should be as the criterion with claims.

Claims (9)

1. a panorama three-dimensional visual sensor, is characterized in that: described vision sensor at least comprises with lower module:
Three wide-angle cameras, wherein any two wide-angle cameras are fixed on a side of described vision sensor, and for stereoscopic vision analysis, the 3rd wide-angle camera is fixed on the opposite side of vision sensor;
Stereopsis module, carries out computer vision analysis for being fixed on the image of two camera collections of a side of described vision sensor described in using;
Panorama splicing processing module, for splicing the image information of whole camera collections.
2. panorama three-dimensional visual sensor as claimed in claim 1, is characterized in that: also comprise:
Range finding supplementary module, the distance near object obtaining with respect to camera, and offer stereopsis module, for auxiliary stereoscopic vision analysis, or offer panorama concatenation module, for optimizing panorama splicing.
3. panorama three-dimensional visual sensor as claimed in claim 2, is characterized in that: described range finding supplementary module is laser ranging module or ultrasonic distance measuring module, or is the infrared transmitter based on structured light technique; And if described range finding supplementary module is the infrared transmitter based on structured light technique, the wide-angle camera being fixed on described vision sensor has the function that receives infrared light.
4. the panorama three-dimensional visual sensor as described in claims 1 to 3 any one, it is characterized in that: the camera that is arranged on vision sensor mould opposite side becomes the certain elevation angle or angle of depression with the optical axis place plane of other two camera lenses for stereoscopic vision analysis, and above mould, increase an auxiliary camera, on the optical axis direction of described auxiliary camera.
5. the panorama three-dimensional visual sensor as described in claims 1 to 3 any one, is characterized in that: also comprise:
Camera control module, the visual information collecting by analysis, near object situation analyzing, adjust vision sensor front end camera towards, or the visual information collecting by analysiss, analyzes omnirange ambient light conditions, the exposure status of control camera;
Sensor localization module, analyzes this vision sensor initial attitude, position, motion state and orientation information by several transducers such as electronic compass, accelerometer or gyroscopes.
6. the panorama three-dimensional visual sensor as described in claims 1 to 3 any one, is characterized in that: described in be fixed on vision sensor one side the angle of two wide-angle cameras for stereoscopic vision analysis for being less than 80 degree.
7. a using method for panorama three-dimensional visual sensor as claimed in claim 1, is characterized in that: comprise the following steps:
Step 1): camera carries out IMAQ
Step 2): utilize two images that camera collection arrives for stereoscopic vision analysis, carry out the stereoscopic vision analysis of image;
Step 3): utilize all camera collections be arranged on vision sensor mould to image carry out the panorama splicing of image;
Step 4): carry out omnidirectional visual analysis.
8. a using method for panorama three-dimensional visual sensor as claimed in claim 3, is characterized in that: comprise the following steps:
Step 1): camera collection image;
Step 2): by being arranged on the depth measurement device of the nearlyer side of camera relative distance, measure near object distance;
Step 3): two camera collections that can be used for stereoscopic vision analysis in comprehensive step 1 to image, the information that step 2 obtains carry out stereoscopic vision analysis;
Step 4): utilize be arranged on camera collections all on vision sensor mould to image carry out the panorama splicing of image;
Step 5): carry out omnidirectional visual analysis.
9. a using method for panorama three-dimensional visual sensor as claimed in claim 7 or 8, is characterized in that: also comprise: use the several sensor localization attitudes such as electronic compass, accelerometer or gyroscope, position, towards etc. information.
CN201410266573.9A 2014-06-16 2014-06-16 Panoramic three-dimensional visual sensor and using method CN104079916A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410266573.9A CN104079916A (en) 2014-06-16 2014-06-16 Panoramic three-dimensional visual sensor and using method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410266573.9A CN104079916A (en) 2014-06-16 2014-06-16 Panoramic three-dimensional visual sensor and using method

Publications (1)

Publication Number Publication Date
CN104079916A true CN104079916A (en) 2014-10-01

Family

ID=51600927

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410266573.9A CN104079916A (en) 2014-06-16 2014-06-16 Panoramic three-dimensional visual sensor and using method

Country Status (1)

Country Link
CN (1) CN104079916A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104570577A (en) * 2015-02-12 2015-04-29 深圳市圆周率软件科技有限责任公司 720-degree panoramic camera
CN105574838A (en) * 2014-10-15 2016-05-11 上海弘视通信技术有限公司 Image registration and splicing method of multi-view camera and device thereof
CN106382918A (en) * 2016-08-31 2017-02-08 胡钊政 Three-dimensional visual sensor
CN106559664A (en) * 2015-09-30 2017-04-05 成都理想境界科技有限公司 The filming apparatus and equipment of three-dimensional panoramic image
CN106713898A (en) * 2017-03-02 2017-05-24 深圳看到科技有限公司 Three-lens panoramic 3D camera
WO2017107263A1 (en) * 2015-12-23 2017-06-29 北京凌宇智控科技有限公司 Three-dimensional space detection system, positioning method and system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101339309A (en) * 2008-08-01 2009-01-07 浙江大学 Convertible display screen and its three-dimensional display device
CN101651784A (en) * 2009-09-24 2010-02-17 上海交通大学 Video tracking system of panoramic pan-tilt-zoom camera
CN102289144A (en) * 2011-06-30 2011-12-21 浙江工业大学 Intelligent three-dimensional (3D) video camera equipment based on all-around vision sensor
CN103118230A (en) * 2013-02-28 2013-05-22 腾讯科技(深圳)有限公司 Panorama acquisition method, device and system
CN103200357A (en) * 2012-01-04 2013-07-10 苏州科泽数字技术有限公司 Method and device for constructing panorama staring web camera
US20130271579A1 (en) * 2012-04-14 2013-10-17 Younian Wang Mobile Stereo Device: Stereo Imaging, Measurement and 3D Scene Reconstruction with Mobile Devices such as Tablet Computers and Smart Phones

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101339309A (en) * 2008-08-01 2009-01-07 浙江大学 Convertible display screen and its three-dimensional display device
CN101651784A (en) * 2009-09-24 2010-02-17 上海交通大学 Video tracking system of panoramic pan-tilt-zoom camera
CN102289144A (en) * 2011-06-30 2011-12-21 浙江工业大学 Intelligent three-dimensional (3D) video camera equipment based on all-around vision sensor
CN103200357A (en) * 2012-01-04 2013-07-10 苏州科泽数字技术有限公司 Method and device for constructing panorama staring web camera
US20130271579A1 (en) * 2012-04-14 2013-10-17 Younian Wang Mobile Stereo Device: Stereo Imaging, Measurement and 3D Scene Reconstruction with Mobile Devices such as Tablet Computers and Smart Phones
CN103118230A (en) * 2013-02-28 2013-05-22 腾讯科技(深圳)有限公司 Panorama acquisition method, device and system

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105574838A (en) * 2014-10-15 2016-05-11 上海弘视通信技术有限公司 Image registration and splicing method of multi-view camera and device thereof
CN105574838B (en) * 2014-10-15 2018-09-14 上海弘视通信技术有限公司 The image registration of more mesh cameras and joining method and its device
CN104570577A (en) * 2015-02-12 2015-04-29 深圳市圆周率软件科技有限责任公司 720-degree panoramic camera
CN104570577B (en) * 2015-02-12 2018-06-05 沈靖程 A kind of 720 degree of panoramic cameras
CN106559664A (en) * 2015-09-30 2017-04-05 成都理想境界科技有限公司 The filming apparatus and equipment of three-dimensional panoramic image
WO2017107263A1 (en) * 2015-12-23 2017-06-29 北京凌宇智控科技有限公司 Three-dimensional space detection system, positioning method and system
CN106382918A (en) * 2016-08-31 2017-02-08 胡钊政 Three-dimensional visual sensor
CN106713898A (en) * 2017-03-02 2017-05-24 深圳看到科技有限公司 Three-lens panoramic 3D camera

Similar Documents

Publication Publication Date Title
KR101761751B1 (en) Hmd calibration with direct geometric modeling
CA2961921C (en) Camera calibration method using a calibration target
US10529141B2 (en) Capturing and aligning three-dimensional scenes
JP6496323B2 (en) System and method for detecting and tracking movable objects
EP3099056A1 (en) Method and apparatus for displaying a light field based image on a user's device, and corresponding computer program product
EP3190781B1 (en) Autofocus method, device and electronic apparatus
US10116922B2 (en) Method and system for automatic 3-D image creation
JP6687204B2 (en) Projection image generation method and apparatus, and mapping method between image pixels and depth values
US20170134713A1 (en) Image calibrating, stitching and depth rebuilding method of a panoramic fish-eye camera and a system thereof
Zhu et al. The multivehicle stereo event camera dataset: An event camera dataset for 3D perception
KR20150086469A (en) Multi-dimensional data capture of an environment using plural devices
CN101419055B (en) Space target position and pose measuring device and method based on vision
US20120237114A1 (en) Method and apparatus for feature-based stereo matching
CN107113415B (en) The method and apparatus for obtaining and merging for more technology depth maps
US20150326783A1 (en) Three dimensional image capture system for imaging building facades using a digital camera, a near-infrared camera, and laser range finder
CN103150715B (en) Image mosaic processing method and processing device
JP2014529727A (en) Automatic scene calibration
EP2757346B1 (en) Method for measuring a height profile of a vehicle passing on the street
EP2993894B1 (en) Image capturing method and electronic apparatus
EP1792282B1 (en) A method for automated 3d imaging
JP2012533222A (en) Image-based surface tracking
CN104217439B (en) Indoor visual positioning system and method
JP5260705B2 (en) 3D augmented reality provider
CN103901895B (en) Target positioning method based on unscented FastSLAM algorithm and matching optimization and robot
CN106993181B (en) More VR/AR equipment collaboration systems and Synergistic method

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20141001

RJ01 Rejection of invention patent application after publication