CN110740238A - light splitting HDR camera applied to mobile robot SLAM field - Google Patents

light splitting HDR camera applied to mobile robot SLAM field Download PDF

Info

Publication number
CN110740238A
CN110740238A CN201911017598.4A CN201911017598A CN110740238A CN 110740238 A CN110740238 A CN 110740238A CN 201911017598 A CN201911017598 A CN 201911017598A CN 110740238 A CN110740238 A CN 110740238A
Authority
CN
China
Prior art keywords
beam splitter
cmos chip
image
spectroscopic
main beam
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911017598.4A
Other languages
Chinese (zh)
Other versions
CN110740238B (en
Inventor
吕恩利
王飞仁
刘妍华
郭嘉明
李斌
苏秋双
赵伟伟
吴鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China Agricultural University
Original Assignee
South China Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China Agricultural University filed Critical South China Agricultural University
Priority to CN201911017598.4A priority Critical patent/CN110740238B/en
Publication of CN110740238A publication Critical patent/CN110740238A/en
Application granted granted Critical
Publication of CN110740238B publication Critical patent/CN110740238B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/51Housings
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/54Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/55Optical parts specially adapted for electronic image sensors; Mounting thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/73Circuitry for compensating brightness variation in the scene by influencing the exposure time
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/741Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

本发明公开了一种应用于移动机器人SLAM领域的分光HDR相机,包括:壳体以及安装在壳体前端的镜头,在所述壳体中设置有分光系统,包括:设置在所述镜头正后方的主分光镜,主分光镜的分光面倾斜于所述镜头的光轴,主分光镜后方设置有第一副分光镜,第一副分光镜的分光面与主分光镜的分光面垂直;位于主分光镜上方设置有第二副分光镜,第二副分光镜的分光面与主分光镜的镜面平行;在第一副分光镜的下方、后方,第二副分光镜的后方、上方分别设置一个焊接在PCB基板上的CMOS芯片,所有PCB基板连接至一个FPGA芯片。本发明可以减小整体曝光时间,降低图像拖影现象的发生,由于采用相同时间和空间的曝光方式,适用于室外大场景和室内小场景。

Figure 201911017598

The invention discloses a spectroscopic HDR camera applied to the field of mobile robot SLAM, comprising: a casing and a lens mounted on the front end of the casing; a spectroscopic system is arranged in the casing, including: being arranged directly behind the lens The main beam splitter, the beam splitting surface of the main beam splitter is inclined to the optical axis of the lens, the rear of the main beam splitter is provided with a first secondary beam splitter, and the beam splitting surface of the first secondary beam splitter is perpendicular to the beam splitting surface of the main beam splitter; A second pair of beam splitters is arranged above the main beam splitter, and the beam splitting surface of the second pair beam splitter is parallel to the mirror surface of the main beam splitter; below and behind the first pair of beam splitters, and behind and above the second pair of beam splitters are respectively arranged A CMOS chip soldered on a PCB substrate, all PCB substrates are connected to an FPGA chip. The present invention can reduce the overall exposure time and reduce the occurrence of image smear phenomenon, and is suitable for large outdoor scenes and small indoor scenes due to the use of exposure methods in the same time and space.

Figure 201911017598

Description

一种应用于移动机器人SLAM领域的分光HDR相机A Spectral HDR Camera Applied in the Field of Mobile Robot SLAM

技术领域technical field

本发明涉及移动机器人定位与导航领域,具体涉及一种应用于移动机器人同时定位与地图构建(SLAM)领域的高动态范围成像(HDR)相机。The invention relates to the field of positioning and navigation of mobile robots, in particular to a high dynamic range imaging (HDR) camera applied to the field of simultaneous positioning and map construction (SLAM) of mobile robots.

背景技术Background technique

基于图像的定位算法是自主移动机器人领域内的热门问题,它是解决移动机器人运动规划与控制的前提条件。基于图像的同时定位与地图构建(SimultaneousLocalization and Mapping,SLAM)能够在未知的环境下,通过分析图像序列,在估计出机器人自身位姿的同时恢复出环境的地图。由于此技术只依赖机器人自身的图像传感器,不需要对环境改造和人工标记,同时相机具有成本较低等有点,因此SLAM技术得到了国内外的广泛关注。视觉SLAM主要依赖的传感器是相机,因此相机的成像质量直接关系着SLAM的定位精度,然而,传统相机由于光宽容度有限,在一些环境下例如:透过窗户的阳光、物体的阴影等,相机得到的图像会存在局部的曝光过度以及局部曝光不足的情况,丢失了图像大部分的亮部或者暗部信息,不利于SLAM算法对于特征点的提取,而特征提取的精度直接关乎整个SLAM算法的精度。高阶相机虽然光宽容度较高,但是其成本往往也较高,应用在移动机器人上非常不经济、普通相机虽然可以在静态的环境下通过连续曝光的方式得到高动态范围成像(High Dynamic Range Imaging,HDR)图像,然而由于移动机器人的运动的存在,会造成普通相机在HDR模式下有较为严重的拖影。因此,提出一种性价比高的高动态范围相机对于SLAM的应用具有重要的意义。Image-based localization algorithm is a popular problem in the field of autonomous mobile robots, and it is a prerequisite for solving the motion planning and control of mobile robots. Image-based Simultaneous Localization and Mapping (SLAM) can recover a map of the environment while estimating the robot's own pose by analyzing image sequences in an unknown environment. Since this technology only relies on the image sensor of the robot itself, it does not require environmental modification and manual marking, and the camera has the advantages of low cost, so SLAM technology has received extensive attention at home and abroad. The main sensor of visual SLAM is the camera, so the image quality of the camera is directly related to the positioning accuracy of SLAM. However, due to the limited light latitude of traditional cameras, in some environments, such as sunlight through windows, shadows of objects, etc., the camera The obtained image will have partial overexposure and partial underexposure, losing most of the bright or dark information of the image, which is not conducive to the extraction of feature points by the SLAM algorithm, and the accuracy of feature extraction is directly related to the accuracy of the entire SLAM algorithm. . Although high-end cameras have high light latitude, their cost is often high, and it is very uneconomical to apply to mobile robots. Although ordinary cameras can obtain high dynamic range imaging (High Dynamic Range) through continuous exposure in a static environment. Imaging, HDR) images, however, due to the motion of the mobile robot, it will cause serious smearing of ordinary cameras in HDR mode. Therefore, it is of great significance to propose a cost-effective high dynamic range camera for the application of SLAM.

发明内容SUMMARY OF THE INVENTION

本发明的目的是提供一种应用于移动机器人SLAM领域的分光HDR相机,主要用以解决传统相机在HDR模式下会存在运动模糊的问题。The purpose of the present invention is to provide a spectroscopic HDR camera applied in the field of mobile robot SLAM, which is mainly used to solve the problem of motion blur in traditional cameras in HDR mode.

为了实现上述任务,本发明采用以下技术方案:In order to realize the above-mentioned tasks, the present invention adopts the following technical solutions:

一种应用于移动机器人SLAM领域的分光HDR相机,包括:壳体以及安装在壳体前端的镜头,在所述壳体中设置有分光系统,包括:A spectroscopic HDR camera applied in the field of mobile robot SLAM, comprising: a casing and a lens mounted on the front end of the casing, a spectroscopic system is arranged in the casing, including:

设置在所述镜头正后方的主分光镜,主分光镜的分光面倾斜于所述镜头的光轴,主分光镜后方设置有第一副分光镜,第一副分光镜的分光面与主分光镜的分光面垂直;位于主分光镜上方设置有第二副分光镜,第二副分光镜的分光面与主分光镜的镜面平行;The main beam splitter is arranged directly behind the lens, the beam splitter of the main beam splitter is inclined to the optical axis of the lens, a first sub beam splitter is arranged behind the main beam splitter, and the beam splitter of the first sub beam splitter is connected to the main beam The beam splitter surface of the mirror is vertical; a second pair of beam splitters is arranged above the main beam splitter, and the beam splitter surface of the second pair beam splitter is parallel to the mirror surface of the main beam splitter;

所述第一副分光镜的下方设置有第一CMOS芯片,第一副分光镜的后方设置有第二CMOS芯片;所述第二副分光镜的后方设置有第三CMOS芯片,第二副分光镜的上方设置有第四CMOS芯片;其中,每一个CMOS芯片均分别焊接在一块PCB基板上;每一个CMOS芯片对应的PCB基板均连接至FPGA芯片;其中,所述FPGA芯片用于将每一个CMOS芯片获取的图像合成高动态范围的图像。A first CMOS chip is arranged below the first sub-spectroscope, and a second CMOS chip is arranged behind the first sub-spectroscope; a third CMOS chip is arranged behind the second sub-spectroscope, and the second sub-spectroscope is arranged behind A fourth CMOS chip is arranged above the mirror; wherein, each CMOS chip is respectively welded on a PCB substrate; the PCB substrate corresponding to each CMOS chip is connected to the FPGA chip; wherein, the FPGA chip is used for each The images acquired by the CMOS chip are synthesized into high dynamic range images.

进一步地,所述主分光镜、第一副分光镜、第二副分光镜的反射与投射比例均为1:1。Further, the reflection and projection ratios of the main beam splitter, the first sub beam splitter, and the second sub beam splitter are all 1:1.

进一步地,所述主分光镜的分光面与镜头的光轴之间的夹角为45°。Further, the included angle between the beam splitting surface of the main beam splitter and the optical axis of the lens is 45°.

进一步地,所述主分光镜、第一副分光镜和第二副分光镜均为正立方体分光镜。Further, the main beam splitter, the first sub beam splitter and the second sub beam splitter are all cube beam splitters.

外界光线经过所述镜头聚焦后,经过主分光镜,50%的外界光线穿透主分光镜的分光面射入第一副分光镜中,其余50%的外界光线被主分光镜的分光面反射到第二副分光镜中;After the outside light is focused by the lens and passes through the main beam splitter, 50% of the outside light penetrates the beam splitting surface of the main beam splitter and enters the first secondary beam splitter, and the remaining 50% of the outside light is reflected by the beam splitter surface of the main beam splitter. into the second pair of beamsplitters;

进入第一副分光镜中的50%的外界光线,其中的一半外界光线经过第一副分光镜的分光面反射后射入第一CMOS芯片,另一半外界光线穿过第一副分光镜的分光面后射入第二CMOS芯片;50% of the outside light entering the first pair of beamsplitters, half of the outside light is reflected by the beam splitting surface of the first pair of beamsplitters and then enters the first CMOS chip, and the other half of the outside light passes through the beam splitting of the first pair of beamsplitters The second CMOS chip is injected behind the face;

进入第二副分光镜中的其余50%的外界光线,其中的一半外界光线经过第二副分光镜的分光面反射后射入第三CMOS芯片,另一半外界光线穿过第二副分光镜的分光面后射入第四CMOS芯片。The remaining 50% of the external light entering the second sub-beamsplitter, half of which are reflected by the beam splitting surface of the second sub-beamsplitter and then enter the third CMOS chip, and the other half of the ambient light passes through the second sub-beamsplitter. After the beam splitting surface, the fourth CMOS chip is injected.

进一步地,所述FPGA芯片通过数据通信接口与外部通信;当FPGA芯片接收到外部发送的图像请求时,同时向第一CMOS芯片至第四CMOS芯片下达不同曝光时长的曝光指令,第一CMOS芯片至第四CMOS芯片分别通过对应的分光面独立获取图像。Further, the FPGA chip communicates with the outside through a data communication interface; when the FPGA chip receives an image request sent from the outside, it simultaneously issues exposure instructions with different exposure durations to the first CMOS chip to the fourth CMOS chip. To the fourth CMOS chips, images are acquired independently through the corresponding beam splitting surfaces.

进一步地,所述分光HDR相机的工作模式包括弱光模式和强光模式,其中,FPGA芯片通过分析上一次利用每一个CMOS芯片获取的图像中,正常曝光的图像的平均亮度以及曝光时间来确定环境的光线情况,然后在下一次每一个CMOS芯片获取图像之前,选择将工作模式切换在弱光模式或强光模式。Further, the working modes of the spectroscopic HDR camera include a low-light mode and a high-light mode, wherein the FPGA chip determines the average brightness and exposure time of the normally exposed image in the last image obtained by using each CMOS chip. The light conditions of the environment, and then choose to switch the working mode to low light mode or strong light mode before each CMOS chip acquires an image next time.

进一步地,所述FPGA芯片通过系数融合的方式将每一个CMOS芯片获取的图像合成高动态范围的图像,包括:Further, the FPGA chip combines the images obtained by each CMOS chip into a high dynamic range image by means of coefficient fusion, including:

对于每一个CMOS芯片获取的图像,将每一张图像的每一个像素点对应一个融合系数,然后通过加权计算不同图像上相同位置的像素值得到所述高动态范围的图像。For an image acquired by each CMOS chip, each pixel of each image corresponds to a fusion coefficient, and then the high dynamic range image is obtained by weighted calculation of pixel values at the same position on different images.

进一步地,在弱光模式下,所述的曝光指令为:延长曝光时间2档、延长曝光时间1档、正常曝光、减少曝光时间1档,此时能获得过曝光图像2张、正常曝光图像1张、曝光不足图像1张,则分光系统更关注暗部的图像细节;Further, in the low light mode, the exposure instruction is: extend the exposure time by 2 steps, extend the exposure time by 1 step, normal exposure, and reduce the exposure time by 1 step, at this time, 2 overexposed images and 1 normal exposure image can be obtained. 1 image and 1 underexposed image, the spectroscopic system pays more attention to the image details in the dark part;

在强光模式下,所述的曝光指令为:延长曝光时间1档、正常曝光、减少曝光时间1档、减少曝光时间2档,此时能获得过曝光图像1张、正常曝光图像1张、曝光不足图像2张,则分光系统更关注亮部的图像细节。In the strong light mode, the exposure instructions are: extend the exposure time by 1 step, normal exposure, reduce the exposure time by 1 step, and reduce the exposure time by 2 steps. At this time, 1 overexposed image, 1 normal exposure image, If there are 2 underexposed images, the spectroscopic system pays more attention to the image details of the bright parts.

进一步地,所述壳体内部设置有分光系统支架,所述分光系统支架包括对称设置的一对支撑板,所述一对支撑板上对称开设有用于固定所述主分光镜、第一副分光镜和第二副分光镜的L形卡槽;Further, a spectroscopic system bracket is arranged inside the housing, and the spectroscopic system bracket includes a pair of symmetrically arranged support plates, and the pair of support plates are symmetrically provided with a pair of supporting plates for fixing the main beam splitter and the first auxiliary beam splitter. L-shaped slot for mirror and second beam splitter;

当主分光镜、第一副分光镜和第二副分光镜安装到卡槽中之后,所述第一副分光镜下方及后方的支撑板上对称开设有第一固定槽、第二固定槽,分别用于固定安装所述第一CMOS芯片对应的PCB基板和第二CMOS芯片对应的PCB基板;所述第二副分光镜的后方及上方的支撑板上对称开设有第三固定槽、第四固定槽,分别用于固定安装所述第三CMOS芯片对应的PCB基板和第四CMOS芯片对应的PCB基板。After the main spectroscope, the first secondary spectroscope and the second secondary spectroscope are installed in the card slots, the support plates below and behind the first secondary spectroscope are symmetrically provided with a first fixing slot and a second fixing slot, respectively. It is used to fixedly install the PCB substrate corresponding to the first CMOS chip and the PCB substrate corresponding to the second CMOS chip; the support plate behind and above the second sub-spectroscope is symmetrically provided with a third fixing groove and a fourth fixing groove The grooves are respectively used to fix and install the PCB substrate corresponding to the third CMOS chip and the PCB substrate corresponding to the fourth CMOS chip.

进一步地,所述PCB基板与相邻的副分光镜的端面平行且面积相同。Further, the PCB substrate and the end faces of the adjacent secondary beam splitters are parallel and have the same area.

本发明具有以下技术特点:The present invention has the following technical characteristics:

1.本发明的相机为一体化分光系统结构,采用多CMOS同时曝光的方式采集不同曝光参数下的图像合成HDR图像,相比于传统的HDR成像方式,可以减小整体曝光时间,降低图像拖影现象的发生,由于采用相同时间和空间的曝光方式,适用于室外大场景和室内小场景。1. The camera of the present invention has an integrated spectroscopic system structure, and adopts the method of multi-CMOS simultaneous exposure to collect images under different exposure parameters to synthesize HDR images. Compared with the traditional HDR imaging method, the overall exposure time can be reduced, and the image drag can be reduced. Due to the exposure method of the same time and space, it is suitable for large outdoor scenes and small indoor scenes.

2.本发明采用多片低成本的CMOS芯片通过合成的方式得到HDR图像,相比具有高光宽容度的芯片具有巨大的成本优势,有助于降低移动机器人定位系统的成本。2. The present invention uses multiple low-cost CMOS chips to obtain HDR images through synthesis, which has a huge cost advantage compared to chips with high light latitude, and helps reduce the cost of a mobile robot positioning system.

3.相比于普通的相机,本发明的相机具有更高的动态范围,有助于移动机器人的SLAM算法提高定位与地图构建的精度。3. Compared with ordinary cameras, the camera of the present invention has a higher dynamic range, which helps the SLAM algorithm of the mobile robot to improve the accuracy of positioning and map construction.

附图说明Description of drawings

图1为本发明去掉部分壳体后的结构示意图;1 is a schematic structural diagram of the present invention after removing part of the shell;

图2位本发明的分光系统支架的示意图;Fig. 2 is the schematic diagram of the spectroscopic system support of the present invention;

图3为本发明的整体结构示意图;Fig. 3 is the overall structure schematic diagram of the present invention;

图中标号说明:1镜头,2壳体,3第四CMOS芯片,4第二副分光镜,5FPGA芯片,6电路板,7分光系统支架,71支撑板,72卡槽,73第一固定槽,74第二固定槽,75第三固定槽,76第四固定槽,77第五固定槽,8第三CMOS芯片,9第二CMOS芯片,10第一副分光镜,11第一CMOS芯片,12主分光镜。Description of the symbols in the figure: 1 lens, 2 shell, 3 fourth CMOS chip, 4 second secondary beam splitter, 5FPGA chip, 6 circuit board, 7 beam splitting system bracket, 71 support plate, 72 card slot, 73 first fixing slot , 74 second fixing slot, 75 third fixing slot, 76 fourth fixing slot, 77 fifth fixing slot, 8 third CMOS chip, 9 second CMOS chip, 10 first secondary beam splitter, 11 first CMOS chip, 12 main beamsplitters.

具体实施方式Detailed ways

本发明公开了一种应用于移动机器人SLAM邻域的分光HDR相机,如图1所示,包括:壳体2以及安装在壳体2前端的镜头1,例如镜头1可通过螺纹配合的方式固定在壳体2前端;在所述壳体2中设置有分光系统,包括:The present invention discloses a spectroscopic HDR camera applied to the SLAM neighborhood of a mobile robot. As shown in FIG. 1 , it includes: a casing 2 and a lens 1 installed at the front end of the casing 2 , for example, the lens 1 can be fixed by screw fitting At the front end of the housing 2; a spectroscopic system is arranged in the housing 2, including:

设置在所述镜头1正后方的主分光镜12,其中正后方是指主分光镜12的中心线与镜头1的光轴重合;主分光镜12的分光面倾斜于所述镜头1的光轴,主分光镜12后方设置有第一副分光镜10,第一副分光镜10的分光面与主分光镜12的分光面垂直;位于主分光镜12上方设置有第二副分光镜4,第二副分光镜4的分光面与主分光镜12的镜面平行。本实施例中,所述主分光镜12、第一副分光镜10和第二副分光镜4均为大小相同的正立方体分光镜,分光面均在正立方体内部的一侧底边和另一侧顶边之间,分光面与正立方体顶面、底面的夹角均为45°;所述主分光镜12、第一副分光镜10、第二副分光镜4的反射与投射比例均为1:1,即光线沿平行于分光镜底面的方向照射在分光镜的分光面上以后,有一半的光线被分光面反射,而另一半光线则穿过分光面。The main beam splitter 12 is arranged directly behind the lens 1, wherein the right rear means that the center line of the main beam splitter 12 coincides with the optical axis of the lens 1; the beam splitter surface of the main beam splitter 12 is inclined to the optical axis of the lens 1 , a first sub-spectroscope 10 is arranged behind the main beam splitter 12, and the beam splitting surface of the first sub beam splitter 10 is perpendicular to the beam splitter surface of the main beam splitter 12; a second sub beam splitter 4 is arranged above the main beam splitter 12, The beam splitting surface of the secondary beam splitter 4 is parallel to the mirror surface of the main beam splitter 12 . In this embodiment, the main beam splitter 12 , the first sub beam splitter 10 and the second sub beam splitter 4 are all square cube beam splitters with the same size, and the beam splitting surfaces are all on one bottom edge of the square cube and the other Between the side and top edges, the angle between the beam splitting surface and the top and bottom surfaces of the cube is 45°; the reflection and projection ratios of the main beam splitter 12, the first secondary beam splitter 10, and the second secondary beam splitter 4 are all 45°. 1:1, that is, after the light is irradiated on the beam splitter surface of the beam splitter in a direction parallel to the bottom surface of the beam splitter, half of the light is reflected by the beam splitter surface, while the other half of the light passes through the beam splitter surface.

可选地,所述所述主分光镜12的分光面与镜头1的光轴之间的夹角为45°,以便于精确地进行分光。如图1所示,主分光镜12的下端更靠近镜头1一侧。Optionally, the included angle between the beam splitting surface of the main beam splitter 12 and the optical axis of the lens 1 is 45°, so as to accurately perform beam splitting. As shown in FIG. 1 , the lower end of the main beam splitter 12 is closer to the side of the lens 1 .

所述第一副分光镜10的下方设置有第一CMOS芯片11,第一副分光镜10的后方设置有第二CMOS芯片9;所述第二副分光镜4的后方设置有第三CMOS芯片8,第二副分光镜4的上方设置有第四CMOS芯片3;其中,每一个CMOS芯片(包括第一CMOS芯片11至第四CMOS芯片3)均分别焊接在一块PCB基板上,用于接收穿过对应的副分光镜的光线;每一个CMOS芯片对应的PCB基板均连接至FPGA芯片5;所述的对应的PCB基板是指CMOS芯片所焊接的PCB基板,在PCB基板上实现CMOS芯片的外围电路。所述PCB基板与相邻的副分光镜的端面平行且面积相同,在本实施例中,每个PCB基板均为正方形板,其大小与第一副分光镜10、第二副分光镜4的端面面积相同,且中心在同一条轴线上。例如,第一CMOS芯片11对应的PCB基板平行设置在第一副分光镜10的正下方,其大小与第一副分光镜10的底面大小相同,且二者的中心在同一条轴线上;第二CMOS芯片9对应的PCB基板平行设置在第一副分光镜10的正后方,其大小与第一副分光镜10的后端面大小相同。A first CMOS chip 11 is arranged below the first sub-beamsplitter 10 , a second CMOS chip 9 is arranged behind the first sub-beamsplitter 10 , and a third CMOS chip is arranged behind the second sub-beamsplitter 4 8. A fourth CMOS chip 3 is arranged above the second secondary beam splitter 4; wherein, each CMOS chip (including the first CMOS chip 11 to the fourth CMOS chip 3) is welded on a PCB substrate for receiving The light passing through the corresponding secondary beam splitter; the PCB substrate corresponding to each CMOS chip is connected to the FPGA chip 5; the corresponding PCB substrate refers to the PCB substrate to which the CMOS chip is welded, and the CMOS chip is realized on the PCB substrate. Peripheral circuits. The PCB substrate and the end faces of the adjacent sub-beamsplitters are parallel and have the same area. In this embodiment, each PCB substrate is a square plate, the size of which is the same as that of the first sub-beamsplitter 10 and the second sub-beamsplitter 4 . The end faces have the same area and the centers are on the same axis. For example, the PCB substrate corresponding to the first CMOS chip 11 is arranged in parallel directly below the first secondary beam splitter 10, and its size is the same as that of the bottom surface of the first secondary beam splitter 10, and the centers of the two are on the same axis; The PCB substrates corresponding to the two CMOS chips 9 are arranged in parallel directly behind the first sub-beam splitter 10 , and have the same size as the rear surface of the first sub-beam splitter 10 .

在本方案中,外界光线经过所述镜头1聚焦后,经过主分光镜12,50%的外界光线穿透主分光镜12的分光面射入第一副分光镜10中,其余50%的外界光线被主分光镜12的分光面反射到第二副分光镜4中;进入第一副分光镜10中的50%的外界光线,其中的一半外界光线经过第一副分光镜10的分光面反射后射入第一CMOS芯片11,另一半外界光线穿过第一副分光镜10的分光面后射入第二CMOS芯片9;进入第二副分光镜4中的其余50%的外界光线,其中的一半外界光线经过第二副分光镜4的分光面反射后射入第三CMOS芯片8,另一半外界光线穿过第二副分光镜4的分光面后射入第四CMOS芯片3。因此,外界光线经过分光系统后,每个CMOS芯片上能有25%的外界光线。In this solution, after the outside light is focused by the lens 1 and passes through the main beam splitter 12, 50% of the outside light penetrates the beam splitting surface of the main beam splitter 12 and enters the first sub beam splitter 10, and the remaining 50% of the outside light The light is reflected by the beam splitting surface of the main beam splitter 12 into the second sub beam splitter 4; 50% of the external light entering the first sub beam splitter 10, half of which is reflected by the beam splitting surface of the first sub beam splitter 10 After entering the first CMOS chip 11, the other half of the external light passes through the beam splitting surface of the first secondary beam splitter 10 and then enters the second CMOS chip 9; the remaining 50% of the external light entering the second secondary beam splitter 4, wherein Half of the ambient light is reflected by the beam splitting surface of the second sub-beamsplitter 4 and then enters the third CMOS chip 8 , and the other half of the ambient light passes through the beam-splitting surface of the second sub beamsplitter 4 and then enters the fourth CMOS chip 3 . Therefore, after the external light passes through the light splitting system, there can be 25% of the external light on each CMOS chip.

本方案中,所述FPGA芯片5通过电路板6固定在壳体2内部,FPGA芯片5通过数据通信接口与外部通信;PCB基板通过排线与电路板连接,继而与FPGA芯片5通信;其中数据接口例如可以采用USB3.0,所述的外部可以为例如计算机、移动机器人的控制器等。FPGA芯片5用于将每一个CMOS芯片获取的欠曝光、正常曝光、过曝光的图像合成高动态范围的图像。In this solution, the FPGA chip 5 is fixed inside the housing 2 through the circuit board 6, and the FPGA chip 5 communicates with the outside through a data communication interface; the PCB substrate is connected to the circuit board through a cable, and then communicates with the FPGA chip 5; The interface can be, for example, USB3.0, and the external can be, for example, a computer, a controller of a mobile robot, and the like. The FPGA chip 5 is used for synthesizing the under-exposure, normal-exposure, and over-exposure images acquired by each CMOS chip into a high-dynamic-range image.

当FPGA芯片5接收到外部发送的图像请求时,同时向第一CMOS芯片11至第四CMOS芯片3下达不同曝光时长的曝光指令,第一CMOS芯片11至第四CMOS芯片3分别通过对应的分光面独立获取图像;其中,第一CMOS芯片11、第二CMOS芯片9分别通过第一副分光镜10的分光面反射、透射独立获取图像,而第三CMOS芯片8、第四CMOS芯片3分别通过第二副分光镜4的分光面反射、透射独立获取图像。When the FPGA chip 5 receives the image request sent from the outside, it simultaneously issues exposure commands with different exposure durations to the first CMOS chip 11 to the fourth CMOS chip 3 , and the first CMOS chip 11 to the fourth CMOS chip 3 pass through the corresponding beam splitters respectively. The first CMOS chip 11 and the second CMOS chip 9 obtain images independently through reflection and transmission of the beam splitting surface of the first secondary beam splitter 10, respectively, while the third CMOS chip 8 and the fourth CMOS chip 3 pass through the The reflection and transmission of the beam splitting surface of the second sub beam splitter 4 independently acquire images.

所述分光HDR相机的工作模式包括弱光模式和强光模式,其中,FPGA芯片5通过分析上一次利用每一个CMOS芯片获取的图像中,正常曝光的图像的平均亮度以及曝光时间来确定环境的光线情况,然后在下一次每一个CMOS芯片获取图像之前,选择将工作模式切换在弱光模式或强光模式。每个CMOS芯片获取的图像经过处理与合成最终得到所述高动态范围的图像。The working modes of the spectroscopic HDR camera include low-light mode and high-light mode, wherein the FPGA chip 5 determines the environmental conditions by analyzing the average brightness of the normally exposed image and the exposure time in the last image obtained by using each CMOS chip. light conditions, and then choose to switch the working mode to low light mode or strong light mode before each CMOS chip acquires an image next time. The image acquired by each CMOS chip is processed and synthesized to finally obtain the high dynamic range image.

在弱光模式下,所述的曝光指令为:延长曝光时间2档、延长曝光时间1档、正常曝光、减少曝光时间1档,此时能获得过曝光图像2张、正常曝光图像1张、曝光不足图像1张,该情况下分光系统更关注暗部的图像细节;In low light mode, the exposure instructions are: extend the exposure time by 2 steps, extend the exposure time by 1 step, normal exposure, and reduce the exposure time by 1 step. At this time, 2 overexposed images, 1 normal exposure image, One underexposed image, in this case, the spectroscopic system pays more attention to the image details in the dark part;

在强光模式下,所述的曝光指令为:延长曝光时间1档、正常曝光、减少曝光时间1档、减少曝光时间2档,此时能获得过曝光图像1张、正常曝光图像1张、曝光不足图像2张,该情况下分光系统更关注亮部的图像细节。In the strong light mode, the exposure instructions are: extend the exposure time by 1 step, normal exposure, reduce the exposure time by 1 step, and reduce the exposure time by 2 steps. At this time, 1 overexposed image, 1 normal exposure image, There are 2 underexposed images, in which case the spectroscopic system pays more attention to the image details of the bright parts.

其中所述的1档、2档指的是表示曝光的时间程度,例如延长曝光时间1档为延长曝光时间T,则延长曝光时间2档为延长曝光时间2T;减少曝光时间同理;具体的档位可根据实际需求设定。Among them, the 1st file and the 2nd file refer to the degree of exposure time. For example, extending the exposure time by 1 file means extending the exposure time T, then extending the exposure time by 2 files is extending the exposure time 2T; the same is true for reducing the exposure time; Gears can be set according to actual needs.

所述正常曝光的时间的获得根据图像的熵最大原理,图像熵反应的是图像中平均信息量的多少,在每次获取正常曝光图像后,FPGA芯片5统计当前的图像熵值,然后根据熵值调整下一次的曝光时间。使用以下公式计算图像熵:The acquisition of the normal exposure time is based on the principle of maximum entropy of the image. The image entropy reflects the average amount of information in the image. After each acquisition of a normal exposure image, the FPGA chip 5 counts the current image entropy value, and then according to the entropy value to adjust the next exposure time. Image entropy is calculated using the following formula:

Figure BDA0002246188450000061
Figure BDA0002246188450000061

其中pi是像素灰度值为i的像素在图像中出现的概率:where p i is the probability that a pixel with pixel gray value i appears in the image:

pi=ni/M×Np i =n i /M×N

其中M、N分别表示图像中像素的行列数,ni表示灰度值为i的像素在图像中的数量。Among them, M and N represent the number of rows and columns of pixels in the image, respectively, and n i represents the number of pixels with a grayscale value of i in the image.

所述FPGA芯片5通过系数融合的方式将每一个CMOS芯片获取的图像合成高动态范围的图像,具体为首先估计不同曝光图像的融合权重,然后再进行加权平均融合,融合方法遵循以下公式:The FPGA chip 5 combines the images obtained by each CMOS chip into a high dynamic range image by means of coefficient fusion. Specifically, the fusion weights of different exposure images are estimated first, and then weighted average fusion is performed. The fusion method follows the following formula:

其中Wn(x,y)为融合权重,In(x,y)代表不同的图像,n表示图像序号。where W n (x, y) is the fusion weight, I n (x, y) represents different images, and n represents the image sequence number.

即,对于每一个CMOS芯片获取的图像,将每一张图像的每一个像素点对应一个融合系数,然后通过加权计算不同图像上相同位置的像素值得到所述高动态范围的图像。That is, for an image obtained by each CMOS chip, each pixel of each image corresponds to a fusion coefficient, and then the high dynamic range image is obtained by weighted calculation of pixel values at the same position on different images.

如图1所示,为了便于分光系统的布设,所述壳体2内部设置有分光系统支架7,所述分光系统支架7包括对称设置的一对支撑板71,所述的对称设置是指这一对支撑板71相互平行且对应;所述一对支撑板71上对称开设有用于固定所述主分光镜12、第一副分光镜10和第二副分光镜4的L形卡槽;如图2所示,由于主分光镜12、第一副分光镜10、第二副分光镜4从位置关系上来看第一副分光镜10位于主分光镜12后方,第二副分光镜4位于主分光镜12上方,故形成在所述支撑板71上的L形卡槽能很好地固定这些分光镜;具体地,将这些分光镜按照位置放置好之后,使分光镜的两侧分别卡入到所述一对支撑板71上的L形卡槽中即可。As shown in FIG. 1 , in order to facilitate the arrangement of the spectroscopic system, a spectroscopic system bracket 7 is provided inside the housing 2 , and the spectroscopic system bracket 7 includes a pair of support plates 71 arranged symmetrically. The symmetrical arrangement refers to this A pair of support plates 71 are parallel to each other and correspond to each other; the pair of support plates 71 are symmetrically provided with L-shaped slots for fixing the main beam splitter 12, the first sub beam splitter 10 and the second sub beam splitter 4; for example As shown in FIG. 2 , since the main beam splitter 12 , the first sub beam splitter 10 , and the second sub beam splitter 4 are positioned behind the main beam splitter 12 , the second sub beam splitter 4 is located in the main beam splitter 4 Above the spectroscope 12, the L-shaped slot formed on the support plate 71 can well fix these spectroscopes; into the L-shaped slot on the pair of support plates 71 .

当主分光镜12、第一副分光镜10和第二副分光镜4安装到卡槽中之后,所述第一副分光镜10下方及后方的支撑板71上对称开设有第一固定槽73、第二固定槽74,如图1、图2所示,第一固定槽73、第二固定槽74均为条形槽,分别用于固定安装所述第一CMOS芯片对应的PCB基板和第二CMOS芯片对应的PCB基板;具体地,使PCB基板的两侧分别卡入到所述一对支撑板71上的固定槽中,即对PCB基板进行了有效固定,也就对PCB基板上的CMOS芯片进行了有效固定,使其能在设定的精确位置接收经过分光镜的光线。所述第二副分光镜4的后方及上方的支撑板71上对称开设有第三固定槽75、第四固定槽76,分别用于固定安装所述第三CMOS芯片对应的PCB基板和第四CMOS芯片对应的PCB基板。After the main beam splitter 12 , the first sub beam splitter 10 and the second sub beam splitter 4 are installed in the card slots, the support plates 71 below and behind the first sub beam splitter 10 are symmetrically provided with first fixing grooves 73 , The second fixing slot 74, as shown in FIG. 1 and FIG. 2, both the first fixing slot 73 and the second fixing slot 74 are strip-shaped slots, which are respectively used for fixing and installing the PCB substrate corresponding to the first CMOS chip and the second fixing slot 74. The PCB substrate corresponding to the CMOS chip; specifically, the two sides of the PCB substrate are respectively snapped into the fixing grooves on the pair of support plates 71, that is, the PCB substrate is effectively fixed, and the CMOS on the PCB substrate is effectively fixed. The chip is effectively fixed so that it can receive the light passing through the beamsplitter at the precise position set. A third fixing slot 75 and a fourth fixing slot 76 are symmetrically opened on the support plate 71 behind and above the second secondary beam splitter 4 , respectively for fixing the PCB substrate corresponding to the third CMOS chip and the fourth fixing slot 76 . The PCB substrate corresponding to the CMOS chip.

进一步地,如图2所示,在第四固定槽76右侧的支撑板71上还对称开设有第五固定槽77,结合图1,第五固定槽用于FPGA芯片5的电路板6。Further, as shown in FIG. 2 , the support plate 71 on the right side of the fourth fixing slot 76 is also symmetrically provided with a fifth fixing slot 77 . Referring to FIG. 1 , the fifth fixing slot is used for the circuit board 6 of the FPGA chip 5 .

本实施例中,在进行镜头1与壳体2螺纹连接时,螺纹型号为工业镜头1的统用C型接口,镜头1的参数为:焦距5mm、光圈1.8、适配CMOS最大像面1/2”、水平视角64°、垂直视角50°、镜头1畸变-0.33%、镜头1长度58mm、镜头1最大分辨率10M、镜头1接口C-mount。In this embodiment, when the lens 1 and the housing 2 are threadedly connected, the thread type is the common C-type interface of the industrial lens 1, and the parameters of the lens 1 are: focal length 5mm, aperture 1.8, and the maximum image surface suitable for CMOS 1/ 2", horizontal viewing angle 64°, vertical viewing angle 50°, lens 1 distortion -0.33%, lens 1 length 58mm, lens 1 maximum resolution 10M, lens 1 interface C-mount.

第一CMOS芯片11至第四CMOS芯片3采用相同的型号,本实施例中为ONSemiconductor MT9V034C12STM,图像分辨率752H×480V、最大像面1/3”、像素大小6.0um×6.0um、灰度成像、全局快门、最大帧率60Hz、10位ADC分辨率、动态范围大于55dB。此CMOS的成本较低、同时具有较高的像素大小、有助于提高图像的动态范围。其中每一个CMOS芯片对应的PCB基板的规格相同,例如PCB基板的厚度为1.2mm,尺寸为20mm×20mm。The first CMOS chip 11 to the fourth CMOS chip 3 are of the same model, which is ON Semiconductor MT9V034C12STM in this embodiment, with an image resolution of 752H×480V, a maximum image surface of 1/3”, a pixel size of 6.0um×6.0um, and grayscale imaging. , global shutter, maximum frame rate 60Hz, 10-bit ADC resolution, dynamic range greater than 55dB. The cost of this CMOS is lower, and at the same time, it has a higher pixel size, which helps to improve the dynamic range of the image. Each CMOS chip corresponds to The specifications of the PCB substrate are the same, for example, the thickness of the PCB substrate is 1.2mm and the size is 20mm×20mm.

主分光镜12、第一分光镜、第二分光镜的规格相同,优选尺寸为20mm×20mm×20mm。The main beam splitter 12 , the first beam splitter, and the second beam splitter have the same specifications, and the preferred size is 20 mm×20 mm×20 mm.

FPGA芯片5与外部通过USB3.0接口连接通信,用以向各个CMOS芯片发送曝光指令,同时接收、处理、合成图像,并将最终合成的图像通过USB3.0接口发送给外部。例如,FPGA芯片5的型号可以为:Xilinx Zynq 7020,其中可编辑逻辑单元为85K,具有充足的计算资源处理4片CMOS芯片的数据。同时此芯片内部包含一个双核心ARM Cortex-A9 SOC计算单元,可以进行通用计算,用来计算图像熵、曝光时间等。FPGA芯片5在每次获取CMOS芯片的图像数据后对当前环境进行测光,统计正常曝光图像的灰度分布,即0-255每个灰度值出现的概率,然后计算正常曝光图像的信息熵,遵循信息熵最大的原则,动态调整下一帧正常曝光的时间。根据下一帧正常曝光的时间确定当前的模式是处于弱光模式和强光模式。The FPGA chip 5 connects and communicates with the outside through the USB3.0 interface, so as to send exposure instructions to each CMOS chip, receive, process, and synthesize images at the same time, and send the final synthesized image to the outside through the USB3.0 interface. For example, the model of the FPGA chip 5 may be: Xilinx Zynq 7020, wherein the programmable logic unit is 85K, and it has sufficient computing resources to process data of four CMOS chips. At the same time, the chip contains a dual-core ARM Cortex-A9 SOC computing unit, which can perform general calculations to calculate image entropy, exposure time, etc. The FPGA chip 5 measures the current environment after each acquisition of the image data of the CMOS chip, counts the grayscale distribution of the normal exposure image, that is, the probability of each grayscale value from 0 to 255, and then calculates the information entropy of the normal exposure image. , follow the principle of maximum information entropy, and dynamically adjust the normal exposure time of the next frame. According to the normal exposure time of the next frame, it is determined that the current mode is in the low light mode and the strong light mode.

本实施例使用4片性价比较高的低阶CMOS芯片配合分光系统,实现了对环境的同时同位置曝光,经过图像合成得到HDR图像,解决了传统相机采集图像光宽容度不足以及普通相机在HDR模式下的运动模糊问题,适用于室内小场景和室外大场景,较高端相机成本优势突出,有助于提高移动机器人的SLAM精度。This embodiment uses 4 low-level CMOS chips with high cost performance and a light splitting system to achieve simultaneous exposure to the environment at the same position, and HDR images are obtained through image synthesis, which solves the problem of insufficient light latitude of images captured by traditional cameras and problems of HDR by ordinary cameras. The motion blur problem in the mode is suitable for small indoor scenes and large outdoor scenes. The cost advantage of higher-end cameras is prominent, which helps to improve the SLAM accuracy of mobile robots.

Claims (10)

1.一种应用于移动机器人SLAM领域的分光HDR相机,包括:壳体(2)以及安装在壳体(2)前端的镜头(1),在所述壳体(2)中设置有分光系统,其特征在于,所述分光系统包括:1. A spectroscopic HDR camera applied to the field of mobile robot SLAM, comprising: a casing (2) and a lens (1) mounted on the front end of the casing (2), wherein a spectroscopic system is provided in the casing (2) , characterized in that the spectroscopic system includes: 设置在所述镜头(1)正后方的主分光镜(12),主分光镜(12)的分光面倾斜于所述镜头(1)的光轴,主分光镜(12)后方设置有第一副分光镜(10),第一副分光镜(10)的分光面与主分光镜(12)的分光面垂直;位于主分光镜(12)上方设置有第二副分光镜(4),第二副分光镜(4)的分光面与主分光镜(12)的镜面平行;The main beam splitter (12) is arranged directly behind the lens (1), the beam splitting surface of the main beam splitter (12) is inclined to the optical axis of the lens (1), and a first beam splitter (12) is arranged behind the main beam splitter (12). A secondary beam splitter (10), the light splitting surface of the first secondary beam splitter (10) is perpendicular to the beam splitting surface of the main beam splitter (12); a second secondary beam splitter (4) is arranged above the main beam splitter (12), The beam splitting surfaces of the second pair of beam splitters (4) are parallel to the mirror surface of the main beam splitter (12); 所述第一副分光镜(10)的下方设置有第一CMOS芯片(11),第一副分光镜(10)的后方设置有第二CMOS芯片(9);所述第二副分光镜(4)的后方设置有第三CMOS芯片(8),第二副分光镜(4)的上方设置有第四CMOS芯片(3);其中,每一个CMOS芯片均分别焊接在一块PCB基板上;每一个CMOS芯片对应的PCB基板均连接至FPGA芯片(5);其中,所述FPGA芯片(5)用于将每一个CMOS芯片获取的图像合成高动态范围的图像。A first CMOS chip (11) is provided below the first secondary beam splitter (10), and a second CMOS chip (9) is provided behind the first secondary beam splitter (10); the second secondary beam splitter (10) is provided with a second CMOS chip (9). 4) a third CMOS chip (8) is arranged at the rear, and a fourth CMOS chip (3) is arranged above the second secondary beam splitter (4); wherein, each CMOS chip is respectively welded on a PCB substrate; each The PCB substrate corresponding to one CMOS chip is connected to the FPGA chip (5); wherein, the FPGA chip (5) is used for synthesizing the image acquired by each CMOS chip into a high dynamic range image. 2.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述主分光镜(12)、第一副分光镜(10)、第二副分光镜(4)的反射与投射比例均为1:1。2. The spectroscopic HDR camera applied to the field of mobile robot SLAM according to claim 1, wherein the main beam splitter (12), the first secondary beam splitter (10), the second secondary beam splitter (4) The ratio of reflection and projection is 1:1. 3.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述主分光镜(12)、第一副分光镜(10)和第二副分光镜(4)均为正立方体分光镜。3. The spectroscopic HDR camera applied in the field of mobile robot SLAM according to claim 1, wherein the main beam splitter (12), the first secondary beam splitter (10) and the second secondary beam splitter (4) All are cube beamsplitters. 4.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述主分光镜(12)的分光面与镜头(1)的光轴之间的夹角为45°;4. The spectroscopic HDR camera applied to the field of mobile robot SLAM as claimed in claim 1, wherein the angle between the spectroscopic surface of the main beam splitter (12) and the optical axis of the lens (1) is 45 °; 外界光线经过所述镜头(1)聚焦后,经过主分光镜(12),50%的外界光线穿透主分光镜(12)的分光面射入第一副分光镜(10)中,其余50%的外界光线被主分光镜(12)的分光面反射到第二副分光镜(4)中;After the outside light is focused by the lens (1) and passes through the main beam splitter (12), 50% of the outside light penetrates the beam splitting surface of the main beam splitter (12) and enters the first sub beam splitter (10), and the remaining 50% % of the external light is reflected by the beam splitting surface of the main beam splitter (12) into the second sub beam splitter (4); 进入第一副分光镜(10)中的50%的外界光线,其中的一半外界光线经过第一副分光镜(10)的分光面反射后射入第一CMOS芯片(11),另一半外界光线穿过第一副分光镜(10)的分光面后射入第二CMOS芯片(9);50% of the ambient light entering the first secondary beam splitter (10), half of the ambient light is reflected by the beam splitting surface of the first secondary beam splitter (10) and then injected into the first CMOS chip (11), and the other half of the ambient light enters the first CMOS chip (11) After passing through the beam splitting surface of the first secondary beam splitter (10), it is injected into the second CMOS chip (9); 进入第二副分光镜(4)中的其余50%的外界光线,其中的一半外界光线经过第二副分光镜(4)的分光面反射后射入第三CMOS芯片(8),另一半外界光线穿过第二副分光镜(4)的分光面后射入第四CMOS芯片(3)。The remaining 50% of the ambient light entering the second sub-beamsplitter (4), half of the ambient light is reflected by the beam splitting surface of the second sub-beamsplitter (4) and then enters the third CMOS chip (8), and the other half of the ambient light enters the third CMOS chip (8) The light enters the fourth CMOS chip (3) after passing through the beam splitting surface of the second secondary beam splitting mirror (4). 5.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述FPGA芯片(5)通过数据通信接口与外部通信;当FPGA芯片(5)接收到外部发送的图像请求时,同时向第一CMOS芯片(11)至第四CMOS芯片(3)下达不同曝光时长的曝光指令,第一CMOS芯片(11)至第四CMOS芯片(3)分别通过对应的分光面独立获取图像。5. The spectroscopic HDR camera applied to the field of mobile robot SLAM as claimed in claim 1, wherein the FPGA chip (5) communicates with the outside through a data communication interface; when the FPGA chip (5) receives an externally sent When an image is requested, exposure commands with different exposure durations are issued to the first CMOS chip (11) to the fourth CMOS chip (3) at the same time, and the first CMOS chip (11) to the fourth CMOS chip (3) respectively pass through the corresponding light splitting surfaces Acquire images independently. 6.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述分光HDR相机的工作模式包括弱光模式和强光模式,其中,FPGA芯片(5)通过分析上一次利用每一个CMOS芯片获取的图像中,正常曝光的图像的平均亮度以及曝光时间来确定环境的光线情况,然后在下一次每一个CMOS芯片获取图像之前,选择将工作模式切换在弱光模式或强光模式。6. The spectroscopic HDR camera applied to the field of mobile robot SLAM as claimed in claim 1, wherein the working mode of the spectroscopic HDR camera comprises a weak light mode and a strong light mode, wherein the FPGA chip (5) is analyzed by analyzing In the image obtained by each CMOS chip last time, the average brightness of the normally exposed image and the exposure time are used to determine the light condition of the environment, and then before each CMOS chip captures the image next time, choose to switch the working mode to low light mode or Strong light mode. 7.如权利要求5所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,在弱光模式下,所述的曝光指令为:延长曝光时间2档、延长曝光时间1档、正常曝光、减少曝光时间1档,此时能获得过曝光图像2张、正常曝光图像1张、曝光不足图像1张,则分光系统更关注暗部的图像细节;7. The spectroscopic HDR camera applied to the field of mobile robot SLAM as claimed in claim 5, wherein in the low light mode, the exposure instruction is: extend the exposure time by 2 steps, extend the exposure time by 1 step, normal Expose and reduce the exposure time by 1 stop, at this time, 2 overexposed images, 1 normal exposure image, and 1 underexposed image can be obtained, and the spectroscopic system pays more attention to the image details in the dark part; 在强光模式下,所述的曝光指令为:延长曝光时间1档、正常曝光、减少曝光时间1档、减少曝光时间2档,此时能获得过曝光图像1张、正常曝光图像1张、曝光不足图像2张,则分光系统更关注亮部的图像细节。In the strong light mode, the exposure instructions are: extend the exposure time by 1 step, normal exposure, reduce the exposure time by 1 step, and reduce the exposure time by 2 steps. At this time, 1 overexposed image, 1 normal exposure image, If there are 2 underexposed images, the spectroscopic system pays more attention to the image details of the bright parts. 8.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述FPGA芯片(5)通过系数融合的方式将每一个CMOS芯片获取的图像合成高动态范围的图像,包括:8. The spectroscopic HDR camera applied in the field of mobile robot SLAM according to claim 1, wherein the FPGA chip (5) synthesizes the image obtained by each CMOS chip into a high dynamic range image by means of coefficient fusion ,include: 对于每一个CMOS芯片获取的图像,将每一张图像的每一个像素点对应一个融合系数,然后通过加权计算不同图像上相同位置的像素值得到所述高动态范围的图像。For an image acquired by each CMOS chip, each pixel of each image corresponds to a fusion coefficient, and then the high dynamic range image is obtained by weighted calculation of pixel values at the same position on different images. 9.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述壳体(2)内部设置有分光系统支架(7),所述分光系统支架(7)包括对称设置的一对支撑板(71),所述一对支撑板(71)上对称开设有用于固定所述主分光镜(12)、第一副分光镜(10)和第二副分光镜(4)的L形卡槽;9. The spectroscopic HDR camera applied in the field of mobile robot SLAM according to claim 1, wherein the shell (2) is provided with a spectroscopic system bracket (7), and the spectroscopic system bracket (7) comprises A pair of support plates (71) are symmetrically arranged, and the pair of support plates (71) are symmetrically provided with a pair of support plates (71) for fixing the main beam splitter (12), the first sub beam splitter (10) and the second sub beam splitter ( 4) L-shaped card slot; 当主分光镜(12)、第一副分光镜(10)和第二副分光镜(4)安装到卡槽中之后,所述第一副分光镜(10)下方及后方的支撑板(71)上对称开设有第一固定槽(73)、第二固定槽(74),分别用于固定安装所述第一CMOS芯片(11)对应的PCB基板和第二CMOS芯片(9)对应的PCB基板;所述第二副分光镜(4)的后方及上方的支撑板(71)上对称开设有第三固定槽(75)、第四固定槽(76),分别用于固定安装所述第三CMOS芯片(8)对应的PCB基板和第四CMOS芯片(3)对应的PCB基板。After the main beam splitter (12), the first sub beam splitter (10) and the second sub beam splitter (4) are installed in the card slot, the support plates (71) below and behind the first sub beam splitter (10) A first fixing groove (73) and a second fixing groove (74) are symmetrically opened on the upper part, which are respectively used for fixing and installing the PCB substrate corresponding to the first CMOS chip (11) and the PCB substrate corresponding to the second CMOS chip (9). ; A third fixing slot (75) and a fourth fixing slot (76) are symmetrically provided on the rear and upper support plates (71) of the second sub-spectroscope (4), respectively for fixing and installing the third fixing slot (76). A PCB substrate corresponding to the CMOS chip (8) and a PCB substrate corresponding to the fourth CMOS chip (3). 10.如权利要求1所述的应用于移动机器人SLAM领域的分光HDR相机,其特征在于,所述PCB基板与相邻的副分光镜的端面平行且面积相同。10 . The spectroscopic HDR camera applied in the field of mobile robot SLAM according to claim 1 , wherein the PCB substrate and the end faces of the adjacent secondary beam splitters are parallel and have the same area. 11 .
CN201911017598.4A 2019-10-24 2019-10-24 A Spectral HDR Camera Applied in the Field of Mobile Robot SLAM Active CN110740238B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911017598.4A CN110740238B (en) 2019-10-24 2019-10-24 A Spectral HDR Camera Applied in the Field of Mobile Robot SLAM

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911017598.4A CN110740238B (en) 2019-10-24 2019-10-24 A Spectral HDR Camera Applied in the Field of Mobile Robot SLAM

Publications (2)

Publication Number Publication Date
CN110740238A true CN110740238A (en) 2020-01-31
CN110740238B CN110740238B (en) 2021-05-11

Family

ID=69271236

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911017598.4A Active CN110740238B (en) 2019-10-24 2019-10-24 A Spectral HDR Camera Applied in the Field of Mobile Robot SLAM

Country Status (1)

Country Link
CN (1) CN110740238B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115942072A (en) * 2022-11-21 2023-04-07 北京航空航天大学 Arbitrary exposure time image acquisition device and method under a dual-processor architecture

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101631202A (en) * 2008-07-16 2010-01-20 肖长诗 Method for acquiring images with super-wide dynamic range
CN101888487A (en) * 2010-06-02 2010-11-17 中国科学院深圳先进技术研究院 High dynamic range video imaging system and image generation method
CN203414717U (en) * 2013-08-23 2014-01-29 重庆米森科技有限公司 Device for frequency-division processing of short-wave infrared and visible light of shooting/photographing equipment
CN105872311A (en) * 2016-05-30 2016-08-17 深圳Tcl数字技术有限公司 High-dynamic-range picture switching method and device
CN207036260U (en) * 2017-06-02 2018-02-23 苏州优函信息科技有限公司 Modularization push-broom type visible ray/near infrared imaging spectrometer
CN108280817A (en) * 2018-01-15 2018-07-13 维沃移动通信有限公司 A kind of image processing method and mobile terminal
CN109068067A (en) * 2018-08-22 2018-12-21 Oppo广东移动通信有限公司 Exposal control method, device and electronic equipment
CN109218628A (en) * 2018-09-20 2019-01-15 Oppo广东移动通信有限公司 Image processing method, device, electronic device and storage medium
CN208890910U (en) * 2018-10-26 2019-05-21 杭州海康威视数字技术股份有限公司 A kind of video camera

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101631202A (en) * 2008-07-16 2010-01-20 肖长诗 Method for acquiring images with super-wide dynamic range
CN101888487A (en) * 2010-06-02 2010-11-17 中国科学院深圳先进技术研究院 High dynamic range video imaging system and image generation method
CN203414717U (en) * 2013-08-23 2014-01-29 重庆米森科技有限公司 Device for frequency-division processing of short-wave infrared and visible light of shooting/photographing equipment
CN105872311A (en) * 2016-05-30 2016-08-17 深圳Tcl数字技术有限公司 High-dynamic-range picture switching method and device
CN207036260U (en) * 2017-06-02 2018-02-23 苏州优函信息科技有限公司 Modularization push-broom type visible ray/near infrared imaging spectrometer
CN108280817A (en) * 2018-01-15 2018-07-13 维沃移动通信有限公司 A kind of image processing method and mobile terminal
CN109068067A (en) * 2018-08-22 2018-12-21 Oppo广东移动通信有限公司 Exposal control method, device and electronic equipment
CN109218628A (en) * 2018-09-20 2019-01-15 Oppo广东移动通信有限公司 Image processing method, device, electronic device and storage medium
CN208890910U (en) * 2018-10-26 2019-05-21 杭州海康威视数字技术股份有限公司 A kind of video camera

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115942072A (en) * 2022-11-21 2023-04-07 北京航空航天大学 Arbitrary exposure time image acquisition device and method under a dual-processor architecture
CN115942072B (en) * 2022-11-21 2024-07-16 北京航空航天大学 Image acquisition device and method for arbitrary exposure time under dual-processor architecture

Also Published As

Publication number Publication date
CN110740238B (en) 2021-05-11

Similar Documents

Publication Publication Date Title
CN109218628B (en) Image processing method, device, electronic device and storage medium
CN109671106B (en) Image processing method, device and equipment
CN110611750A (en) A night scene high dynamic range image generation method, device and electronic equipment
CN103780840A (en) High-quality imaging double camera shooting and imaging device and method thereof
CN110619593A (en) Double-exposure video imaging system based on dynamic scene
CN110493524B (en) A light metering adjustment method, device, device and storage medium
CN110198418B (en) Image processing method, device, storage medium and electronic device
WO2018153313A1 (en) Stereoscopic camera and height acquisition method therefor and height acquisition system
CN103905731B (en) A kind of wide dynamic images acquisition method and system
CN105472265B (en) A kind of device and method obtaining high dynamic range images
CN106679676B (en) A single-field multifunctional optical sensor and its implementation method
CN110430370B (en) Image processing method, device, storage medium and electronic device
CN112361990B (en) Laser pattern extraction method and device, laser measurement equipment and system
CN109361853B (en) Image processing method, image processing device, electronic equipment and storage medium
CN107995396B (en) Two camera modules and terminal
WO2021109409A1 (en) Image capturing method and device, apparatus, and storage medium
CN102404495A (en) Method for adjusting shooting parameters of digital camera
WO2018214838A1 (en) Surveillance image capture method, apparatus, and system
WO2020006078A1 (en) Entropy maximization based auto-exposure
WO2022000974A1 (en) Infrared image processing method, apparatus and device, and storage medium
CN114430462B (en) Unmanned aerial vehicle autonomous photographing parameter adjusting method, device, equipment and storage medium
CN114286024B (en) Optical polarization information model construction method and device based on dynamic vision sensor
CN110266967B (en) Image processing method, image processing device, storage medium and electronic equipment
CN110740238A (en) light splitting HDR camera applied to mobile robot SLAM field
WO2016154873A1 (en) Terminal device and photographing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant