WO2023202387A1

WO2023202387A1 - Optical imaging module, optical imaging system, and terminal device

Info

Publication number: WO2023202387A1
Application number: PCT/CN2023/086660
Authority: WO
Inventors: 陈廷爱; 王庆平; 杨沫
Original assignee: 华为技术有限公司
Priority date: 2022-04-18
Filing date: 2023-04-06
Publication date: 2023-10-26
Also published as: CN116953898A

Abstract

An optical imaging module, an optical imaging system, and a terminal device, which are used to solve the problem in the prior art of not being able to simultaneously achieve an ultra-large field of view and relatively low optical distortion. The present application can be applied to fields such as security monitoring, autonomous driving or intelligent driving. The optical imaging module comprises: a first lens assembly used to transmit first light of a first field of view to a light combining assembly, and a second lens assembly used to transmit second light of a second field of view to the light combining assembly, the first field of view partially overlapping the second field of view, and an angle α between a first optical axis of the first lens assembly and a second optical axis of the second lens assembly satisfying the condition 0° < α < 180°, or 180° < α < 360°. The light combining assembly is configured to mix the first light and the second light to obtain third light. A third lens assembly is used to focus the third light to a probe assembly. The probe assembly is used to form an image according to the focused third light. Therefore, imaging of an ultra-large field of view can be achieved, and optical distortion of a formed image can be reduced.

Description

An optical imaging module, optical imaging system and terminal equipment

Cross-references to related applications

This application requires the priority of a Chinese patent application submitted to the China Patent Office on April 18, 2022, with the application number 202210407071.8 and the application name "An optical imaging module, optical imaging system and terminal equipment", and its entire content has been approved This reference is incorporated into this application.

Technical field

The present application relates to the field of optical imaging technology, and in particular, to an optical imaging module, an optical imaging system and a terminal device.

Background technique

With the development of science and technology, users have higher and higher requirements for the imaging function of optical lenses. For example, imaging with a very large field of view is required. However, conventional optical lenses cannot achieve ultra-wide field of view shooting, and require the use of specially designed optical lenses, such as fisheye lenses (see Figure 1a), panoramic reflection lenses (see Figure 1b) or panoramic ring lenses (see Figure 1c )wait.

Although these optical lenses can capture images with a large field of view, the captured images have large optical distortion. Not only does it distort the edges of the image, but it also affects the resolution of the edge image. Currently, these specially designed optical lenses can achieve imaging with a very large field of view, but the obtained images have large optical distortion. Among them, optical distortion refers to the ratio of the difference between the real image height (y_chief) and the ideal image height (y_ref) and the ideal image height, which can be expressed by the following formula 1, and the ideal image height can be expressed by the following formula 2.
Optical distortion=100%×(y_chief-y_ref)/y_ref Formula 1
y_ref＝f×tanθ Formula 2

Among them, f represents the focal length and θ represents half of the field of view.

To sum up, how to achieve imaging with a very large field of view while making the optical distortion of the captured images smaller is an urgent technical problem that needs to be solved.

Contents of the invention

This application provides an optical imaging module, an optical imaging system and a terminal device, which can not only realize imaging with a large field of view, but also reduce the optical distortion of the captured image.

In a first aspect, the application provides an optical imaging module. The optical imaging module includes a first lens component, a second lens component, a third lens component, a light combining component and a detection component. The first light of the first lens component The angle between the axis and the second optical axis of the second lens component is greater than 0° and less than 180°, or greater than 180° and less than 360°. The first lens component is used to propagate the first light ray from the first field of view to the light combining component; the second lens component is used to propagate the second light ray from the second field of view to the light combining component. The first field of view and the third light combining component The two fields of view partially overlap; the light combining component is used to mix the first light and the second light to obtain a third light; the third lens component is used to focus the third light from the light combining component to the detection component; the detection component is used to An image is formed based on the focused third ray.

Based on the above solution, the first light rays in the first field of view can be collected into the light combining group as much as possible through the first lens assembly. in the transmission aperture of the component; the second light of the second field of view can be collected as much as possible into the transmission aperture of the light combining component through the second lens component; the light combining component mixes the first light and the second light to obtain a third light , to achieve the aliasing of the first field of view and the second field of view; the first light ray and the second light ray are coupled to the same detection component, thereby unmixing and separating the smaller first field of view and the second field of view. , and then splicing, etc., can not only achieve imaging with a large field of view, but also reduce the optical distortion of the formed image. For example, the optical distortion is less than 10% and the combined field of view is greater than 190°. Furthermore, the image formed based on the first field of view and the image formed based on the second field of view can be unmixed, separated and displayed separately, so that one of the fields of view can be flexibly selected as the observation field, which can be applied to directional or peripheral imaging. Visual observation.

In a possible implementation, the angle between the first optical axis and the second optical axis is equal to 90°.

By designing the angle between the first optical axis and the second optical axis to be equal to 90°, it helps to simplify the design and assembly of the optical imaging module.

In a possible implementation, the field of view angle of the first field of view is greater than or equal to 90° and less than or equal to 135°; and/or the field of view angle of the second field of view is greater than or equal to 90° and less than or equal to 135°. 135°.

Further, optionally, the field of view angle of the first field of view is greater than or equal to 98° and less than or equal to 130°; and/or the field of view angle of the second field of view is greater than or equal to 98° and less than or equal to 130°.

Based on this, the optical distortion of the synthetic field of view after combining the first field of view and the second field of view is small, and imaging of an extremely large field of view can be achieved.

In a possible implementation, the first light ray propagated through the first lens component is parallel light or non-parallel light; and/or the second light ray propagated through the second lens component is parallel light or non-parallel light.

Converting the first light of the first field of view into parallel light through the first lens assembly helps to reduce the difficulty of assembling the various lens groups. Converting the first light in the first field of view into non-parallel light through the first lens component helps reduce the size of the entire optical imaging module during design.

In a possible implementation, the light combining component includes a polarizing light splitting element, which is used to reflect the first polarized light in the first light ray to the third lens component and transmit the second polarized light in the second light ray. To the third lens component, the polarization state of the first polarized light is different from the polarization state of the second polarized light.

The polarization splitting element can be used to mix the first polarized light in the first light ray and the second polarized light in the second light ray, thereby obtaining the third light ray.

Further, optionally, the polarizing light splitting element includes a first light combining surface, and the first light combining surface is coated with a polarizing light splitting film or etched with a metal wire grid.

In a possible implementation, the detection component includes a first filter layer and a first photosensitive layer, the first filter layer includes N first filter blocks, and the first filter block includes n first filter units. , at least two of the n first filter units allow the third light to pass through different polarization states, N and n are both integers greater than 1; the first photosensitive layer includes P first pixel blocks, and the A pixel block includes p first pixels, p is greater than or equal to n, and P is a positive integer.

By the number of first pixels included in the first pixel block being greater than or equal to the number of first filter units included in the first filter block, it can be ensured that each first pixel block can detect all polarized light (i.e., first polarized light). light and second polarized light). Moreover, when p is greater than n, multiple first pixels can correspond to one first filter unit, that is, pixel merging can be achieved, thereby helping to increase the signal-to-noise ratio of the formed image.

In a possible implementation, the detection component further includes a second filter layer; the second filter layer includes M second filter blocks, the second filter block includes m second filter units, and mth second filter units. At least two of the two filter units allow the third light to pass through different wavelength ranges, and M and m are both integers greater than 1; a second filter unit corresponds to a first filter unit and belongs to the same second filter unit. The m first filter units corresponding to the m second filter units of the filter block allow the The polarization states of the three light rays passing through are the same; or, the n second filter units corresponding to the n first filter units belonging to the same first filter block allow the third light ray to pass through the same wavelength range.

By dividing the first pixel block based on the number n of first filter units included in the first filter block and the number m of second filter units included in the second filter block, it can be ensured that each first pixel block can be detected The first filter layer allows all polarization states of light (ie, first polarized light and second polarized light) to pass through, and the second filter layer allows all wavelength bands to pass through. In this way, the first light and the second light can completely cover the polarization sensor, thereby improving the utilization rate of the polarization sensor. Furthermore, by collecting one frame of original image based on this detection component, an image with a larger field of view and smaller optical distortion can be obtained, which can be applied to scenes such as video streaming. It also does not require high-capacity image processing bandwidth, helping to save processor computing power.

In a possible implementation, the light combining component is a spectrum splitting element, and the spectrum splitting element is used to reflect k light rays of the first waveband in the first light ray to the third lens assembly, and to reflect k light rays in the second light ray. A second waveband of light is transmitted to the second lens component, a first waveband of light is corresponding to a second waveband of light, and k is an integer greater than 1.

The spectral splitting element can be used to mix part of the first light (i.e., k light rays of the first wave band) and part of the second light (i.e., k light rays of the second wave band), thereby obtaining the third light.

Further, optionally, the spectrum splitting element includes a second light combining surface, and the second light combining surface includes a multi-passband spectral film.

In a possible implementation, the detection component includes a third filter layer and a second photosensitive layer; the third filter layer includes Q third filter blocks, and the third filter block includes at least 2k third filter blocks. Unit, 2k third filter units allow the third light to pass through different wavelength ranges, Q and q are positive integers; the second photosensitive layer includes Q second pixel blocks, and the second pixel block includes at least 2k second pixels , a second pixel block corresponds to a third filter block, and a second pixel corresponds to a third filter unit.

By collecting one frame of original image through this detection component, an image with a larger field of view and smaller optical distortion can be obtained, which can be applied to scenarios such as video streaming. It also does not require high-capacity image processing bandwidth, helping to save processor computing power.

In a possible implementation, the light combining component is specifically configured to reflect the first light to the third lens component during the first period, and to transmit the second light to the third lens component during the second period.

Exemplarily, the light combining component includes a dichroic mirror coated with an electrically controlled film; or a deflection reflector; or a dichroic prism, a first switch and a second switch, wherein the first switch is located between the first lens component and the dichroic prism. , the second switch is located between the second lens assembly and the dichroic prism. Further, the first switch includes a first electronically controlled switch or a first liquid crystal light valve; and/or the second switch includes a second electronically controlled switch or a second liquid crystal light valve.

In a possible implementation, the detection component includes a fourth filter layer and a third photosensitive layer; the fourth filter layer includes H fourth filter blocks, and the fourth filter block includes h fourth filter units , at least two of the h fourth filter units allow the third light to pass through different wavelength ranges, H and h are both integers greater than 1; the third photosensitive layer includes H third pixel blocks, The third pixel block includes h third pixels, one third pixel block corresponds to one fourth filter block, and one fourth filter unit corresponds to one third pixel.

This detection component is compatible with existing image sensors. Moreover, by collecting 2 frames of original images (that is, collecting one frame in the first period and one frame in the second period), an image with a larger field of view and smaller optical distortion can be obtained, which can be applied to scenarios such as video streaming. Moreover, high-capacity image processing bandwidth is not required, helping to save processor computing power.

In a second aspect, this application provides an optical imaging system, including two optical imaging modules of the first aspect or any one of the first aspects, and the detection components of the two optical imaging modules are connected through a bus.

In a third aspect, the present application provides a terminal device. The terminal device includes the above-mentioned first aspect or the method in the first aspect. Any optical imaging module; or the terminal device includes the above second aspect or any optical imaging system in the second aspect.

Further, optionally, the terminal device may also include a processor, and the processor may be used to control the imaging of the optical imaging module.

The technical effects that can be achieved by any one of the above second to third aspects can be referred to the description of the beneficial effects in the above first aspect, and will not be repeated here.

Description of the drawings

Figure 1a is a schematic structural diagram of a fisheye lens in the prior art;

Figure 1b is a schematic structural diagram of a panoramic ring lens in the prior art;

Figure 1c is a schematic structural diagram of a panoramic reflection lens in the prior art;

Figure 2a is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into a smartphone;

Figure 2b is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into a smart tablet;

Figure 2c is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into a smart bracelet;

Figure 2d is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into a camera;

Figure 2e is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into a sweeping robot;

Figure 2f is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into a vehicle;

Figure 2g is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into AR glasses;

Figure 2h is a schematic diagram of an application scenario in which an optical imaging module provided by this application is integrated into a roadside unit;

Figure 3 is a schematic structural diagram of an optical imaging module provided by this application;

Figure 4a is a schematic structural diagram of a first lens assembly provided by this application;

Figure 4b is a schematic structural diagram of another first lens assembly provided by this application;

Figure 5 is a schematic diagram of the angle between the first optical axis and the second optical axis provided by the present application;

Figure 6a is a schematic structural diagram of a third lens assembly provided by the present application;

Figure 6b is a schematic structural diagram of another third lens assembly provided by this application;

Figure 7a is a schematic diagram of the simulation results of a field curvature provided by this application;

Figure 7b is a schematic diagram of the simulation results of optical distortion provided by this application;

Figure 8a is a schematic diagram of the simulation results of another field curvature provided by this application;

Figure 8b is a schematic diagram of another simulation result of optical distortion provided by this application;

Figure 9 is a schematic diagram of a polarizing beam splitter provided by the present application;

Figure 10 is a spectroscopic schematic diagram of a spectrum spectrometer provided by this application;

Figure 11 is a spectroscopic diagram of another light combining component provided by this application;

Figure 12 is a schematic structural diagram of a detection component provided by this application;

Figure 13 is a schematic diagram of the corresponding relationship between a first filter block and a second filter block provided by this application;

Figure 14a is a schematic diagram of the corresponding relationship between another first filter block and a second filter block provided by this application;

Figure 14b is a schematic diagram of a first filter block in a first filter layer provided by this application;

Figure 15 is a schematic diagram of the corresponding relationship between another first filter block and a second filter block provided by this application;

Figure 16 is a schematic structural diagram of a detection component provided by this application;

Figure 17a is a schematic diagram of a third filter layer provided by this application;

Figure 17b is a schematic diagram of another third filter layer provided by this application;

Figure 18 is a schematic structural diagram of a third image sensor provided by this application;

Figure 19a is a schematic diagram of a fourth filter layer provided by this application;

Figure 19b is a schematic diagram of another fourth filter layer provided by this application;

Figure 19c is a schematic diagram of another fourth filter layer provided by this application;

Figure 19d is a schematic diagram of another fourth filter layer provided by this application;

Figure 20 is a schematic structural diagram of a detection component provided by this application;

Figure 21a is a schematic structural diagram of an optical imaging module provided by this application;

Figure 21b is a schematic structural diagram of an optical imaging module provided by this application;

Figure 22 is a schematic structural diagram of an optical imaging system provided by this application;

Figure 23 is a schematic structural diagram of a terminal device provided by this application.

Detailed ways

The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Below, possible application scenarios of this application are introduced. It should be noted that these introductions are for the convenience of understanding by those skilled in the art and do not limit the scope of protection claimed by this application.

In a possible application scenario, the optical imaging module in this application can be integrated into a terminal device or provided in a component of the terminal device. The terminal device can be, for example, a smartphone (see Figure 2a), a smart tablet (see Figure 2b), a smart bracelet (see Figure 2c), a smart home device (see the camera shown in Figure 2d), or smart manufacturing equipment , game consoles, robots (see the sweeping robot shown in Figure 2e) or intelligent transportation equipment (such as automated guided vehicles (AGV) or unmanned transport vehicles, etc.). For example, taking an optical imaging module integrated into a smartphone (which can be called an ultra-wide-angle smartphone), taking photos with an ultra-wide-angle smartphone can help users choose the best shot with the help of the large-field image's understanding of light, shadow, and scenes. angle and shooting position, so as to obtain better shooting results. Alternatively, when using ultra-wide-angle smartphones or game consoles to play games, large-field images can be used to help outdoor users release their focus, allowing outdoor users to focus on the game screen while walking while avoiding dangers. Take the optical imaging module integrated in the camera (which can be called an ultra-wide-angle camera) as an example. The ultra-wide-angle camera can be used in security monitoring, such as panoramic monitoring or environmental monitoring.

In another possible application scenario, the optical imaging module can also be integrated on vehicles (such as unmanned vehicles, smart vehicles, electric vehicles, digital vehicles, etc.) or intelligent transportation equipment as ultra-wide-angle vehicle cameras, as shown in Figure 2f . Ultra-wide-angle vehicle-mounted cameras can obtain measurement information such as the distance of surrounding objects in real time or periodically, thereby providing necessary information for lane correction, vehicle distance maintenance, reversing and other operations. Since the ultra-wide-angle vehicle-mounted camera can observe the surrounding environment with the help of large-field imaging (such as front-view short focus >135°, surround view >180°), it can achieve: a) target recognition and classification, such as various lane line recognition, traffic light recognition, and Traffic sign recognition, etc.; b) Passable space detection (freespace), for example, the safe boundary (travelable area) for vehicles can be divided, mainly for vehicles, ordinary road edges, side stone edges, and boundaries where no obstacles are visible , dividing unknown boundaries, etc.; c) The ability to detect lateral moving targets, such as the detection and tracking of pedestrians and vehicles crossing intersections; d) Positioning and map creation, such as visual simultaneous positioning and map construction (simultaneous localization) and mapping, SLAM) technology positioning and map creation, etc. Ultra-wide-angle vehicle cameras can be used in fields such as unmanned driving, autonomous driving, assisted driving, smart driving or connected cars.

In another possible application scenario, the optical imaging module in this application can also be integrated into a near eye display (NED) device (which can be called an ultra-wide-angle NED device). The ultra-wide-angle NED device can, for example, be an enhanced Augmented reality (AR) equipment or virtual reality (VR) equipment, AR equipment may include but is not limited to AR glasses or AR helmets, and VR devices may include but is not limited to VR glasses or VR helmets. See Figure 2g takes AR glasses as an example. Users can wear AR glasses to play games, watch videos, participate in virtual meetings, live broadcasts, or video shopping, etc. Ultra-wide-angle NED devices can use the illumination information obtained from larger field-of-view images to assist in enhancing the lighting and shadows of generated AR virtual objects, or help NED devices generate multiple interactive virtual objects in a large field-of-view space.

It should be noted that the above application scenarios are only examples. The optical imaging module provided in this application can also be applied in a variety of other scenarios, and is not limited to the above example scenarios. For example, optical imaging modules can also be installed on drones as airborne cameras. For another example, the optical imaging module can also be installed on roadside traffic equipment (such as roadside unit (RSU)) as a roadside traffic camera, see Figure 2h, thus enabling intelligent vehicle-road collaboration, etc. In addition, in the application scenarios given above, the form and position of the optical imaging module are only examples. The optical imaging module can also be set in other possible positions, and the optical imaging module can also be set in other possible forms. This application There are no restrictions on this.

Based on the above content, the optical imaging module proposed in this application will be described in detail below with reference to Figures 3 to 22.

As shown in Figure 3, it is a schematic structural diagram of an optical imaging module provided by this application. The optical imaging module may include a first lens component, a second lens component, a third lens component, a light combining component and a detection component. The angle α between the first optical axis of the first lens component and the second optical axis of the second lens component is greater than 0° and less than 180°, or greater than 180° and less than 360°, please refer to Figure 5 below. It can also be understood that the first optical axis and the second optical axis are not parallel (that is, α is not equal to 180°) and do not overlap (that is, α is not equal to 0°). The first lens component is used to propagate the first light from the first field of view (or sub-field of view 1) to the light combining component. The second lens component is used to propagate the second light from the second field of view (or sub-field of view 2) to the light combining component. Wherein, the angle β between the center line of the first field of view and the center line of the second field of view is the same as the angle α between the first optical axis and the second optical axis. The light combining component is used to mix the first light and the second light to obtain a third light. It can also be understood that part of the first light and part of the second light are mixed into the third light after passing through the combined light component. The third lens component is used to focus the third light from the light combining component to the detection component. The detection component is used to form an image according to the focused third light. It should be noted that the names of the light combining components in this application are only examples, and the light combining components can also split light first and then mix.

The first field of view is different from the second field of view, and the field angle of the first field of view and the field of view angle of the second field of view may be the same or different. Exemplarily, the field of view angle of the first field of view (or the full field of view angle of the first field of view) is greater than or equal to 90° and less than or equal to 135°. Further, the field of view angle of the first field of view is Greater than 98° and less than 130°; and/or, the field of view angle of the second field of view (or the full field of view angle of the second field of view) is greater than or equal to 90° and less than or equal to 135°, further, The field of view angle of the second field of view is greater than 98° and less than 130°. For example, the field of view angle of the first field of view is 90°, 100°, 105°, 110°, 120°, 125°, 130° or 135°, etc., and the field of view angle of the second field of view is 90°, 100°. , 105°, 110°, 120°, 125°, 130° or 135°, etc. Further, the first field of view partially overlaps with the second field of view. In this way, the synthesis of the first field of view and the second field of view can be achieved (for example, it can be unmixed, separated, re-stitched, etc.). For example, 10% of the first field of view overlaps with 10% of the second field of view, and the field of view angle of the first field of view and the field of view angle of the second field of view are both 100°. Based on this, the first field of view and the second field of view overlap. The combined field of view angle of the two fields of view is 190°. For another example, the field of view angle of the first field of view is 100°, the field of view angle of the second field of view is 120°, and 5° of the first field of view overlaps with 5° of the second field of view. Based on this, the first field of view The combined field of view angle of the first field and the second field of view is 215°. For another example, if the field of view angle of the first field of view is the same as that of the second field of view, and the first field of view and the second field of view need to overlap by 15%, then the angle of the first field of view and the second field of view must be 15%. The minimum field of view angle can be 96°.

In a possible implementation, the first lens component and the second lens component may also be called a front lens, and the third lens component may also be called a rear lens. The third lens assembly is located at a certain distance behind the exit of the light combining assembly. It can be understood that the optical imaging module may not include a third lens component. Based on this, the first lens component is required to focus the first light to the detection component, and the second lens component is required to focus the second light to the detection component.

Based on the above optical imaging module, the first light ray of the first field of view can be collected as much as possible into the transmission aperture of the light combining component through the first lens component; the second light ray of the second field of view can be collected through the second lens component Collect as much as possible into the transmission aperture of the light combining component; the light combining component mixes the first light and the second light to obtain the third light, achieving the aliasing of the first field of view and the second field of view; the first light and the second light The light is coupled to the same detection component, so that by unmixing, separating, and re-splicing the smaller first field of view and the second field of view, it is possible to achieve imaging of a very large field of view and reduce the resulting image. optical distortion. For example, the optical distortion is less than 10% and the combined field of view is greater than 190°. Furthermore, the image formed based on the first field of view and the image formed based on the second field of view can be displayed separately after unmixing and separation, so that one of the fields of view can be flexibly selected as the observation field, which can be applied to directional or peripheral vision. observe.

Each functional component shown in Figure 3 will be introduced and explained separately below to provide an exemplary specific implementation solution.

1. Lens components

In a possible implementation, the optical imaging module includes a first lens component and a second lens component, and may further include a third lens component. The third lens component and the second lens component may be the same as the first lens component, or they may be different. The differences between the third lens assembly and the second lens assembly and the first lens assembly may include, but are not limited to, differences in the number of lenses included and/or optical parameters. Among them, the optical parameters may include but are not limited to the radius, thickness, refractive index, abbe number, material, etc. of the lens.

In order to facilitate the explanation of the solution, the first lens component is taken as an example for detailed introduction below.

In a possible implementation, the first lens component includes at least one lens. The lens included in the first lens assembly may be a spherical lens or an aspherical lens. It can also be understood that the first lens assembly may include a single spherical lens, or a single aspherical lens, or a combination of multiple spherical lenses, or a combination of multiple aspherical lenses, or at least one spherical lens and at least one aspherical lens. A combination of spherical lenses. It can be understood that the lenses may include, but are not limited to, concave lenses and convex lenses; further, there are many different types of convex lenses and concave lenses, for example, convex lenses include biconvex lenses, plano-convex lenses, and meniscus lenses, etc., and concave lenses include biconcave lenses, plano-concave lenses, and meniscus-convex lenses. wait. The first lens assembly formed by a combination of multiple spherical lenses and/or aspherical lenses helps to improve imaging quality and reduce aberrations.

Further, optionally, the material of the lens in the first lens assembly may be optical materials such as glass, resin, or crystal. When the material of the lens is resin, it helps to reduce the quality of the optical imaging module. When the material of the lens is glass, it helps to further improve the image quality. Further, in order to effectively suppress temperature drift, the first lens assembly includes at least one lens made of glass material. It should be understood that when the first lens assembly includes at least three lenses, some of the lenses may be made of resin, some of the lenses may be made of glass, and some of the lenses may be made of crystal; or some of the lenses may be made of resin and some of the lenses may be made of crystal. The material is glass; or some of the lenses are made of glass and some of the lenses are crystal; or all of the lenses are made of resin; or all of the lenses are made of glass; or all of the lenses are made of crystal; this application does not limit this.

Please refer to FIG. 4a, which is a schematic structural diagram of a first lens component provided by the present application. In this example, the first lens component includes two lenses. It should be noted that the two lenses included in the first lens assembly can also be of other shapes, please refer to Figure 4b. Based on the first lens assembly shown in Figure 4b, the field of view angle of the first field of view may be 100°.

In a possible implementation, the first lens component may be a component with optical power that can converge the first light rays in the first field of view, that is, the first light rays entering the light combining component are non-parallel rays. Alternatively, the first lens component is a component without optical power. Based on this, the first light ray entering the light combining component is a parallel light ray. Due to engineering constraints and limitations of the receiving aperture of the light combining component, the maximum field of view of the first light entering the light combining component is usually set to ≤35°.

Taking the first lens assembly and the second lens assembly as an example, please refer to FIG. 5 , which exemplarily shows the angle α between the first optical axis of the first lens assembly and the second optical axis of the second lens assembly. . In this example, the angle α=90° between the first optical axis and the second optical axis is taken as an example. It should be noted that the angle α between the first optical axis and the second optical axis can also be other angles greater than 0° and less than 180°, for example, 45°, 90°, 100°, 120°, 130° etc.; or, the angle α between the first optical axis and the second optical axis can also be other angles greater than 180° and less than 360°, for example, 190°, 200°, 240°, etc., which are no longer the same here. List one.

Please refer to Figure 6a and Figure 6b, which are schematic structural diagrams of two third lens assemblies provided by this application. In this example, the third lens assembly includes three lenses. It should be noted that the third lens assembly may include more or less lenses than three lenses. Regarding the lenses, please refer to the aforementioned introduction of the lenses in the first lens assembly, which will not be described again this time.

Taking the first lens component and the second lens component as shown in Figure 4a above and the third lens component as shown in Figure 6a as an example to simulate field curvature and optical distortion, the reference wavelength used for field curvature simulation is 486.1 Nanometer (nm), 587.5nm, 656.3nm. It can be understood that the reference wavelength used can also be 470nm, 510nm, 555nm, 610nm, 650nm, etc. The simulated field curvature results can be seen in Figure 7a below, and the optical distortion can be seen in Figure 7b below. It can be seen from Figure 7a that the field curvature is relatively reasonable. Therefore, the aberrations based on the first lens assembly and the second lens assembly shown in FIG. 4a and the third lens assembly shown in FIG. 6a are small and can be easily corrected. It can be determined from Figure 7b that the optical distortion based on the first lens assembly and the second lens assembly shown in Figure 4a and the third lens assembly shown in Figure 6a are both small, and the maximum optical distortion is less than 27%, indicating that the optical The optical distortion of the imaging module is small.

Taking the first lens component and the second lens component both having the structure shown in Figure 4b above and the third lens component having the structure shown in Figure 6b above as an example to simulate field curvature and optical distortion, the reference wavelength used in the field curvature simulation is 486.1 Nanometer (nm), 587.5nm, 656.3nm. The simulated field curvature results can be seen in Figure 8a below, and the optical distortion can be seen in Figure 8b below. It can be seen from Figure 8a that the field curvature is relatively reasonable. Therefore, the aberrations based on the first lens assembly and the second lens assembly shown in FIG. 4b and the third lens assembly shown in FIG. 6b are small and can be easily corrected. It can be determined from Figure 8b that the optical distortion based on the first lens assembly and the second lens assembly shown in Figure 4b and the third lens assembly shown in Figure 6b are both small, and the maximum optical distortion is less than 14%, indicating that the optical The optical distortion of the imaging module is also smaller.

2. Combined light components

In a possible implementation, the light combining component can mix the first light from the first lens component and the second light from the second lens component to obtain the third light. In other words, the first light and the second light are mixed into the third light after passing through the light combining component.

Based on the light splitting principle of the light combining component, three possible structures are exemplarily shown below.

Structure 1, the light combining component includes a polarizing light splitting element.

In a possible implementation, the polarization splitting element performs light splitting based on the polarization states of the received first light and the second light. For example, the polarizing beam splitter element may be a polarizing beam splitter (PBS). Please refer to Figure 9, which is a schematic diagram of a polarizing beam splitter provided in this application. The polarizing beam splitter can be made by plating one or more layers of polarizing light splitting films or etching metal wire grids on the inclined surface of the right-angle prism (which can be called the first light combined surface), and then bonding them together through an adhesive layer. Taking advantage of the property that when the beam is incident at the Brewster angle, the transmittance of P polarized light (indicated by the arrow) is 1 and the transmittance of S polarized light (indicated by the solid point) is less than 1, after the light beam passes through the film multiple times at the Brewster angle, An optical element that completely transmits the P polarization component and reflects most of the S polarization component (at least 90%). For example, the polarizing beam splitter can divide the incident light (including P-polarized light and S-polarized light) into horizontally polarized light (ie, P-polarized light) and vertically polarized light (ie, S-polarized light). Among them, P-polarized light passes completely, S-polarized light is reflected at an angle of 45 degrees, and the exit direction of S-polarized light is at an angle of 90 degrees to the exit direction of P-polarized light. It can also be understood that PBS has transmission and reflection characteristics. Generally, the reflectance for S-polarized light is above 99.5%, and the transmittance for P-polarized light is above 91%. PBS can realize that the wavelength range of the light splitting film is relatively wide and the degree of polarization is relatively high.

Specifically, the polarization splitting element is used to reflect the first polarized light in the first light ray to the third lens component, and transmit the second polarized light in the second light ray to the third lens. component, the polarization state of the first polarized light is different from the polarization state of the second polarized light. Further, optionally, the first polarized light and the second polarized light are polarized lights whose polarization states are perpendicular to each other. For example, the first polarized light is S-polarized light, the second polarized light is P-polarized light, the S-polarized light of the first light is reflected by the first light combining surface of the polarizing beam splitting element to the third lens component, and the second light of The P-polarized light is transmitted to the third lens component by the first light combining surface of the polarization splitting element. It can be understood that the third light ray includes S-polarized light of the first light ray and P-polarized light of the second light ray.

It can be understood that the polarizing beam splitting prism given above is only an example, and the polarizing beam splitting element may also be a polarizing beam splitting plate, for example. The polarizing light splitting plate is formed by plating one or more layers of polarizing light splitting films (or polarizing light splitting films) on the surface of the glass plate (which can be called the first light combining surface) or etching a metal wire grid. This application does not limit the specific form of the polarizing light splitting element. All forms that can realize the functions of the polarizing light splitting element in this application are within the protection scope of this application.

It should be noted that the optical imaging module may also include a light-absorbing structure that can absorb the P-polarized light of the first light transmitted by the first light combining surface of the polarization splitting element and the S-polarized light of the reflected second light. , thereby reducing interference from unnecessary light when the optical imaging module is imaging.

Structure 2, the light combining component includes a spectrum splitting element.

In a possible implementation, the spectral light splitting element performs light splitting based on the wavelength bands of the received first light and the second light. It can be understood that the colors of light corresponding to different wavelength bands are also different. Therefore, the spectrum splitting element can also be understood as performing light splitting based on the colors of the received first light and the second light. Exemplary. The spectrum spectroscopic element can be plated with a multi-passband spectral film (or called a color separation film) on the second spectroscopic interface. The multi-passband spectral film can separate monochromatic light or narrow-band complex color light in different bands from the complex color light. . Specifically, the wavelength range allowed to be transmitted or reflected by each passband in the multi-passband spectral film can be selected according to actual needs.

Specifically, the spectrum splitting element is used to reflect k light rays of the first wavelength band among the first light rays to the third lens assembly, and transmit k light rays of the second wavelength band among the second light rays to the third lens assembly. In the second lens component, a light of the first waveband corresponds to a light of the second waveband, and k is an integer greater than 1. Furthermore, the k first wave bands have different wavelength ranges, and the k second wave bands have different wavelength ranges. It can be understood that the third rays include k rays of the first waveband and k rays of the second waveband.

Please refer to Figure 10, which is a schematic diagram of a spectroscopic spectrometer provided by this application. In this example, the multi-passband spectral film R1R2G1G2B1B2 coated on the second light-combination surface of the spectrum splitter element is taken as an example. The multi-passband spectrum film R1R2G1G2B1B2 represents the band corresponding to the reflection color R1, the band corresponding to the color G1 and the color B1 of the spectrum spectrometer. The corresponding band, the reflected band can be collectively called the first band; and the band corresponding to the transmitted color R2, the band corresponding to the color G2 and the band corresponding to the color B2, the transmitted band can be collectively called the second band. The band corresponding to each color in Figure 10 can be represented by a rectangular block, and the abscissa represents the wavelength. In other words, the color of the spectral spectroscopic element that reflects the three first wavebands of the first light is represented as R1G1B1, and the color of the three second wavebands of the second light that is transmitted by the spectroscopic element. The color is represented by R2G2B2. Based on this, the third light ray may include the light ray corresponding to R1G1B1 in the first light ray and the light ray corresponding to R2G2B2 in the second light ray. Furthermore, the first band of color R1 corresponds to the second band of color R2. Color R1 and color R2 can synthesize (or approximately synthesize) the color R in the conventional Bayer image sensor, or they can also be other defined colors R; color G1 The first band corresponds to the second band of color G2. Color G1 and color G2 can synthesize (or approximately synthesize) the color G in the conventional Bayer image sensor, or they can also be other defined colors G; the first band of color B1 Corresponding to the second band of color B2, color B1 and color B2 can synthesize (or approximately synthesize) the color B in the conventional Bayer image sensor, or they can also be other defined colors B. Further, optionally, the center wavelength of the first band may be greater than the center wavelength of the corresponding second band, or the center wavelength of the first band may be less than the center wavelength of the corresponding second band, where "1" in R1G1B1 indicates relative Short wavelength (λ), "2" represents a relatively long wavelength.

It can be understood that the multi-passband spectrum film of the second spectroscopic interface of the spectrum splitting element can also be R2R1G2G1B2B1 (representing reflection R2G2B2, transmission R1G1B1), or R1R2Y1Y2B1B2 (representing reflection R1Y1B1, transmission R2Y2B2), or C1C2M1M2Y1Y2 (representing reflection C1M1Y1, Transmission C2M2Y2), we will not list them one by one this time. In this application, there are no restrictions on which wavebands the spectrum splitter element transmits and which wavebands it reflects. The multi-passband spectral film can be selected according to actual needs.

It should be noted that the first light ray may include light rays corresponding to R1R2G1G2B1B2, and the second light rays may also include light rays corresponding to R1R2G1G2B1B2. The optical imaging module may also include a light-absorbing structure that can absorb light corresponding to R2G2B2 in the first light and light corresponding to R1G1B1 in the second light.

Structure 3, the light combining component includes time-division multiplexed light splitting elements.

In a possible implementation, the light combining component can also perform light splitting based on time periods. Specifically, the light combining component is specifically configured to reflect the first light to the third lens component during the first period, and transmit the second light to the third lens component during the second period.

As follows, three possible time-division multiplexed optical splitting elements are exemplified.

Structure 3.1, the time-division multiplexed light splitting element includes a light splitting prism, a first switch (shutter) and a second switch.

It can also be understood that the time-division multiplexed light splitting element includes a light splitting prism configured with a first switch and a second switch, and the light splitting prism is an ordinary light splitting prism. Please refer to Figure 11, which is a schematic light spectrum diagram of another light combining component provided by this application. The light combining component includes a first switch, a second switch and a beam splitting prism. The first switch is located between the first lens assembly and the dichroic prism and is used to control whether the first light ray passes through; the second switch is located between the second lens assembly and the dichroic prism and is used to control whether the second light ray passes through. Please refer to (a) in Figure 11. During the first period, the first switch is turned on (the dotted line indicates that the switch is in the open state), the second switch is closed (the solid line indicates that the switch is in the off state), and the first light can pass through The first switch is reflected to the third lens assembly through the dichroic prism, and the second light is blocked by the second switch and cannot enter the dichroic prism; please refer to (b) in Figure 11. During the second period, the first switch is turned off, and the second switch When it is turned on, the second light can pass through the second switch and be transmitted to the third lens assembly through the dichroic prism. The first light is blocked by the first switch and cannot enter the dichroic prism.

In a possible implementation, the first switch includes a first electronically controlled switch or a first liquid crystal light valve; and/or the second switch includes a second electronically controlled switch or a second liquid crystal light valve.

Structure 3.2, the light combining component includes a deflectable reflector.

In a possible implementation, during the first period, the reflective surface of the deflectable mirror faces the first lens assembly, reflects the first light to the third lens assembly, and blocks the second light from entering the third lens assembly. . Specifically, the angle between the reflective surface of the deflectable mirror and the first optical axis of the first lens assembly is equal to (90°-α/2). For example, when α=90°, the angle between the reflective surface of the deflectable mirror and the first optical axis of the first lens assembly is equal to 45°, and the deflectable reflective light The reflective surface of the mirror can reflect all the first light to the third lens group. In the second period, the reflective surface of the deflectable mirror is used to block the first light from entering the third lens assembly and allow the second light to be transmitted to the third lens assembly. For example, when α=90°, the reflective surface of the polarizable reflective light mirror is parallel to the second optical axis of the second lens component and perpendicular to the first optical axis of the first lens component. It should be noted that the rotation point of the deflectable reflector cannot be at the intersection of the first optical axis and the second optical axis, but should be located close to the first lens assembly to ensure that the second light is not blocked during the second period. and prevent the first light from entering the third lens assembly. It can also be understood that during the second period, the polarizable reflector needs to meet two conditions: 1) allow the second light to enter the third lens assembly; 2) prevent the first light from entering the third lens assembly.

Structure 3.3, the light combining component includes a beam splitter coated with an electrically controlled film.

In the first period, the electrically controlled film that controls the beam splitter reflects the first light to the third lens assembly and blocks the second light from entering the third lens assembly; in the second period, the electrically controlled film that controls the beam splitter reflects the second light to the third lens assembly. Transmit to the third lens component and block the first light from entering the third lens component.

It should be noted that the optical imaging module may also include a light-absorbing structure that can absorb invalid light at any time. The second light at the first time is invalid light, and the first light at the second time is also invalid light. This reduces the interference of unnecessary light when the optical imaging module is imaging.

3. Detection components

In a possible implementation, the third light ray can be focused on the detection component, and the detection component can form an image based on the focused third light ray. Further, optionally, the detection component can photoelectrically convert the received third light to obtain an electrical signal, and form an image based on the electrical signal.

As follows, based on the encoding method of the detection component, three possible detection component types are exemplarily shown.

Type A, the detection component includes a first image sensor.

In a possible implementation, the first image sensor is a polarization sensor, and the pixels of the polarization sensor are polarization pixels. If the detection component is type A, the light combining component is the above-mentioned structure one. In other words, the detection component is a polarization sensor, and the light combining component is a polarization splitter element. Further, the polarization transmittance of the polarized pixel is the same as the polarization transmittance of the third light obtained after passing through the light combining component. For example, the third light ray includes P polarized light and S polarized light, and the polarized pixel can respond to P polarized light or S polarized light.

Please refer to Figure 12, which is a schematic structural diagram of a detection component provided by this application. The detection component is a polarization sensor, and the polarization sensor includes a first filter layer and a first photosensitive layer. Further, the polarization sensor may further include a second filter layer. Wherein, the first filter layer is a polarization filter layer, and the second filter layer is a color filter layer. The color filter layer may be, for example, a color mosaic filter layer. It should be noted that the order of the first filter layer and the second filter layer can be interchanged, that is, the second filter layer is located between the first filter layer and the first photosensitive layer.

Wherein, the first filter layer includes N first filter blocks, the first filter block is the smallest repeatable block of the first filter layer, and the first filter block includes n first filter units, so At least two of the n first filter units allow the third light to pass through different polarization states, and both N and n are integers greater than 1. Please refer to (1) in Figure 13 or (1) in Figure 14a. The first filter block includes two first filter units. One filter unit allows the P-polarized light in the third light to pass (vertical line Filling means that P polarized light is allowed to pass), and the other first filter unit allows S polarized light in the third light to pass (horizontal line filling means that S polarized light is allowed to pass). Further, optionally, the second filter layer includes M second filter blocks, the second filter block is the smallest repeatable block of the second filter layer, and the second filter block includes m second filter blocks. In the optical unit, at least two of the m second filter units allow the third light to pass through different wavelength ranges, and M and m are both integers greater than 1. Please refer to (2) in Figure 13 or (2) in Figure 14a. The second filter block includes three filter units. The three filter units allow the third light to pass through different wavebands. R means allow Through red light, G means green light is allowed to pass through, and B means blue light is allowed to pass through. Wherein, one second filter unit corresponds to one first filter unit.

In order to ensure that each first pixel block can detect all polarization states of light and all wavelength bands, the first photosensitive layer can be based on the number n of the first filter units included in the first filter block and the second filter block. The number m of second filter units included divides the first pixel block, for example, the first pixel block includes a first number of pixels p=n×m. Wherein, the first pixel block is the smallest repeatable block in the first photosensitive layer.

In a possible implementation manner, the m first filter units corresponding to the m second filter units belonging to the same second filter block allow the third light to pass through the same polarization state. Combining (1) and (2) in Figure 13, the three first filter units in the first filter layer corresponding to the three second filter units RGB included in the second filter block allow the polarized light to pass through evenly. It is P-polarized light in the third light ray or both are S-polarized light. Correspondingly, the first pixel block includes 2×3 first pixels. Combining (1) and (2) in Figure 14a, the three first filter units in the first filter layer corresponding to the three second filter units RGB included in the second filter block allow the polarized light to pass through evenly. It is P-polarized light in the third light ray or both are S-polarized light. Correspondingly, the first pixel block includes 3×2 first pixels. In this way, one polarization state can correspond to all wavebands, or one waveband can correspond to all polarization states. It can also be understood that each first pixel can detect all polarized light (ie, P polarized light and S polarized light) and all wavelength bands of light that are allowed to pass through.

Alternatively, the n second filter units corresponding to the n first filter units belonging to the same first filter block allow the third light to pass through the same wavelength range. Please refer to Figure 14b. The first filter block includes four first filter units. One filter unit allows the P-polarized light in the third light to pass (the filled vertical line indicates that the P-polarized light is allowed to pass), and the other filter unit allows the P-polarized light in the third light to pass through. One filter unit allows the S-polarized light in the third light to pass (the horizontal line filling indicates that the S-polarized light is allowed to pass), and the other first filter unit allows the P-polarized light and S-polarized light in the third light to pass (to the left The tilted filling means allowing P polarized light and S polarized light to pass through at the same time), and the first filter unit allows the S polarized light and P polarized light in the third light to pass through (the filling tilted to the right means allowing S polarized light at the same time) and P polarized light transmission). Furthermore, the four first filter units allow the corresponding four second filter units to allow the third light to pass through the same wavelength range. For example, the second filter layer is a monochromatic filter layer, and the monochromatic filter layer refers to Only the light of a certain color in the third light is allowed to pass; or it can be a full-color filter layer. The full-color filter layer means that the light of white light is allowed to pass. Based on this, the number m of the second filter units included in the second filter block can be understood as equal to 1. Further, the number of pixels included in the first pixel block is p=n×1.

It should be noted that the first photosensitive layer divides the first pixel block according to the number n of first filter units included in the first filter block and the number m of second filter units included in the second filter block. There may also be other Possible settings. Please refer to (1) and (2) in Figure 15. The first filter block includes 4 first filter units, the second filter block includes 4 second filter units, and the first pixel block includes 4×4 the first pixel. I won’t list them one by one this time. In this way, the first light and the second light can completely cover the polarization sensor, thereby improving the utilization of the polarization sensor.

Type B, the detection component includes a second image sensor.

In a possible implementation, the second image sensor is a spectrum sensor, and the pixels of the spectrum sensor are spectrum pixels. Correspondingly, the light combining component is the above-mentioned structure 2. In other words, the detection component is a spectrum sensor, and the light combining component is a spectrum splitter element.

Please refer to Figure 16, which is a schematic structural diagram of a detection component provided by this application. The detection component is a spectrum sensor, and the spectrum sensor includes a third filter layer and a second photosensitive layer. The third filter layer may be a color filter layer. For the color filter layer, please refer to the relevant introduction in type A mentioned above, which will not be described again here.

Wherein, the third filter layer includes Q third filter blocks, Q is a positive integer, the third filter block is the smallest repeatable block of the third filter layer, and the third filter block includes at least 2k third filter blocks. optical unit, a third filter unit is used to receive from The third light of the spectrum splitting element is a light of a first wavelength band or a light of a second wavelength band, and the 2k third filter units allow different wavelength bands of the third light to pass through. In a possible implementation manner, the number of third filter units included in the third filter block is the same as the number of passbands of the multi-passband spectral film coated on the second light combining surface. For example, in the above-mentioned multi-passband spectral film R1R2G1G2B1B2 plated on the second combined light surface, the third filter block includes 6 third filter units. These 6 third filter units allow R1, R2, G1, G2, The light corresponding to B1 and B2 passes through, and the distribution pattern of the six third filter units is shown in Figure 17a. Correspondingly, a second pixel block includes 2×3 pixels. Further, the second photosensitive layer includes Q second pixel blocks, the second pixel block includes at least 2k second pixels, one second pixel block corresponds to a third filter block, and one second pixel corresponds to a third filter block. light unit. It should be noted that the distribution mode of the six third filter units shown in Figure 17a is only a possible example, and it can also be any other possible distribution mode, which is not limited in this application.

The multi-passband spectral film R1R2G1G2B1B2 is plated with the second combined light surface. The third filter block includes 4×4 third filter units. These 16 third filter units allow R1, R2, G1, G2, The light corresponding to B1 and B2 passes through. The distribution of these 16 third filter units can be seen in Figure 17b. Correspondingly, a second pixel block includes 4×4 second pixels, and one second pixel corresponds to one third filter unit. Filter unit. It should be noted that the smallest repeatable third filter block in the third filter layer may also include a greater number of third filter units than in the above-mentioned figure 17b, which is not limited in this application. In addition, the number of third filter units included in the third filter block may be different from the number of passbands of the multi-passband spectral film coated on the second light combining surface.

Based on the above-mentioned type A and type B detection components, by collecting one frame of original image, an image with a larger field of view and smaller optical distortion can be obtained, which can be applied to scenes such as video streaming. It also does not require high-capacity image processing bandwidth, helping to save processor computing power.

Type C, the detection component includes a third image sensor.

In a possible implementation, the third image sensor may be a common image sensor (ie, a Bayer image sensor). Correspondingly, the light combining component is the above-mentioned structure three.

In a possible implementation, the third image sensor may include a fourth filter layer and a third photosensitive layer, see FIG. 18 . Wherein, the fourth filter layer may be a color filter layer. The fourth filter layer includes H fourth filter blocks, and the fourth filter block is the smallest repeatable block of the fourth filter layer. The fourth filter block includes h fourth filter units, at least two of the h fourth filter units allow the third light to pass through different wavebands, H and h are both is an integer greater than 1. Please refer to Figure 19a, which is a schematic diagram of a fourth filter layer provided by the present application. For example, the fourth filter layer includes 3×3 fourth filter blocks. The fourth filter block includes 2×2 fourth filter units, which can be expressed as RGGB, and R indicates that the fourth filter unit allows the fourth filter unit to The light in the red band of the three light rays passes through, G indicates that the fourth filter unit allows the light in the green band of the third light ray to pass, and B indicates that the fourth filter unit allows the light in the blue band of the third light ray to pass through. It should be noted that the 2×2 fourth filter units included in the fourth filter block can be expressed as RYYB, R indicates that the fourth filter unit allows the light in the red band of the third light to pass, and Y indicates that the fourth The filter unit allows the light in the yellow band of the third light to pass through, and B indicates that the fourth filter unit allows the light in the blue band of the third light to pass through. Further, the third photosensitive layer includes H third pixel blocks, the third pixel block is the smallest repeatable block of the third photosensitive layer, the third pixel block includes h third pixels, and one third pixel block corresponds to one third pixel block. Four filter blocks, one fourth filter unit corresponds to one third pixel.

It should be noted that the fourth filter layer can also have other possible distributions. For example, the fourth filter layer may include 1×3 fourth filter units, which may be represented as RGB, where R indicates that the fourth filter unit allows light in the red band of the third light to pass, and G indicates that the fourth filter unit Allow the green band light in the third light to pass through, B represents the fourth filter unit Allow the light in the blue band of the third ray to pass, see Figure 19b or Figure 19c. For another example, the fourth filter layer may also include 4×4 fourth filter units, see Figure 19d, in which the 4 fourth filter units all allow the light in the red band of the third light to pass, and 8 fourth Each of the filter units allows the green light in the third light to pass through, and the four fourth filter units all allow the blue light in the third light to pass through.

It should be noted that the fourth filter layer may also be a monochromatic filter layer, which only allows light of a certain color in the third light to pass; or it may be a full-color (black and white) filter. For the light layer, please refer to Figure 20. The full-color filter layer allows white light to pass through.

Based on the above-mentioned type C detection component, by collecting 2 frames of original images (that is, collecting one frame in the first period and collecting one frame in the second period), an image with a larger field of view and smaller optical distortion can be obtained, which can be applied to video Streaming media and other scenarios. It also does not require high-capacity image processing bandwidth, helping to save processor computing power.

It can be understood that the shapes of the first filter layer, the second filter layer, the third filter layer and the fourth filter layer given above are only examples. The above-mentioned filter units can also be in other geometrically symmetrical shapes (such as regular hexagons or rectangles). The filter units can be closely arranged to form a filter layer, and the filter layer can be in a rectangular or square shape. When the filter unit is a square, the filter units are closely arranged to form a filter layer that can be square or rectangular; when the filter unit is a regular hexagon, the filter units are closely arranged to form a whole filter layer It can be roughly rectangular or square in shape, and the edges may be uneven.

Based on the above content, two specific implementation methods of the above optical imaging module are given below in combination with the specific hardware structure. In order to further understand the structure of the above-mentioned optical imaging module and the implementation process of the imaging process of the optical imaging module. It should be noted that, if there are no special instructions or logical conflicts among the above-mentioned modules, they can be combined to form other possible optical imaging modules according to their internal logical relationships.

As shown in Figure 21a and Figure 21b, they are schematic structural diagrams of two optical imaging modules provided by this application. The optical imaging module may include a first lens component, a second lens component, a light combining component, a third lens component and a detection component. In this example, it is assumed that the angle between the first optical axis of the first lens component and the second optical axis of the second lens component is equal to 90°. The first light ray in the first field of view is propagated to the light combining component through the first lens component, and the second light ray in the second field of view is propagated to the light combining component through the light combining component. The light combining component mixes the first light ray and the second light ray. , the third light ray is obtained, the third light ray is focused to the detection component through the third lens component, and the detection component forms an image based on the focused third ray of light. For a detailed introduction to the first lens component, the second lens component, the light combining component, the third lens component and the detection component, please refer to the foregoing relevant descriptions and will not be described again this time.

After the image is obtained based on the above optical imaging module, the first image S1 corresponding to the first field of view and the second image S2 corresponding to the second field of view can be unmixed, separated, calibrated, spliced, etc. in a corresponding manner, and then synthesized Large field of view images that can be directly observed. For example, the distortion of the synthesized image is controllable, the field of view of the synthesized image is >190°, and the optical distortion is <10%.

In order to facilitate the explanation of the solution, take the light combining component as the polarizing light splitting element of the above-mentioned structure 1 as an example, the first filter layer in the detection component as shown in Figure 14b as above, and the second filter layer as a monochromatic filter layer. For example, the first pixel block includes 2×2 pixels. Each first pixel block in the detection component can detect the first light ray in the first field of view and the second light ray in the second field of view. Among them, one first pixel among the four first pixels in the first pixel block can detect S-polarized light, one first pixel can detect P-polarized light, and two first pixels can simultaneously detect S-polarized light and P-polarized light can be separated into the first image S1 of the first field of view and the second image S2 of the second field of view using the following formula 3.

Among them, I ₀ indicates that S-polarized light is detected, I ₉₀ indicates that P-polarized light is detected, I ₁₃₅ indicates that P-polarized light and S-polarized light are detected at the same time, and I ₄₅ indicates that P-polarized light and S-polarized light are detected at the same time.

Further, by performing the following processing on the first image S1 and the second image S2, a large field of view image that can be directly viewed by the user can be generated.

Step a, image preprocessing. It mainly includes feature point extraction and identification, internal parameter reading, distortion correction, brightness adjustment, etc.

Step b, image registration. Mainly includes coordinate conversion and perspective matrix solution.

Step c, image synthesis. Mainly including image fusion, boundary processing, etc.

Step d, image display. Mainly includes data transmission and display.

Based on the architecture and functional principles of the optical imaging module described above, this application can also provide an optical imaging system. Please refer to Figure 22, which is an optical imaging system provided by this application. The optical imaging system includes two optical imaging modules in any of the above embodiments. Further, the two detection components of the two imaging modules can be connected through a bus.

In a possible implementation, the optical imaging system is equivalent to rotating one of the optical imaging modules 180° around the imaging plane of the detection component. It can also be understood that the imaging surfaces (or photosensitive surfaces) of the detection components of the two optical imaging modules are opposite to each other.

Based on this optical imaging system, 360° imaging can be achieved. Moreover, the detection components of the two optical imaging modules are connected through a bus, and the data of the two detection components can be output together through the bus, which helps to avoid asynchronization between the two detection components, so that synchronization can be achieved in hardware.

Based on the architecture and functional principles of the optical imaging module described above, this application can also provide a terminal device. Referring to Figure 23, the terminal device may include at least one processor 2301 and the optical imaging module 2302 in any of the above embodiments. Further, optionally, the optical imaging module may also include a memory 2303. Memory 2303 is used to store programs or instructions. The processor is used to call programs or instructions to control the imaging of the above-mentioned optical imaging module. Processor 2301 executes instructions stored in a non-transitory computer-readable medium such as memory 2303. For the optical imaging module 2302, please refer to the above related introduction, and will not be described again here. The processor 2301 may also be a plurality of computing devices that control individual components or subsystems of the terminal device 2300 in a distributed manner.

The processor 2301 may be a circuit with signal (or data) processing capabilities. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a central processing unit (Central Processing Unit, CPU). , microprocessor, graphics processing unit (GPU) (can be understood as a microprocessor), or digital signal processor (digital signal processor, DSP), etc.; in another implementation, the processor can A certain function is realized through the logical relationship of the hardware circuit. The logical relationship of the hardware circuit is fixed or can be reconstructed. For example, the processor is an application-specific integrated circuit (ASIC) or a programmable logic device. , PLD) implemented hardware circuit, such as FPGA. In a reconfigurable hardware circuit, the process of the processor loading the configuration file and realizing the hardware circuit configuration can be understood as the process of the processor loading instructions to realize the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU). Tensor processing unit (TPU), deep learning processing unit (DPU), etc. Although FIG. 23 functionally illustrates the processor, memory, and other elements of processor 2301 in the same block, one of ordinary skill in the art will understand that the processor and memory may not actually be stored in the same physical enclosure. multiple processors or memories within. For example, the memory may be a hard drive or other storage medium located in a housing different from processor 2301. For another example, the processor can also be remote from the terminal device but can communicate wirelessly with the terminal device.

In some embodiments, memory 2303 may contain instructions (eg, program logic) that may be read by processor 2301 to perform various functions of terminal device 2300, including the functions described above. Memory 2303 may also contain additional instructions, including instructions for sending data to, receiving data from, interacting with, and/or controlling other systems of the end device. In addition to instructions, the memory 2303 can also store data, such as image information acquired by the optical imaging module 2302.

The memory can be, for example, random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically erasable EPROM, EEPROM), register, hard disk, mobile hard disk, CD-ROM or any other form of storage media well known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium.

It should be noted that the functional framework of the terminal device shown in Figure 23 is just an example. In other examples, the terminal device 2300 may include more, less, or different devices, and each device may include more, less or different components. In addition, the devices and components shown can be combined or divided in any way, which is not specifically limited in this application.

For example, the terminal device may be a vehicle (such as an unmanned vehicle, a smart vehicle, an electric vehicle, or a digital vehicle, etc.), a robot, a mapping device, a drone, or a smart home device (such as a television, a sweeping robot, or a smart desk lamp). , audio system, intelligent lighting system, electrical control system, home background music, home theater system, intercom system, or video surveillance, etc.), intelligent manufacturing equipment (such as industrial equipment), intelligent transportation equipment (such as AGV, unmanned transport vehicle) , or trucks, etc.), or smart terminals (mobile phones, watches, computers, tablets, PDAs, desktops, headsets, speakers, wearable devices, vehicle-mounted devices, virtual reality devices, augmented reality devices, etc.), etc.

In the various embodiments of this application, if there is no special explanation or logical conflict, the terms and/or descriptions between different embodiments are consistent and can be referenced to each other. The technical features in different embodiments are based on their inherent Logical relationships can be combined to form new embodiments.

In this application, "vertical" does not mean absolute verticality, and certain engineering errors may be allowed. "At least one" means one or more, and "plurality" means two or more. "And/or" describes the association of associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural. “At least one of the following” or similar expressions thereof refers to any combination of these items, including any combination of a single item (items) or a plurality of items (items). For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c" ”, where a, b, c can be single or multiple. In the text description of this application, the character "/" generally indicates that the related objects are in an "or" relationship. In the formula of this application, the character "/" indicates that the related objects are in a "division" relationship. In addition, in this application, the word "exemplarily" is used to mean an example, illustration or explanation. Any embodiment or design described herein as "example" is not intended to be construed as preferred or advantageous over other embodiments or designs. Alternatively, it can be understood that the use of the word "example" is intended to present concepts in a specific manner and does not constitute a limitation on this application.

It can be understood that the various numerical numbers involved in this application are only for convenience of description and are not used to limit the scope of the embodiments of this application. The size of the serial numbers of the above processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic. The terms "first", "second" and other similar expressions are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "including" and "having" and any variations thereof are intended to cover a non-exclusive inclusion, for example, the inclusion of a series of steps or units. Methods, systems, products or devices are not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such processes, methods, products or devices.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, and all of them should be covered. within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An optical imaging module, characterized in that it includes a first lens component, a second lens component, a third lens component, a light combining component and a detection component, and the first optical axis of the first lens component is in contact with the second lens component. The angle between the second optical axes of the lens assembly is greater than 0° and less than 180°, or greater than 180° and less than 360°;

The first lens component is used to propagate the first light from the first field of view to the light combining component;

The second lens component is used to propagate the second light from the second field of view to the light combining component, and the first field of view partially overlaps with the second field of view;

The light combining component is used to mix the first light and the second light to obtain a third light;

The third lens assembly is used to focus the third light from the light combining assembly to the detection assembly;

The detection component is used to form an image according to the focused third light.
The module of claim 1, wherein the angle between the first optical axis and the second optical axis is equal to 90°.
The module of claim 1 or 2, wherein the viewing angle of the first field of view is greater than or equal to 90° and less than or equal to 135°; and/or,

The field of view angle of the second field of view is greater than or equal to 90° and less than or equal to 135°.
The module according to any one of claims 1 to 3, wherein the first light propagated through the first lens component is parallel light or non-parallel light; and/or,

The second light propagated through the second lens component is parallel light or non-parallel light.
The module according to any one of claims 1 to 4, wherein the light combining component includes a polarizing light splitting element;

The polarization splitting element is used to reflect the first polarized light in the first light ray to the third lens component, and transmit the second polarized light in the second light ray to the third lens component, The polarization state of the first polarized light is different from the polarization state of the second polarized light.
The module of claim 5, wherein the polarizing light splitting element includes a first light combining surface, and the first light combining surface is plated with a polarizing light splitting film or etched with a metal wire grid.
The module of claim 5 or 6, wherein the detection component includes a first filter layer and a first photosensitive layer;

The first filter layer includes N first filter blocks, the first filter block includes n first filter units, and at least two of the n first filter units are first filter units. The polarization states that allow the third light to pass through are different, and both N and n are integers greater than 1;

The first photosensitive layer includes P first pixel blocks, the first pixel block includes p first pixels, p is greater than or equal to n, and P is a positive integer.
The module of claim 7, wherein the detection component further includes a second filter layer;

The second filter layer includes M second filter blocks, the second filter block includes m second filter units, and at least two of the m second filter units allow all The third light ray passes through different waveband ranges, and the M and m are both integers greater than 1;

Among them, one second filter unit corresponds to one first filter unit, and the m first filter units corresponding to the m second filter units belonging to the same second filter block allow the polarization of the third light to pass. The states are the same, or the n second filter units corresponding to the n first filter units belonging to the same first filter block allow the third light to pass through the same wavelength range.
The module according to any one of claims 1 to 4, characterized in that the light combining component is a spectrum splitting element;

The spectrum splitting element is used to reflect k light rays of the first wavelength band among the first light rays to the third lens assembly, and transmit k light rays of the second wavelength band among the second light rays to In the second lens component, a light of the first wave band corresponds to a light of the second wave band, and the k is an integer greater than 1.
The module of claim 9, wherein the spectrum splitting element includes a second light combining surface, and the second light combining surface includes a multi-passband spectral film.
The module of claim 9 or 10, wherein the detection component includes a third filter layer and a second photosensitive layer;

The third filter layer includes Q third filter blocks, the third filter block includes at least 2k third filter units, and the 2k third filter units allow the third light to pass through The band ranges are different, and the Q and q are positive integers;

The second photosensitive layer includes Q second pixel blocks, the second pixel block includes at least 2k second pixels, one second pixel block corresponds to a third filter block, and one second pixel corresponds to a third filter block. light unit.
The module according to any one of claims 1 to 4, characterized in that the light combining component is specifically used for:

The first light is reflected to the third lens component during a first period, and the second light is transmitted to the third lens component during a second period.
The module of claim 12, wherein the light combining component includes any of the following:

Spectroscope coated with electrically controlled film;

deflection reflector;

A dichroic prism, a first switch and a second switch. The first switch is located between the first lens assembly and the dichroic prism. The second switch is located between the second lens assembly and the dichroic prism. .
The module of claim 13, wherein the first switch includes a first electronically controlled switch or a first liquid crystal light valve; and/or,

The second switch includes a second electronically controlled switch or a second liquid crystal light valve.
The module according to any one of claims 12 to 14, wherein the detection component includes a fourth filter layer and a third photosensitive layer;

The fourth filter layer includes H fourth filter blocks, the fourth filter block includes h fourth filter units, and at least two of the h fourth filter units are fourth filter units. The wavelength ranges allowed to pass through the third light ray are different, and H and h are both integers greater than 1;

The third photosensitive layer includes H third pixel blocks, the third pixel block includes h third pixels, one third pixel block corresponds to a fourth filter block, and one fourth filter unit corresponds to a third pixels.
An optical imaging system, characterized by comprising two optical imaging modules according to any one of claims 1 to 15;

Wherein, the detection components of the two optical imaging modules are connected through a bus.
A terminal device, characterized by comprising a processor and an optical imaging module as claimed in any one of claims 1 to 15;

The processor is used to control imaging of the optical imaging module.