CN114402579A - Light filtering device, camera module, mobile terminal and image acquisition method - Google Patents

Light filtering device, camera module, mobile terminal and image acquisition method Download PDF

Info

Publication number
CN114402579A
CN114402579A CN201980100439.2A CN201980100439A CN114402579A CN 114402579 A CN114402579 A CN 114402579A CN 201980100439 A CN201980100439 A CN 201980100439A CN 114402579 A CN114402579 A CN 114402579A
Authority
CN
China
Prior art keywords
filter
frame
image sensor
red
optical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201980100439.2A
Other languages
Chinese (zh)
Inventor
周彦汝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Publication of CN114402579A publication Critical patent/CN114402579A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00Mountings, adjusting means, or light-tight connections, for optical elements

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Optical Filters (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

A filter device (10), a camera module (210), a mobile terminal (3100) and an image acquisition method are provided. The filter device (10) comprises a rotating shaft (11), a support frame (12) and a filter assembly (13). The shaft (11) comprises an axis (110); the support frame (12) is arranged on the rotating shaft (11) and can rotate around the axis (110); the optical filter component (13) arranged on the support frame (12) comprises a plurality of optical filters (130) and can rotate around the axis (110).

Description

Light filtering device, camera module, mobile terminal and image acquisition method Technical Field
The application relates to the technical field of images, in particular to a light filtering device, a camera module, a mobile terminal and an image acquisition method.
Background
Since the pixels in the image sensor can only sense the intensity of light, but cannot distinguish the wavelength of the light (i.e. the color of the light), a color filter needs to be added to each pixel in the image sensor to achieve the color image acquisition. The color image obtained by this method needs to be subjected to processing such as interpolation. However, the colors restored by interpolation are different from the actual colors to some extent, and the colors of the obtained color image are not accurate enough.
Disclosure of Invention
The embodiment of the application provides a light filtering device, a camera module, a mobile terminal and an image acquisition method.
One aspect of the present application provides a light filtering apparatus, which includes a rotating shaft, a supporting frame and a light filter assembly. The shaft includes an axis; the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the support frame. The optical filter assembly comprises a plurality of optical filters, at least part of the optical filters can pass through light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor.
Another aspect of the present application provides a camera module, which includes an image sensor and a filter. The light filtering device is arranged on the light path of the image sensor, and the image sensor can receive light rays passing through the light filtering device. The optical filtering device comprises a rotating shaft, a supporting frame and an optical filter component. The shaft includes an axis; the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the support frame. The optical filter assembly comprises a plurality of optical filters, at least part of the optical filters can pass through light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor.
In another aspect, the present application provides a mobile terminal including a housing and a camera module, wherein the camera module is combined with the housing. The camera module comprises an image sensor and a filtering device. The light filtering device is arranged on the light path of the image sensor, and the image sensor can receive light rays passing through the light filtering device. The optical filtering device comprises a rotating shaft, a supporting frame and an optical filter component. The shaft includes an axis; the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the support frame. The optical filter assembly comprises a plurality of optical filters, at least part of the optical filters can pass through light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor.
In yet another aspect, the present application provides an image collecting method for a camera module, where the camera module includes an image sensor and a filter device, the filter device is disposed on a light path of the image sensor, and the image sensor is capable of receiving light passing through the filter device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the optical filter comprises at least one red optical filter, at least one green optical filter and at least one blue optical filter, wherein the red optical filter is used for filtering light rays except red light rays, the green optical filter is used for filtering light rays except green light rays, and the blue optical filter is used for filtering light rays except blue light rays; the image acquisition method comprises the following steps: rotating the support frame to enable the optical filters to sequentially cover the image sensor; in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, and the original image comprises at least one red original image corresponding to the red optical filter, at least one green original image corresponding to the green optical filter and at least one blue original image corresponding to the blue optical filter; and fusing at least one frame of the red original image, at least one frame of the green original image and at least one frame of the blue original image into at least one frame of color target image.
In yet another aspect, the present application provides an image collecting method for a camera module, where the camera module includes an image sensor and a filter device, the filter device is disposed on a light path of the image sensor, and the image sensor is capable of receiving light passing through the filter device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the optical filter comprises at least one red optical filter, at least one green optical filter, at least one blue optical filter and at least one white optical filter, wherein the red optical filter is used for filtering light except red light, the green optical filter is used for filtering light except green light, and the blue optical filter is used for filtering light except blue light; the white filter is used for filtering light except visible light; the image acquisition method comprises the following steps: rotating the support frame to enable the optical filters to sequentially cover the image sensor; in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, wherein the original image comprises at least one red original image corresponding to the red optical filter, at least one green original image corresponding to the green optical filter, at least one blue original image corresponding to the blue optical filter, and at least one gray original image corresponding to the white optical filter; and fusing at least one frame of the red original image, at least one frame of the green original image, at least one frame of the blue original image and at least one frame of the gray original image into at least one frame of the color target image.
In yet another aspect, the present application provides an image collecting method for a camera module, where the camera module includes an image sensor and a filter device, the filter device is disposed on a light path of the image sensor, and the image sensor is capable of receiving light passing through the filter device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the filter comprises at least one red filter, at least one yellow filter and at least one blue filter, wherein the red filter is used for filtering light rays except red light rays, the yellow filter is used for filtering light rays except yellow light rays, and the blue filter is used for filtering light rays except blue light rays; the image acquisition method comprises the following steps: rotating the support frame to enable the optical filters to sequentially cover the image sensor; in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, and the original image comprises at least one red original image corresponding to the red optical filter, at least one green original image corresponding to the yellow optical filter and at least one blue original image corresponding to the blue optical filter; and fusing at least one frame of the red original image, at least one frame of the yellow original image and at least one frame of the blue original image into at least one frame of color target image.
In yet another aspect, the present application provides an image collecting method for a camera module, where the camera module includes an image sensor and a filter device, the filter device is disposed on a light path of the image sensor, and the image sensor is capable of receiving light passing through the filter device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the optical filter comprises at least one red optical filter, at least one yellow optical filter, at least one blue optical filter and at least one white optical filter, wherein the red optical filter is used for filtering light except red light, the yellow optical filter is used for filtering light except yellow light, and the blue optical filter is used for filtering light except blue light; the white filter is used for filtering light except visible light; the image acquisition method comprises the following steps: rotating the support frame to enable the optical filters to sequentially cover the image sensor; in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, wherein the original image comprises at least one red original image corresponding to the red optical filter, at least one yellow original image corresponding to the yellow optical filter, at least one blue original image corresponding to the blue optical filter, and at least one gray original image corresponding to the white optical filter; and fusing at least one frame of the red original image, at least one frame of the yellow original image, at least one frame of the blue original image and at least one frame of the gray original image into at least one frame of the color target image.
Additional aspects and advantages of embodiments of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of embodiments of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic view of a filter device according to an embodiment of the present disclosure;
FIG. 2 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 3 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 4 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 5 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 6 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 7 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 8 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 9 is a schematic view of yet another optical filtering apparatus according to an embodiment of the present application;
FIG. 10 is a schematic view of yet another optical filtering apparatus according to an embodiment of the present application;
FIG. 11 is a graph showing the wavelength and spectral response of light;
FIG. 12 is a schematic view of another filter device according to an embodiment of the present disclosure;
FIG. 13 is a schematic view of yet another filter arrangement in an embodiment of the present application;
FIG. 14 is a schematic view of yet another optical filtering apparatus according to an embodiment of the present application;
FIG. 15 is a schematic view of yet another optical filtering apparatus according to an embodiment of the present application;
FIG. 16 is a schematic view of yet another filter device in an embodiment of the present application;
FIG. 17 is a schematic view of yet another optical filtering apparatus according to an embodiment of the present application;
FIG. 18 is a schematic view of yet another optical filtering apparatus according to an embodiment of the present application;
FIG. 19 is a schematic view of yet another optical filtering apparatus according to an embodiment of the present application;
FIG. 20 is a schematic view of another filter device according to an embodiment of the present disclosure;
fig. 21 is a schematic view of a camera module according to an embodiment of the present disclosure;
fig. 22 is a schematic view of another camera module according to an embodiment of the present disclosure;
FIG. 23 is a flow chart of an image capture method in an embodiment of the present application;
FIG. 24 is a flow chart of a method of image acquisition in certain embodiments of the present application;
FIG. 25 is a flow chart of yet another method of image acquisition in certain embodiments of the present application;
FIG. 26 is a flow chart of yet another method of image acquisition in an embodiment of the present application;
FIG. 27 is a flow chart of yet another method of image acquisition in certain embodiments of the present application;
FIG. 28 is a schematic diagram illustrating the principles of an image acquisition method in some embodiments of the present application;
FIG. 29 is a flow chart of yet another method of image acquisition in certain embodiments of the present application;
FIG. 30 is a flow chart of yet another method of image acquisition in certain embodiments of the present application;
fig. 31 is a schematic diagram of a mobile terminal according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be further described below with reference to the accompanying drawings. The same or similar reference numbers in the drawings identify the same or similar elements or elements having the same or similar functionality throughout.
In addition, the embodiments of the present application described below in conjunction with the accompanying drawings are exemplary and are only for the purpose of explaining the embodiments of the present application, and are not to be construed as limiting the present application.
In this application, unless expressly stated or limited otherwise, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through intervening media. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
Referring to fig. 1, the present application provides a filter device 10, in which the filter device 10 includes a rotating shaft 11, a supporting frame 12 and a filter assembly 13. The shaft 11 includes an axis 110. The support frame 12 is mounted on the shaft 11 and is rotatable about an axis 110. The optical filter assembly 13 is mounted on the supporting frame 12, the optical filter assembly 13 includes a plurality of optical filters 130, at least some of the optical filters 130 can pass through light beams of different wavelength bands, and the plurality of optical filters 130 are distributed around the rotating shaft 11 at intervals. When the support frame 12 rotates around the axis 110, the plurality of filters 130 also rotate around the axis 110, so that any one of the plurality of filters 130 can selectively cover the image sensor 20 (shown in fig. 21 or 22).
Referring to fig. 21 and 22, the present application provides a camera module 210, wherein the camera module 210 includes a filter device 10 and an image sensor 20. The filter device 10 is disposed on an optical path of the image sensor 20, and the image sensor 20 can receive light passing through the filter device 10. Referring to fig. 1, the filter device 10 includes a shaft 11, a supporting frame 12 and a filter assembly 13. The shaft 11 includes an axis 110. The support frame 12 is mounted on the shaft 11 and is rotatable about an axis 110. The optical filter assembly 13 is mounted on the supporting frame 12, the optical filter assembly 13 includes a plurality of optical filters 130, at least some of the optical filters 130 can pass through light beams of different wavelength bands, and the plurality of optical filters 130 are distributed around the rotating shaft 11 at intervals. When the support frame 12 rotates around the axis 110, the plurality of filters 130 also rotate around the axis 110 to enable any one of the plurality of filters 130 to selectively cover the image sensor 20.
Referring to fig. 31, the present application provides a mobile terminal 3100, where the mobile terminal 3100 includes a housing 3102 and a camera module 210, and the camera module 210 is combined with the housing 3102. Referring to fig. 21 and 22, the present application provides a camera module 210, wherein the camera module 210 includes a filter device 10 and an image sensor 20. The filter device 10 is disposed on an optical path of the image sensor 20, and the image sensor 20 can receive light passing through the filter device 10. Referring to fig. 1, the filter device 10 includes a shaft 11, a supporting frame 12 and a filter assembly 13. The shaft 11 includes an axis 110. The support frame 12 is mounted on the shaft 11 and is rotatable about an axis 110. The optical filter assembly 13 is mounted on the supporting frame 12, the optical filter assembly 13 includes a plurality of optical filters 130, at least some of the optical filters 130 can pass through light beams of different wavelength bands, and the plurality of optical filters 130 are distributed around the rotating shaft 11 at intervals. When the support frame 12 rotates around the axis 110, the plurality of filters 130 also rotate around the axis 110 to enable any one of the plurality of filters 130 to selectively cover the image sensor 20.
Referring to fig. 21 to 23, the present application provides an image capturing method for a camera module 210, wherein the camera module 210 includes a filter device 10 and an image sensor 20. The filter device 10 is disposed on an optical path of the image sensor 20, and the image sensor 20 can receive light passing through the filter device 10. Referring to fig. 1, the filter device 10 includes a shaft 11, a supporting frame 12 and a filter assembly 13. The shaft 11 includes an axis 110. The support frame 12 is mounted on the shaft 11 and is rotatable about an axis 110. The optical filter assembly 13 is mounted on the supporting frame 12, the optical filter assembly 13 includes a plurality of optical filters 130, at least some of the optical filters 130 can pass through light beams of different wavelength bands, and the plurality of optical filters 130 are distributed around the rotating shaft 11 at intervals. When the support frame 12 rotates around the axis 110, the plurality of filters 130 also rotate around the axis 110 to enable any one of the plurality of filters 130 to selectively cover the image sensor 20. The filter 130 includes at least one red filter 131, at least one green filter 132 and at least one blue filter 133, the red filter 131 is used for filtering light except for red light, the green filter 132 is used for filtering light except for green light, and the blue filter 133 is used for filtering light except for blue light. The image acquisition method comprises the following steps:
2301: rotating the support frame 12 to make the plurality of filters 130 sequentially cover the image sensor 20;
2302: during the period that any one of the filters 130 covers the image sensor 20, the image sensor 20 is exposed to obtain an original image corresponding to the filter 130 covered thereon, where the original image includes at least one red original image corresponding to the red filter 131, at least one green original image corresponding to the green filter 132, and at least one blue original image corresponding to the blue filter 133; and
2303: and fusing the at least one frame of red original image, the at least one frame of green original image and the at least one frame of blue original image into at least one frame of color target image.
Referring to fig. 21, 22 and 26, the present application provides an image capturing method for a camera module 210, in which the camera module 210 includes a filter 10 and an image sensor 20. The filter device 10 is disposed on an optical path of the image sensor 20, and the image sensor 20 can receive light passing through the filter device 10. The filtering device 10 may be identical to the filtering device 120 shown in fig. 12, and specifically, referring to fig. 12, the filtering device 120 includes a rotating shaft 1201, a supporting frame 1202 and a filter assembly 1203. The spindle 1201 includes an axis 12010. The support frame 1202 is mounted on the rotating shaft 1201 and can rotate around the axis 12010. The optical filter assembly 1203 is mounted on the supporting frame 1202, the optical filter assembly 1203 includes a plurality of optical filters 12030, at least a portion of the optical filters 12030 can pass through light rays with different wavelength bands, and the optical filters 12030 are distributed around the rotating shaft 1201 at intervals. When the support frame 1202 rotates around the axis 12010, the plurality of filters 12030 also rotate around the axis 12010, so that any one of the plurality of filters 12030 can selectively cover the image sensor 20. The filters 12030 include at least one red filter 12031, at least one green filter 12032, at least one blue filter 12033, and at least one white filter 12034. The red filter 12031 is used for filtering light except red light, the green filter 12032 is used for filtering light except green light, the blue filter 12033 is used for filtering light except blue light, and the white filter 12034 is used for filtering light except visible light. The image acquisition method comprises the following steps:
2601: rotating the support frame 1202 to make the plurality of filters 12030 sequentially cover the image sensor 20;
2602: during a period that any one of the filters 12030 covers the image sensor 20, the image sensor 20 is exposed to obtain an original image corresponding to the filter 12030 covered thereon, where the original image includes at least one frame of red original image corresponding to the red filter 12031, at least one frame of green original image corresponding to the green filter 12032, at least one frame of blue original image corresponding to the blue filter 12033, and at least one frame of gray original image corresponding to the white filter 12034; and
2603: at least one frame of red original image, at least one frame of green original image, at least one frame of blue original image and at least one frame of gray original image are fused into at least one frame of color target image.
Referring to fig. 21, 22 and 29, the present application provides an image capturing method for a camera module 210, in which the camera module 210 includes a filter 10 and an image sensor 20. The filter device 10 is disposed on an optical path of the image sensor 20, and the image sensor 20 can receive light passing through the filter device 10. The filter device 10 may be identical to the filter device 150 shown in fig. 15, and specifically, referring to fig. 15, the filter device 150 includes a rotating shaft 1501, a supporting frame 1502, and a filter assembly 1503. The spindle 1501 includes an axis 15010. The support 1502 is mounted on a pivot 1501 and is capable of rotating about an axis 15010. The filter assembly 1503 is mounted on the support 1502, the filter assembly 1503 includes a plurality of filters 15030, at least some of the filters 15030 can pass light of different wavelength bands, and the plurality of filters 15030 are spaced around the rotation axis 1501. When the support 1502 rotates around the axis 15010, the plurality of filters 15030 also rotate around the axis 15010 to enable any one of the plurality of filters 15030 to selectively cover the image sensor 20. The filter 15030 includes at least one red filter 15031, at least one yellow filter 15035, and at least one blue filter 15033. The red filter 15031 is used for filtering light except red light, the yellow filter 15035 is used for filtering light except yellow light, and the blue filter 15033 is used for filtering light except blue light. The image acquisition method comprises the following steps:
2901: rotating the support frame 1502 so that the plurality of filters 15030 sequentially cover the image sensor 20;
2902: during the period that any one of the filters 15030 covers the image sensor 20, the image sensor 20 is exposed to acquire an original image corresponding to the filter 15030 covered thereon, the original image including at least one frame of red original image corresponding to the red filter 15031, at least one frame of yellow original image corresponding to the yellow filter 15035, and at least one frame of blue original image corresponding to the blue filter 15033; and
2903: and fusing at least one frame of red original image, at least one frame of yellow original image and at least one frame of blue original image into at least one frame of color target image.
Referring to fig. 21, 22 and 30, the present application provides an image capturing method for a camera module 210, in which the camera module 210 includes a filter 10 and an image sensor 20. The filter device 10 is disposed on an optical path of the image sensor 20, and the image sensor 20 can receive light passing through the filter device 10. The filtering device 10 may be identical to the filtering device 180 shown in fig. 18, and specifically, referring to fig. 18, the filtering device 180 includes a rotating shaft 181, a supporting frame 182, and a filter assembly 183. The shaft 181 includes an axis 1810. The supporting frame 182 is mounted on the rotating shaft 181 and can rotate around the axis 1810. The filter assembly 183 is mounted on the supporting frame 182, the filter assembly 183 includes a plurality of filters 1830, at least some of the filters 1830 can pass light of different wavelength bands, and the plurality of filters 1830 are spaced around the rotating shaft 181. As the support bracket 182 rotates about the axis 1810, the plurality of filters 1830 also rotate about the axis 1810 to enable any one of the plurality of filters 1830 to selectively cover the image sensor 20. The filters 1830 include at least one red filter 1831, at least one yellow filter 1835, at least one blue filter 1833, and at least one white filter 1834. The red filter 1831 is used for filtering light other than red light, the yellow filter 1835 is used for filtering light other than yellow light, the blue filter 1833 is used for filtering light other than blue light, and the white filter 1834 is used for filtering light other than visible light. The image acquisition method comprises the following steps:
3001: rotating the support frame 182 such that the plurality of filters 1830 sequentially cover the image sensor 20;
3002: during the period that any one of the filters 1830 covers the image sensor 20, the image sensor 20 is exposed to light to acquire an original image corresponding to the filter 1830 covered thereon, where the original image includes at least one red original image corresponding to the red filter 1831, at least one yellow original image corresponding to the yellow filter 1835, at least one blue original image corresponding to the blue filter 1833, and at least one gray original image corresponding to the white filter 1834; and
3003: at least one frame of red original image, at least one frame of yellow original image, at least one frame of blue original image and at least one frame of gray original image are fused into at least one frame of color target image.
Embodiments of the present application are further described below with reference to the accompanying drawings.
FIG. 11 is a graph of the wavelength and spectral response of light transmitted through a red filter, a green filter, and a blue filter. As can be seen from fig. 11, the red filter can pass visible light with a wavelength range of 560nm to 670nm, and mainly transmits red light with a wavelength range of 580nm to 650nm, the green filter can pass visible light with a wavelength range of 470nm to 620nm, and mainly transmits green light with a wavelength range of 480nm to 590nm, the blue filter can pass visible light with a wavelength range of 420nm to 520nm, and mainly transmits blue light with a wavelength range of 430nm to 510 nm.
Since the pixels in the image sensor can only sense the intensity of light, the wavelength of the light cannot be resolved, that is, the pixels in the image sensor cannot resolve the color of the light. Therefore, in the related art, a red filter, a color filter, and a blue filter are generally disposed on pixels at different positions in an image sensor, respectively, to acquire a color image. The scheme increases the processing difficulty of the pixels by arranging the optical filter for each pixel in the image sensor, and the utilization rate of light energy is low because each pixel can only transmit light rays with the color corresponding to the optical filter. Meanwhile, the color image acquired by this method needs to be processed by interpolation or the like. However, the colors restored by interpolation are different from the actual colors to some extent, and the colors of the obtained color image are not accurate enough.
For the above reasons, one aspect of the present application provides a filter device 10, as shown in fig. 1, the filter device 10 includes a rotating shaft 11, a supporting frame 12 and a filter assembly 13. The shaft 11 includes an axis 110, and the axis 110 is perpendicular to the plane of the support frame 12. The support 12 is mounted on the shaft 11 and the support 12 is rotatable about an axis 110. The filter assembly 13 is mounted on the support frame 12, and the filter assembly 13 includes a plurality of filters 130, at least some of the filters 130 can pass light of different wavelength bands, for example, the red filter 131 can only pass light of a red wavelength band; the green filter 132 can pass only light of a green wavelength band; the blue filter 133 can pass only light of a blue wavelength band. The plurality of filters 130 are spaced around the rotating shaft 11, and when the supporting frame 12 rotates around the rotating shaft 11, the plurality of filters 130 also rotate around the axis 110, so that any one of the plurality of filters 130 can selectively cover the image sensor 20 (shown in fig. 21 or 22).
In some embodiments, referring to fig. 1, the supporting frame 12 may be fixedly connected to the rotating shaft 11, that is, the supporting frame 12 and the rotating shaft 11 rotate at the same speed and in the same direction. The connection manner of the support frame 12 and the rotating shaft may be at least one of welding, clamping, or screwing, which is not limited herein. In other embodiments, the support 12 may be rotatably connected to the shaft 11, that is, the support 12 and the shaft 11 rotate asynchronously, for example, the shaft 11 is stationary and only the support 12 rotates around the axis 110; or, the support frame 12 and the rotating shaft 11 rotate in the same direction at different speeds; or, the support frame 12 and the rotating shaft 11 rotate at the same speed in different directions; or, the support frame 12 and the rotating shaft 11 rotate in different directions and at different speeds.
In some embodiments, as shown in fig. 1 to 9, the center of the supporting frame 12 may be located on the axis 110 of the rotating shaft 11; alternatively, as shown in fig. 10, the center of the supporting frame 12 may not be located on the axis (not shown) of the rotating shaft 11.
In some embodiments, referring to fig. 1, the plurality of filters 130 are uniformly distributed around the rotating shaft 11. The filter 130 includes a red filter 131, a green filter 132, and a blue filter 133.
In one example, referring to fig. 2, the supporting frame 12 is a transparent solid structure, and the material of the supporting frame 12 may be plastic, etc., but the supporting frame 12 may also be other transparent materials. The supporting frame 12 is a polygonal solid structure, such as a triangular light-transmitting solid structure, the supporting frame 12 of the triangular structure has three vertexes A, B and C, each optical filter 130 is disposed corresponding to the vertexes A, B and C of the supporting frame 12, each optical filter 130 may be directly mounted at the vertexes A, B and C of the supporting frame 12, or may be other structures, such as a tray, a carrying frame, etc., which are mounted at the vertexes A, B and C of the supporting frame 12 first, and then the optical filter 130 is mounted on the tray, the carrying frame (which needs to be hollow or have high light transmittance), etc. The orthographic projection of the center of each filter 130 on the support frame 12 coincides with the corresponding vertex, and the vertical distances from the centers of any two filters 130 to the axis 110 are the same. Specifically, the orthographic projection of the center of the red filter 131 on the support 12 coincides with the corresponding vertex a, the orthographic projection of the center of the green filter 132 on the support 32 coincides with the corresponding vertex B, and the orthographic projection of the center of the blue filter 133 on the support 12 coincides with the corresponding vertex C. The orthogonal projection of the center of the red filter 131 on the support frame 12 to the axis 110 is equal to the orthogonal distance I1 of the orthogonal projection of the center of the green filter 132 on the support frame 12 to the axis 110, I2; the orthogonal projection of the center of the red filter 131 on the support frame 12 is equal to the orthogonal distance I1 of the orthogonal projection of the center of the blue filter 133 on the support frame 12 to the axis 110, I3; the orthogonal projection of the center of the green filter 132 on the support 12 to the axis 110 is equal to the orthogonal distance I2 of the orthogonal projection of the center of the blue filter 133 on the support 12 to the axis 110, I3. I.e., I1 ═ I2 ═ I3.
In another example, referring to fig. 3, the supporting frame 12 has a polygonal structure, and the vertex of the polygonal structure is opened with a mounting frame 120, specifically, the supporting frame 12 is opened with a plurality of hollow mounting frames 120, for example, the plurality of mounting frames 120 includes a first mounting frame 121, a second mounting frame 122, and a third mounting frame 123. The support frame 12 includes a first side 124, a second side 125, and a third side 126. The first side 124 connects the first mounting frame 121 and the second mounting frame 122, the second side 125 connects the second mounting frame 122 and the third mounting frame 123, and the third side 126 connects the first mounting frame 121 and the third mounting frame 123. The center of each filter 130 coincides with the corresponding vertex of the support frame 12, and the perpendicular distances from the centers of any two filters 130 to the axis 110 are the same. Specifically, if the first side 124, the second side 125, and the third side 126 are extended until two of them intersect to form a virtual triangle, the center of the first mounting frame 121, the center of the second mounting frame 122, and the center of the third mounting frame 123 coincide with three vertices of the triangle, respectively. The red filter 131, the green filter 132, and the blue filter 133 may be mounted (housed) in the first mounting frame 121, the second mounting frame 122, and the third mounting frame 123, respectively, and at this time, the center of the red filter 131, the center of the green filter 132, and the center of the blue filter 133 may coincide with the three vertices of the triangle, respectively. Similarly, the perpendicular distance I1 (not shown) from the center of the red filter 131 to the axis 110 is equal to the perpendicular distance I2 (not shown) from the center of the green filter 132 to the axis 110; the perpendicular distance I1 (not shown) from the center of the red filter 131 to the axis 110 is equal to the perpendicular distance I3 (not shown) from the center of the blue filter 133 to the axis 110; the perpendicular distance I2 (not shown) from the center of the green filter 132 to the axis 110 is equal to the perpendicular distance I3 (not shown) from the center of the blue filter 133 to the axis 110. I.e., I1 ═ I2 ═ I3. In this case, the supporting frame 12 may be made of a transparent material or a non-transparent material.
When the supporting frame 12 is a polygonal solid structure, the shape of the optical filter 130 is not limited, and may be any shape such as a circle, a square, a pentagon, an ellipse, etc.; when the supporting frame 12 is a 120-polygon structure with a mounting frame at the vertex, the shape of the optical filter 130 is not limited, and may be any shape such as a circle, a square, a pentagon, an ellipse, etc., but needs to be matched with the shape of the mounting frame 120, that is, when the mounting frame 120 is a circle, the optical filter 130 is also a circle; when the mounting frame 120 is square, the filter 130 is also square. The supporting frame 12 may have a regular polygonal structure, such as a regular triangle structure or a regular pentagon structure.
When the image sensor is used, if the target filter 130 (any one of the red filter 131, the green filter 132, and the blue filter 133) is to be covered above the image sensor 20, so that the image sensor 20 receives the light filtered by the target filter 13 to form an image, the support frame 12 is only required to be rotated to drive the target filter 130 to move above the image sensor 20. Specifically, when the support frame 12 is fixedly connected with the rotating shaft 11, the support frame 12 can be rotated by rotating the rotating shaft 11; when the support frame 12 is rotatably connected to the rotating shaft 11, the support frame 12 can be driven by the driving device to rotate relative to the rotating shaft 11. When the support frame 12 has a regular polygon structure, if the plurality of filters 13 are sequentially switched to correspond to the image sensor 20, the angles of rotation of the support frame 12 are equal, and the control is easier.
The filters may include a plurality of red filters, a plurality of green filters, and a plurality of blue filters. For example, referring to fig. 4, in the filter device 40, the filter 430 includes a first red filter 4311, a second red filter 4312, a first green filter 4321, a second green filter 4322, a first blue filter 4331 and a second blue filter 4332. In this case, in an example, the supporting frame 42 may be a regular hexagonal transparent solid structure having six vertices A, B, C, D, E and F, each of the filters 430 is disposed corresponding to the six vertices A, B, C, D, E and F of the supporting frame 42, each of the filters 430 may be directly mounted at the vertices A, B, C, D, E and F of the supporting frame 42, or may be other structures, such as a tray, a carrying frame, etc., which are mounted at the vertices A, B, C, D, E and F of the supporting frame 42 first, and then the filters 430 are mounted on the tray, the carrying frame (which needs to be hollow or have high transmittance), etc. The orthographic projection of the center of each filter 430 on the support frame 42 coincides with the corresponding vertex, and the vertical distances from the centers of any two filters 430 to the axis 410 are the same. Specifically, an orthogonal projection of the center of the first red filter 4311 on the support 42 coincides with the corresponding vertex a, an orthogonal projection of the center of the first green filter 4321 on the support 42 coincides with the corresponding vertex B, an orthogonal projection of the center of the first blue filter 4331 on the support 42 coincides with the corresponding vertex C, an orthogonal projection of the center of the second red filter 4312 on the support 42 coincides with the corresponding vertex D, an orthogonal projection of the center of the second green filter 4322 on the support 42 coincides with the corresponding vertex E, and an orthogonal projection of the center of the second blue filter 4332 on the support 42 coincides with the corresponding vertex F. The perpendicular distance I1 from the center of the first red filter 4311 to the axis 410, the perpendicular distance I2 from the center of the first green filter 4321 to the axis 410, the perpendicular distance I3 from the center of the first blue filter 4331 to the axis 410, the perpendicular distance I4 from the center of the second red filter 4312 to the axis 410, the perpendicular distance I5 from the center of the second green filter 4322 to the axis 410, and the perpendicular distance I6 from the center of the second blue filter 4332 to the axis 410 are all the same. That is, I1 ═ I2 ═ I3 ═ I4 ═ I5 ═ I6.
In another example, the supporting frame 42 may also be a regular hexagon structure with a mounting frame at its vertex, and specifically, six hollow mounting frames are disposed on the supporting frame 42. The support bracket 42 includes six sides. Six sides are respectively connected with two adjacent mounting frames. The center of each filter 430 coincides with the corresponding vertex of the support frame 42, and the perpendicular distances from the centers of any two filters 430 to the axis 410 are the same. Specifically, if the six sides are extended until two sides intersect to form a virtual hexagon, the centers of the six mounting frames coincide with six vertices of the hexagon respectively. When the first red filter 4311, the second red filter 4312, the first green filter 4321, the second green filter 4322, the first blue filter 4331 and the second blue filter 4332 are respectively mounted (accommodated) in six mounting frames, the center of the first red filter 4311, the center of the second red filter 4312, the center of the first green filter 4321, the center of the second green filter 4322, the center of the first blue filter 4331 and the center of the second blue filter 4332 respectively coincide with six vertices of a hexagon. Similarly, the perpendicular distance I1 from the center of the first red filter 4311 to the axis 410, the perpendicular distance I2 from the center of the first green filter 4321 to the axis 410, the perpendicular distance I3 from the center of the first blue filter 4331 to the axis 410, the perpendicular distance I4 from the center of the second red filter 4312 to the axis 410, the perpendicular distance I5 from the center of the second green filter 4322 to the axis 410, and the perpendicular distance I6 from the center of the second blue filter 4332 to the axis 410 are all the same. That is, I1 ═ I2 ═ I3 ═ I4 ═ I5 ═ I6. In this case, the supporting frame 42 may be made of a transparent material or a non-transparent material.
The filter device 40 has the same beneficial effects and using processes as the filter device 10, and further, since the filter device 40 includes a plurality of red filters, a plurality of green filters and a plurality of blue filters, the angle of rotation required for the supporting frame 42 is smaller each time the filters 430 are switched, the switching is quicker and the imaging speed is faster. For example, in the optical filter device 10, if the red filter 131 is placed over the image sensor 20 in the initial position, and the green filter 132 is placed over the image sensor 20 in order to switch the placement of the red filter 131 over the image sensor 20, the support frame 12 needs to be rotated by 120 degrees. In the filter device 40, if the first red filter 4311 is supposed to cover the image sensor 20 at the initial position, and the first green filter 4321 is supposed to cover the image sensor 20 when the first red filter 4311 is supposed to cover the image sensor 20, the supporting frame 42 only needs to rotate 60 degrees, and the switching is quicker.
In some embodiments, each filter is circular, each filter being tangent to both of the adjacent edges at the corresponding vertex; or the projection of each filter on the plane where the support frame is located is tangent to two adjacent edges at the corresponding vertex. Referring to fig. 5, in an exemplary filter device 50, the filters 530 are circular, each filter 530 is disposed on the support frame 52, and two adjacent edges of the filter 530 at the corresponding vertex of the support frame 52 are tangent to each other. Specifically, the filter 530 includes a red filter 531, a green filter 532, and a blue filter 533, and the red filter 531, the green filter 532, and the blue filter 533 are circular. The supporting frame 52 comprises a first edge 521, a second edge 523 and a third edge 525, the green filter 532 is arranged between the first edge 521 and the second edge 523, and the green filter 532 is tangent to the first edge 521 and the second edge 523; the red filter 531 is disposed between the first edge 521 and the third edge 525, and the red filter 531 is tangent to the first edge 521 and the third edge 525; the blue filter 533 is disposed between the second edge 523 and the third edge 525, and the blue filter 533 is tangent to the second edge 523 and the third edge 525. The optical filter 530 may be directly tangent to two adjacent edges at the corresponding vertex of the support frame 52, or the support frame 52 may be provided with a hollow mounting frame that is tangent to two adjacent edges at the corresponding vertex, and the optical filter 530 is installed in the mounting frame and then tangent to two adjacent edges at the corresponding vertex of the support frame 52. In another example of the filtering apparatus 50, the filters 530 are circular, and other structures, such as a tray, a carrying frame, etc., are first installed at the vertices A, B and C of the supporting frame 52, and then the filters 530 are installed on the tray, the carrying frame (which needs to be hollow or have high transmittance), etc., so that the projection of each filter 530 on the plane of the supporting frame 52 is tangent to the two adjacent edges at the corresponding vertex. Specifically, the projection of the green filter 532 on the plane of the supporting frame 52 is tangent to the first edge 521 and the second edge 523; the projection of the red filter 531 on the plane of the support frame 52 is tangent to the first side 521 and the third side 525; the projection of the blue filter 533 on the plane of the supporting frame 52 is tangent to the second side 523 and the third side 525. The filter device 50 has the same beneficial effects and using processes as those of the filter device 10, and further, compared with the filter device 10, on one hand, the projection of the filter 530 is closer to the range of the plane of the supporting frame 52, and the space occupied by the filter 530 is smaller, so that the structure of the filter device 50 is more compact and the filter device is easier to install in the camera module 210; on the other hand, the supporting frame 52 provides support and protection for the optical filter 530, so as to prevent the optical filter 530 from deforming or moving during the rotation process, and improve the service life of the optical filter device 50.
Referring to fig. 6, each filter 630 is circular, and each filter 630 is tangent to two adjacent edges at the corresponding vertex; or under the condition that the projection of each filter 630 on the plane where the support frame 62 is located is tangent to the two adjacent edges at the corresponding vertex, the support frame 62 may have a regular hexagon structure, and the filter 630 may also include a plurality of red filters (6311, 6312), a plurality of green filters (6321, 6322), and a plurality of blue filters (6331, 6332), and the specific structure of the support frame 62 is please refer to the specific structure of the support frame 52 of the filter device 50 in fig. 5, which is not described herein again. The filter device 60 has the same beneficial effects and using processes as those of the filter device 50, and further, since the filter device 60 includes a plurality of red filters (6311, 6312), a plurality of green filters (6321, 6322) and a plurality of blue filters (6331, 6332), the angle of rotation required by the supporting frame 62 is smaller, the switching is quicker and the imaging speed is faster each time the filters 630 are switched.
Referring to fig. 7, in the optical filtering device 70, the supporting frame 72 is a triangle-like structure, the supporting frame 72 includes a plurality of sides, two adjacent sides are connected by an arc, the optical filter 730 is disposed between the two adjacent sides, and a circumferential portion of the optical filter 730 coincides with the corresponding arc. Specifically, the filter 730 includes a red filter 731, a green filter 732, and a blue filter 733, and the red filter 731, the green filter 732, and the blue filter 733 are circular. The supporting frame 72 includes a first side 721, a second side 723, a third side 725, a first arc 722, a second arc 724, and a third arc 726, the first arc 722 connects the first side 721 with the third side 723, the second arc 724 connects the third side 723 with the third side 725, and the third arc 726 connects the first side 721 with the third side 725. The green filter 732 is disposed between the first side 721 and the second side 723 and a circumferential portion of the green filter 732 coincides with the first arc 722; the red filter 731 is disposed between the first side 721 and the third side 725, and a circumferential portion of the red filter 731 coincides with the third arc 726; the blue filter 733 is disposed between the second side 723 and the third side 725, and a circumferential portion of the blue filter 733 coincides with the second arc line 724. The filter device 70 has the same beneficial effects and using processes as those of the filter device 50, and further, since the supporting frame 72 of the filter device 70 has a triangle-like structure, the supporting frame 72 includes a plurality of sides, and two adjacent sides are connected by an arc, the space occupied by the supporting frame 72 is smaller, so that the structure of the filter device 70 is more compact and the filter device is easier to be installed in the camera module 210.
Referring to fig. 8, in the light filtering device 80, the supporting frame 82 may have a regular hexagon-like structure, and the filter 830 may also include a plurality of red filters (8311, 8312), a plurality of green filters (8321, 8322) and a plurality of blue filters (8331, 8332), and the specific structure of the supporting frame 82 is combined with the specific structure of the supporting frame 72 of the light filtering device 70 in fig. 7, which is not described herein again. The filter device 80 has the same beneficial effects and using processes as the filter device 70, and further, since the filter device 80 includes a plurality of red filters (8311, 8312), a plurality of green filters (8321, 8322) and a plurality of blue filters (8331, 8332), the angle of rotation required by the support frame 82 is smaller each time the filters 830 are switched, the switching is quicker and the imaging speed is faster.
Referring to fig. 9, in the filter device 90, the supporting frame 92 may have a circular structure, the filters 930 are also circular, and each filter 930 is inscribed in the supporting frame 92, or a projection of each filter 930 on a plane where the supporting frame 92 is located is inscribed in the supporting frame 92. Specifically, the filter 930 includes a red filter 931, a green filter 932, and a blue filter 933. In one example, the filter 930 can be directly inscribed within the holder 929. For example, the red filter 931 is directly inscribed in the support frame 92, the green filter 932 is directly inscribed in the support frame 92, the blue filter 933 is directly inscribed in the support frame 92, and any two filters 930 are circumscribed, for example, the red filter 931 is circumscribed with the green filter 932, the red filter 931 is circumscribed with the blue filter 933, and the blue filter 933 is circumscribed with the green filter 932. In another example, the supporting frame 92 is provided with a plurality of hollow mounting frames, each mounting frame is internally tangent to the outer circumference of the supporting frame 92, and each optical filter 930 is internally tangent to the supporting frame 92 after being mounted in the corresponding mounting frame. In another example, the filters 930 are circular, and other structures, such as a tray, a carrying frame, etc., are first installed on the supporting frame 92, and then the filters 930 are installed on the tray, the carrying frame (which needs to be hollow or have high transmittance), etc., so that the projection of each filter 930 on the plane where the supporting frame 92 is located is inscribed in the supporting frame 92, for example, the projection of the red filter 931 on the plane where the supporting frame 92 is located is directly inscribed in the supporting frame 92, the projection of the green filter 932 on the plane where the supporting frame 92 is located is directly inscribed in the supporting frame 92, and the projection of the blue filter 933 on the plane where the supporting frame 92 is located is directly inscribed in the supporting frame 92. The filter device 90 has the same beneficial effects and using processes as those of the filter device 70, and further, since the supporting frame 92 of the filter device 90 has a circular structure, the space occupied by the supporting frame 92 is smaller, so that the structure of the filter device 90 is more compact and the filter device is easier to install in the camera module 210. Moreover, any two filters 930 are circumscribed, so that the plurality of filters 930 are closer to each other, and the space occupied by the plurality of filters 930 is smaller, further making the structure of the filter device 90 more compact.
Similarly, in other embodiments, the filters may also include a plurality of red filters, a plurality of green filters and a plurality of blue filters, adjacent filters are circumscribed with each other, and each filter is still inscribed in (the outer circumference of) the supporting frame 92. At this time, compared to the filter device 90 shown in fig. 9, the filter device in the present embodiment includes a plurality of red filters, a plurality of green filters, and a plurality of blue filters, and the angle of rotation required for the support frame is smaller, the switching is faster, and the imaging speed is faster each time the filters are switched.
Referring to fig. 10, in the light filtering device 100, the supporting frame 102 may be a fan-shaped structure, and the filter 1030 includes a red filter 1031, a green filter 1032 and a blue filter 1033. The supporting frame 102 includes a first radius edge 1021, a second radius edge 1022, a first arc edge 1023 and a second arc edge 1024. The first arc edge 1023 and the second arc edge 1024 are connected with a first radius edge 1021 and a second radius edge 1022, and the rotating shaft 101 is arranged at the center of the fan and connected with the first radius edge 1021 and the second radius edge 1022. In one example, the plurality of filters 1030 are each disposed between the first arc edge 1023 and the second arc edge 1024. Wherein, the red filter 1031 is directly inscribed with the first arc edge 1023 and directly circumscribed with the second arc edge 1024; the green filter 1032 is directly internally tangent to the first arc edge 1023 and directly externally tangent to the second arc edge 1024; the blue filter 1033 is directly inscribed in the first arc edge 1023 and directly circumscribed with the second arc edge 1024. Similarly, the filter 1030 may also include a plurality of red filters, a plurality of green filters and a plurality of blue filters, which are not described herein.
In another example, the supporting frame 102 is provided with a plurality of hollow mounting frames, each mounting frame is directly internally tangent to the first arc edge 1023 and directly externally tangent to the second arc edge 1024, and each optical filter 1030 is installed in the corresponding mounting frame and then is internally tangent to the first arc edge 1023 and externally tangent to the second arc edge 1024.
In yet another example, the filter 1030 is circular. Other structures (such as a tray, a carrying frame, etc.) are firstly installed on the supporting frame 102, and then the optical filters 1030 are installed on the tray, the carrying frame (which needs to be hollow or have high light transmittance), etc., so that the projection of each optical filter 1030 on the plane where the supporting frame 102 is located is internally tangent to the supporting frame 102, for example, the projection of the red optical filter 1031 on the plane where the supporting frame 102 is located is internally tangent to the first arc edge 1023 and externally tangent to the second arc edge 1024, the projection of the green optical filter 1032 on the plane where the supporting frame 102 is located is internally tangent to the first arc edge 1023 and externally tangent to the second arc edge 1024, and the projection of the blue optical filter 1033 on the plane where the supporting frame 92 is located is internally tangent to the first arc edge 1023 and externally tangent to the second arc edge 1024.
Since the supporting frame 102 is a fan-shaped structure, on one hand, the first arc edge 1023 and the second arc edge 1024 clamp the optical filter 1030 therebetween, so that the optical filter 1030 is more firmly and stably mounted; on the other hand, the supporting frame 102 does not need to rotate for one circle, so that all the filters 1030 in the supporting frame 102 can be arranged on the light path where the image sensor 20 is located at least once, and the time and space for idling of the filter device 100 are reduced.
Referring to fig. 12, the filter 1203 includes at least one red filter 12031, at least one green filter 12032, at least one blue filter 12033, and at least one white filter 12034, wherein the white filter 12034 can transmit all visible light. The structure of the support frame 1202 may be the same as the structure of the support frame 12 in fig. 1 to 3 or the support frame 42 in fig. 4. The differences include: the filter 1203 includes a red filter 12031, a green filter 12032, a blue filter 12033, and a white filter 12034, the support frame 120 is a quadrilateral structure having four vertices A, B, C and D, wherein the red filter 12031, the green filter 12032, the blue filter 12033, and the white filter 12034 are correspondingly disposed on the four vertices A, B, C and D of the support frame 1202, and the vertical distances from the centers of any two filters 1203 to the axis 12010 are the same, for example, the vertical distance I1 from the center of the red filter 12032 to the axis 12010, the vertical distance I2 from the center of the green filter 12032 to the axis 12010, the vertical distance I3 from the center of the blue filter 12033 to the axis 12010, and the vertical distance I4 from the center of the white filter 12034 to the axis 12010 are the same. That is, I1 ═ I2 ═ I3 ═ I4. For the parts with the same structure, please refer to the description of fig. 1 to fig. 4, which will not be described herein again. The advantageous effects of the filtering device 120 are the same as those of the filtering device 10 in fig. 1 to 3, and are not described herein again. Further, the addition of the white filter 12034 to the filter 120 can increase the brightness of the obtained image. Specifically, when the white filter 12034 is turned into the optical path of the image sensor 20, the image sensor 20 can obtain the brightness information because the white filter 12034 can pass all visible rays, thereby improving the brightness of the image.
Referring to fig. 13, the structure of the filtering device 130 may be substantially the same as that of the filtering device 50 in fig. 5 or the filtering device 60 in fig. 6. The differences include: the filter 1303 includes a red filter 13031, a green filter 13032, a blue filter 13033, and a white filter 13034, and the red filter 13031, the green filter 13032, the blue filter 13033, and the white filter 13034 are all circular. The support 1302 comprises a first side 13021, a second side 13023, a third side 13025 and a fourth side 13027, wherein a green filter 13032 is arranged between the first side 13021 and the second side 13023, and the green filter 13032 is tangent to the first side 13021 and the second side 13023; the red filter 13031 is arranged between the first side 13021 and the fourth side 13027, and the red filter 13031 is tangent with the first side 13021 and the fourth side 13027; the blue filter 13033 is arranged between the second side 13023 and the third side 13025, and the blue filter 13033 is tangent to the second side 13023 and the third side 13025; the white filter 13034 is disposed between the third side 13025 and the fourth side 13027 and the white filter 13034 is tangent to the third side 13025 and the fourth side 13027. Other structures are substantially the same as the filtering device 50 in fig. 5 or the filtering device 60 in fig. 6, please refer to the descriptions of fig. 5 to fig. 6, and are not repeated herein. The advantageous effects of the filtering device 130 are the same as those of the filtering devices 50 and 60 in fig. 5 and 6, and are not described herein again. Further, the addition of the white filter 13034 in the filtering device 130 can increase the brightness of the obtained image. Specifically, when the white filter 13034 is turned into the optical path of the image sensor 20, the image sensor 20 can obtain the brightness information and improve the brightness of the image because the white filter 13034 can pass all visible rays.
Referring to fig. 14, the structure of the filtering device 140 may be substantially the same as that of the filtering device 70 in fig. 7 or the filtering device 80 in fig. 8. The differences include: the filter 1403 includes a red filter 14031, a green filter 14032, a blue filter 14033, and a white filter 14034, and the red filter 14031, the green filter 14032, the blue filter 14033, and the white filter 14034 are all circular. Support frame 1402 includes first edge 14021, second edge 14023, third edge 14025, fourth edge 14027, first arc 14022, second arc 14024, third arc 14026, and fourth arc 14028. The green filter 14032 is disposed between the first edge 14021 and the second edge 14023 and a circumferential portion of the green filter 14032 coincides with the first arc 14022; the red filter 14031 is disposed between the first edge 14021 and the fourth edge 14027 and a circumferential portion of the red filter 14031 coincides with the fourth arc 14028; the blue filter 14033 is disposed between the second edge 14023 and the third edge 14025 and a circumferential portion of the blue filter 14033 coincides with the second arc 14024; the white filter 14034 is disposed between the third side 14025 and the fourth side 14027 and a circumferential portion of the white filter 14034 coincides with the third arc 14026. The other structure is substantially the same as the filtering device 70 in fig. 7 or the filtering device 80 in fig. 8, please refer to the description of fig. 7 to 8, and the description thereof is omitted. The advantageous effects of the filtering device 140 are the same as those of the filtering devices 70 and 80 in fig. 7 and 8, and are not described herein again. Further, the addition of the white filter 14034 in the filtering device 140 can increase the brightness of the obtained image. Specifically, when the white filter 14034 is turned into the optical path of the image sensor 20, the image sensor 20 can obtain the brightness information and improve the brightness of the image because the white filter 14034 can pass all visible rays.
Similarly, when the filters include a plurality of red filters, a plurality of green filters, a plurality of blue filters and a plurality of white filters, in some embodiments, the supporting frame may have a circular structure similar to the structure shown in fig. 9 and the matching relationship with the filters. In other embodiments, the supporting frame has a fan-shaped structure similar to the structure shown in fig. 10 and the matching relationship with the plurality of filters.
Referring to fig. 15, the structure of the filter device 150 may be substantially the same as the structure of the filter device 120 in fig. 12, except that: filter 1503 includes at least one red filter 15031, at least one yellow filter 15035 (two or more filters are shown in fig. 15), and at least one blue filter 15033.
Referring to fig. 16, the structure of the filtering device 160 may be substantially the same as that of the filtering device 130 in fig. 13, except that: the filters 1603 include at least one red filter 16031, at least one yellow filter 16035 (two, or one or more, as shown in fig. 16), and at least one blue filter 16033.
Referring to fig. 17, the structure of the filter device 170 may be substantially the same as the structure of the filter device 140 in fig. 14, except that: the filters 1703 include at least one red filter 17031, at least one yellow filter 17035 (two are shown in fig. 17, or one or more), and at least one blue filter 17033.
Referring to fig. 18, the structure of the filtering device 180 may be substantially the same as that of the filtering device 120 in fig. 12, except that: the filters 1803 include at least one red filter 18031, at least one yellow filter 18035, at least one blue filter 18033, and at least one white filter 18034.
Referring to fig. 19, the structure of the filter 190 may be substantially the same as that of the filter 130 in fig. 13, except that: the filters 1903 include at least one red filter 19031, at least one yellow filter 19035, at least one blue filter 19033, and at least one white filter 19034.
Referring to fig. 20, the structure of the filter device 200 may be substantially the same as the structure of the filter device 140 in fig. 14, except that: filters 2003 include at least one red filter 20031, at least one yellow filter 20035, at least one blue filter 20033, and at least one white filter 20034.
The structure of the support frame is not limited to the structure described in the above embodiment, and may be other structures. The structure of the filter is not limited to the structure described in the above embodiment, and may be another structure. The position where the filters are mounted on the supporting frame is not limited to the mounting positions listed in the above embodiments, and may be other positions as long as the vertical distances from the centers of the filters to the axis are the same, so that any one of the filters can selectively cover the image sensor when the filters rotate.
Referring to fig. 21 and 22, the present application further provides a camera module 210. The camera module 210 includes the image sensor 20, the lens 22 and the filter device in any of the embodiments. That is, the filtering device may be any one of the filtering device 10, the filtering device 40, the filtering device 50, the filtering device 60, the filtering device 70, the filtering device 80, the filtering device 90, the filtering device 100, the filtering device 120, the filtering device 130, the filtering device 140, the filtering device 150, the filtering device 160, the filtering device 170, the filtering device 180, the filtering device 190, and the filtering device 200. Taking the image capturing module 210 as an example of the filter device 10, the filter device 10 is disposed on the optical path of the image sensor 20, and the image sensor 20 can receive the light passing through the filter device 10. In one embodiment, as shown in fig. 21, the filter device 10 is disposed below the lens 22 and the image sensor 20 is disposed below the filter device 10 along the light receiving direction of the image sensor 20. All light enters the filter device 10 from the lens 22, the filter device 10 filters the light, only the light of the color corresponding to the filter 130 rotated to the lower side of the lens 22 is retained, and the single color light passing through the filter device 10 enters the image sensor 20 to be imaged to obtain an original image of the single color.
In another embodiment, as shown in fig. 22, the filter device 10 is disposed above the lens 22 and the image sensor 20 is disposed below the lens 22 along the light receiving direction of the image sensor 20. All light rays enter the filter device 10, the filter device 10 performs a filtering operation on all light rays, only light rays of a color corresponding to the filter 130 rotated above the lens 22 are retained, single color light rays passing through the filter device 10 enter the lens 22, and the image sensor 20 receives the single color light rays passing through the lens 30 to perform imaging to obtain an original image of the single color.
Referring to fig. 21 to 23 and fig. 1, the present application further provides an image capturing method for the camera module 210. The image capturing method will be described by taking the image capturing module 210 including the filter device 10 and the image sensor 20 as an example, and other image capturing methods applied to the filter device 50, the filter device 70, and the filter device 90 are the same and will not be described separately. Specifically, the filter device 10 includes a red filter 131, a green filter 132 and a blue filter 133. The image acquisition method comprises the following steps:
2301: rotating the support frame 12 to make the plurality of filters 130 sequentially cover the image sensor 20;
2302: during the period that any one of the filters 130 covers the image sensor 20, the image sensor 20 is exposed to obtain an original image corresponding to the filter 130 covered thereon, where the original image includes at least one red original image corresponding to the red filter 131, at least one green original image corresponding to the green filter 132, and at least one blue original image corresponding to the blue filter 133; and
2303: and fusing the at least one frame of red original image, the at least one frame of green original image and the at least one frame of blue original image into at least one frame of color target image.
The camera module 210 may further include a processing chip (not shown), and the step 2303 is performed in the processing chip, that is, the processing chip fuses the at least one red original image, the at least one green original image, and the at least one blue original image into the at least one color target image.
In one example, the support frame 12 of the filter device 10 is rotated such that the plurality of filters 130 on the support frame 12 sequentially cover the image sensor 20. For example, the support frame 12 on the filter device 10 is rotated in a counterclockwise direction, so that the red filter 131 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of red original image; rotating the support frame 12 in the counterclockwise direction to make the green filter 132 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of green original image; the support frame 12 is rotated counterclockwise, so that the blue filter 133 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of blue original image. The processing chip fuses a frame of red original image, a frame of green original image and a frame of blue original image into a frame of color target image. The support frame 12 may also rotate clockwise to obtain the blue original image, the green original image, and the red original image in sequence, and the rotation direction of the support frame 12 is not limited.
Similarly, the supporting frame 12 may be rotated in the same direction for multiple times to obtain multiple frames of red original images, multiple frames of green original images, and multiple frames of blue original images. For example, a plurality of frames of single-color original images are synthesized into one frame of single-color intermediate image, and then a plurality of frames of single-color intermediate images with different colors are synthesized into one frame of color target image. Specifically, referring to fig. 24, the step 2303 of fusing at least one red original image frame, at least one green original image frame and at least one blue original image frame into at least one color target image frame includes:
23031: synthesizing a plurality of frames of red original images into a frame of red intermediate image, synthesizing a plurality of frames of green original images into a frame of green intermediate image, and synthesizing a plurality of frames of blue original images into a frame of blue intermediate image; and
23032: and synthesizing a frame of red intermediate image, a frame of green intermediate image and a frame of blue intermediate image into a frame of color target image.
For example, a plurality of frames of single-color original images are synthesized into a plurality of frames of color intermediate images, and then the plurality of frames of color intermediate images are fused and synthesized into one frame of color target image. Specifically, please refer to fig. 25, step 2303: fusing at least one frame of red original image, at least one frame of green original image and at least one frame of blue original image into at least one frame of color target image, comprising:
23033: fusing a plurality of frames of red original images, a plurality of frames of green original images and a plurality of frames of blue original images into a plurality of frames of color intermediate images; and
23034: and fusing the multi-frame color intermediate images into a frame color target image.
In another example, referring to fig. 4, the filter device 40 includes a plurality of red filters 431, a plurality of green filters 432, and a plurality of blue filters 433. For example, the supporting frame 42 on the filtering device 40 is rotated clockwise, so that the first red filter 4311 covers the image sensor 20, and the image sensor 20 is exposed to obtain a red original image of one frame; rotating the supporting frame 42 clockwise to make the first green filter 4321 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of green original image; rotating the supporting frame 42 clockwise to make the first blue filter 4331 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of blue original image; rotating the supporting frame 42 on the filtering device 40 clockwise to make the second red filter 4312 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of red original image; then, the supporting frame 42 is rotated clockwise to make the second green filter 4322 cover the image sensor 20, and the image sensor 20 is exposed to obtain a frame of green original image; then, the supporting frame 42 is rotated clockwise to make the second blue filter 4332 cover the image sensor 20, and the image sensor 20 is exposed to obtain a frame of blue original image; the processing chip fuses the two frames of red original images, the two frames of green original images and the two frames of blue original images into at least one frame of color target image. The support bracket 42 may also be rotated in a counterclockwise direction, and the rotation direction of the support bracket 12 is not limited. Other image capturing methods applied to the filtering device 60 and the filtering device 80 are the same and will not be described separately.
Similarly, the process of combining the multiple single-color original images into at least one color target image may be the same as steps 23031 and 23032, or may be the same as steps 23033 and 23034, and is not repeated herein. In the filtering device 40, the filtering device 60 or the filtering device 80, the color image can be acquired twice by rotating the support frame for one circle, so that the image acquisition speed is increased.
In an example, referring to fig. 10, the supporting frame 102 in the optical filter device 100 is a fan-shaped structure, the supporting frame 102 on the optical filter device 100 is rotated to make the red filter 1031 cover the image sensor 20, and the image sensor 20 is exposed to obtain a frame of red original image; rotating the support frame 102 in the direction (counterclockwise) of the green filter 1032 which is closer to the red filter 1031, so that the green filter 1032 covers the image sensor 20, and the image sensor 20 is exposed to obtain a green original image of one frame; then, the supporting frame 102 is rotated along the same direction, so that the blue filter 1033 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of blue original image; the processing chip fuses a frame of red original image, a frame of green original image and a frame of blue original image into a frame of color target image. In the optical filtering device 100, the supporting frame 102 does not need to rotate for one circle to complete one-time image acquisition, so that the idle time of the optical filtering device 100 is reduced.
Similarly, a plurality of single-color images with the same color can be obtained by rotating the supporting frame 102 with a fan-shaped structure for a plurality of times. For example, the support frame 102 is rotated in the same direction all the time to obtain a plurality of single-color original images of the same color. Specifically, referring to fig. 10 and 21, the supporting frame 102 on the optical filter device 100 is rotated counterclockwise, so that the red filter 1031 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of red original image; continuing to rotate the support frame 102 in the counterclockwise direction, so that the green filter 1032 covers the image sensor 20, and the image sensor 20 is exposed to obtain a green original image of one frame; then, the supporting frame 102 is rotated in the counterclockwise direction, so that the blue optical filter 1033 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of blue original image; continuing to rotate the support frame 102 in the counterclockwise direction, so that the red filter 1031 covers the image sensor 20 again, and exposing the image sensor 20 to obtain a frame of red original image; rotating the support frame 102 in the counterclockwise direction to make the green filter 1032 cover the image sensor 20 again, and exposing the image sensor 20 to obtain a frame of green original image; the supporting frame 102 is rotated counterclockwise, so that the blue filter 1033 covers the image sensor 20 again, and the image sensor 20 is exposed to obtain a frame of blue original image. The processing chip fuses the two frames of red original images, the two frames of green original images and the two frames of blue original images into at least one frame of color target image. Similarly, the process of combining the multiple single-color original images into at least one color target image may be the same as steps 23031 and 23032, or may be the same as steps 23033 and 23034, and is not repeated herein.
For example, the supporting frame 12 with a fan-shaped structure can also be rotated back and forth to obtain a plurality of single-color original images with the same color. Specifically, referring to fig. 10 and 21, the supporting frame 102 on the optical filter device 100 is rotated counterclockwise, so that the red filter 1031 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of red original image; continuing to rotate the support frame 102 in the counterclockwise direction, so that the green filter 1032 covers the image sensor 20, and the image sensor 20 is exposed to obtain a green original image of one frame; continuing to rotate the support frame 102 in the counterclockwise direction, so that the blue filter 1033 covers the image sensor 20, and exposing the image sensor 20 to obtain a frame of blue original image; keeping the support frame 102 still, i.e. keeping the blue filter 1033 covering the image sensor 20, and exposing the image sensor 20 again to obtain a frame of blue original image; rotating the supporting frame 102 in a clockwise direction to make the green filter 1032 cover the image sensor 20, and exposing the image sensor 20 to obtain a green original image of one frame; the support frame 102 is further rotated counterclockwise, so that the red filter 1031 covers the image sensor 20, and the image sensor 20 is exposed to obtain a red original image of one frame. The processing chip fuses the two frames of red original images, the two frames of green original images and the two frames of blue original images into at least one frame of color target image. Similarly, the process of combining the multiple single-color original images into at least one color target image may be the same as steps 23031 and 23032, or may be the same as steps 23033 and 23034, and is not repeated herein. The filter device 100 uses the supporting frame 102 with the sector structure and adopts a reciprocating rotation mode to acquire multiple color image acquisition, so that the idle time of the filter device 100 is reduced, and the space occupied by the filter device 100 is also reduced.
Referring to fig. 21, 22, 12 and 26, the present application further provides an image capturing method for the camera module 210. The image capturing method is described by taking the image capturing module 210 including the filter device 120 and the image sensor 20 as an example. Other image capturing methods applied to the filtering device 130 and the filtering device 140 are the same and will not be described separately. Specifically, the filtering device 120 includes at least one red filter 12031, at least one green filter 12032, at least one blue filter 12033, and at least one white filter 12034. The image acquisition method comprises the following steps:
2601: rotating the support frame 1202 to make the plurality of filters 12030 sequentially cover the image sensor 20;
2602: during a period when any one of the filters 12030 covers the image sensor, the image sensor 20 is exposed to obtain an original image corresponding to the filter 12030 covered thereon, where the original image includes at least one red original image corresponding to the red filter 12031, at least one green original image corresponding to the green filter 12032, at least one blue original image corresponding to the blue filter 12033, and at least one gray original image corresponding to the white filter 12034; and
2603: at least one frame of red original image, at least one frame of green original image, at least one frame of blue original image and at least one frame of gray original image are fused into at least one frame of color target image.
The camera module 210 may further include a processing chip (not shown), and the step 2603 is performed in the processing chip, that is, the processing chip fuses the at least one red original image, the at least one green original image, the at least one blue original image, and the at least one gray original image into the at least one color target image.
In one example, the support frame 1202 of the filtering device 120 is rotated such that the plurality of filters 12030 on the support frame 1202 sequentially cover the image sensor 20. For example, the support frame 1202 on the filter device 120 is rotated clockwise, so that the red filter 12031 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of red original image; rotating the support frame 1202 in the clockwise direction to make the green filter 12032 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of green original image; rotating the support frame 1202 in the clockwise direction to make the blue filter 12033 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of blue original image; rotating the support frame 1202 in the clockwise direction to make the white filter 12034 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame gray level original image; the processing chip fuses a frame of red original image, a frame of green original image, a frame of blue original image and a frame of gray original image into a frame of color target image. The supporting frame 1202 may also rotate in the counterclockwise direction to obtain the gray-scale original image, the blue original image, the green original image, and the red original image in sequence, and the rotation direction of the supporting frame 1202 is not limited.
Referring to fig. 27, step 2603: fusing at least one frame of red original image, at least one frame of green original image, at least one frame of blue original image and at least one frame of gray original image into at least one frame of color target image, comprising:
26031: fusing at least one frame of red original image, at least one frame of green original image and at least one frame of blue original image into at least one frame of color intermediate image; and
26032: and correcting the brightness information of at least one frame of color intermediate image by using at least one frame of gray original image to obtain at least one frame of color target image.
Specifically, as shown in fig. 28, after the processing chip fuses one frame of red original image, one frame of green original image, and one frame of blue original image into one frame of color intermediate image, the color and brightness of the color intermediate image are separated to obtain a color-brightness separated image, where L in the color-brightness separated image in fig. 28 represents brightness, and CLR represents color. Specifically, assuming that the single-color pixel a is a red pixel R, the single-color pixel B is a green pixel G, and the single-color pixel C is a blue pixel Bu: (1) the processing chip can convert the color intermediate image in the RGB space into a color and brightness separated image in the YCrCb space, wherein Y in the YCrCb is the brightness L in the color and brightness separated image, and Cr and Cb in the YCrCb are the color CLR in the color and brightness separated image; (2) the processing chip can also convert the RGB color intermediate image into a Lab space color separation image, wherein L in Lab is the brightness L in the color separation image, and a and b in Lab are the color CLR in the color separation image. Note that L + CLR in the color-separated image shown in fig. 28 does not indicate that the pixel value of each pixel is formed by adding L and CLR, and only the pixel value indicating each pixel is formed by L and CLR.
Subsequently, the processing chip fuses the brightness of the luminance separated image and the brightness of the grayscale original image. For example, the pixel value of each panchromatic pixel W in the grayscale original image is the brightness value of each panchromatic pixel, and the processing chip may add L of each pixel in the color-brightness separated image to W of the panchromatic pixel at the corresponding position in the grayscale original image, so as to obtain the pixel value after brightness correction. The processing chip forms a brightness-corrected color-brightness separation image according to the plurality of brightness-corrected pixel values, and converts the brightness-corrected color-brightness separation image into a color target image by using color space conversion.
Since the white filter is disposed in the filtering device 120, the white filter can assist in acquiring the gray-scale original image in which the brightness information is recorded, and the brightness of the color intermediate image is corrected according to the brightness information of the gray-scale original image to acquire the color target image with higher brightness.
Similarly, the supporting frame 1202 may be rotated multiple times in the same direction to obtain multiple frames of red original images, multiple frames of green original images, multiple frames of blue original images, and multiple frames of gray original images. The multi-frame red original image, the multi-frame green original image, and the multi-frame blue original image may be synthesized into at least one frame of color intermediate image, and the synthesizing process may be the same as steps 23031 and 23032, or may be the same as steps 23033 and 23034, which are not described herein again. Then, the brightness information of the synthesized color intermediate image can be corrected by using one frame of the multi-frame gray scale original image to obtain at least one frame of color target image. Or, after weighting the multi-frame gray scale original image, correcting the brightness information of the synthesized color intermediate image by using the weighted brightness value pair to obtain at least one frame of color target image.
The image capturing method of the filter 1203 in the filtering device 120 including the plurality of red filters 12031, the plurality of green filters 12032, the plurality of blue filters 12033, and the plurality of white filters 12034 is the same as the image capturing method of the plurality of red filters, the plurality of green filters, and the plurality of blue filters applied to the filtering device 40 or the filtering device 60, which is not described herein again; the image capturing method of the fan-shaped support frame 1202 of the optical filter device 120 is similar to the image capturing method applied to the optical filter device 100, and is not repeated herein.
Referring to fig. 21, 22, 15 and 29, the present application further provides an image capturing method for the camera module 210. The image capturing method will be described by taking as an example that the camera module 210 includes the filter device 10 (shown in fig. 1) and the image sensor 20, and the structure of the filter device 10 is the same as that of the filter device 150, and other image capturing methods applied to the filter device 160 and the filter device 170 are the same and will not be described separately. The filtering device 150 includes at least one red filter 15031, at least one yellow filter 15035, and at least one blue filter 15033. The image acquisition method comprises the following steps:
2901: rotating the support frame 1502 so that the plurality of filters 15030 sequentially cover the image sensor 20;
2902: during the period that any one of the filters 15030 covers the image sensor 20, the image sensor 20 is exposed to acquire an original image corresponding to the filter 15030 covered thereon, wherein the original image includes at least one red original image corresponding to the red filter 15031, at least one yellow original image corresponding to the yellow filter 15035, and at least one blue original image corresponding to the blue filter 15033; and
2903: and fusing at least one frame of red original image, at least one frame of yellow original image and at least one frame of blue original image into at least one frame of color target image.
The camera module 210 may further include a processing chip (not shown), and step 2903 is performed in the processing chip, that is, the processing chip fuses at least one red original image frame, at least one yellow original image frame, and at least one blue original image frame into at least one color target image frame.
In one example, the support frame 1502 of the filtering device 150 is rotated such that the plurality of filters 15030 on the support frame 1502 sequentially cover the image sensor 20. For example, the support 1502 on the filter device 150 is rotated clockwise to make the red filter 15031 cover the image sensor 20, and the image sensor 20 is exposed to obtain a frame of red original image; rotating the support frame 1502 clockwise to enable the yellow filter 15035 to cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of yellow original image; rotating the support frame 1502 clockwise to enable the blue filter 15033 to cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of blue original image; the support 1502 is rotated clockwise to cover the yellow filter 15035 on the image sensor 20, and the image sensor 20 is exposed to obtain a yellow original image of one frame. The processing chip fuses one frame of red original image, two frames of yellow original images and one frame of blue original image into one frame of color target image. The support frame 1502 may also rotate in the counterclockwise direction to obtain the yellow original image, the blue original image, the yellow original image, and the red original image in sequence, and the rotation direction of the support frame 1502 is not limited.
Similarly, the supporting frame 1502 may be rotated multiple times in the same direction to obtain multiple frames of red original images, multiple frames of yellow original images, and multiple frames of blue original images, and the multiple frames of single-color original images are combined into at least one frame of color target image. The process of synthesizing at least one frame of color target image from multiple frames of single-color original images may be the same as steps 23031 and 23032, or may be the same as steps 23033 and 23034, and is not repeated here.
The image capturing method of the filter 1503 in the filtering device 150 including the plurality of red filters 15031, the plurality of yellow filters 15035, and the plurality of blue filters 15033 is the same as the image capturing method of the plurality of red filters, the plurality of green filters, and the plurality of blue filters applied to the filtering device 40 or the filtering device 60, and is not described herein again; the image capturing method of the supporting frame 1502 of the optical filter device 150 is similar to the image capturing method applied to the optical filter device 100, and is not repeated herein.
Referring to fig. 18 and 30, the present application further provides an image capturing method for the camera module 210. The image capturing method is described by taking the image capturing module 210 including the filter device 180 and the image sensor 20 as an example, and other image capturing methods applied to the filter device 190 and the filter device 200 are the same and will not be described separately. The filter device 180 includes at least one red filter 18031, at least one yellow filter 18035, at least one blue filter 18033, and at least one white filter 18034. The image acquisition method comprises the following steps:
3001: rotating the support frame 1802 so that the plurality of filters 18030 sequentially cover the image sensor 20;
3002: during a period that any one of the filters 18030 covers the image sensor 20, the image sensor 20 is exposed to obtain an original image corresponding to the filter 18030 covered thereon, where the original image includes at least one red original image corresponding to the red filter 18031, at least one yellow original image corresponding to the yellow filter 18035, at least one blue original image corresponding to the blue filter 18033, and at least one gray original image corresponding to the white filter 18034; and
3003: at least one frame of red original image, at least one frame of yellow original image, at least one frame of blue original image and at least one frame of gray original image are fused into at least one frame of color target image.
The camera module 210 further includes a processing chip (not shown), and the step 3003 is performed in the processing chip, that is, the processing chip fuses at least one red original image frame, at least one yellow original image frame, at least one blue original image frame, and at least one gray original image frame into at least one color target image frame.
In one example, the support 1802 of the filter device 180 is rotated such that the plurality of filters 18030 on the support 1802 sequentially cover the image sensor 20. For example, the support 1802 on the filter device 180 is rotated clockwise, so that the red filter 1831 covers the image sensor 20, and the image sensor 20 is exposed to obtain a frame of red original image; rotating the support frame 1802 clockwise to enable the yellow filter 18035 to cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of yellow original image; rotating the support frame 1802 clockwise to make the blue filter 18033 cover the image sensor 20, and exposing the image sensor 20 to obtain a frame of blue original image; the support 1802 is then rotated clockwise to cover the image sensor 20 with the white filter 18034, and the image sensor 20 is exposed to obtain a gray-scale original image of one frame. The processing chip fuses a frame of red original image, a frame of yellow original image, a frame of blue original image and a frame of gray original image into a frame of color target image. The support 1802 may also rotate in the counterclockwise direction to obtain the grayscale original image, the blue original image, the yellow original image, and the red original image in turn, and the rotation direction of the support 1202 is not limited.
Similarly, the support 1802 may be rotated again in the same direction for multiple times to obtain multiple frames of red original images, multiple frames of yellow original images, multiple frames of blue original images, and multiple frames of gray original images. The multi-frame red original image, the multi-frame green original image, and the multi-frame blue original image may be synthesized into at least one frame of color intermediate image, and the synthesizing process may be the same as steps 23031 and 23032, or may be the same as steps 23033 and 23034, which are not described herein again. Then, the brightness information of the synthesized color intermediate image can be corrected by using one frame of the multi-frame gray scale original image to obtain at least one frame of color target image. Or, after weighting the multi-frame gray scale original image, correcting the brightness information of the synthesized color intermediate image by using the weighted brightness value pair to obtain at least one frame of color target image.
The image collecting method of the optical filter 1803 in the optical filter device 180 including the multiple red optical filters 18031, the multiple yellow optical filters 18035, the multiple blue optical filters 18033 and the multiple white optical filters 18034 is the same as the image collecting method of the multiple red optical filters, the multiple green optical filters, the multiple blue optical filters and the multiple white optical filters applied to the optical filter device 120 or the optical filter devices 130 and 140, and is not described herein again; the image capturing method of the supporting frame 1802 in the optical filtering device 180 with a fan-shaped structure is similar to the image capturing method applied to the optical filtering device 100, and is not repeated herein.
Referring to fig. 31, the present application further provides a mobile terminal 3100, where the mobile terminal 3100 may be a mobile phone, a tablet computer, a notebook computer, an intelligent wearable device (such as an intelligent watch, an intelligent bracelet, an intelligent glasses, an intelligent helmet, etc.), a head display device, a virtual reality device, and the like, which are not limited herein. The mobile terminal 3100 includes a housing 3102 and a camera assembly 210. The housing 3102 is coupled to the camera head assembly 210. The camera assembly 210 may be any of the camera assemblies described above.
In the description herein, reference to the description of the terms "certain embodiments," "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In the description of the present application, "a plurality" means at least two, e.g., two, three, unless specifically limited otherwise.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations of the above embodiments may be made by those of ordinary skill in the art within the scope of the present application, which is defined by the claims and their equivalents.

Claims (22)

  1. A light filtering device, comprising:
    a shaft including an axis;
    the support frame is arranged on the rotating shaft and can rotate around the axis; and
    install filter assembly on the support frame, filter assembly includes a plurality of filters, and at least part the light of filter through different wave bands is a plurality of the filter encircles pivot interval distribution, works as the support frame encircles when the axis rotates, and is a plurality of the filter also encircles the axis rotates to can make selectively a plurality ofly any one in the filter covers image sensor.
  2. The filter device according to claim 1, wherein the support frame is fixedly connected to the rotation shaft; or
    The support frame is rotatably connected with the rotating shaft.
  3. The filter device of claim 1, wherein a plurality of the filters are uniformly spaced around the rotation axis.
  4. A filter device according to claim 3, wherein the support frame is centred on said axis.
  5. The filter device according to claim 4, wherein the supporting frame has a polygonal structure, each filter is disposed corresponding to a vertex of the supporting frame, and the vertical distances from the centers of any two filters to the axis are the same.
  6. The filter device according to claim 5, wherein an orthographic projection of a center of each filter on the support frame coincides with the corresponding vertex; or
    The center of each filter is superposed with the vertex corresponding to the support frame.
  7. A filter device according to claim 5, wherein each of said filters is circular,
    each optical filter is tangent to two adjacent edges at the corresponding vertex; or
    The projection of each filter on the plane where the support frame is located is tangent to two adjacent edges at the corresponding vertex.
  8. The filter device of claim 4, wherein the support frame includes a plurality of sides, adjacent sides being connected by an arc, each filter having a circular shape and being disposed between the adjacent sides,
    the circumferential part of each optical filter is coincided with the arc line; or
    The circumferential part of the projection of each filter on the plane of the support frame is coincident with the arc line.
  9. The filter device of claim 4, wherein the support frame has a circular structure, each filter has a circular shape,
    each filter is internally tangent to the support frame; or
    The projection of each filter on the plane where the support frame is located is internally tangent to the support frame.
  10. The filter device according to claim 1, wherein the supporting frame has a fan-shaped structure and comprises two radius edges, a first arc edge and a second arc edge, the first arc edge and the second arc edge are both connected to the two radius edges, and the rotating shaft is disposed at a center of the fan-shaped structure and is connected to both the radius edges;
    the plurality of optical filters are arranged between the first arc edge and the second arc edge, and each optical filter is internally tangent to the first arc edge and externally tangent to the second arc edge; or
    The projections of the optical filters on the support frame are positioned between the first arc edge and the second arc edge, and the projection of each optical filter on the support frame is internally tangent to the first arc edge and externally tangent to the second arc edge.
  11. The filter device according to any one of claims 1 to 10, wherein the plurality of filters include:
    the red filter is used for filtering light rays except the red light rays;
    the green filter is used for filtering light rays except the green light rays; and
    and the blue filter is used for filtering light rays except the blue light rays.
  12. The filter device according to any one of claims 1 to 10, wherein the plurality of filters include:
    the red filter is used for filtering light rays except the red light rays;
    the green filter is used for filtering light rays except the green light rays;
    the blue filter is used for filtering light rays except the blue light rays; and
    at least one white filter for filtering light other than visible light.
  13. The filter device according to any one of claims 1 to 10, wherein the plurality of filters include:
    the red filter is used for filtering light rays except the red light rays;
    the yellow filter is used for filtering light rays except the yellow light rays; and
    and the blue filter is used for filtering light rays except the blue light rays.
  14. The filter device according to any one of claims 1 to 10, wherein the plurality of filters include:
    the red filter is used for filtering light rays except the red light rays;
    the yellow filter is used for filtering light rays except the yellow light rays;
    the blue filter is used for filtering light rays except the blue light rays; and
    at least one white filter for filtering light other than visible light.
  15. The utility model provides a module of making a video recording which characterized in that includes:
    an image sensor; and
    the light filtering device of any one of claims 1 to 14, disposed in an optical path of the image sensor, the image sensor being capable of receiving light passing through the light filtering device.
  16. The camera module of claim 15, further comprising a lens, wherein the lens and the filter are sequentially disposed along a light receiving direction of the image sensor, and the image sensor is capable of receiving light sequentially passing through the lens and the filter.
  17. The camera module of claim 15, further comprising a lens, wherein the filter device and the lens are sequentially disposed along a light receiving direction of the image sensor, and the image sensor is capable of receiving light rays sequentially passing through the filter device and the lens.
  18. A mobile terminal, comprising:
    a housing; and
    the camera module of any of claims 15-17, in combination with the housing.
  19. An image acquisition method is used for a camera module and is characterized in that the camera module comprises an image sensor and a light filtering device, the light filtering device is arranged on a light path of the image sensor, and the image sensor can receive light rays passing through the light filtering device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the optical filter comprises at least one red optical filter, at least one green optical filter and at least one blue optical filter, wherein the red optical filter is used for filtering light rays except red light rays, the green optical filter is used for filtering light rays except green light rays, and the blue optical filter is used for filtering light rays except blue light rays; the image acquisition method comprises the following steps:
    rotating the support frame to enable the optical filters to sequentially cover the image sensor;
    in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, and the original image comprises at least one red original image corresponding to the red optical filter, at least one green original image corresponding to the green optical filter and at least one blue original image corresponding to the blue optical filter; and
    and fusing at least one frame of the red original image, at least one frame of the green original image and at least one frame of the blue original image into at least one frame of color target image.
  20. An image acquisition method is used for a camera module and is characterized in that the camera module comprises an image sensor and a light filtering device, the light filtering device is arranged on a light path of the image sensor, and the image sensor can receive light rays passing through the light filtering device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the optical filter comprises at least one red optical filter, at least one green optical filter, at least one blue optical filter and at least one white optical filter, wherein the red optical filter is used for filtering light rays except red light rays, the green optical filter is used for filtering light rays except green light rays, and the blue optical filter is used for filtering light rays except blue light rays; the white filter is used for filtering light except visible light; the image acquisition method comprises the following steps:
    rotating the support frame to enable the optical filters to sequentially cover the image sensor;
    in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, wherein the original image comprises at least one red original image corresponding to the red optical filter, at least one green original image corresponding to the green optical filter, at least one blue original image corresponding to the blue optical filter, and at least one gray original image corresponding to the white optical filter; and
    and fusing at least one frame of the red original image, at least one frame of the green original image, at least one frame of the blue original image and at least one frame of the gray original image into at least one frame of the color target image.
  21. An image acquisition method is used for a camera module and is characterized in that the camera module comprises an image sensor and a light filtering device, the light filtering device is arranged on a light path of the image sensor, and the image sensor can receive light rays passing through the light filtering device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the filter comprises at least one red filter, at least one yellow filter and at least one blue filter, wherein the red filter is used for filtering light rays except red light rays, the yellow filter is used for filtering light rays except yellow light rays, and the blue filter is used for filtering light rays except blue light rays; the image acquisition method comprises the following steps:
    rotating the support frame to enable the optical filters to sequentially cover the image sensor;
    in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, and the original image comprises at least one red original image corresponding to the red optical filter, at least one yellow original image corresponding to the yellow optical filter and at least one blue original image corresponding to the blue optical filter; and
    and fusing at least one frame of the red original image, at least one frame of the yellow original image and at least one frame of the blue original image into at least one frame of color target image.
  22. An image acquisition method is used for a camera module and is characterized in that the camera module comprises an image sensor and a light filtering device, the light filtering device is arranged on a light path of the image sensor, and the image sensor can receive light rays passing through the light filtering device; the optical filter device comprises a rotating shaft, a supporting frame and an optical filter assembly; the rotating shaft comprises an axis, and the support frame is arranged on the rotating shaft and can rotate around the axis; the optical filter assembly is arranged on the supporting frame and comprises a plurality of optical filters, at least part of the optical filters can pass light rays of different wave bands, the optical filters are distributed around the rotating shaft at intervals, and when the supporting frame rotates around the axis, the optical filters also rotate around the axis so as to selectively enable any one of the optical filters to cover the image sensor; the optical filter comprises at least one red optical filter, at least one yellow optical filter, at least one blue optical filter and at least one white optical filter, wherein the red optical filter is used for filtering light except red light, the yellow optical filter is used for filtering light except yellow light, and the blue optical filter is used for filtering light except blue light; the white filter is used for filtering light except visible light; the image acquisition method comprises the following steps:
    rotating the support frame to enable the optical filters to sequentially cover the image sensor;
    in the period that any one optical filter covers the image sensor, the image sensor is exposed to obtain an original image corresponding to the optical filter covered on the image sensor, wherein the original image comprises at least one red original image corresponding to the red optical filter, at least one yellow original image corresponding to the yellow optical filter, at least one blue original image corresponding to the blue optical filter, and at least one gray original image corresponding to the white optical filter; and
    and fusing at least one frame of the red original image, at least one frame of the yellow original image, at least one frame of the blue original image and at least one frame of the gray original image into at least one frame of the color target image.
CN201980100439.2A 2019-12-19 2019-12-19 Light filtering device, camera module, mobile terminal and image acquisition method Pending CN114402579A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/126585 WO2021120098A1 (en) 2019-12-19 2019-12-19 Optical filtering apparatus, image capture module, mobile terminal and image collection method

Publications (1)

Publication Number Publication Date
CN114402579A true CN114402579A (en) 2022-04-26

Family

ID=76476773

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201980100439.2A Pending CN114402579A (en) 2019-12-19 2019-12-19 Light filtering device, camera module, mobile terminal and image acquisition method

Country Status (2)

Country Link
CN (1) CN114402579A (en)
WO (1) WO2021120098A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113630527A (en) * 2020-05-08 2021-11-09 南昌欧菲光电技术有限公司 Light filtering assembly, camera module, electronic equipment and shooting method
CN114967289B (en) * 2022-06-16 2023-09-26 苏州华星光电技术有限公司 Color wheel module, brightness correction device and method of display panel and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101750696A (en) * 2008-12-03 2010-06-23 鸿富锦精密工业(深圳)有限公司 Lens module
CN201658379U (en) * 2010-02-04 2010-12-01 深圳市乐德康科技有限公司 Medical image information acquisition system
CN202975565U (en) * 2012-11-28 2013-06-05 贾岩 Photography equipment and an optical filter bearing device of the photography equipment
CN203275287U (en) * 2013-06-04 2013-11-06 沈阳仪表科学研究院有限公司 Portable optical filter color wheel type multispectral imaging system
CN105100566A (en) * 2015-05-29 2015-11-25 努比亚技术有限公司 Camera module, photographing device and multispectral imaging method
CN106105189A (en) * 2014-02-14 2016-11-09 空客防务与空间有限公司 Multi-optical spectrum image collecting equipment including filter wheel
CN107147832A (en) * 2017-05-31 2017-09-08 广东欧珀移动通信有限公司 The CCD camera assembly and electronic equipment of electronic equipment
CN109951624A (en) * 2019-04-12 2019-06-28 武汉鸿瑞达信息技术有限公司 A kind of imaging camera system and method based on filter halo

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5650832A (en) * 1995-06-13 1997-07-22 Texas Instruments Incorporated Multimode color wheel for display device
CN107728280A (en) * 2017-09-30 2018-02-23 广东欧珀移动通信有限公司 Filtering apparatus, imaging sensor and imaging modules
CN110381244B (en) * 2019-08-26 2021-02-02 浙江大华技术股份有限公司 Camera and method for improving image quality under low illumination

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101750696A (en) * 2008-12-03 2010-06-23 鸿富锦精密工业(深圳)有限公司 Lens module
CN201658379U (en) * 2010-02-04 2010-12-01 深圳市乐德康科技有限公司 Medical image information acquisition system
CN202975565U (en) * 2012-11-28 2013-06-05 贾岩 Photography equipment and an optical filter bearing device of the photography equipment
CN203275287U (en) * 2013-06-04 2013-11-06 沈阳仪表科学研究院有限公司 Portable optical filter color wheel type multispectral imaging system
CN106105189A (en) * 2014-02-14 2016-11-09 空客防务与空间有限公司 Multi-optical spectrum image collecting equipment including filter wheel
CN105100566A (en) * 2015-05-29 2015-11-25 努比亚技术有限公司 Camera module, photographing device and multispectral imaging method
CN107147832A (en) * 2017-05-31 2017-09-08 广东欧珀移动通信有限公司 The CCD camera assembly and electronic equipment of electronic equipment
CN109951624A (en) * 2019-04-12 2019-06-28 武汉鸿瑞达信息技术有限公司 A kind of imaging camera system and method based on filter halo

Also Published As

Publication number Publication date
WO2021120098A1 (en) 2021-06-24

Similar Documents

Publication Publication Date Title
JP6027560B2 (en) Automatic tracking imaging device
US7050120B2 (en) Display device with cooperative color filters
EP4027639A1 (en) Image processing method, image processing device, and photographing device
CN114402579A (en) Light filtering device, camera module, mobile terminal and image acquisition method
JPH02128593A (en) Optical chopper assembly
CN107576396A (en) The program control multispectral imaging method and device of mobile phone
US20050046703A1 (en) Color calibration in photographic devices
CN107515045A (en) A kind of Mobile phone portable formula multispectral imaging method and device
WO2004057858A1 (en) A color imaging system and a method in a color imaging system
WO2017148109A1 (en) Color imaging device, and method for obtaining color images
US20200267291A1 (en) Reduced optical crosstalk plenoptic imaging device, corresponding method, computer program product, computer-readable carrier medium and apparatus
CN110868526A (en) Shooting module, shooting method and electronic equipment
US4588384A (en) Fixed-star projecting machine for planetarium
CN110798603B (en) Camera module and electronic equipment
CN102474649A (en) Three-dimensional imaging device and optical transmission plate
JP2005303524A (en) Camera for calibration and calibration system
WO2023202387A1 (en) Optical imaging module, optical imaging system, and terminal device
CN101414037A (en) System and method for detecting color wheel optical filter
CN215931247U (en) Optical measurement equipment
JP2011029903A (en) Digital camera system with attached polarizer
EP4022886A1 (en) Infrared light-guided portrait relighting
CN113728618A (en) Camera device
CN114441041B (en) Filter type imaging colorimeter
JP3805631B2 (en) Multifocal omnidirectional imaging device
CN113452968B (en) Image processing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination