WO2024051418A1 - 基于光谱信息的拍照方法和电子设备 - Google Patents

基于光谱信息的拍照方法和电子设备 Download PDF

Info

Publication number
WO2024051418A1
WO2024051418A1 PCT/CN2023/111389 CN2023111389W WO2024051418A1 WO 2024051418 A1 WO2024051418 A1 WO 2024051418A1 CN 2023111389 W CN2023111389 W CN 2023111389W WO 2024051418 A1 WO2024051418 A1 WO 2024051418A1
Authority
WO
WIPO (PCT)
Prior art keywords
ambient light
information
spectral
spectral information
sensor
Prior art date
Application number
PCT/CN2023/111389
Other languages
English (en)
French (fr)
Inventor
李丽
黄志雷
王宇
Original Assignee
北京与光科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京与光科技有限公司 filed Critical 北京与光科技有限公司
Publication of WO2024051418A1 publication Critical patent/WO2024051418A1/zh

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02—Details
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/617—Upgrading or updating of programs or applications for camera control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/73—Colour balance circuits, e.g. white balance circuits or colour temperature control

Definitions

  • the present application relates to the field of spectroscopy technology, and more specifically, to a photographing method and electronic equipment based on spectral information.
  • the color temperature detection in ambient light is inaccurate when taking a photo, it will affect the actual imaging effect of the photo, causing things in the picture to deviate from their original colors. For example, if there is a blue sofa in the environment when taking a picture, the image taken will appear gray or other colors when some color temperature detection is inaccurate.
  • Embodiments of the present application provide a photographing method and electronic device based on spectral information, which adjust photographing parameters through the spectral information of ambient light collected and restored by a spectral sensor and the ambient light information based on spectral information, and can adjust photographing parameters more accurately. parameter.
  • a photographing method based on spectral information including: using a spectral sensor to obtain spectral information of the first ambient light; determining information of the second ambient light based on the spectral information of the first ambient light; And, adjusting the photographing parameters based on the second ambient light information.
  • Determining the information of the second ambient light includes: determining the color temperature information of the second ambient light based on the spectral information of the first ambient light.
  • determining the color temperature information of the second ambient light based on the spectral information of the first ambient light includes: based on the spectral information of the first ambient light including at least position information and spectral information. Determine the color temperature information of the second ambient light at different locations or areas.
  • the photographing method based on spectral information after determining the color temperature information of the second ambient light at different locations or areas based on the spectral information of the first ambient light including at least position information and spectral information, it further includes: based on the The color temperature information of the second ambient light in the different locations or areas is used to manage multiple light sources to be managed located in the different locations or areas.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining the light source of the second ambient light based on the spectral information of the first ambient light to taking pictures. The distance of the device to determine the spatial information of the light source relative to the camera device.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining different positions of the second ambient light based on the spectral information of the first ambient light or The weight parameter of the area's light source in the second ambient light.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining the difference of the second ambient light based on the spectral information of the first ambient light. The location or area's light source and its type.
  • the spectral sensor includes a first spectral sensor disposed at the position of the first photographing device on the screen display side of the electronic device and a third spectral sensor disposed on the back cover side of the electronic device.
  • a second spectrum sensor at the position of the second camera device.
  • using a spectral sensor to obtain spectral information of the first ambient light includes: determining distance information between the target photographed object and the electronic device; and, in response to the distance information being less than a preset distance, using A spectral sensor on the same side of the electronic device as the target object acquires spectral information of the first ambient light and light reflected from the surface of the target object, and uses a sensor on the same side of the electronic device opposite to the target object.
  • the spectrum sensor on the side acquires the spectrum information of the first ambient light.
  • the field of view range of the spectrum sensor is not smaller than the field of view range of the lens of the camera device used to photograph the target object.
  • the first spectral sensor and the second spectral sensor have different field of view angles.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining the information of the second ambient light based on the acquired spectral information of the first spectral sensor and the second spectral sensor. Describe the reflection spectrum information of the target object.
  • adjusting photographing parameters based on the information of the second ambient light includes: matching the spectral information of the second ambient light with preset spectral information to determine whether the current ambient light source is consistent with the preset spectral information. Whether the predetermined data in the spectral library matches; and, in response to the current ambient light source matching the predetermined data in the spectral library, adjusting the photographing parameters according to the spectral information of the current ambient light source and the information of the second ambient light.
  • using a spectral sensor to obtain the spectral information of the first ambient light includes: when it is determined that the photographing preview interface of the photographing device is opened, adjusting the posture of the photographing device; and, obtaining the photographing Spectral information of the first ambient light collected in multiple postures of the device.
  • the spectral sensor includes uniform light components with different cross-sectional thicknesses.
  • an electronic device including: a spectrum sensor for obtaining spectral information of the first ambient light; and a photographing device for adjusting the photographing method based on the spectral information as described above. Take photos with the camera parameters.
  • the photographing method and electronic device based on spectral information provided by the embodiments of the present application adjust the photographing parameters through the spectral information of the ambient light collected and restored by the spectrum sensor and the ambient light information based on the spectral information, so that the photographing parameters can be adjusted more accurately.
  • Figure 1 illustrates a schematic process of a photographing method based on spectral information according to an embodiment of the present application. picture.
  • FIG. 2 illustrates a schematic diagram of a spectrum chip used for calculating reconstructed spectra according to an embodiment of the present application.
  • FIG. 3 illustrates a schematic configuration diagram of a spectral sensor according to an embodiment of the present application.
  • FIG. 4 illustrates a schematic diagram of ambient light generated by multiple light sources according to an embodiment of the present application.
  • FIG. 5 illustrates a schematic diagram of an electronic device including multiple camera devices according to an embodiment of the present application.
  • FIG. 6 illustrates a schematic diagram of a first spectral image acquired by a first spectral sensor according to an embodiment of the present application.
  • FIG. 7 illustrates a schematic diagram of a second spectrum image acquired by a second spectrum sensor according to an embodiment of the present application.
  • FIG. 8 illustrates a schematic diagram of a light uniformity component of a spectrum sensor according to an embodiment of the present application.
  • Figure 9 illustrates a schematic diagram of an electronic device according to an embodiment of the present application.
  • Figure 1 illustrates a schematic flow chart of a photographing method based on spectral information according to an embodiment of the present application.
  • the photographing method based on spectral information includes the following steps: S10, use a spectral sensor to obtain spectral information of the first ambient light; S20, determine the first ambient light based on the spectral information of the first ambient light. information about the second ambient light; and, S30, adjust the photographing parameters based on the information about the second ambient light.
  • the spectrum sensor may include a spectrum chip, and the spectrum chip includes a filter structure and an image sensor.
  • the filter structure is located on the photosensitive path of the image sensor.
  • the filter structure is Broadband filter structure in frequency domain or wavelength domain. The pass spectra of different wavelengths of each filter structure are not exactly the same.
  • Filter structures can be metasurfaces, photonic crystals, nanopillars, multilayer films, dyes, quantum dots, MEMS (microelectromechanical systems), FP etalon (FP etalon), cavity layer (resonant cavity layer), waveguide layer (waveguide layer) layer), diffractive elements and other structures or materials with filtering properties.
  • the filter structure can It is the light modulation layer in Chinese patent CN201910699962.3, as shown in Figure 2.
  • FIG. 2 illustrates a schematic diagram of a spectrum chip used for calculating reconstructed spectra according to an embodiment of the present application.
  • the spectrum chip 100 includes a light modulation layer 110 and an image sensing layer 120, wherein each modulation unit 111 of the light modulation layer 110 and each sensor of the image sensing layer 120
  • the units 121 are respectively arranged on the light modulation layer and the image sensing layer correspondingly up and down.
  • Each modulation unit is provided with at least one modulation sub-unit, and each modulation sub-unit is provided with a plurality of light modulation layers that pass through the light modulation layer. Modulation holes, each modulation hole in the same modulation subunit is arranged into a two-dimensional graphic structure with specific arrangement rules.
  • the spectrum chip is based on the modulation effect of the modulation unit array in optoelectronics on light of different wavelengths, and can collect image information of multiple spectra at the same time; therefore, the modulation structures corresponding to different coordinates can obtain the spectra corresponding to different light source areas in the ambient light.
  • Information including but not limited to central wavelength, color temperature, chromaticity, etc.
  • the light source information at different regional positions can be judged based on the coordinate position of the modulation structure, several light sources in the ambient light can be judged based on the obtained spectral information of the ambient light.
  • the central wavelength, color temperature, and color coordinates of the light source etc. can be recovered through spectroscopy.
  • the spectrum sensor may further include an optical system.
  • the optical system is located on the photosensitive path of the image sensor. After the light is adjusted by the optical system and then modulated through the filter structure, it is received by the image sensor. Obtain a spectral response; wherein the optical system may be an optical component such as a lens assembly.
  • FIG. 3 illustrates a schematic configuration diagram of a spectrum sensor according to an embodiment of the present application.
  • the image sensor may be a CMOS image sensor (CIS), CCD, array light detector, etc.
  • the spectrum sensor also includes a data processing unit, which may be an MCU, CPU, GPU, FPGA, NPU, ASIC, etc., which can export the data generated by the image sensor to the outside for processing.
  • the image sensor measures the spectral response
  • it is transmitted to the data processing unit to calculate the spectral information recovered by the incident light.
  • the process is described in detail as follows:
  • the intensity signals of the incident light at different wavelengths ⁇ are recorded as The transmission spectra of the group structural units are different from each other.
  • one physical pixel is used, that is, one physical pixel corresponds to a group of structural units, However, it is not limited to this. In other embodiments, a group of multiple physical pixels may also correspond to a group of structural units.
  • a physical pixel When corresponding to a structural unit, it can be understood as the measured light intensity corresponding to m "physical pixels", which is a vector with a length of m.
  • A is the light response of the system to different wavelengths, which is determined by two factors: the transmittance of the filter structure and the quantum efficiency of the image sensor.
  • A is a matrix, and each row vector corresponds to the response of a set of structural units to incident light of different wavelengths.
  • the incident light is sampled discretely and uniformly, with a total of n sampling points.
  • the number of columns of A is the same as the number of sampling points of the incident light.
  • x( ⁇ ) is the intensity of the incident light at different wavelengths ⁇ , which is the spectrum of the incident light to be measured.
  • the filter structure can be directly formed on the upper surface of the image sensor, such as quantum dots, nanowires, etc., which directly form the filter structure in the photosensitive area of the sensor. Or materials (nanowires, quantum dots, etc.), taking the filter structure as an example.
  • the filter structure is formed on the upper surface of the raw material, so The transmission spectrum and the response of the image sensor are integrated, that is, it can be understood that the response of the detector and the transmission spectrum are the same curve.
  • the spectral information of the incident light can be obtained through calculation. Then the spectral information
  • the color temperature of the incident light (that is, the measured ambient light) can be restored through the color coordinates.
  • the spectral sensor according to the embodiment of the present application may be a spectral color temperature sensor, used to obtain the color temperature information of the ambient light, and adjust the photographing parameters based on the color temperature information of the ambient light, such as adjusting the white balance of the photographing device.
  • the ambient light information may also include information such as the central wavelength, chromaticity, illuminance, color coordinates, etc. of the ambient light.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining the second ambient light based on the spectral information of the first ambient light. 2. Color temperature information of ambient light.
  • the first ambient light and the second ambient light may be the same ambient light, or they may be different ambient lights.
  • the first ambient light may be part of the ambient light of the second ambient light, such as part of the ambient light of a predetermined position or area of the second ambient light as the first ambient light.
  • the second ambient light may also be part of the first ambient light.
  • first ambient light and the second ambient light may also be ambient light in different locations or regions respectively.
  • first ambient light may be the front camera of a mobile phone.
  • the first ambient light may be the ambient light in the corresponding environment of the rear camera of the mobile phone.
  • the spectral sensor can obtain position information and corresponding spectral information corresponding to different filter structural units, the restored spectral information includes two-dimensional spatial information. Therefore, the spectral information of the ambient light includes at least position information and spectral information, and the color temperature information of the ambient light at different locations (coordinates) or regions can be obtained according to the spectral information of the ambient light.
  • determining the color temperature information of the second ambient light based on the spectral information of the first ambient light includes: based on the at least position information and spectral information.
  • the spectral information of the first ambient light determines the color temperature information of the second ambient light at different locations or areas.
  • the information of the second ambient light obtained by the spectrum sensor includes light source 1 , information of light source 2 and light source 3.
  • the light source 1 is on the upper side of the second ambient light, and the light source 2 and the light source 3 are located below the light source 1 .
  • FIG. 4 illustrates a schematic diagram of ambient light generated by multiple light sources according to an embodiment of the present application.
  • the second ambient light in different positions or areas is determined based on the spectral information of the first ambient light that at least includes position information and spectral information.
  • the color temperature information further includes: managing a plurality of light sources to be managed located in the different locations or areas based on the color temperature information of the second ambient light in the different locations or areas.
  • auxiliary TOF time of flight
  • the weight of the light source in the ambient light can be determined based on the acquired position information and spectral information of the light source of the ambient light, and the photography parameters can be adjusted based on the weight parameters of the light sources at different locations on the ambient light. That is to say, the ambient light generated by light sources in different areas has different effects on photography, so the photography parameters can be adjusted based on the spectral information of the ambient light in different areas.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining the information of the second ambient light based on the spectral information of the first ambient light.
  • the distance between the light source of the second ambient light and the camera device is determined to determine the spatial information of the light source relative to the camera device.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining the information of the second ambient light based on the spectral information of the first ambient light. weight parameters of light sources in different positions or areas of the second ambient light in the second ambient light.
  • the positions and types of different light sources in the second ambient light can also be determined based on the obtained position information and spectral information of the light source of the first ambient light, and based on the position of the light source and type to adjust photography parameters, such as adjusting white balance.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: determining the information of the second ambient light based on the spectral information of the first ambient light.
  • the photographing method based on spectral information according to the embodiment of the present application can be applied to an electronic device having multiple photographing devices, for example, as shown in FIG. 5 .
  • FIG. 5 illustrates a schematic diagram of an electronic device including multiple camera devices according to an embodiment of the present application.
  • the electronic device is provided with a first camera device and a second camera device.
  • the first camera device is disposed on the screen display side, and the second camera device is disposed on the back cover side.
  • At least a first spectrum sensor is provided next to the first camera device, and at least a second spectrum sensor is provided next to the second camera device.
  • the spectral sensor includes a first spectral sensor disposed at the position of the first photographing device located on the screen display side of the electronic device and a first spectral sensor disposed at the screen display side of the electronic device.
  • a second spectrum sensor at the position of the second camera device on one side of the back cover of the device.
  • the electronic device may be a mobile electronic device or a non-mobile electronic device.
  • mobile electronic devices can be mobile phones, tablets, laptops, PDAs, vehicle-mounted electronic devices, wearable devices, netbooks, etc.
  • non-mobile electronic devices can be servers, personal computers (PCs), televisions (televisions), etc. , TV), teller machines or self-service machines, etc., the embodiments of this application are not specifically limited.
  • the distance between the target object and the electronic device can be determined through the distance sensor or the focus situation during shooting.
  • the distance can be determined through the focus information obtained by the camera device.
  • the electronic device may store a preset distance in advance, and when it is less than the preset distance, acquire a spectrum sensor corresponding to another camera device other than the current camera device. That is, when the first spectrum sensor is provided on one side of the target object and the second spectrum sensor is provided on the opposite side, when the distance between the electronic device and the target object is less than the preset distance, the opposite side is turned on. spectral sensor.
  • the second spectrum sensor is at the rear camera, that is, at the position of the second camera.
  • the spectral information of the ambient light collected by the first spectrum sensor is obtained, that is, the spectrum next to the front camera.
  • Spectral information of ambient light collected by the sensor Since the ambient light has different effects on the photo taken at different distances, the closer the photo is taken, the greater the deviation between the color of the photo taken and the color of the actual object. This is because the object itself absorbs and reflects light. And different physics have different abilities to absorb or reflect light, especially objects of different colors have different abilities to reflect or absorb light from different environments. Therefore, when the distance is far away, the spectrum sensor obtains the spectral information of the ambient light.
  • the spectrum sensor on the side of the target object obtains the ambient light and Spectral information of light reflected from the surface of an object. Therefore, the spectral information of the current ambient light can be obtained with the help of the spectral sensor on the opposite side.
  • a spectrum sensor is used to obtain the spectrum of the first ambient light
  • the information includes: determining distance information between the target object and the electronic device; and, in response to the distance information being less than a preset distance, using a spectrum sensor on the same side of the electronic device as the target object to obtain the first Spectral information of ambient light and light reflected from the surface of the target object, and using a spectrum sensor on the side opposite to the target object of the electronic device to obtain spectral information of the first ambient light.
  • the spectrum sensor includes a spectrum sensor on the same side or on the opposite side to the lens of the photographing device used to photograph the target subject.
  • the field of view range of the spectral sensor is not smaller than the field of view range of the lens of the camera device used to photograph the target object.
  • the first spectrum sensor and the second spectrum sensor may have different field of view angles, thereby acquiring ambient light information at different positions and field of view angles, and combining them as photography parameters. Adjustment information.
  • the first spectral sensor and the second spectral sensor have different field of view angles.
  • the obtained spectral information of the first ambient light includes the spectral information of the object's reflected light.
  • the spectral information of the ambient light at the front and rear is obtained through two spectral sensors.
  • the reflection spectrum of the object can be judged based on the obtained spectral information of the ambient light at the front and rear sides.
  • the camera parameters can be adjusted according to the different reflection spectra of different objects. That is, when the distance between the camera device and the target object is less than the preset distance, when the spectrum sensor collects the spectral information of the ambient light in the space corresponding to the focal plane, the collected spectral information includes the reflection of the object within the focus range. Spectral information.
  • FIG. 6 illustrates a schematic diagram of a first spectrum image acquired by a first spectrum sensor according to an embodiment of the present application.
  • FIG. 7 illustrates a schematic diagram of a second spectral image acquired by a second spectral sensor according to an embodiment of the present application.
  • determining the information of the second ambient light based on the spectral information of the first ambient light includes: based on the first spectrum sensor and the second The spectrum information acquired by the spectrum sensor determines the reflection spectrum information of the target object.
  • the photographing parameters are managed according to the currently detected spectral information of the ambient light and the color temperature, chromaticity and other information of the spectrally restored ambient light, such as adjusting the white balance. , or display parameters such as color intensity.
  • adjusting the photographing parameters based on the information of the second ambient light includes: matching the spectral information of the second ambient light with the preset spectral information, To determine whether the current ambient light source matches the predetermined data in the spectrum library; and, in response to the current ambient light source matching the predetermined data in the spectrum library, based on the spectral information of the current ambient light source and the information of the ambient light Adjust camera parameters.
  • using a spectrum sensor to obtain spectral information of ambient light may include: when it is determined that the photograph preview interface of the photographing device is opened, adjusting the posture of the photographing device; and, obtaining the image of the photographing device. Spectral information of the first ambient light collected in multiple postures.
  • these postures can be manually adjusted by the user, and different ambient light information is obtained at different angles.
  • the spectral information of the current actual ambient light is calculated based on the spectral information of the ambient light obtained multiple times, which can reduce the environmental light caused by being too close.
  • the measurement error of the spectral information is obtained.
  • the spectral information of the ambient light is obtained through multiple postures of the front and rear spectral sensors at the same time, and the spectral information of the actual ambient light is calculated.
  • the shooting parameters can be adjusted based on the obtained spectral information of the actual ambient light.
  • the camera and the spectral sensor can also be disposed under the display screen.
  • the spectral sensor can use The structure of the screen acts as a filter structure to perform light modulation and obtain corresponding spectral information.
  • a front-facing camera is provided at the top, and a spectrum sensor can be provided adjacent to the front-facing camera.
  • the spectral sensor can use a spectral image detector or a near-infrared spectrum sensor.
  • the channels of the spectral sensor can be determined by the number of light modulation structures, and dozens of channels can be used for spectral recovery in order to obtain very accurate ambient light information.
  • the spectrum sensor used can be equipped with a filter structure, or it can be partially equipped with a filter structure. An image sensor without a filter structure can obtain the brightness of the ambient light. Information parameters, etc., to assist in better recovery of spectral information.
  • the back of the mobile phone can be equipped with multiple cameras, such as a main camera, a wide-angle camera, a depth camera, etc.; at this time, the spectrum sensor can reuse one of the cameras, for example
  • the depth camera can not only take images, but also obtain the spectral information of ambient light, and can better blur the background. Since the spectral sensor can obtain the spectral information of the corresponding pixels while imaging, it can better distinguish the pixels in the pixels. Background information not only distinguishes RGB color information, but also obtains different material information of the same color in the background through spectral information. Therefore, the background and foreground can be better distinguished and the background blur function can be performed.
  • different light uniforming components can also be configured on the spectrum sensor, or the same light uniforming component has different light uniformity capabilities.
  • FIG. 8 illustrates a schematic diagram of a light uniformity component of a spectrum sensor according to an embodiment of the present application. Therefore, if uniform light components of different thicknesses are used, more position information or light source spectral information in more areas can be obtained.
  • the spectral sensor includes uniform light components with different cross-sectional thicknesses.
  • the spectrum sensor can be placed on the same plane as the camera of the camera, or the spectrum sensor can be placed higher than the plane of the camera. Since the field of view of the spectrum sensor is larger, the spectrum of ambient light acquired will be The more information, the more accurate the ambient light information recovered, so different optical components can be used to make the field of view of the spectral sensor larger, for example, it can exceed 180°; the spectral sensor can be set adjacent to the camera, or it can A greater distance from the camera. There is no special limit on the distance between the spectrum sensor and the camera.
  • the photographing method based on spectral information adjusts the photographing parameters through the spectral information of the ambient light collected and restored by the spectrum sensor and the ambient light information based on the spectral information, so that the photographing parameters can be adjusted more accurately.
  • the color temperature of the ambient light is restored through the spectral image method, and the coordinates of the color temperature in the chromaticity coordinate system are obtained through the color temperature curve, and converted to obtain the correction gain.
  • determine the type of light source by matching and identifying the spectral curve, and then use the intensity value of each channel to calculate the CCT (Correlated Color Temperature) of the current ambient light, and then use the type of light source and color temperature to achieve more accurate adjustment of shooting parameters.
  • CCT Correlated Color Temperature
  • FIG 9 illustrates a schematic diagram of an electronic device according to an embodiment of the present application.
  • the electronic device 1000 includes a central processing unit (Central Processing Unit, CPU) 1001, which can be loaded into random access according to a program stored in a read-only memory (Read-Only Memory, ROM) 1002 or from a storage part.
  • the program in the memory (Random Access Memory, RAM) 1003 performs various appropriate actions and processes.
  • RAM 1003 various programs and data required for system operation are also stored.
  • CPU 1001, ROM 1002 and RAM 1003 are connected to each other through bus 1004.
  • An input/output (I/O) interface 1005 is also connected to bus 1004.
  • the following components are connected to the I/O interface 1005: an input part 1006 including a keyboard, a mouse, etc.; an output part 1007 including a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc., and a speaker, etc. ; a storage part 1008 including a hard disk, etc.; and a communication part 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc.
  • the communication section 1009 performs communication processing via a network such as the Internet.
  • Driver 1010 is also connected to I/O interface 1005 as needed.
  • Removable media 1011 such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 1010 as needed, so that computer programs read therefrom are installed into the storage portion 1008 as needed.
  • the electronic device 1000 may further include at least one spectrum sensor and at least one camera.
  • the photosensitive range of the spectrum sensor corresponds to the target photosensitive area of the camera.
  • the above-mentioned electronic devices may be smart wearable devices, smart terminals, etc., such as smart watches or mobile phones and other electronic devices.
  • the process described above with reference to the flowchart may be implemented as a computer software program.
  • embodiments of the present invention include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via communication portion 1009 and/or installed from removable media 1011.
  • CPU central processing unit
  • the above-mentioned electronic device may be a smart mobile terminal device such as a mobile phone, a tablet computer, or a laptop computer.
  • the above-mentioned electronic device may also be an intelligent terminal device such as a desktop computer.
  • the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), Flash Memory, Optical Fiber, Portable Compact Disk Read-Only Memory (Compact Disc Read-Only Memory,
  • a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block in the block diagram or flowchart illustration, and combinations of blocks in the block diagram or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or may be implemented by special purpose hardware-based systems that perform the specified functions or operations. Achieved by a combination of specialized hardware and computer instructions.
  • each component or each step can be decomposed and/or recombined. These decompositions and/or recombinations shall be considered equivalent versions of this application.

Abstract

本申请涉及一种基于光谱信息的拍照方法和电子设备。该基于光谱信息的拍照方法包括：使用光谱传感器得到第一环境光的光谱信息；基于所述第一环境光的光谱信息确定第二环境光的信息；以及，基于所述第二环境光的信息调整拍照参数。这样，通过光谱传感器采集和恢复的环境光的光谱信息和基于光谱信息的环境光信息来调整拍照参数，可以更加准确地调整拍照参数

Description

基于光谱信息的拍照方法和电子设备 技术领域
本申请涉及光谱技术领域，更为具体地说，涉及一种基于光谱信息的拍照方法和电子设备。
背景技术
近年来随着移动终端的迅速普及和手机影像技术的不断发展，智能手机越来越成为大众最常用的拍摄设备。人们使用智能手机进行拍照得到的照片质量要求越来越高。在人们进行拍照时，环境中的光线会对用户得到的照片质量产生较大的影响。在日常拍照，摄影或图像处理过程中，白平衡能够帮助摄影者达到理想中的画面色调效果，并对影像颜色进行真实的还原。在白平衡的调整控制过程中，需要依据现场日光或灯光的色温情况，对数码设备的颜色值进行精确调控，此时能否精确测定现场的色温情况就变得尤为重要。如果拍照时环境光中的色温检测不准确，则会影响照片的实际成像效果，使得图片中的事物偏离其原本的颜色。例如，如果拍照时环境中的蓝色沙发，在某些检测色温不准确时拍出来的图像显示为灰色或者其他颜色。
因此用户大都不希望照片中的事物偏离其原本的颜色，通常需要对得到的照片进行复杂的修图处理，而且处理的效果可能并不能真实还原事物本身的颜色。因此，目前急需一种可以减少或者改善对拍照颜色影响的方法。
发明内容
本申请实施例提供了一种基于光谱信息的拍照方法和电子设备，其通过光谱传感器采集和恢复的环境光的光谱信息和基于光谱信息的环境光信息来调整拍照参数，可以更加准确地调整拍照参数。
根据本申请的一方面，提供了一种基于光谱信息的拍照方法，包括：使用光谱传感器得到第一环境光的光谱信息；基于所述第一环境光的光谱信息确定第二环境光的信息；以及，基于所述第二环境光的信息调整拍照参数。
在上述基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确 定第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的色温信息。
在上述基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定所述第二环境光的色温信息包括：基于至少包括位置信息及光谱信息的所述第一环境光的光谱信息确定不同位置或者区域的所述第二环境光的色温信息。
在上述基于光谱信息的拍照方法中，在基于至少包括位置信息及光谱信息的所述第一环境光的光谱信息确定不同位置或者区域的所述第二环境光的色温信息之后进一步包括：基于所述不同位置或者区域的所述第二环境光的色温信息来对位于所述不同位置或者区域的多个待管理的光源进行管理。
在上述基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的光源到拍照装置的距离，以确定所述光源相对于所述拍照装置的空间信息。
在上述基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的不同位置或者区域的光源在所述第二环境光中的权重参数。
在上述基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定所述第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的不同位置或者区域的光源及其类型。
在上述基于光谱信息的拍照方法中，所述光谱传感器包括设置在位于电子设备的屏幕显示一侧的第一拍照装置位置处的第一光谱传感器和设置在位于电子设备的背盖一侧的第二拍照装置位置处的第二光谱传感器。
在上述基于光谱信息的拍照方法中，使用光谱传感器得到第一环境光的光谱信息包括：确定目标拍摄物与所述电子设备的距离信息；以及，响应于所述距离信息小于预设距离，使用所述电子设备的与所述目标拍摄物相同侧的光谱传感器获取所述第一环境光及所述目标拍摄物表面反射光的光谱信息，和使用所述电子设备的与所述目标拍摄物相对侧的光谱传感器获取所述第一环境光的光谱信息。
在上述基于光谱信息的拍照方法中，所述光谱传感器的视场范围不小于用于拍摄所述目标拍摄物的拍照装置的镜头的视场范围。
在上述基于光谱信息的拍照方法中，所述第一光谱传感器和所述第二光谱传感器具有不同的视场角。
在上述基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：基于所述第一光谱传感器和所述第二光谱传感器的获取的光谱信息确定所述目标拍摄物的反射光谱信息。
在上述基于光谱信息的拍照方法中，基于所述第二环境光的信息调整拍照参数包括：将所述第二环境光的光谱信息与预置的光谱信息进行匹配，以判断当前的环境光源与光谱库里的预定数据是否匹配；以及，响应于当前的环境光源与光谱库里的预定数据匹配，根据所述当前的环境光源的光谱信息和所述第二环境光的信息调整拍照参数。
在上述基于光谱信息的拍照方法中，使用光谱传感器得到第一环境光的光谱信息包括：当判断打开所述拍照装置的拍照预览界面时，调整所述拍摄装置的姿态；以及，获取所述拍照装置的多个姿态下采集到的第一环境光的光谱信息。
在上述基于光谱信息的拍照方法中，所述光谱传感器包括具有不同截面厚度的匀光组件。
根据本申请的另一方面，提供了一种电子设备，包括：光谱传感器，用于得到第一环境光的光谱信息；以及，拍照装置，用于使用如上所述的基于光谱信息的拍照方法调整的拍照参数进行拍照。
本申请实施例提供的基于光谱信息的拍照方法和电子设备，通过光谱传感器采集和恢复的环境光的光谱信息和基于光谱信息的环境光信息来调整拍照参数，可以更加准确地调整拍照参数。
附图说明
通过阅读下文优选的具体实施方式中的详细描述，本申请各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。说明书附图仅用于示出优选实施方式的目的，而并不认为是对本申请的限制。显而易见地，下面描述的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。而且在整个附图中，用相同的附图标记表示相同的部件。
图1图示了根据本申请实施例的基于光谱信息的拍照方法的示意性流程 图。
图2图示了根据本申请实施例的用于计算重构型光谱的光谱芯片的示意图。
图3图示了根据本申请实施例的光谱传感器的示意性配置图。
图4图示了根据本申请实施例的多个光源生成的环境光的示意图。
图5图示了根据本申请实施例的包括多个拍照装置的电子设备的示意图。
图6图示了根据本申请实施例的第一光谱传感器获取的第一光谱图像的示意图。
图7图示了根据本申请实施例的第二光谱传感器获取的第二光谱图像的示意图。
图8图示了根据本申请实施例的光谱传感器的匀光组件的示意图。
图9图示了根据本申请实施例的电子设备的示意图。
具体实施方式
下面，将参考附图详细地描述根据本申请的示例实施例。显然，所描述的实施例仅仅是本申请的一部分实施例，而不是本申请的全部实施例，应理解，本申请不受这里描述的示例实施例的限制。
图1图示了根据本申请实施例的基于光谱信息的拍照方法的示意性流程图。
如图1所示，根据本申请实施例的基于光谱信息的拍照方法包括以下步骤：S10，使用光谱传感器得到第一环境光的光谱信息；S20，基于所述第一环境光的光谱信息确定第二环境光的信息；以及，S30，基于所述第二环境光的信息调整拍照参数。
在本申请实施例中，所述光谱传感器可以包括光谱芯片，且所述光谱芯片包括滤光结构和图像传感器，所述滤光结构位于所述图像传感器的感光路径上，所述滤光结构为频域或者波长域上的宽带滤光结构。各个滤光结构不同波长的通光谱不完全相同。滤光结构可以是超表面、光子晶体、纳米柱、多层膜、染料、量子点、MEMS(微机电系统)、FP etalon(FP标准具)、cavity layer(谐振腔层)、waveguide layer(波导层)、衍射元件等具有滤光特性的结构或者材料。例如，在本申请实施例中，所述滤光结构可以 是中国专利CN201910699962.3中的光调制层，如图2所示。图2图示了根据本申请实施例的用于计算重构型光谱的光谱芯片的示意图。
基于中国专利CN201921223201.2以及CN201910700328.7的内容可知，光谱芯片100包括光调制层110和图像传感层120，其中光调制层110的每个调制单元111和图像传感层120的每个感应单元121分别上下对应的设置在光调制层和图像传感层上，每个调制单元内分别设有至少一个调制子单元，每个调制子单元内分别设有若干个穿于光调制层内的调制孔，同一调制子单元内的各个调制孔排布成一具有特定排布规律的二维图形结构。所述光谱芯片基于光电子学中的调制单元阵列对不同波长光的调制作用，且能同时采集多个光谱的图像信息；因此对应不同坐标的调制结构，可以获取对应环境光中不同光源区域的光谱信息，包括但不限于中心波长，色温、色度等。
由于不同区域位置的光源信息可以根据调制结构的坐标位置进行判断，因此可以根据获取到的环境光的光谱信息判断所述环境光中有几种光源，所述光源的中心波长、色温、色坐标等都可以通过光谱进行恢复。
下面详细说明如果对环境光谱进行恢复。
如图3所示，所述光谱传感器还可以进一步包括光学系统，所述光学系统位于所述图像传感器的感光路径上，光通过光学系统调整再经由滤光结构进行调制后，被图像传感器接收，获取光谱响应；其中所述光学系统可能是透镜组件等光学组件。这里，图3图示了根据本申请实施例的光谱传感器的示意性配置图。
并且，图像传感器可以是CMOS图像传感器(CIS)、CCD、阵列光探测器等。另外，所述光谱传感器还包括数据处理单元，所述数据处理单元可以是MCU、CPU、GPU、FPGA、NPU、ASIC等处理单元，其可以将图像传感器生成的数据导出到外部进行处理。
例如，图像传感器测得光谱响应后，传入数据处理单元进行入射光恢复光谱信息计算。该过程具体描述如下：
将入射光在不同波长λ下的强度信号记为x(λ)，滤光结构的透射谱曲线记为T(λ)，滤光片(滤光结构)上具有m组的结构单元，每一组结构单元的透射谱互不相同，整体来讲，滤光结构可记为Ti(λ)(i＝1,2,3,…,m)。每一组结构单元下方都有相应的物理像素，探测经过滤光结构调制的光强bi。在本申请实施例中，以一个物理像素，即一个物理像素对应一组结构单元， 但是不限定于此，在其它实施例中，也可以是多个物理像素为一组对应于一组结构单元。
进一步，入射光的频谱分布和图像传感器的测量值之间的关系可以由下式表示：
bi＝∫x(λ)*Ti(λ)*R(λ)dλ
再进行离散化，得
bi＝Σ(x(λ)*Ti(λ)*R(λ))
其中R(λ)为图像传感器的响应，记为：
Ai(λ)＝Ti(λ)*R(λ)
则上式可以扩展为矩阵形式：
其中，bi(i＝1,2,3,…,m)是待测光透过滤光结构后图像传感器的响应，分别对应m个结构单元对应的图像传感器的光强测量值，当一个物理像素对应一个结构单元时，可以理解为m个“物理像素”对应的光强测量值，其是一个长度为m的向量。A是系统对于不同波长的光响应，由滤光结构透射率和图像传感器的量子效率两个因素决定。A是矩阵，每一个行向量对应一组结构单元对不同波长入射光的响应，这里，对入射光进行离散、均匀的采样，共有n个采样点。A的列数与入射光的采样点数相同。这里，x(λ)即是入射光在不同波长λ的光强，也就是待测量的入射光光谱。
在一些实施例中，与上述实施例不同之处在于，所述滤光结构可直接形成于所述图像传感器上表面，例如量子点、纳米线等，其直接在传感器的感光区域形成滤光结构或材料(纳米线、量子点等)，以滤光结构为例，此时，可以理解为所述图像传感器的原材料在加工形成所述图像传感器时，在原材料上表面加工形成滤光结构，所述透射谱和所述图像传感器的响应是一体的，即可以理解为所述探测器的响应和所述透射谱为同一曲线，此时入射光的频谱分布和图像传感器的光强测量值之间的关系可以由下式表示：
bi＝Σ(x(λ)*Ri(λ))
即本实施例中，透射谱Ai(λ)＝Ri(λ)
此时，入射光的频谱分布和图像传感器的光强测量值之间的关系可以由下式表示：
bi＝∫x(λ)*Ti(λ)*Ri(λ)dλ
再进行离散化，得
bi＝Σ(x(λ)*Ti(λ)*Ri(λ))
本实施例中，Ai(λ)＝Ti(λ)*Ri(λ)
即B＝AX
根据已知的透射谱A以及所测得的B待测光透过滤光结构后图像传感器的响应，可以通过计算获得入射光的光谱信息。然后光谱信息X(λ)，与三刺激函数X(λ)、Y(λ)、Z(λ)，分别对应波长相乘后累加，得出三刺激值，X、Y、Z。
那么色坐标x＝X/(X+Y+Z)、Y/(X+Y+Z)
通过色坐标可以进而对入射光(也就是所测的环境光)进行色温恢复。也就是，根据本申请实施例的光谱传感器可以是光谱色温传感器，用于获取环境光的色温信息，并基于所述环境光的色温信息来调整拍照参数，例如调整拍照装置的白平衡。
另外，如上所述，除了所述环境光的色温信息以外，所述环境光信息还可以包括所述环境光的中心波长、色度、照度、色坐标等信息。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的色温信息。
这里，本领域技术人员可以理解，在本申请实施例中，所述第一环境光和所述第二环境光可以是相同的环境光，也可以是不同的环境光。例如，所述第一环境光可以是所述第二环境光的部分环境光，例如所述第二环境光的预定位置或者区域的部分环境光作为第一环境光。或者，所述第二环境光也可以是所述第一环境光的部分环境光。
另外，所述第一环境光和所述第二环境光也可以分别是不同位置或者区域的环境光，例如，如将在下文中进一步详细说明的，所述第一环境光可以是手机的前摄对应的环境下的环境光，而所述第一环境光可以是手机的后摄对应的环境下的环境光。
并且，在本申请实施例中，由于所述光谱传感器可以获得包括对应不同滤光结构单元的位置信息及对应的光谱信息，因此所恢复的光谱信息中包括二维空间信息。因此，所述环境光的光谱信息至少包括位置信息及光谱信息，且可以根据所述环境光的光谱信息获取不同位置(坐标)或者区域的所述环境光的色温信息。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定所述第二环境光的色温信息包括：基于至少包括位置信息及光谱信息的所述第一环境光的光谱信息确定不同位置或者区域的所述第二环境光的色温信息。
例如，在不同位置或者区域的所述第二环境光的色温信息由多个光源引起的情况下，例如，如图4所示，所述光谱传感器获取到的第二环境光的信息包括光源1、光源2和光源3的信息。所述光源1在第二环境光的上侧，所述光源2和光源3位置在光源1的下方。这样，就可以基于所述不同位置或者区域的所述第二环境光的色温信息来对位于所述不同位置或者区域的多个待管理的光源进行管理。这里，图4图示了根据本申请实施例的多个光源生成的环境光的示意图。
也就是，在根据本申请实施例的基于光谱信息的拍照方法中，在基于至少包括位置信息及光谱信息的所述第一环境光的光谱信息确定不同位置或者区域的所述第二环境光的色温信息之后进一步包括：基于所述不同位置或者区域的所述第二环境光的色温信息来对位于所述不同位置或者区域的多个待管理的光源进行管理。
此外，还可以加上辅助TOF(time of flight)等进一步获取到光源到拍照装置的距离，以三维的方式获取光源相对于所述拍照装置的空间信息，并根据不同的空间位置对拍照参数进行调整。进一步地，还可以根据获取的环境光的光源的位置信息及光谱信息，判断光源在环境光中起到的权重，根据不同位置光源对环境光的权重参数对拍照参数进行调整。也就是，不同区域的光源产生的环境光对拍照的影响不同，因此可以根据不同区域的环境光的光谱信息进行拍照参数的调整。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定所述第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的光源到拍照装置的距离，以确定所述光源相对于所述拍照装置的空间信息。
并且，在根据本申请实施例的基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定所述第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的不同位置或者区域的光源在所述第二环境光中的权重参数。
另外，在本申请实施例中，还可以根据获取的第一环境光的光源的位置信息及光谱信息，确定所述第二环境光中的不同光源的位置和类型，并基于所述光源的位置和类型调整拍照参数，例如调整白平衡。
并且，在根据本申请实施例的基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定所述第二环境光的信息包括：基于所述第一环境光的光谱信息确定所述第二环境光的不同位置或者区域的光源及其类型。
在另一实施例中，根据本申请实施例的基于光谱信息的拍照方法可以应用于具有多个拍照装置的电子设备，例如，如图5所示。图5图示了根据本申请实施例的包括多个拍照装置的电子设备的示意图。
如图5所示，所述电子设备设置有第一拍照装置及第二拍照装置，所述第一拍照装置设置于屏幕显示一侧，所述第二拍照装置设置于背盖一侧，所述第一拍照装置旁设置有至少第一光谱传感器，所述第二拍照装置旁设置至少第二光谱传感器。
也就是，在根据本申请实施例的基于光谱信息的拍照方法中，所述光谱传感器包括设置在位于电子设备的屏幕显示一侧的第一拍照装置位置处的第一光谱传感器和设置在位于电子设备的背盖一侧的第二拍照装置位置处的第二光谱传感器。
在本申请实施例中，所述电子设备可以是移动电子设备，也可以为非移动电子设备。例如，移动电子设备可以为手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、可穿戴设备、上网本等，非移动电子设备可以为服务器、个人计算机(personal computer，PC)、电视机(television，TV)、柜员机或者自助机等，本申请实施例不作具体限定。
在本申请实施例中，可以在拍摄的时候通过距离传感器或者对焦的情况判断出所述目标拍摄物与所述电子设备之间的距离，例如，可以通过拍照装置获取的对焦信息进行距离的判断。并且，所述电子设备可以预先存储预设距离，当小于所述预设距离时，获取与当前拍照装置以外的另一拍照装置对应的光谱传感器。也就是，当所述目标拍摄物一侧设置有第一光谱传感器，相对的一侧设置有第二光谱传感器时，当电子设备距离目标拍摄物的距离小于预设距离时，开启相对的一侧的光谱传感器。
具体地，当使用第一拍照装置，例如前摄拍照装置进行拍照时，判断实际拍照时所述目标拍摄物与所述电子设备的距离小于预设距离时，获取所述 第二光谱传感器采集的环境光的光谱信息。这里，所述第二光谱传感器在后摄，也就是第二拍摄装置的位置处。
同样地，当使用第二拍照装置进行拍照时，也就是使用后摄进行拍照时，当达到预设距离时，获取第一光谱传感器采集的环境光的光谱信息，也即是前摄旁的光谱传感器采集的环境光的光谱信息。由于在不同距离拍摄时，环境光对拍照的影响不同，越近时，拍出来的照片的颜色与实际的物体的颜色偏离越大。这是因为物体本身对光就有吸收和反射。且不同的物理对光的吸收或反射能力不同，尤其是不同的颜色的物体对不同环境光的反射或吸收的能力都不相同。因此，当距离较远时，光谱传感器获取到的是环境光的光谱信息，而当距离达到设定距离以及小于设定距离以后，所述目标拍摄物一侧的光谱传感器获取的是环境光及物体表面反射光的光谱信息。因此，借助相对侧的光谱传感器可以获取当前环境光的光谱信息。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，在所述拍照装置包括所述第一拍照装置和所述第二拍照装置的情况下，使用光谱传感器得到第一环境光的光谱信息包括：确定目标拍摄物与所述电子设备的距离信息；以及，响应于所述距离信息小于预设距离，使用所述电子设备的与所述目标拍摄物相同侧的光谱传感器获取所述第一环境光及所述目标拍摄物表面反射光的光谱信息，和使用所述电子设备的与所述目标拍摄物相对侧的光谱传感器获取所述第一环境光的光谱信息。
这里，在本申请实施例中，还需要确定当前拍摄装置的镜头的视场范围与光谱传感器的视场范围，其中所述光谱传感器的视场范围不小于所述镜头的视场范围。这里，所述光谱传感器包括与用于拍摄所述目标拍摄物的拍照装置的镜头相同侧或者相对侧的光谱传感器。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，所述光谱传感器的视场范围不小于用于拍摄所述目标拍摄物的拍照装置的镜头的视场范围。
此外，在本申请实施例中，所述第一光谱传感器和所述第二光谱传感器可以具有不同的视场角，从而获取不同位置及视场角的环境光信息，并进行组合以作为拍照参数调整的信息。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，所述第一光谱传感器和所述第二光谱传感器具有不同的视场角。
并且，如上所述，当拍照装置与目标拍摄物的距离小于预设距离时，获取到的第一环境光的光谱信息中包含物体反射光的光谱信息。这样，通过两个光谱传感器获取前后面的环境光的光谱信息，可以根据获取到的前后两面环境光的光谱信息判断物体的反射光谱，根据不同的物体的反射光谱不同，进行拍照参数的调整。也就是，当拍照装置与目标拍摄物的距离小于预设距离时，当光谱传感器采集所述焦平面对应的空间内环境光的光谱信息时，采集的光谱信息包括焦点范围内被拍摄物的反射光谱信息。
具体地，不同光谱传感器获取的光谱图像如图6和图7所示。其中，图6图示了根据本申请实施例的第一光谱传感器获取的第一光谱图像的示意图。图7图示了根据本申请实施例的第二光谱传感器获取的第二光谱图像的示意图。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，基于所述第一环境光的光谱信息确定所述第二环境光的信息包括：基于所述第一光谱传感器和所述第二光谱传感器的获取的光谱信息确定所述目标拍摄物的反射光谱信息。
另外，在本申请实施例中，在获取所述第二环境光的光谱信息之后，还可以与预置的光谱信息进行匹配，从而判断当前的环境光源与光谱库里的预定数据是否匹配，并且在当前的环境光源与所述光谱库里的预定数据匹配时，根据当前检测到的环境光的光谱信息及光谱恢复的环境光的色温、色度等信息对拍照参数进行管理，例如调整白平衡，或者显示色彩度等参数。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，基于所述第二环境光的信息调整拍照参数包括：将所述第二环境光的光谱信息与预置的光谱信息进行匹配，以判断当前的环境光源与光谱库里的预定数据是否匹配；以及，响应于当前的环境光源与光谱库里的预定数据匹配，根据所述当前的环境光源的光谱信息和所述环境光的信息调整拍照参数。
此外，在本申请实施例中，使用光谱传感器得到环境光的光谱信息可以包括：当判断打开所述拍照装置的拍照预览界面时，调整所述拍摄装置的姿态；和，获取所述拍照装置的多个姿态下采集到的第一环境光的光谱信息。
具体地，这些姿态可以通过用户进行手动调整，不同的角度获取到环境光的信息有所不同。这样，根据多次获取到的环境光的光谱信息进行计算，从而计算得到当前实际环境光的光谱信息，可以减少距离过近带来的环境光 的光谱信息的测量误差，此外，通过前后光谱传感器同时多次多姿态获取环境光的光谱信息，计算实际环境光的光谱信息，可以根据获取到的实际环境光的光谱信息，调节拍摄参数。
此外，在本申请实施例中，在手机的正面和/或背面装配有摄像头以及光谱传感器的情况下，摄像头和光谱传感器也可以设置于显示屏下，这种情况下，所述光谱传感器可以利用屏幕的结构作为滤光结构，进行光调制，并获取对应的光谱信息。
例如，在手机的正面，在顶端设置有前置摄像头，在前置摄像头的相邻位置可以设置有光谱传感器。其中，光谱传感器可以采用光谱图像探测器，近红外光谱传感器。光谱传感器的通道可以是根据光调制结构的数量确定有多通道，可以几十个通道进行光谱恢复，以便获得非常准确的环境光信息。当然如果环境光比较单一也可以采用十几通道的光谱传感器，所采用的光谱传感器可以设置有滤光结构，也可以部分设置滤光结构，不设计滤光结构的图像传感器可以获取环境光的亮度信息参数等，以辅助更好的恢复光谱信息。
当然，在另外的一些示例中，手机的背面上可以装配有多个摄像头，例如同时装配有主摄像头、广角摄像头、深度摄像头等；此时，光谱传感器可以复用其中一颗摄像头，例如复用深度摄像头既可以进行图像拍照，又可以获取环境光的光谱信息，且可以更好的进行背景虚化，由于光谱传感器可以在成像的同时获取对应像素的光谱信息，可以更好的区分像素中的背景信息，不仅仅区别RGB颜色信息，还可以通过光谱信息获取背景中相同颜色的不同物质信息，因此可以更好的区分背景及前景，进行背景虚化的功能。
进一步的，在本申请实施例中，还可以在光谱传感器上配置不同的匀光组件，或者同一匀光组件上的匀光能力不同。
这里，由于不同的匀光能对环境光的进入的光能量不同，因此到达调制结构上的光的能量也不相同，因此匀光能力越强的匀光组件，对应空间中的环境光就会显示较为统一的光谱信息，匀光能力较差一些的匀光组件可以更好的区分不同位置不同区域的光谱信息。图8图示了根据本申请实施例的光谱传感器的匀光组件的示意图。因此，若采用不同厚度的匀光组件，可以获取更多位置信息或更多区域的光源光谱信息。
因此，在根据本申请实施例的基于光谱信息的拍照方法中，所述光谱传感器包括具有不同截面厚度的匀光组件。
进一步地，在电子设备上，可以使光谱传感器与拍照装置的摄像头在同一平面上，也可以使得光谱传感器高于摄像头的平面，由于光谱传感器的视场角越大，获取到的环境光的光谱信息越多，恢复的环境光的信息就越准确，因此可以采用不同的光学组件使得光谱传感器的视场角变大，例如可以超过180°；光谱传感器可以与摄像头相邻设置，或者，也可以与摄像头相距较远的距离。对光谱传感器与摄像头之间的距离不做特殊限定。
因此，根据本申请实施例的基于光谱信息的拍照方法通过光谱传感器采集和恢复的环境光的光谱信息和基于光谱信息的环境光信息来调整拍照参数，可以更加准确地调整拍照参数。
例如，通过光谱图像方式对环境光进行色温恢复，且通过色温曲线获取色温在色度坐标系中的坐标，进行转换得到校正增益。或者，通过对光谱曲线进行匹配识别确定光源类型，进而利用各通道的强度值进行当前环境光的CCT(Correlated Color Temperature，相关色温)计算，进而利用光源类型、色温实现更加准确的拍摄参数的调节。
图9图示了根据本申请实施例的电子设备的示意图。如图9所示，电子设备1000包括中央处理单元(Central Processing Unit，CPU)1001，其可以根据存储在只读存储器(Read-Only Memory，ROM)1002中的程序或者从储存部分加载到随机访问存储器(Random Access Memory，RAM)1003中的程序而执行各种适当的动作和处理。在RAM 1003中，还存储有系统操作所需的各种程序和数据。CPU 1001、ROM1002以及RAM 1003通过总线1004彼此相连。输入/输出(Input/Output，I/O)接口1005也连接至总线1004。
以下部件连接至I/O接口1005：包括键盘、鼠标等的输入部分1006；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal Display，LCD)等以及扬声器等的输出部分1007；包括硬盘等的储存部分1008；以及包括诸如LAN(Local AreaNetwork，局域网)卡、调制解调器等的网络接口卡的通信部分1009。通信部分1009经由诸如因特网的网络执行通信处理。驱动器1010也根据需要连接至I/O接口1005。可拆卸介质1011，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器1010上，以便于从其上读出的计算机程序根据需要被安装入储存部分1008。
如图9所示，电子设备1000还可以包括至少一个光谱传感器，至少一个摄像头。所述光谱传感器的感光范围与所述摄像头的目标感光区域对应。 举例来说，上述的电子设备可以是智能穿戴设备、智能终端等，例如智能手表或者手机等电子设备。
特别地，根据本申请实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本发明的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分1009从网络上被下载和安装，和/或从可拆卸介质1011被安装。在该计算机程序被中央处理单元(CPU)1001执行时，执行本申请的系统中限定的各种功能。
具体来说，上述的电子设备可以是手机、平板电脑或者笔记本电脑等智能移动终端设备。或者，上述的电子设备也可以是台式电脑等智能终端设备。
需要说明的是，本发明实施例所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，
CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本发明中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本发明中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、有线等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本发明各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上结合具体实施例描述了本申请的基本原理，但是，需要指出的是，在本申请中提及的优点、优势、效果等仅是示例而非限制，不能认为这些优点、优势、效果等是本申请的各个实施例必须具备的。另外，上述公开的具体细节仅是为了示例的作用和便于理解的作用，而非限制，上述细节并不限制本申请为必须采用上述具体的细节来实现。
本申请中涉及的器件、装置、设备、系统的方框图仅作为例示性的例子并且不意图要求或暗示必须按照方框图示出的方式进行连接、布置、配置。如本领域技术人员将认识到的，可以按任意方式连接、布置、配置这些器件、装置、设备、系统。诸如“包括”、“包含”、“具有”等等的词语是开放性词汇，指“包括但不限于”，且可与其互换使用。这里所使用的词汇“或”和“和”指词汇“和/或”，且可与其互换使用，除非上下文明确指示不是如此。这里所使用的词汇“诸如”指词组“诸如但不限于”，且可与其互换使用。
还需要指出的是，在本申请的装置、设备和方法中，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应视为本申请的等效方案。
提供所公开的方面的以上描述以使本领域的任何技术人员能够做出或者使用本申请。对这些方面的各种修改对于本领域技术人员而言是非常显而易见的，并且在此定义的一般原理可以应用于其他方面而不脱离本申请的范围。因此，本申请不意图被限制到在此示出的方面，而是按照与在此公开的原理和新颖的特征一致的最宽范围。
为了例示和描述的目的已经给出了以上描述。此外，此描述不意图将本申请的实施例限制到在此公开的形式。尽管以上已经讨论了多个示例方面和实施例，但是本领域技术人员将认识到其某些变型、修改、改变、添加和子组合。

Claims (16)

  1. 一种基于光谱信息的拍照方法，其特征在于，包括：
    使用光谱传感器得到第一环境光的光谱信息；
    基于所述第一环境光的光谱信息确定第二环境光的信息；以及，
    基于所述第二环境光的信息调整拍照参数。
  2. 根据权利要求1所述的基于光谱信息的拍照方法，其中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：
    基于所述第一环境光的光谱信息确定所述第二环境光的色温信息。
  3. 根据权利要求2所述的基于光谱信息的拍照方法，其中，基于所述第一环境光的光谱信息确定所述第二环境光的色温信息包括：
    基于至少包括位置信息及光谱信息的所述第一环境光的光谱信息确定不同位置或者区域的所述第二环境光的色温信息。
  4. 根据权利要求2所述的基于光谱信息的拍照方法，其中，在基于至少包括位置信息及光谱信息的所述第一环境光的光谱信息确定不同位置或者区域的所述第二环境光的色温信息之后进一步包括：
    基于所述不同位置或者区域的所述第二环境光的色温信息来对位于所述不同位置或者区域的多个待管理的光源进行管理。
  5. 根据权利要求1所述的基于光谱信息的拍照方法，其中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：
    基于所述第一环境光的光谱信息确定所述第二环境光的光源到拍照装置的距离，以确定所述光源相对于所述拍照装置的空间信息。
  6. 根据权利要求1所述的基于光谱信息的拍照方法，其中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：
    基于所述第一环境光的光谱信息确定所述第二环境光的不同位置或者区域的光源在所述第二环境光中的权重参数。
  7. 根据权利要求1所述的基于光谱信息的拍照方法，其中，基于所述第一环境光的光谱信息确定所述第二环境光的信息包括：
    基于所述第一环境光的光谱信息确定所述第二环境光的不同位置或者区域的光源及其类型。
  8. 根据权利要求1所述的基于光谱信息的拍照方法，其中，所述光谱传感器包括设置在位于电子设备的屏幕显示一侧的第一拍照装置位置处的第一光谱传感器和设置在位于电子设备的背盖一侧的第二拍照装置位置处的第二光谱传感器。
  9. 根据权利要求8所述的基于光谱信息的拍照方法，其中，使用光谱传感器得到第一环境光的光谱信息包括：
    确定目标拍摄物与所述电子设备的距离信息；以及，
    响应于所述距离信息小于预设距离，使用所述电子设备的与所述目标拍摄物相同侧的光谱传感器获取所述第一环境光及所述目标拍摄物表面反射光的光谱信息，和使用所述电子设备的与所述目标拍摄物相对侧的光谱传感器获取所述第一环境光的光谱信息。
  10. 根据权利要求8所述的基于光谱信息的拍照方法，其中，所述光谱传感器的视场范围不小于用于拍摄所述目标拍摄物的拍照装置的镜头的视场范围。
  11. 根据权利要求8所述的基于光谱信息的拍照方法，其中，所述第一光谱传感器和所述第二光谱传感器具有不同的视场角。
  12. 根据权利要求8所述的基于光谱信息的拍照方法，其中，基于所述第一环境光的光谱信息确定第二环境光的信息包括：
    基于所述第一光谱传感器和所述第二光谱传感器的获取的光谱信息确定所述目标拍摄物的反射光谱信息。
  13. 根据权利要求1所述的基于光谱信息的拍照方法，其中，基于所述第二环境光的信息调整拍照参数包括：
    将所述第二环境光的光谱信息与预置的光谱信息进行匹配，以判断当前的环境光源与光谱库里的预定数据是否匹配；以及，
    响应于当前的环境光源与光谱库里的预定数据匹配，根据所述当前的环境光源的光谱信息和所述第二环境光的信息调整拍照参数。
  14. 根据权利要求1所述的基于光谱信息的拍照方法，其中，使用光谱传感器得到第一环境光的光谱信息包括：
    当判断打开所述拍照装置的拍照预览界面时，调整所述拍摄装置的姿态；以及，
    获取所述拍照装置的多个姿态下采集到的第一环境光的光谱信息。
  15. 根据权利要求1所述的基于光谱信息的拍照方法，其中，所述光谱传感器包括具有不同截面厚度的匀光组件。
  16. 一种电子设备，包括：
    光谱传感器，用于得到第一环境光的光谱信息；以及
    拍照装置，用于使用如权利要求1到15中任意一项所述的基于光谱信息的拍照方法调整的拍照参数进行拍照。
PCT/CN2023/111389 2022-09-07 2023-08-07 基于光谱信息的拍照方法和电子设备 WO2024051418A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211105648.6 2022-09-07
CN202211105648.6A CN117729412A (zh) 2022-09-07 2022-09-07 基于光谱信息的拍照方法和电子设备

Publications (1)

Publication Number Publication Date
WO2024051418A1 true WO2024051418A1 (zh) 2024-03-14

Family

ID=90192022

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/111389 WO2024051418A1 (zh) 2022-09-07 2023-08-07 基于光谱信息的拍照方法和电子设备

Country Status (2)

Country Link
CN (1) CN117729412A (zh)
WO (1) WO2024051418A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018058873A1 (zh) * 2016-09-29 2018-04-05 宇龙计算机通信科技(深圳)有限公司 拍照方法及装置
CN111551266A (zh) * 2020-05-25 2020-08-18 吉林求是光谱数据科技有限公司 一种基于多光谱图像探测技术的环境色温测试方法及系统
CN113194303A (zh) * 2021-06-29 2021-07-30 深圳小米通讯技术有限公司 图像白平衡方法、装置、电子设备和计算机可读存储介质
US20210372920A1 (en) * 2018-05-04 2021-12-02 Peter Ehbets Handheld non-contact multispectral measurement device with position correction
US20220006994A1 (en) * 2020-07-01 2022-01-06 Spectricity White balance compensation using a spectral sensor system
CN114746728A (zh) * 2019-11-27 2022-07-12 ams传感器德国有限公司 环境光源分类

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018058873A1 (zh) * 2016-09-29 2018-04-05 宇龙计算机通信科技(深圳)有限公司 拍照方法及装置
US20210372920A1 (en) * 2018-05-04 2021-12-02 Peter Ehbets Handheld non-contact multispectral measurement device with position correction
CN114746728A (zh) * 2019-11-27 2022-07-12 ams传感器德国有限公司 环境光源分类
CN111551266A (zh) * 2020-05-25 2020-08-18 吉林求是光谱数据科技有限公司 一种基于多光谱图像探测技术的环境色温测试方法及系统
US20220006994A1 (en) * 2020-07-01 2022-01-06 Spectricity White balance compensation using a spectral sensor system
CN113194303A (zh) * 2021-06-29 2021-07-30 深圳小米通讯技术有限公司 图像白平衡方法、装置、电子设备和计算机可读存储介质

Also Published As

Publication number Publication date
CN117729412A (zh) 2024-03-19

Similar Documents

Publication Publication Date Title
US9866748B2 (en) System and method for controlling a camera based on processing an image captured by other camera
JP7371081B2 (ja) 夜景撮影方法、装置、電子機器および記憶媒体
WO2021238843A1 (zh) 一种基于多光谱图像探测技术的环境色温测试方法及系统
US20160227100A1 (en) Dual camera systems and methods for rapid 3a convergence and high dynamic range exposure metering
WO2020038074A1 (zh) 曝光控制方法、装置以及电子设备
US20170230585A1 (en) Systems and methods for implementing seamless zoom function using multiple cameras
CA2864763C (fr) Procede et dispositif de mesure de la couleur d'un objet
US9060110B2 (en) Image capture with tunable polarization and tunable spectral sensitivity
US20160212333A1 (en) Picture taking device comprising a plurality of camera modules
US10187566B2 (en) Method and device for generating images
US20170150126A1 (en) Photographing device and operating method of the same
US20120213407A1 (en) Image capture and post-capture processing
TW201316760A (zh) 攝像裝置、及其之控制方法
US20120019669A1 (en) Systems and methods for calibrating image sensors
US20100123804A1 (en) Emotion-based image processing apparatus and image processing method
US20230325999A1 (en) Image generation method and apparatus and electronic device
US9877004B2 (en) Color-mixture-ratio calculation device and method, and imaging device
US8654210B2 (en) Adaptive color imaging
US9635247B2 (en) Method of displaying a photographing mode by using lens characteristics, computer-readable storage medium of recording the method and an electronic apparatus
WO2024051418A1 (zh) 基于光谱信息的拍照方法和电子设备
WO2023071933A1 (zh) 相机拍摄参数调整方法、装置及电子设备
US20120212636A1 (en) Image capture and post-capture processing
US8547447B2 (en) Image sensor compensation
JP2020191546A (ja) 画像処理装置、画像処理方法、およびプログラム
US20240071275A1 (en) Calibration system for display apparatus and operating method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23862116

Country of ref document: EP

Kind code of ref document: A1