WO2023207443A1 - Remote spectral imaging system and method - Google Patents

Remote spectral imaging system and method Download PDF

Info

Publication number
WO2023207443A1
WO2023207443A1 PCT/CN2023/083095 CN2023083095W WO2023207443A1 WO 2023207443 A1 WO2023207443 A1 WO 2023207443A1 CN 2023083095 W CN2023083095 W CN 2023083095W WO 2023207443 A1 WO2023207443 A1 WO 2023207443A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
data
color
display
image data
Prior art date
Application number
PCT/CN2023/083095
Other languages
French (fr)
Chinese (zh)
Inventor
崔开宇
徐晟�
黄翊东
张巍
冯雪
刘仿
Original Assignee
清华大学
北京与光科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202210476278.0A external-priority patent/CN117014586A/en
Priority claimed from CN202210869460.2A external-priority patent/CN117478867A/en
Application filed by 清华大学, 北京与光科技有限公司 filed Critical 清华大学
Publication of WO2023207443A1 publication Critical patent/WO2023207443A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present application relates to the field of computer technology, and in particular to a remote spectral imaging system and method.
  • the transmission process of remote images can be divided into three steps: image acquisition, signal transmission, and image reconstruction.
  • the image reconstruction step can use display technologies such as laser liquid crystal display to achieve high-definition and high-fidelity color reproduction based on the different color channel intensity values obtained by the color camera.
  • RGB and other multi-channel cameras are used to collect color image information of actual scenes, and then the terminal screen is displayed based on the transmitted image information.
  • current imaging technology focuses on how to obtain more color image information and high-performance transmission, but does not consider terminal display, so the accuracy of image display in actual situations cannot be guaranteed.
  • This application provides a remote spectral imaging system and method to solve the defect of color distortion of remote images in the existing technology and improve the color restoration of remote images.
  • This application provides a terminal detection system and method for a remote spectral imaging system to solve the defect of color distortion in actual image display in the prior art and improve the color accuracy of image display.
  • This application provides a remote spectral imaging system, including: an image generation subsystem and a remote display subsystem;
  • the image generation subsystem is configured to obtain first image data of the scene to be imaged based on spectral imaging, and send the first image data to the remote display subsystem, and the remote display subsystem displays the first image data;
  • Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data, and the second image data is displayed by the remote display subsystem.
  • the remote spectral imaging system further includes a color difference detection subsystem
  • the color difference detection subsystem is used to shoot the display image of the first image data, obtain a comparison image, and obtain the image color difference value based on the comparison image and the first image data.
  • the color difference detection subsystem includes a spectrum imaging module, an image matching module and a color difference calculation module;
  • the spectral imaging module is configured to capture the display image of the first image data based on spectral imaging to obtain the comparison image;
  • the image matching module is used to perform image matching on the comparison image and the first image data to obtain a matching pixel point set;
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes The first pixel point in the first image data and the second pixel point in the comparison image;
  • the color difference calculation unit is used to obtain the color difference value of the first pixel point and the second pixel point in each of the pixel point pairs, and obtain the image color difference value.
  • the remote display subsystem includes a display module and a color difference compensation module
  • the display module is used to display the first image data and the second image data
  • the color difference compensation module is configured to perform color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the image generation subsystem includes a spectral imaging device, a color processing module and an encoding module;
  • the spectral imaging device is used to obtain spectral data of the scene to be imaged and send the spectral data to the color processing module;
  • the color processing module is used to receive the spectrum data and perform color processing on the spectrum data to obtain color data; the color processing module is also used to send the color data to the encoding module;
  • the encoding module is used to receive the color data and encode the color data to obtain first image data.
  • the spectral imaging device includes a filter structure, an image sensor and a data processing unit, the filter structure is disposed on the photosensitive path of the image sensor; the transmittance curve of the filter structure is based on The spectral characteristics of the scene to be imaged are determined;
  • the filter structure is used to modulate the incident light of the scene to be imaged to obtain a modulated light signal
  • the image sensor is used to receive the modulated light signal and obtain light intensity data
  • the data processing unit is configured to receive light intensity data sent by the image sensor, and perform spectral recovery based on the light intensity data to obtain spectral data corresponding to the scene to be imaged.
  • the color processing module is used to perform processing on the incident light spectrum Color processing, obtaining color data, including:
  • the color processing module is used to perform color processing on the incident light spectrum based on a wide color gamut standard to obtain color data.
  • system further includes: a data transmission module
  • the data transmission module includes any of the following: a first transmission module, a second transmission module and a third transmission module;
  • the first transmission module is connected to the spectral imaging device and the color processing module, and is used to send the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module is connected to the color processing module and the encoding module, and is used to send the color data obtained by the color processing module to the encoding module;
  • the third transmission module is connected to the encoding module and the remote display subsystem, and is used to send the first image data obtained by the encoding module to the remote display subsystem.
  • the data transmission module is used to transmit data at a data rate of no less than 8 bit/s.
  • the remote display subsystem includes a terminal detection system, which includes: a display terminal, a colorimeter and a calibration module;
  • the display terminal is used to display the first image data
  • the colorimeter is used to obtain the color data of the display image of the first image data
  • the calibration module is configured to obtain, based on the color data of the first image data and the color data of the display image of the first image data, the difference between the first image data and the display image of the first image data. Mapping relations;
  • the calibration module is also used to perform color calibration on the display terminal based on the mapping relationship.
  • the color data is any one of chromaticity values, RGB values, and spectral data.
  • the display terminal is also used to receive and display the target image.
  • the display terminal includes a first display unit and a second display unit:
  • the first display unit is used to display the target image
  • the second display unit is used to display the first image data.
  • the first display unit is a first split-screen area in the display terminal
  • the second display unit is a second split-screen area in the display terminal.
  • This application also provides a terminal detection method based on the above-mentioned remote spectral imaging system, including:
  • Color calibration is performed on the display terminal based on the mapping relationship.
  • the color data is any one of chromaticity values, RGB values, and spectral data.
  • the method further includes:
  • the display terminal receives the target image
  • the color calibration of the display terminal based on the mapping relationship also includes:
  • the display terminal displays the target image.
  • the first image data is displayed by a second display unit
  • the method also includes:
  • the display terminal receives a target image, and the target image is displayed by the first display unit;
  • the obtaining the color data of the display image of the first image data includes:
  • the color data based on the first image data and the first image data Display the color data of the image, and obtain the mapping relationship between the first image data and the display image of the first image data, including:
  • the color calibration of the display terminal based on the mapping relationship includes:
  • the target image is obtained based on spectral data of the target imaging object.
  • the target image is obtained by color processing the spectral data; and/or
  • the target image is obtained through data transmission at a data rate of no less than 8 bit/s.
  • This application also provides a remote spectral imaging method based on the above-mentioned remote spectral imaging system, including:
  • Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data.
  • obtaining the image color difference value based on the display image of the first image data and the first image data includes:
  • the display image of the first image data is photographed to obtain a comparison image, and the image color difference value is obtained based on the comparison image and the first image data.
  • photographing the display image of the first image data, obtaining a comparison image, and obtaining the image color difference value based on the comparison image and the first image data includes:
  • the display image of the first image data is photographed to obtain the Described comparison image
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes a pair of pixel points in the first image data.
  • the color difference value of the first pixel point and the second pixel point in each pair of pixel points is obtained to obtain the image color difference value.
  • performing color difference compensation on the first image data based on the image color difference value to obtain the second image data includes:
  • Color difference compensation is performed on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the scene to be imaged is photographed based on spectral imaging to obtain first image data, including:
  • the color data is encoded to obtain first image data.
  • encoding the color data to obtain color data includes:
  • Color processing is performed on the incident light spectrum based on a wide color gamut standard to obtain color data.
  • the method includes any of the following data transmission steps:
  • the first transmission module sends the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module sends the color data obtained by the color processing module to the encoding module
  • the third transmission module sends the first image data obtained by the encoding module to the remote display subsystem.
  • data transmission is performed at a data rate of no less than 8 bit/s.
  • the remote spectral imaging system and method provided by this application can, on the one hand, collect pictures through spectral imaging through the image generation subsystem, and obtain accurate spectral information and color information; on the other hand, the first image data can be compared with the color difference detection subsystem through the color difference detection subsystem. The images are matched and calibrated to restore the real spatial color information of the scene to be imaged, improving the color restoration of remote image display.
  • the terminal detection system and method for a remote spectral imaging system uses the first image data as a color reference to obtain a display image of the first image data actually presented by the display terminal, compare the two, and establish a mapping Relationship, the display terminal is calibrated and errors are eliminated based on the mapping relationship, which avoids the color display error of the display terminal and further improves the accuracy of the imaging system.
  • Figure 1 is a schematic structural diagram of a remote spectral imaging system provided by an embodiment of the present application
  • Figure 2 is a schematic structural diagram of a spectral imaging device provided by an embodiment of the present application.
  • Figure 3 is a functional block diagram of a spectrum chip provided by an embodiment of the present application.
  • Figure 4 is a schematic structural diagram of a spectrum chip provided by an embodiment of the present application.
  • Figure 5 is a schematic structural diagram of a terminal detection system for a remote spectral imaging system provided by an embodiment of the present application
  • Figure 6 is one of the flow diagrams of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application
  • Figure 7 is a terminal detection for a remote spectral imaging system provided by an embodiment of the present application. Flowchart 2 of the method;
  • Figure 8 is the third schematic flowchart of the terminal detection method for the remote spectral imaging system provided by the embodiment of the present application.
  • Figure 9 is a schematic flowchart of a remote spectral imaging method provided by an embodiment of the present application.
  • Figure 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the accuracy of the acquired information must be ensured at the collection end.
  • the loss of information must be avoided during the transmission process.
  • the accuracy must be ensured during the display process. Any deficiency has a certain probability of affecting the accuracy of the final display.
  • Figure 1 is a schematic structural diagram of a remote spectral imaging system provided by an embodiment of the present application.
  • the remote spectral imaging system provided by an embodiment of the present application includes: an image generation subsystem 110 and a remote display subsystem 120;
  • the image generation subsystem 110 is configured to obtain first image data of the scene to be imaged based on spectral imaging, and send the first image data to the remote display subsystem 120, and the remote display subsystem 120 displays the first image data;
  • the scene to be imaged is located at the image collection site, and the image generation subsystem 110 captures the scene to be imaged through a spectral imaging device to obtain spectral data; further, the spectral data is transmitted, preferably during the transmission process.
  • the spectral data is subjected to color processing, converted into a color space, and then compressed and encoded to obtain a display image.
  • the image generation subsystem described in this application may include a collection end for collecting data and a transmission end for transmitting to the display system, that is, the collection end may be implemented as the spectrum imaging device to Collect spectral data; and then further use the transmission end to transmit the data with high fidelity, for example, perform high-bit data transmission on the spectral data. That is, after acquiring the spectral data of each pixel of the image, in order to minimize the loss of color information in the image, high-bit data transmission is performed in the embodiment of the present application. In this field, usually no less than 8 Bit data transmission can be called high-bit data transmission.
  • the high-bit data transmission may be 10-bit, 12-bit, or 16-bit data transmission.
  • color processing of the transmitted spectral data may include: the image processing process is performed in a gamut-independent color space, and there is no compression or loss of color gamut during the image processing process.
  • Spectral data needs to be converted into a color space, such as the BT.2020 color gamut space, through colorimetric calculation and color characterization.
  • color characterization can use polynomial transformation, lookup table or neural network methods.
  • color characterization refers to establishing a correspondence between chromaticity values and camera output values.
  • Encode color-processed image data (spectral data). That is to say, after front-end signal collection and digital image processing, the loss of image color accuracy can be minimized.
  • the processed image data Encoding results in high-fidelity display images.
  • the remote display subsystem 120 is configured to receive the image sent by the image generation subsystem 110
  • the first image data and display the first image data that is, the remote display subsystem 120 displays based on the first image data; generally speaking, encoding can be performed on the remote display subsystem, also It can be performed in the image generation subsystem; further explanation is needed.
  • the first image data generated by the image generation subsystem 110 of this application can be conventional RGB, black and white pictures, or spectral data or spectral imaging.
  • the remote display subsystem 120 is provided at a remote image display location.
  • the remote display subsystem 120 may be a display device, such as a portable display, a 4K display, or an 8K display.
  • the spectral imaging device used to obtain spectral data in this application and the display device used to display it are implemented as remote settings.
  • users can be provided with high color gamut and high fidelity image experience to the maximum extent.
  • the spectral imaging device can be set up in the consultation room, and the display device can be set up in a certain consultation room outside the consultation room.
  • the doctor can obtain the spatial color information of the real medical scene to the greatest extent through the display device, and there is It is helpful to improve the diagnostic accuracy of doctors.
  • the remote spectral imaging system provided by the embodiment of the present application is also used to obtain the image color difference between the display image of the first image data and the first image data based on the first image data and the display image of the first image data. value; wherein, the display image of the first image data is a display image displayed by the remote display subsystem 120 based on the first image data.
  • the remote display subsystem 120 has an image display function and can display the first image data in the form of an image.
  • the display image of the first image data refers to the picture in which the first image data is actually displayed by the remote display subsystem under the influence of the device's own parameters and external environmental factors (such as external light).
  • the display content of an image data is the first image data under the influence of the device's own parameters and external environmental factors (such as external light).
  • the remote spectral imaging system provided by the embodiment of the present application is also used to perform color difference compensation on the first image data based on the image color difference value, obtain the second image data, and obtain the second image data based on the image color difference value.
  • the remote display subsystem displays the second image data.
  • a color difference detection device (such as the color difference detection subsystem provided in the embodiment of the present application) is used to pre-obtain the image color difference value of the display image of the first image data and the first image data.
  • the first image data may include multiple
  • the first image data can be a detection picture including 16777216 colors.
  • the image color difference value corresponding to each color of the 16777216 colors can be obtained.
  • the remote display subsystem performs color difference compensation on the display image of the first image data by using the image color difference values corresponding to each of the 16777216 colors obtained in advance.
  • a color difference detection device (such as the color difference detection subsystem provided in the embodiment of the present application) is used to obtain the display image of the first image data and the image color difference value of the first image data in real time. By displaying the first image data The image is compared with the first image data to obtain the image color difference value of the first image data, and real-time color difference compensation is performed on the first image data based on the image color difference value.
  • the remote spectral imaging system further includes a color difference detection subsystem, and the color difference detection subsystem is configured to obtain a display of the first image data based on the first image data and a display image of the first image data. The image color difference value between the image and the first image data;
  • the remote display subsystem in the remote spectral imaging system may be used to perform color difference compensation on the first image data based on the image color difference value, obtain second image data, and use the remote display subsystem to perform color difference compensation on the first image data based on the image color difference value.
  • the display subsystem displays the second image data.
  • the present application further provides a color difference detection subsystem.
  • the color difference detection subsystem 130 is used to shoot the display image of the first image data and obtain a comparison image. Based on the comparison image and the Obtain the image color difference value from the first image data, and send the image color difference value to the remote display subsystem 120.
  • the image (comparison image) of the first image data displayed on the remote display subsystem 120 can be obtained, and then compared with the actual first image data, the color difference (color difference) between the two can be obtained. difference);
  • the displayed image of the first image data refers to the display content displayed by the remote display subsystem 120 based on the first image data.
  • the color difference detection subsystem 130 is located at the remote image display location.
  • the image color difference value may include RGB value or LAB color difference value (L represents the brightness of the color, A represents the red-green value, B represents the yellow-blue value) and other value color spaces that represent the color difference of objects.
  • the remote display subsystem 120 is also configured to receive the image color difference value sent by the color difference detection subsystem 130, perform color difference compensation on the first image data based on the image color difference value, and obtain the second image data. , and display the second image data.
  • the color difference detection subsystem described in this application can be integrated into the remote spectral imaging system; it can also be independently communicated and connected to the remote spectral imaging system, that is, it is only used to capture the first image data to obtain the comparison image. Then it is transmitted to the remote spectral imaging system through communication to obtain the image color difference value.
  • the color value of the first image data is 100. Since the ambient light of the remote image display place is sufficient, the color value of the actual displayed image is improved.
  • the color value of the comparison image is 105.
  • the color difference value of the image is the difference between the first image data and the contrast value.
  • the difference value of the image is -5. Based on the difference value -5, the first image data is compensated for the color difference.
  • the remote display subsystem 120 obtains and displays the second image data with a color value of 95. After the second image data is combined with the ambient light, the actual display closer to the first image data.
  • the remote spectral imaging system can obtain accurate spectral information and color information (ie, obtain accurate spectral data) through image generation subsystem 110 through spectral imaging; it can also achieve high fidelity transmission and encoding.
  • the first image data and the comparison image are matched and calibrated through the color difference detection subsystem 130 to restore the real spatial color information of the scene to be imaged, thereby improving the color restoration degree of remote image display.
  • the remote image second image
  • the improvement of the color authenticity of the data will help improve the accuracy and effectiveness of users' judgment of remote images, and improve the efficiency, accuracy and credibility of telemedicine.
  • the image generation subsystem 110 includes
  • the spectral imaging device is used to obtain spectral data of the scene to be imaged and send the spectral data to the color processing module;
  • the color processing module is used to receive the spectral data sent by the spectral imaging device and perform color processing on the spectral data to obtain color data; the color processing module is also used to send the spectral data to the encoding module. color data;
  • the encoding module is configured to receive the color data sent by the color processing module, and encode the color data to obtain first image data.
  • the spectral imaging device can obtain the light intensity data of the incident light of the scene to be imaged.
  • the light intensity data corresponds to the position of the spectral pixels in the spectral imaging device.
  • the data processing module of the spectral imaging device can obtain the light intensity data corresponding to each spectral pixel.
  • the spectral transmittance information, spectral intensity data and spectral imaging device parameter information invert the spectral data of the incident light of the scene to be imaged.
  • the color processing module performs color processing on the spectral data through methods such as color matrix transformation or color space mapping, and enables the encoding module to encode and obtain the first image data.
  • FIG. 2 is a schematic structural diagram of a spectral imaging device provided by an embodiment of the present application.
  • the spectral imaging device includes a filter structure 210, an image sensor 220 and a data processing unit 230.
  • the filter structure 210 is disposed on the On the photosensitive path of the image sensor 220; the transmittance curve of the filter structure 210 can be determined based on the spectral characteristics of the scene to be imaged;
  • the filter structure 210 is used to modulate the incident light of the scene to be imaged to obtain a modulated light signal
  • the image sensor 220 is used to receive the modulated light signal and obtain light intensity data
  • the data processing unit 230 is configured to receive the light intensity data sent by the image sensor. and perform spectral recovery based on the light intensity data to obtain spectral data corresponding to the scene to be imaged.
  • the filter structure 210 includes at least one structural unit, and the structural unit includes at least two micro-nano structures.
  • the structural units in the filter structure 210 can emit light in the target wavelength range according to the corresponding transmittance curve, that is, the incident light enters the structural unit and is modulated to obtain a modulated optical signal.
  • the filter structure 210 may be a broadband filter structure 210 in the frequency domain or wavelength domain.
  • the filter structure 210 can be a metasurface, a photonic crystal, a nanopillar, a multilayer film, a dye, a quantum dot, a microelectromechanical system (MEMS), an FP etalon, a cavity layer, a waveguide layer ( Structures or materials with light filtering properties such as waveguide layer) or diffractive elements.
  • the transmittance curves of each different structural unit are different from each other.
  • the transmittance curve of a structural unit represents the spectral transmittance of the structural unit to incident light of different wavelengths.
  • the transmittance curve of the structural unit is determined based on the spectral characteristics of the scene to be imaged.
  • the transmittance of the structural unit is higher in the band corresponding to the absorption peak of the scene to be imaged, which enables the filter structure 210 to have a distinguishing effect on the spectral characteristics of different states of the scene to be imaged. .
  • the image sensor 220 may be a CMOS image sensor (CMOS image sensor, CIS), a charge coupled device (Charge coupled device, CCD), or an array light detector, etc.
  • the image sensor 220 can convert the optical signal into an electrical signal to obtain the detection image and light intensity data corresponding to the structural unit one-to-one.
  • the detection image includes multiple detection positions, and each detection position corresponds to one or more structural units, so each detection position corresponds to the light intensity data of its corresponding structural unit.
  • the spectral imaging device further includes an optical system, which is disposed on the photosensitive path of the image sensor 220;
  • the optical system is used to adjust the optical path of incident light.
  • the optical signal is adjusted by the optical system and then modulated by the filter structure 210, and then is received by the image sensor 220 to obtain the light intensity data; wherein the optical system can be a lens component, a uniform light component or a collimation component. and other optical systems, capable of incident Light is focused, homogenized or collimated.
  • the data processing unit may be a micro control unit (Micro Controller Unit, MCU), a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU), Processing units such as Field Programmable Gate Array (FPGA), Neural Network Processing Unit (NPU) or Application Specific Integrated Circuit (ASIC).
  • MCU Micro Controller Unit
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • FPGA Field Programmable Gate Array
  • NPU Neural Network Processing Unit
  • ASIC Application Specific Integrated Circuit
  • the above-mentioned spectral imaging device is a spectral imaging device based on computational spectrum.
  • this application is based on computational spectral imaging technology, which can use the transmission spectrum curve to achieve broad-spectrum modulation of incident light, thereby making the acquired spatial information richer and more accurate.
  • the spectral response is transmitted to the data processing unit 230 for spectral recovery calculation.
  • the process is described as follows:
  • the intensity signals of the incident light at different wavelengths ⁇ are denoted as x( ⁇ ), and the transmission spectrum curve of the filter structure 210 is denoted as T( ⁇ ).
  • Structural units the transmission spectra obtained by incident light passing through each structural unit in a structural group are different from each other.
  • One physical pixel can correspond to a structural unit, or a group of multiple physical pixels can correspond to a structural unit.
  • At least two different structural units constitute a structural group, and a structural group and the physical pixel corresponding to the structural group constitute a spectral pixel.
  • the transmission spectrum that the spectral imaging device can use for spectral recovery is called the effective transmission spectrum.
  • the effective transmission spectrum refers to the transmission spectrum composed of different structural units. When the structural units are the same, the same will appear in the system response coefficient A matrix. vector, resulting in a situation where the spectrum cannot be recovered.
  • the number of effective transmission spectra Ti ( ⁇ ) of the filter structure 210 and the number of structural units can be to be consistent, the transmission spectrum of the filter structure 210 is artificially set, tested or calculated according to certain rules according to the accuracy requirements and speed requirements of scene recognition or spectral recovery to be imaged. Therefore, the effective transmission spectrum of the filter structure 210 is The number can be less than the number of structural units (for example, if there are repetitions in the structural units, the transmission spectrum of the repeated structural units is an invalid transmission spectrum), or even more than the number of structural units (for example, new ones can be obtained by combining the structural units). transmission spectrum).
  • this application can use at least one spectral pixel to restore the image. That is to say, the spectroscopic device in this application can restore the spectral curve and perform spectral imaging based on the spectral response.
  • R( ⁇ ) is the quantum efficiency of the image sensor 220
  • Strong data respectively corresponds to the light intensity measurement values of the image sensor 220 corresponding to m structural units.
  • a physical pixel corresponds to a structural unit, it can be understood as the light intensity measurement values corresponding to m physical pixels, which is a length of m vector.
  • the system response coefficient A is the light response of the system to different wavelengths, which is determined by the transmittance of the filter structure 210 and the image sensor
  • the quantum efficiency of 220 is determined by two factors.
  • A is a matrix, and each row vector corresponds to the response of a structural unit to incident light of different wavelengths.
  • the incident light is sampled discretely and uniformly, with a total of n sampling points.
  • the number of columns of A is the same as the number of sampling points of the incident light.
  • x( ⁇ ) is the intensity of the incident light at different wavelengths ⁇ , which is the spectrum of the incident light to be measured.
  • the color processing module is used to perform color processing on the spectral data.
  • Obtaining the color data includes:
  • the color processing module is used to perform color processing on the incident light spectrum based on a wide color gamut standard to obtain color data.
  • Color processing is performed on the transmitted incident light spectral data.
  • it is often necessary to convert the image information into a certain color space for processing, which will cause the loss of the color information of the image during the processing.
  • color processing selects a device-independent color space with a larger color gamut.
  • a color space larger than sRGB is used as a color space for color processing by the color processing module.
  • the number of colors that this color space can cover is much larger than other color spaces, which can greatly reduce the loss of data during transmission and processing and maximize high-fidelity color restoration.
  • the remote spectral imaging system provided by the embodiment of the present application adopts a wide color gamut standard color space.
  • the number of colors that the wide color gamut standard color space can cover is much larger than that of the traditional color space.
  • the second color space can be reduced. The loss of image data during transmission and processing is achieved to achieve high-fidelity color restoration to the maximum extent.
  • the encoding module encodes the color-processed spectral data (ie, color data) to obtain the first image data.
  • system further includes a data transmission module to transmit spectral data in a high-bit manner;
  • the data transmission module includes any of the following: a first transmission module, a second transmission module and a third transmission module;
  • the first transmission module is connected to the spectral imaging device and the color processing module. Connection, used to send the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module is connected to the color processing module and the encoding module, and is used to send the color data obtained by the color processing module to the encoding module;
  • the third transmission module is connected to the encoding module and the remote display subsystem, and is used to send the first image data obtained by the encoding module to the remote display subsystem.
  • the spectral imaging device in the image generation subsystem 110 is located at the image collection location, and the color processing module in the image generation subsystem 110 is located at the remote image display location.
  • the spectrum imaging device and the color processing module are connected through a first transmission module is connected.
  • the color processing module in the image generation subsystem 110 is located at the image collection location, and the encoding module in the image generation subsystem 110 is located at the remote image display location.
  • the color processing module and encoding module are connected through a second transmission module. connected.
  • the encoding module is installed at the image collection location, and the encoding module and the remote display subsystem 120 are connected through a third transmission module.
  • the embodiments of this application do not limit the spatial location of the data processing module and color processing module in the image generation subsystem. In actual applications, users can consider the hardware data processing capabilities or network transmission speed of the image collection place and the remote image display place. Factors can be set flexibly.
  • the data transmission module is used to transmit data at a data rate of no less than 8 bit/s.
  • high-bit data transmission is performed on the spectral data or first image data generated in the image generation subsystem 110, that is, after obtaining the spectral data or first image data, the color information in the image is minimized.
  • high-bit data transmission is performed.
  • a data transmission rate of not less than 8 bits per second can usually be called high-bit data transmission.
  • the high-bit data transmission may be 10-bit, 12-bit, or 14-bit data transmission.
  • the remote spectral imaging system provided by the embodiment of the present application can Use remote communication technologies such as 5G to achieve high-throughput and high-spatial-temporal resolution hyperspectral image transmission, and realize real-time transmission of high-throughput information to ensure the real-time nature of remote image transmission.
  • the color difference detection subsystem 130 includes a spectrum imaging module, an image matching module and a color difference calculation module;
  • the spectral imaging module is configured to capture the display image of the first image data based on spectral imaging to obtain the comparison image;
  • the spectral imaging module can be a spectral imaging device or a spectrometer or other spectral equipment. It captures the display image of the first image data, obtains the spectral information of the display image of the first image data, and obtains the comparison image based on the spectral information. .
  • the spectral equipment and spectral imaging methods adopted by the spectral imaging module and the spectral imaging device may be different.
  • the image matching module is used to perform image matching on the comparison image and the first image data to obtain a matching pixel point set;
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes The first pixel point in the first image data and the second pixel point in the comparison image;
  • the image matching module can perform image matching on the comparison image and the first image data based on an image processing algorithm.
  • the embodiment of the present application does not limit the matching method.
  • the matched comparison image has a one-to-one correspondence with the pixels of the first image data.
  • the color difference calculation unit is used to obtain the color difference value of the first pixel point and the second pixel point in each of the pixel point pairs, and obtain the image color difference value.
  • the color difference calculation unit calculates the color difference value of each pixel corresponding to the comparison image and the first image data to obtain the image color difference value.
  • the remote display subsystem 120 includes a display module and a color difference compensation module
  • the display module is used to display the first image data and the second image data
  • the color difference compensation module is configured to perform color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the first image data includes 4 pixel points: A, B, C and D.
  • the information of each pixel is as follows:
  • the comparison image includes four pixels that correspond one-to-one with the first image data: A’, B’, C’ and D’.
  • the information of each pixel is as follows:
  • the image color difference value is:
  • the color difference compensation module performs color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value, that is, adding the image color difference value to the first image data to obtain the second image data.
  • the second image The data includes 4 pixels A”, B”, C” and D”.
  • the information of each pixel is as follows:
  • the second image data displayed by the display module under the influence of environmental factors such as ambient light or device display parameters, has a color display effect close to that of the scene to be imaged. true color.
  • the processed image data As the last link of the solution, the encoding and display are also very important for the effect of image display.
  • the color space needs to be selected according to the storage requirements when compressing and encoding image data.
  • the image data is still compressed and encoded as described above.
  • the storage bit depth is set to a high number of bits, such as 10 bits, to ensure the level transition of the image color to the greatest extent, and then transmit Display to the back-end display system.
  • the embodiments of the present application model the spectral color gamut (that is, obtain the first image data based on the spectral information of the scene to be imaged) and the display end color gamut modeling (that is, at the remote image display end, by converting the first image data displayed by the remote display subsystem to The actual picture of the image data is calibrated with the first image data generated by the image generation subsystem 110 of the image acquisition end).
  • the color gamut representation (second image data) in the remote display subsystem controls the remote display subsystem to display the second image data so that the displayed picture is as close as possible to the real space color of the scene to be imaged.
  • Embodiments of the present application can also directly transmit incident light spectrum information at different locations in the scene space to be imaged collected by the spectrum imaging device to the remote display subsystem, providing fine spectral features that are not easily detectable by human vision, and assisting users in remote image judgment. .
  • this application can reasonably improve the efficiency, accuracy and credibility of telemedicine and promote the equal distribution of medical resources.
  • the snapshot spectrum chip can be used to collect spectral and image information of the actual scene, and then through high-speed data transmission, based on the collected light Spectral information is used to display the color of the image on the display terminal.
  • the embodiment of the present application uses a spectrum chip at the actual scene picture collection end.
  • Figure 3 is a functional block diagram of the spectrum chip provided by the embodiment of the present application.
  • Figure 4 is a schematic structural diagram of the spectrum chip provided by the embodiment of the present application, as shown in Figures 2 and 3
  • the spectrum chip includes a filter structure and an image sensor. The filter structure is formed on the optical path of the image sensor.
  • the spectrum imaging chip is used to obtain spectral information of each point in space of the field scene, that is, the The step is to obtain the spatial spectral data obtained through the spectral imaging chip.
  • the incident light is modulated by the filtering light structure, and then received by the image sensor to obtain the corresponding spectral information, and then processed through the spectral recovery algorithm to obtain the corresponding spectral data.
  • the filter structure is composed of multiple structural units, each structural unit includes at least one micro-nano structure, and the micro-nano structure can be implemented as a modulation hole as shown in Figure 4.
  • the spectral data is subjected to high-bit data transmission, that is, after the spectral data of each pixel of the image is obtained, in order to minimize the loss of color information in the image. It should be noted that the spectral information obtained by the image sensor can also be transmitted directly.
  • remote communication technologies such as 5G are used to realize high-throughput and high-spatial-temporal resolution hyperspectral image transmission, and realize real-time transmission of high-throughput information to ensure the real-time nature of the system.
  • the BT.2020 color gamut space can be used throughout the process.
  • the color gamut and spectral color gamut of the display terminal are first modeled, and then the obtained real-time high-throughput spectral data is used to calculate and obtain the representation of the spatial color information of the field scene in the display system color gamut, and control the display system display to make it as close as possible to the spatial color of the actual scene.
  • this application can be applied to telemedicine scenarios.
  • Existing telemedicine images cannot achieve accurate color reproduction, which will affect the user experience.
  • image color distortion may lead to misjudgment by users.
  • To ensure the accuracy of remote images it needs to be achieved from multiple angles. First, the accuracy of the acquired information must be ensured at the collection end. Secondly, the loss of information must be avoided during the transmission process. Finally, the accuracy must be ensured during display. All three must Any deficiencies have a certain probability of affecting the accuracy of the final display.
  • embodiments of the present application provide a terminal detection system for any of the above remote spectral imaging systems, which is used to detect the color display error in the remote spectral imaging system, and perform calibration and error elimination to further improve the imaging solution. accuracy.
  • the terminal detection system can also be applied to other imaging systems, such as 4K high-definition image transmission imaging systems.
  • FIG. 5 is a schematic structural diagram of a terminal detection system for a remote spectral imaging system provided by an embodiment of the present application.
  • the terminal detection system includes: a display terminal 510, a colorimeter 520 and a calibration module 530;
  • the display terminal 510 and the colorimeter 520 can be set up separately, and the calibration module 530 can be set up independently, or can be set up inside the display terminal 510 .
  • the display terminal 510 is used to display the first image data
  • the colorimeter 520 is used to obtain the color data of the display image of the first image data
  • the display terminal 510 has a display function and can display the first image data in the form of an image.
  • the display image of the first image data is a display image displayed by the display terminal 510 based on the first image data.
  • the colorimeter 520 may be a colorimeter detector or a spectrometer, etc.
  • the colorimeter is used to measure measurement parameter values such as RGB values or chromaticity values of the display image of the first image data.
  • the display image of the first image data refers to the image obtained by the display terminal 510 actually displaying the first image data under the influence of factors such as the device itself and/or the external environment (such as external light).
  • the screen is the screen displayed by the display terminal 510 under actual circumstances that is visible to the user or can be collected by other devices.
  • the color data of the display image of the first image data may include quantified data representing the color of the image, such as hue data, saturation data, and brightness data.
  • the calibration module 530 is configured to obtain a difference between the first image data and the display image of the first image data based on the color data of the first image data and the color data of the display image of the first image data. mapping relationship;
  • the color data of the first image data is obtained in advance when acquiring the first image data. Since the first image data corresponds to the display image of the first image data, each pixel in the first image data The position information and color data information are in one-to-one correspondence with the display image of the first image data. Based on the corresponding relationship, the mapping relationship between the first image data and the display image of the first image data can be obtained.
  • the mapping relationship is Refers to the deviation relationship of color data corresponding to the same position of the first image data and the display image of the first image data.
  • the resolution information of the first image data is n*m pixels.
  • a coordinate system is established with the center of the first image data as the origin. With 1 pixel as the unit spacing, n*m positions can be obtained.
  • Each position Corresponding to one color data; similarly, n*m positions can also be obtained correspondingly to the display image of the first image data, each position corresponds to one color data, the pixel position of the first image data and the display image of the first image data.
  • the positions of can correspond one to one, so the color data of the first image data and the color data in the display image of the first image data can correspond one to one, thereby constructing a mapping relationship, that is, according to the reference parameter values obtained in advance in the first image data
  • a mapping relationship is established with the measurement parameter value obtained by measuring the display image of the first image data.
  • the corresponding method of the first image data and the display image of the first image data may include but is not limited to AI image recognition, image equal distance division, etc.
  • the pixel color data at coordinates (5,36) in the first image data is saturation 30, brightness 10, and color value 36;
  • the pixel color data at coordinates (5,36) in the display image of the first image data is Saturation 35, brightness 15, color value 56;
  • pixels (5,36) can be obtained During actual display, the deviation from the first image data is saturation +5, brightness +5, and color value +20. That is, the mapping relationship at (5,36) is: saturation +5, brightness +5, and color value +20.
  • the calibration module 530 is also used to perform color calibration on the display terminal based on the mapping relationship.
  • color calibration refers to correcting the actual displayed color data to a real color.
  • the real color refers to the color when the data is generated, that is, the color that is not affected by the device itself or the external environment.
  • photometric compensation, white balance compensation, and chromaticity compensation can be performed on the display terminal.
  • the same color as pixel (5,36) in the display terminal can be adjusted based on the mapping relationship: saturation -5, brightness -5, color value -20, such that Its actual display color is the same as the real color.
  • the calibration module 530 can adjust the spectral data based on the mapping relationship, such as adjusting the spectral data of the color corresponding to the pixel (5, 36), so that the display terminal displays The display image is closer to the real color.
  • the terminal detection system for a remote spectral imaging system uses the first image data as a color reference to obtain the display image of the first image data presented by the display terminal, compares the two, and establishes a mapping relationship based on The mapping relationship calibrates and eliminates errors on the display terminal, avoiding color display errors on the display terminal and further improving the accuracy of the imaging system.
  • the above-mentioned color data is any one of chromaticity values, RGB values, and spectral data.
  • the colorimeter 520 measures the chromaticity value of the displayed image of the first image data; the colorimeter 520 may also be a spectrometer, and the colorimeter 520 directly measures the image spectrum curve corresponding to the displayed image of the first image data ( Spectral data), and then establish a mapping relationship based on the measurement parameter value (color data of the display image of the first image data) and the reference parameter value (color data of the first image data), and then perform remote mapping based on the mapping relationship.
  • the spectral data of the spectral imaging system is calibrated, the spectral data of the remote spectral imaging system is used to generate a target image, and then the target image is displayed through the display terminal, thereby eliminating imaging errors caused by the display terminal.
  • the color data of the display image of the first image data and the color data of the first image data are not necessarily the same type of parameters.
  • the measuring instrument is a screen colorimeter
  • the first image is measured.
  • the data displays the chromaticity value of the image
  • the reference parameter value provided by the first image data is the RGB value
  • the mapping relationship between the known RGB value and the measured chromaticity value can be established, and then through the The mapping relationship is used to adjust the spectral data so that the display image displayed on the display terminal is closer to the real color.
  • the display terminal is also used to receive and display the target image.
  • the target image can be obtained after screen collection, transmission and color processing in the remote spectral imaging system, and the display terminal can receive the target image reconstructed after processing; in one embodiment, the target image can be obtained through screen collection in the remote spectral imaging system. and transmit the collected spectral data to the display terminal.
  • the calibration module 530 can adjust the spectral data based on the mapping relationship
  • the display terminal 510 reconstructs the target image based on the adjusted spectral data and displays the target image, so that the displayed target The image is close to the real color.
  • the display terminal includes a first display unit and a second display unit:
  • the first display unit is used to display the target image
  • the second display unit is used to display the first image data.
  • the first display unit and the second display unit may be two independent sub-display terminals, and the display screen characteristics of the first display unit and the second display unit should be consistent (or meet a preset consistency threshold), that is, The performance of the display screens corresponding to the first display unit and the second display unit needs to be relatively close.
  • the first display unit and the second display unit may be tested and display parameters adjusted in advance.
  • the colorimeter obtains the corresponding measurement parameter value by photographing the first image data displayed by the second display unit, and then based on the known reference parameter value
  • the real-time mapping relationship is obtained, the spectral data is adjusted in real time through the real-time mapping relationship, and then displayed on the first display unit, thereby reducing the impact of the display system and ambient light on spectral imaging.
  • the display terminal may be implemented as a display screen.
  • Embodiments of the present application use the first image data as a detection standard for the imaging system to achieve calibration of the imaging error and ambient light deviation of the display system.
  • the first image data is used to establish a mapping relationship between the actual imaging chromaticity value of the display terminal and the reference standard, and the display is adjusted according to the mapping relationship; further, during the imaging display process, a display unit is set for real-time detection and adjustment of the first image data.
  • the first display unit is a first split-screen area in the display terminal
  • the second display unit is a second split-screen area in the display terminal.
  • the first display unit and the second display unit are preferably implemented as two areas of the display terminal, that is, the same display terminal is divided into screens or an area is set to display the first image data.
  • the embodiment of the present application does not limit the size of the split-screen area.
  • the terminal detection system for remotely mounted spectral imaging system avoids the introduction of different parameters of the two display units due to the separate setting of the first display unit and the second display unit by splitting the screen of the same display terminal.
  • the new display error maintains the consistency of the device displaying the first image data and the display image displaying the first image data, further improving the accuracy of the imaging system.
  • the terminal detection method for a remote spectrum imaging system provided by this application is described below.
  • the terminal detection method for a remote spectrum imaging system described below and the terminal detection system for a remote spectrum imaging system described above can correspond to each other. .
  • the terminal detection method for the remote spectral imaging system provided by the embodiment of the present application is described below with reference to FIGS. 6-8 .
  • Figure 6 is one of the flow diagrams of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 6, an embodiment of the present application provides a terminal detection method for a remote spectral imaging system, including :
  • Step 610 Obtain the color data of the display image of the first image data
  • Step 620 Based on the color data of the first image data and the color data of the display image of the first image data, obtain the mapping relationship between the first image data and the display image of the first image data;
  • Step 630 Perform color calibration on the display terminal based on the mapping relationship.
  • the terminal detection method for a remote spectral imaging system uses the first image data as a color reference to obtain a display image of the first image data actually presented by the display terminal, compare the two, and establish a mapping relationship.
  • the display terminal is calibrated and errors are eliminated based on the mapping relationship, which avoids color display errors on the display terminal and further improves the accuracy of the imaging system.
  • the color data is any one of chromaticity values, RGB values, and spectral data.
  • the target image is obtained based on spectral data of the target imaging object.
  • the target imaging object is the actual scene of the picture to be transmitted, that is, the real picture that is not affected by the picture to be transmitted (before imaging, it can be called the picture to be imaged, the two are the same).
  • the spectral data of the target imaging object can be obtained through the snapshot spectrum chip introduced in the embodiment of the present application. Refer to the above and the introduction of Figures 3 and 4, which will not be described again here.
  • the target image is obtained by color processing the spectral data; and/or
  • the target image is obtained through data transmission at a data rate of no less than 8 bit/s.
  • the embodiment of the present application does not limit the order of color processing and data transmission.
  • FIG. 7 is a schematic flow chart of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in FIG. 7 , the first image data and the third image data are obtained. After displaying the mapping relationship between images, the image data also includes:
  • the display terminal receives the target image
  • the color calibration of the display terminal based on the mapping relationship also includes:
  • the display terminal displays the target image.
  • the first image data can be preset and displayed on the display terminal in the embodiment of the present application, and the mapping relationship can be obtained in advance through the terminal detection method for the remote spectral imaging system shown in FIG. 6 .
  • a spectrum chip is used to obtain spectral image data (i.e., spectral data) of the actual scene of the picture to be transmitted, high-bit data transmission is performed on the spectral data, color processing is performed on the transmitted spectral data, and the color processed data is obtained
  • the image data (that is, the target image) is adjusted according to the mapping relationship according to the color-processed image data, and then the target image is encoded and displayed.
  • mapping relationship in the embodiment of the present application is obtained in advance.
  • the pre-obtained mapping relationship is mainly based on the display deviation of the display terminal itself and the impact of ambient light on imaging when obtaining the mapping relationship.
  • the deviation caused by the display terminal itself is relatively stable. If the ambient light is also relatively stable, the mapping relationship only needs to be obtained before imaging and display. It can be applied to color correction of subsequent display terminals, such as color correction in scenarios where the display terminal displays remote spectral imaging or spectral video.
  • the image color displayed by different models of display terminals based on the same image data may deviate, and there is also a certain deviation from the color perceived by the human eye in the actual scene; the same display terminal under different ambient lights, Final display color effects will also vary.
  • the terminal detection method for the remote spectral imaging system provided by the embodiment of the present application measures the display image of the first image data and obtains the mapping relationship according to the first image data, which can eliminate the effects of the display terminal itself and the ambient light on the display terminal. Imaging effect.
  • Figure 8 is a third schematic flowchart of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 8, the first image data is displayed by the second display unit;
  • the method also includes:
  • the display terminal receives a target image, and the target image is displayed by the first display unit;
  • the obtaining the color data of the display image of the first image data includes:
  • Obtaining a mapping relationship between the first image data and the display image of the first image data based on the color data of the first image data and the color data of the display image of the first image data includes: :
  • the color calibration of the display terminal based on the mapping relationship includes:
  • the first image data is displayed by the second display unit.
  • the color data of the displayed image of the first image data may change.
  • the current color data of the image refers to the color data of the display image of the first image data obtained in real time; for example, when color correction needs to be performed on the display terminal, the color data of the display image of the first image data is obtained.
  • the real-time mapping relationship is obtained based on the current color data of the display image of the first image data and the color data of the first image data obtained in real time, and can reflect the display deviation of the display image of the first image data and the first image data at the current moment.
  • a spectrum chip is used to obtain spectral image data (i.e., spectral data) of the actual scene of the picture to be transmitted, high-bit data transmission is performed on the spectral data, and color isolation is performed on the transmitted spectral data.
  • spectral image data i.e., spectral data
  • color isolation is performed on the transmitted spectral data.
  • the color-processed image data i.e., the target image.
  • the high-fidelity transmission image is displayed on the first display unit, and the currently displayed high-fidelity transmission image may be the target image at the previous moment; the first image data is displayed on the second display unit, and the high-fidelity transmission image shown in Figure 6 is used to display the first image data.
  • the terminal detection method of the remote spectral imaging system obtains the real-time mapping relationship between the first image data and the display image of the first image data in real time.
  • the color-processed image data and the real-time mapping relationship can be obtained at the same time, the color-processed image data can be adjusted according to the real-time mapping relationship, and then the target image can be encoded and displayed.
  • the image colors displayed by different models of display terminals based on the same image data may deviate, and there is also a certain deviation from the color perceived by the human eye in the actual scene; the same display terminal under different ambient lights will ultimately The display color effects will also be different.
  • the first image data is displayed by the second display unit. According to the first image data and the first The display image of the image data can obtain the real-time mapping relationship, which can eliminate the influence of the display performance of the display terminal itself and the ambient light on the imaging effect in real time.
  • the long-range spectral imaging method provided by the present application is described below.
  • the long-range spectral imaging method described below and the long-range spectral imaging system described above can be mutually referenced.
  • FIG. 9 is a schematic flowchart of a remote spectral imaging method provided by an embodiment of the present application. As shown in Figure 9, an embodiment of the present application provides a remote spectral imaging method based on the above-mentioned remote spectral imaging system, including:
  • Step 910 Shoot the scene to be imaged based on spectral imaging (especially computational spectral imaging technology) to obtain first image data;
  • Step 920 Display the first image data, and obtain an image color difference value based on the display image of the first image data and the first image data;
  • Step 930 Perform color difference compensation on the first image data based on the image color difference value to obtain second image data.
  • obtaining the image color difference value based on the display image of the first image data and the first image data includes:
  • the display image of the first image data is photographed to obtain a comparison image, and the image color difference value is obtained based on the comparison image and the first image data.
  • photographing the display image of the first image data, obtaining a comparison image, and obtaining the image color difference value based on the comparison image and the first image data includes:
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes a pair of pixel points in the first image data.
  • performing color difference compensation on the first image data based on the image color difference value to obtain the second image data includes:
  • Color difference compensation is performed on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the scene to be imaged is photographed based on spectral imaging to obtain first image data, including:
  • the color data is encoded to obtain first image data.
  • performing color processing on the spectral data to obtain color data includes:
  • Color processing is performed on the incident light spectrum based on a wide color gamut standard to obtain the color data.
  • the method includes any of the following data transmission steps:
  • the first transmission module sends the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module sends the color data obtained by the color processing module to the encoding module
  • the third transmission module sends the first image data obtained by the encoding module to the remote display subsystem.
  • data transmission is performed at a data rate of no less than 8 bit/s.
  • the image color difference value described in this application can be adjusted in real time according to any of the first image data and the corresponding display image, thereby obtaining the second image data.
  • the image color difference value can be obtained through one or more measurements, and then Any first image data is compensated according to the image color difference value to obtain corresponding second image data. That is, the image color difference value can be a fixed value or a dynamic change.
  • the first image data and the second image data in this application can be displayed on different remote display subsystems, that is, the remote display subsystem can be implemented as two screens to achieve display.
  • the first The screen is used to display the first image data
  • the second screen is used to display the final required high-fidelity second image data.
  • the performance of the first screen and the second screen is close, that is, the same image can be displayed in the same environment. near.
  • the first screen and the second screen may also be different areas of the same screen.
  • the remote spectral imaging method uses spectral imaging to directly obtain the spatial spectral information of the scene to be imaged, avoiding the information loss caused by dimensionality reduction when obtaining spectral dimension information by traditional cameras such as RGB color cameras, as well as different lighting and environments. Deviations in image white balance under color temperature, and at the same time, the actual display effect is calibrated and matched at the image reconstruction end (remote display subsystem) to further improve the accuracy of image color restoration of remote images and restore the spatial color information of the scene to be imaged to the greatest extent.
  • Figure 10 illustrates a schematic diagram of the physical structure of an electronic device.
  • the electronic device may include: a processor (processor) 1010, a communications interface (Communications Interface) 1020, a memory (memory) 1030 and a communication bus 1040.
  • the processor 1010, the communication interface 1020, and the memory 1030 complete communication with each other through the communication bus 1040.
  • the processor 1010 can call logic instructions in the memory 1030 to perform any of the above methods.
  • the above-mentioned logical instructions in the memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
  • the technical solution of this application is essentially or The part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes a number of instructions to enable a computer device (which can be a personal computer). Computer, server, or network device, etc.) executes all or part of the steps of the methods described in various embodiments of this application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .
  • the present application also provides a computer program product.
  • the computer program product includes a computer program.
  • the computer program can be stored on a non-transitory computer-readable storage medium.
  • the computer program can Perform any of the above methods.
  • the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is implemented when executed by a processor to perform any of the above methods.
  • the device embodiments described above are only illustrative.
  • the units described as separate components may or may not be physically separated.
  • the components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.
  • each embodiment can be implemented by software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute various embodiments or certain parts of the embodiments. method described.

Abstract

The present application provides a remote spectral imaging system and method. The system comprises: an image generation subsystem and a remote display subsystem. The image generation subsystem is used to obtain, on the basis of spectral imaging, first image data of a scene to be imaged, and the remote display subsystem displays the first image data; on the basis of the first image data and a display image of the first image data, an image chromatic aberration value between the display image of the first image data and the first image data is obtained, chromatic aberration compensation is performed on the first image data on the basis of the image chromatic aberration value to obtain second image data, and the remote display subsystem displays the second image data; and the chromatic aberration value may be obtained by a chromatic aberration detection subsystem performing calculation on the basis of the first image data and the display image in the remote display subsystem. According to the remote spectral imaging system and method provided in the present application, the color rendition degree of remote image display can be improved.

Description

远程光谱成像系统及方法Remote spectral imaging system and method
相关申请的交叉引用Cross-references to related applications
本申请要求于2022年04月29日提交的申请号为202210476278.0，名称为“远程光谱成像系统及方法”，以及于2022年07月22日提交的申请号为202210869460.2，名称为“用于成像系统的终端检测系统及方法和远程光谱成像系统”的中国专利申请的优先权，其通过引用方式全部并入本文。This application requires an application number 202210476278.0 submitted on April 29, 2022, titled "Remote Spectral Imaging System and Method", and an application number 202210869460.2 submitted on July 22, 2022, titled "For Imaging Systems" "Terminal detection system and method and remote spectral imaging system" Chinese patent application priority, which is fully incorporated herein by reference.
技术领域Technical field
本申请涉及计算机技术领域，尤其涉及一种远程光谱成像系统及方法。The present application relates to the field of computer technology, and in particular to a remote spectral imaging system and method.
背景技术Background technique
在远程画面传输中，画面的色彩高保真度和画面高清晰度是重要的评价标准，例如在医疗领域中，前者能够实现精确显示治疗区域的生理状态细微差别，后者则可以满足医疗高精准度的需要。远程画面的传输过程可分为画面采集、信号传输、画面重建三个步骤。其中画面重建步骤可根据彩色相机获取的不同色彩通道强度值，利用激光液晶显示等显示技术实现画面的高清晰度及色彩高保真度重现。In remote image transmission, the high color fidelity and high definition of the image are important evaluation criteria. For example, in the medical field, the former can accurately display the subtle differences in the physiological status of the treatment area, while the latter can meet the high precision of medical treatment. degree of need. The transmission process of remote images can be divided into three steps: image acquisition, signal transmission, and image reconstruction. The image reconstruction step can use display technologies such as laser liquid crystal display to achieve high-definition and high-fidelity color reproduction based on the different color channel intensity values obtained by the color camera.
然而现有技术中，由于激光液晶等显示技术的色域限制和不同环境色温的影响，画面颜色白平衡和画面重建还原度具有一定的失真和限制。因此，现有的远程画面显示技术，无法做到色彩上的精准复现。However, in the existing technology, due to the color gamut limitations of display technologies such as laser liquid crystal and the influence of different environmental color temperatures, there are certain distortions and limitations in the picture color white balance and picture reconstruction restoration. Therefore, existing remote screen display technology cannot accurately reproduce colors.
目前，现有的成像及图像传输系统中，往往采用基于RGB多通道成像方案：利用RGB等多通道相机对实际场景进行彩色图像信息的采集，然后在终端根据传输的图像信息进行终端画面显示。但是目前成像技术中重点关注如何获取更多的彩色图像信息以及高性能的传输，但未考虑终端显示，因此无法保证图像在实际情况下的显示准确性。 Currently, existing imaging and image transmission systems often use RGB-based multi-channel imaging solutions: RGB and other multi-channel cameras are used to collect color image information of actual scenes, and then the terminal screen is displayed based on the transmitted image information. However, current imaging technology focuses on how to obtain more color image information and high-performance transmission, but does not consider terminal display, so the accuracy of image display in actual situations cannot be guaranteed.
发明内容Contents of the invention
本申请提供一种远程光谱成像系统及方法，用以解决现有技术中远程画面色彩失真的缺陷，提高远程画面色彩还原度。This application provides a remote spectral imaging system and method to solve the defect of color distortion of remote images in the existing technology and improve the color restoration of remote images.
本申请提供一种用于远程光谱成像系统的终端检测系统及方法，用以解决现有技术中图像实际显示颜色失真的缺陷，提高图像显示的颜色准确性。This application provides a terminal detection system and method for a remote spectral imaging system to solve the defect of color distortion in actual image display in the prior art and improve the color accuracy of image display.
本申请提供一种远程光谱成像系统，包括：图像生成子系统和远程显示子系统；This application provides a remote spectral imaging system, including: an image generation subsystem and a remote display subsystem;
所述图像生成子系统用于基于光谱成像获取待成像场景的第一图像数据，并向所述远程显示子系统发送第一图像数据，由所述远程显示子系统显示所述第一图像数据；The image generation subsystem is configured to obtain first image data of the scene to be imaged based on spectral imaging, and send the first image data to the remote display subsystem, and the remote display subsystem displays the first image data;
基于所述第一图像数据和所述第一图像数据的显示图像获取所述第一图像数据的显示图像与所述第一图像数据的图像色差值；Obtaining an image color difference value between the display image of the first image data and the first image data based on the first image data and the display image of the first image data;
基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，并由所述远程显示子系统显示所述第二图像数据。Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data, and the second image data is displayed by the remote display subsystem.
在一个实施例中，所述远程光谱成像系统还包括色差检测子系统；In one embodiment, the remote spectral imaging system further includes a color difference detection subsystem;
所述色差检测子系统用于对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值。The color difference detection subsystem is used to shoot the display image of the first image data, obtain a comparison image, and obtain the image color difference value based on the comparison image and the first image data.
在一个实施例中，所述色差检测子系统包括光谱成像模块、图像匹配模块和色差计算模块；In one embodiment, the color difference detection subsystem includes a spectrum imaging module, an image matching module and a color difference calculation module;
所述光谱成像模块用于基于光谱成像对所述第一图像数据的显示图像进行拍摄，获得所述对比图像；The spectral imaging module is configured to capture the display image of the first image data based on spectral imaging to obtain the comparison image;
所述图像匹配模块用于对所述对比图像和所述第一图像数据进行图像匹配，获得匹配像素点集；所述匹配像素点集中包括多个像素点对，每个所述像素点对包括所述第一图像数据中的第一像素点和所述对比图像中的第二像素点； The image matching module is used to perform image matching on the comparison image and the first image data to obtain a matching pixel point set; the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes The first pixel point in the first image data and the second pixel point in the comparison image;
所述色差计算单元用于获取每个所述像素点对中的第一像素点与第二像素点的色差值，获得图像色差值。The color difference calculation unit is used to obtain the color difference value of the first pixel point and the second pixel point in each of the pixel point pairs, and obtain the image color difference value.
在一个实施例中，所述远程显示子系统包括显示模块和色差补偿模块；In one embodiment, the remote display subsystem includes a display module and a color difference compensation module;
所述显示模块用于显示所述第一图像数据和所述第二图像数据；The display module is used to display the first image data and the second image data;
所述色差补偿模块用于基于所述图像色差值对每个所述像素点对中的第一像素点进行色差补偿，获得第二图像数据。The color difference compensation module is configured to perform color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
在一个实施例中，所述图像色差值为RGB值。In one embodiment, the image color difference values are RGB values.
在一个实施例中，所述图像生成子系统包括光谱成像装置、颜色处理模块和编码模块；In one embodiment, the image generation subsystem includes a spectral imaging device, a color processing module and an encoding module;
所述光谱成像装置用于获取待成像场景的光谱数据，并向所述颜色处理模块发送所述光谱数据；The spectral imaging device is used to obtain spectral data of the scene to be imaged and send the spectral data to the color processing module;
所述颜色处理模块用于接收所述光谱数据，并对所述光谱数据进行颜色处理，获得颜色数据；所述颜色处理模块还用于向所述编码模块发送所述颜色数据；The color processing module is used to receive the spectrum data and perform color processing on the spectrum data to obtain color data; the color processing module is also used to send the color data to the encoding module;
所述编码模块用于接收所述颜色数据，并对所述颜色数据进行编码获得第一图像数据。The encoding module is used to receive the color data and encode the color data to obtain first image data.
在一个实施例中，所述光谱成像装置包括滤光结构、图像传感器和数据处理单元，所述滤光结构设置于所述图像传感器的感光路径上；所述滤光结构的透射率曲线是基于待成像场景的光谱特征确定的；In one embodiment, the spectral imaging device includes a filter structure, an image sensor and a data processing unit, the filter structure is disposed on the photosensitive path of the image sensor; the transmittance curve of the filter structure is based on The spectral characteristics of the scene to be imaged are determined;
所述滤光结构用于对待成像场景的入射光进行调制，获得调制光信号；The filter structure is used to modulate the incident light of the scene to be imaged to obtain a modulated light signal;
所述图像传感器用于接收所述调制光信号，并获得光强数据；The image sensor is used to receive the modulated light signal and obtain light intensity data;
所述数据处理单元用于接收所述图像传感器发送的光强数据，并基于所述光强数据进行光谱恢复，获得所述待成像场景对应的光谱数据。The data processing unit is configured to receive light intensity data sent by the image sensor, and perform spectral recovery based on the light intensity data to obtain spectral data corresponding to the scene to be imaged.
在一个实施例中，所述颜色处理模块用于对所述入射光光谱进行 颜色处理，获得颜色数据，包括：In one embodiment, the color processing module is used to perform processing on the incident light spectrum Color processing, obtaining color data, including:
所述颜色处理模块用于基于广色域标准对所述入射光光谱进行颜色处理，获得颜色数据。The color processing module is used to perform color processing on the incident light spectrum based on a wide color gamut standard to obtain color data.
在一个实施例中，所述系统还包括：数据传输模块；In one embodiment, the system further includes: a data transmission module;
所述数据传输模块包括以下任一：第一传输模块、第二传输模块和第三传输模块；The data transmission module includes any of the following: a first transmission module, a second transmission module and a third transmission module;
所述第一传输模块与所述光谱成像装置和所述颜色处理模块相连接，用于向所述颜色处理模块发送所述光谱成像装置获得的所述光谱数据；The first transmission module is connected to the spectral imaging device and the color processing module, and is used to send the spectral data obtained by the spectral imaging device to the color processing module;
所述第二传输模块与所述颜色处理模块和所述编码模块相连接，用于向所述编码模块发送所述颜色处理模块获得的所述颜色数据；The second transmission module is connected to the color processing module and the encoding module, and is used to send the color data obtained by the color processing module to the encoding module;
所述第三传输模块与所述编码模块和所述远程显示子系统相连接，用于向所述远程显示子系统发送所述编码模块获得的第一图像数据。The third transmission module is connected to the encoding module and the remote display subsystem, and is used to send the first image data obtained by the encoding module to the remote display subsystem.
在一个实施例中，所述数据传输模块用于以不低于8bit/s的数据率进行数据传输。In one embodiment, the data transmission module is used to transmit data at a data rate of no less than 8 bit/s.
在一个实施例中，所述远程显示子系统包括终端检测系统，所述终端检测系统包括：显示终端、测色仪和校准模块；In one embodiment, the remote display subsystem includes a terminal detection system, which includes: a display terminal, a colorimeter and a calibration module;
所述显示终端用于显示所述第一图像数据；The display terminal is used to display the first image data;
所述测色仪用于获取所述第一图像数据的显示图像的颜色数据；The colorimeter is used to obtain the color data of the display image of the first image data;
所述校准模块用于基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系；The calibration module is configured to obtain, based on the color data of the first image data and the color data of the display image of the first image data, the difference between the first image data and the display image of the first image data. Mapping relations;
所述校准模块还用于基于所述映射关系对所述显示终端进行颜色校准。The calibration module is also used to perform color calibration on the display terminal based on the mapping relationship.
在一个实施例中，所述颜色数据为色度值、RGB值和光谱数据中的任意一种。 In one embodiment, the color data is any one of chromaticity values, RGB values, and spectral data.
在一个实施例中，所述显示终端还用于接收并显示目标图像。In one embodiment, the display terminal is also used to receive and display the target image.
在一个实施例中，所述显示终端包括第一显示单元和第二显示单元：In one embodiment, the display terminal includes a first display unit and a second display unit:
所述第一显示单元用于显示所述目标图像；The first display unit is used to display the target image;
所述第二显示单元用于显示所述第一图像数据。The second display unit is used to display the first image data.
在一个实施例中，所述第一显示单元为所述显示终端中的第一分屏区域，所述第二显示单元为所述显示终端中的第二分屏区域。In one embodiment, the first display unit is a first split-screen area in the display terminal, and the second display unit is a second split-screen area in the display terminal.
本申请还提供一种基于上述远程光谱成像系统的终端检测方法，包括：This application also provides a terminal detection method based on the above-mentioned remote spectral imaging system, including:
获取第一图像数据的显示图像的颜色数据；Obtain the color data of the display image of the first image data;
基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系；Based on the color data of the first image data and the color data of the display image of the first image data, obtain a mapping relationship between the first image data and the display image of the first image data;
基于所述映射关系对显示终端进行颜色校准。Color calibration is performed on the display terminal based on the mapping relationship.
在一个实施例中，所述颜色数据为色度值、RGB值和光谱数据中的任意一种。In one embodiment, the color data is any one of chromaticity values, RGB values, and spectral data.
在一个实施例中，所述获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系之后，还包括：In one embodiment, after obtaining the mapping relationship between the first image data and the display image of the first image data, the method further includes:
显示终端接收目标图像；The display terminal receives the target image;
所述基于所述映射关系对显示终端进行颜色校准，之后还包括：The color calibration of the display terminal based on the mapping relationship also includes:
所述显示终端显示所述目标图像。The display terminal displays the target image.
在一个实施例中，所述第一图像数据由第二显示单元显示；In one embodiment, the first image data is displayed by a second display unit;
所述方法还包括：The method also includes:
显示终端接收目标图像，所述目标图像由第一显示单元显示；The display terminal receives a target image, and the target image is displayed by the first display unit;
所述获取第一图像数据的显示图像的颜色数据包括：The obtaining the color data of the display image of the first image data includes:
获取所述第一图像数据的显示图像的当前颜色数据；Obtain the current color data of the display image of the first image data;
所述基于所述第一图像数据的颜色数据与所述第一图像数据的 显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系，包括：The color data based on the first image data and the first image data Display the color data of the image, and obtain the mapping relationship between the first image data and the display image of the first image data, including:
基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的当前颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的实时映射关系；Based on the color data of the first image data and the current color data of the display image of the first image data, obtain a real-time mapping relationship between the first image data and the display image of the first image data;
所述基于所述映射关系对显示终端进行颜色校准包括：The color calibration of the display terminal based on the mapping relationship includes:
基于所述实时映射关系对所述显示终端进行实时颜色校准。Perform real-time color calibration on the display terminal based on the real-time mapping relationship.
在一个实施例中，所述目标图像是基于目标成像对象的光谱数据获得的。In one embodiment, the target image is obtained based on spectral data of the target imaging object.
在一个实施例中，所述目标图像是通过对所述光谱数据进行颜色处理获得的；和/或In one embodiment, the target image is obtained by color processing the spectral data; and/or
所述目标图像是通过不低于8bit/s的数据率进行数据传输获得的。The target image is obtained through data transmission at a data rate of no less than 8 bit/s.
本申请还提供一种基于上述的远程光谱成像系统的远程光谱成像方法，包括：This application also provides a remote spectral imaging method based on the above-mentioned remote spectral imaging system, including:
基于光谱成像对待成像场景进行拍摄，获得第一图像数据；Shoot the scene to be imaged based on spectral imaging to obtain the first image data;
显示所述第一图像数据，基于所述第一图像数据的显示图像和所述第一图像数据，获取图像色差值；Display the first image data, and obtain an image color difference value based on the display image of the first image data and the first image data;
基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据。Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data.
在一个实施例中，所述基于所述第一图像数据的显示图像和所述第一图像数据，获取图像色差值包括：In one embodiment, obtaining the image color difference value based on the display image of the first image data and the first image data includes:
对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值。The display image of the first image data is photographed to obtain a comparison image, and the image color difference value is obtained based on the comparison image and the first image data.
在一个实施例中，所述对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值包括：In one embodiment, photographing the display image of the first image data, obtaining a comparison image, and obtaining the image color difference value based on the comparison image and the first image data includes:
基于光谱成像对所述第一图像数据的显示图像进行拍摄，获得所 述对比图像；Based on spectral imaging, the display image of the first image data is photographed to obtain the Described comparison image;
对所述对比图像和所述第一图像数据进行图像匹配，获得匹配像素点集；所述匹配像素点集中包括多个像素点对，每个所述像素点对包括所述第一图像数据中的第一像素点和所述对比图像中的第二像素点；Perform image matching on the comparison image and the first image data to obtain a matching pixel point set; the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes a pair of pixel points in the first image data. The first pixel point and the second pixel point in the comparison image;
获取每个所述像素点对中的第一像素点与第二像素点的色差值，获得图像色差值。The color difference value of the first pixel point and the second pixel point in each pair of pixel points is obtained to obtain the image color difference value.
在一个实施例中，所述基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，包括：In one embodiment, performing color difference compensation on the first image data based on the image color difference value to obtain the second image data includes:
基于所述图像色差值对每个所述像素点对中的第一像素点进行色差补偿，获得第二图像数据。Color difference compensation is performed on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
在一个实施例中，所述图像色差值为RGB值。In one embodiment, the image color difference values are RGB values.
在一个实施例中，所述基于光谱成像对待成像场景进行拍摄，获得第一图像数据，包括：In one embodiment, the scene to be imaged is photographed based on spectral imaging to obtain first image data, including:
获取待成像场景的光谱数据；Obtain spectral data of the scene to be imaged;
对所述光谱数据进行颜色处理，获得颜色数据；Perform color processing on the spectral data to obtain color data;
对所述颜色数据进行编码获得第一图像数据。The color data is encoded to obtain first image data.
在一个实施例中，所述对所述颜色数据进行编码获得颜色数据，包括：In one embodiment, encoding the color data to obtain color data includes:
基于广色域标准对所述入射光光谱进行颜色处理，获得颜色数据。Color processing is performed on the incident light spectrum based on a wide color gamut standard to obtain color data.
在一个实施例中，所述方法包括以下任一数据传输步骤：In one embodiment, the method includes any of the following data transmission steps:
第一传输模块向颜色处理模块发送光谱成像装置获得的所述光谱数据；The first transmission module sends the spectral data obtained by the spectral imaging device to the color processing module;
第二传输模块向编码模块发送所述颜色处理模块获得的所述颜色数据；The second transmission module sends the color data obtained by the color processing module to the encoding module;
第三传输模块向远程显示子系统发送所述编码模块获得的第一图像数据。 The third transmission module sends the first image data obtained by the encoding module to the remote display subsystem.
在一个实施例中，所述数据传输步骤中以不低于8bit/s的数据率进行数据传输。In one embodiment, in the data transmission step, data transmission is performed at a data rate of no less than 8 bit/s.
本申请提供的远程光谱成像系统及方法，一方面通过图像生成子系统通过光谱成像进行画面采集，能够获得精确的光谱信息和色彩信息；另一方面通过色差检测子系统将第一图像数据与对比图像进行匹配与校准，还原待成像场景的真实空间颜色信息，提高了远程图像显示的色彩还原度。The remote spectral imaging system and method provided by this application can, on the one hand, collect pictures through spectral imaging through the image generation subsystem, and obtain accurate spectral information and color information; on the other hand, the first image data can be compared with the color difference detection subsystem through the color difference detection subsystem. The images are matched and calibrated to restore the real spatial color information of the scene to be imaged, improving the color restoration of remote image display.
本申请实施例提供的用于远程光谱成像系统的终端检测系统及方法，以第一图像数据作为颜色参考，获取显示终端实际呈现的第一图像数据的显示图像，将两者进行对比，建立映射关系，基于映射关系对显示终端进行校准和误差消除，避免了显示终端的颜色显示误差，进一步提升成像系统的精准度。The terminal detection system and method for a remote spectral imaging system provided by embodiments of the present application uses the first image data as a color reference to obtain a display image of the first image data actually presented by the display terminal, compare the two, and establish a mapping Relationship, the display terminal is calibrated and errors are eliminated based on the mapping relationship, which avoids the color display error of the display terminal and further improves the accuracy of the imaging system.
附图说明Description of the drawings
为了更清楚地说明本申请或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the technical solutions in this application or the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are of the present invention. For some embodiments of the application, those of ordinary skill in the art can also obtain other drawings based on these drawings without exerting creative efforts.
图1是本申请实施例提供的远程光谱成像系统结构示意图；Figure 1 is a schematic structural diagram of a remote spectral imaging system provided by an embodiment of the present application;
图2是本申请实施例提供的光谱成像装置结构示意图；Figure 2 is a schematic structural diagram of a spectral imaging device provided by an embodiment of the present application;
图3是本申请实施例提供的光谱芯片的原理框图；Figure 3 is a functional block diagram of a spectrum chip provided by an embodiment of the present application;
图4是本申请实施例提供的光谱芯片的结构示意图；Figure 4 is a schematic structural diagram of a spectrum chip provided by an embodiment of the present application;
图5是本申请实施例提供的用于远程光谱成像系统的终端检测系统的结构示意图；Figure 5 is a schematic structural diagram of a terminal detection system for a remote spectral imaging system provided by an embodiment of the present application;
图6是本申请实施例提供的用于远程光谱成像系统的终端检测方法的流程示意图之一；Figure 6 is one of the flow diagrams of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application;
图7是本申请实施例提供的用于远程光谱成像系统的终端检测 方法的流程示意图之二；Figure 7 is a terminal detection for a remote spectral imaging system provided by an embodiment of the present application. Flowchart 2 of the method;
图8是本申请实施例提供的用于远程光谱成像系统的终端检测方法的流程示意图之三；Figure 8 is the third schematic flowchart of the terminal detection method for the remote spectral imaging system provided by the embodiment of the present application;
图9是本申请实施例提供的远程光谱成像方法的流程示意图；Figure 9 is a schematic flowchart of a remote spectral imaging method provided by an embodiment of the present application;
图10是本申请实施例提供的电子设备的结构示意图。Figure 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
具体实施方式Detailed ways
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application. , not all examples. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.
RGB等多通道相机在画面获取时，相当于将空间对应位置的高维光谱信息做了简单的三维投影，那么在画面颜色，也即光谱信息的视觉体现的采集获取上，不可避免的会丢失很多信息，因此就导致在画面信息的采集端无法保证色彩信息的精准获取；另一方面，由于激光液晶等显示技术的色域限制和不同环境色温的影响，画面颜色白平衡和画面重建还原度也有一定的失真和限制。因此，现有的远程画面(例如远程医疗画面)，无法做到色彩上的精准复现，从而会影响用户体验，如在医疗场景中画面颜色失真可能会导致用户误判。When acquiring images with multi-channel cameras such as RGB, it is equivalent to performing a simple three-dimensional projection of the high-dimensional spectral information at the corresponding position in space. Therefore, the color of the image, that is, the visual representation of the spectral information, will inevitably be lost. There is a lot of information, so the accurate acquisition of color information cannot be guaranteed at the picture information collection end; on the other hand, due to the color gamut limitations of display technologies such as laser liquid crystal and the influence of different environmental color temperatures, the picture color white balance and picture reconstruction restoration degree There are also certain distortions and limitations. Therefore, existing remote images (such as telemedicine images) cannot achieve accurate color reproduction, which will affect the user experience. For example, in medical scenarios, image color distortion may lead to user misjudgment.
要确保远程画面的精度，需要从多个角度共同去实现，首先要在采集端确保获取的信息的精度，其次在传输过程中避免信息的丢失，最后在显示过程中也要确保精度，三者任一不足都有一定概率影响最后显示的精度。To ensure the accuracy of remote images, it needs to be achieved from multiple angles. First, the accuracy of the acquired information must be ensured at the collection end. Secondly, the loss of information must be avoided during the transmission process. Finally, the accuracy must be ensured during the display process. Any deficiency has a certain probability of affecting the accuracy of the final display.
下面结合图1-图2描述本申请实施例提供的远程光谱成像系统。The remote spectral imaging system provided by the embodiment of the present application will be described below with reference to Figures 1-2.
图1是本申请实施例提供的远程光谱成像系统结构示意图，如图1所示，本申请实施例提供的远程光谱成像系统包括：图像生成子系统110和远程显示子系统120； Figure 1 is a schematic structural diagram of a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 1, the remote spectral imaging system provided by an embodiment of the present application includes: an image generation subsystem 110 and a remote display subsystem 120;
所述图像生成子系统110用于基于光谱成像获取待成像场景的第一图像数据，并向所述远程显示子系统120发送第一图像数据，由远程显示子系统120显示第一图像数据；The image generation subsystem 110 is configured to obtain first image data of the scene to be imaged based on spectral imaging, and send the first image data to the remote display subsystem 120, and the remote display subsystem 120 displays the first image data;
在一个实施例中，待成像场景位于图像采集地，图像生成子系统110通过光谱成像装置对待成像场景进行拍摄获得光谱数据；进一步，对所述光谱数据进行传输，优选地，在传输过程中对所述光谱数据进行颜色处理，转化到颜色空间，再进行压缩和编码获取显示图像。In one embodiment, the scene to be imaged is located at the image collection site, and the image generation subsystem 110 captures the scene to be imaged through a spectral imaging device to obtain spectral data; further, the spectral data is transmitted, preferably during the transmission process. The spectral data is subjected to color processing, converted into a color space, and then compressed and encoded to obtain a display image.
需要说明的是，本申请所述图像生成子系统可以为包含用以采集数据的采集端和用以传输至显示系统的传输端，即所述采集端可以实施为所述光谱成像装置，用以采集光谱数据；再进一步用传输端对数据进行高保真的传输，例如，对所述光谱数据进行高比特数据传输。也就是，在获取图像的每个像素的光谱数据以后，为了最大限度地减小图像中颜色信息的丢失，在本申请实施例中进行高比特数据传输，在本领域内，通常不低于8比特的数据传输都可以被称为高比特数据传输。例如，高比特数据传输可以是10比特、12比特、或者16比特的数据传输。It should be noted that the image generation subsystem described in this application may include a collection end for collecting data and a transmission end for transmitting to the display system, that is, the collection end may be implemented as the spectrum imaging device to Collect spectral data; and then further use the transmission end to transmit the data with high fidelity, for example, perform high-bit data transmission on the spectral data. That is, after acquiring the spectral data of each pixel of the image, in order to minimize the loss of color information in the image, high-bit data transmission is performed in the embodiment of the present application. In this field, usually no less than 8 Bit data transmission can be called high-bit data transmission. For example, the high-bit data transmission may be 10-bit, 12-bit, or 16-bit data transmission.
进一步，可以对传输的光谱数据进行颜色处理包括：所述图像处理过程在色域无关的颜色空间中进行，且所述图像处理过程中没有色域的压缩和损失。光谱数据需要经过色度学计算和颜色特性化，来转换到颜色空间，比如BT.2020色域空间。其中，颜色特性化可以使用多项式转换、查找表或者神经网络方法。这里，颜色特性化指的是建立色度值与相机输出值之间的对应关系。Further, color processing of the transmitted spectral data may include: the image processing process is performed in a gamut-independent color space, and there is no compression or loss of color gamut during the image processing process. Spectral data needs to be converted into a color space, such as the BT.2020 color gamut space, through colorimetric calculation and color characterization. Among them, color characterization can use polynomial transformation, lookup table or neural network methods. Here, color characterization refers to establishing a correspondence between chromaticity values and camera output values.
对已进行颜色处理的图像数据(光谱数据)进行编码。也就是，在经过前端信号采集和数字图像处理之后，可以最大限度地减少图像色彩精度的损失，而作为完整的全链路的高色域和高保真颜色管理方案，根据已处理的图像数据的编码可获取高保真的显示图像。Encode color-processed image data (spectral data). That is to say, after front-end signal collection and digital image processing, the loss of image color accuracy can be minimized. As a complete full-link high color gamut and high-fidelity color management solution, according to the processed image data Encoding results in high-fidelity display images.
所述远程显示子系统120用于接收所述图像生成子系统110发送 的所述第一图像数据，并显示所述第一图像数据，即所述远程显示子系统120基于第一图像数据进行显示；一般来讲，编码可以在所述远程显示子系统上进行，亦可以在图像生成子系统进行；需要进一步解释，本申请所述图像生成子系统110生成的所述第一图像数据，可以是常规的RGB、黑白等图片，亦可以是光谱数据或光谱成像。The remote display subsystem 120 is configured to receive the image sent by the image generation subsystem 110 The first image data and display the first image data, that is, the remote display subsystem 120 displays based on the first image data; generally speaking, encoding can be performed on the remote display subsystem, also It can be performed in the image generation subsystem; further explanation is needed. The first image data generated by the image generation subsystem 110 of this application can be conventional RGB, black and white pictures, or spectral data or spectral imaging.
在一个实施例中，远程显示子系统120设置于远程图像显示地，远程显示子系统120可以为显示设备，如便携显示器、4K显示器或8K显示器等。In one embodiment, the remote display subsystem 120 is provided at a remote image display location. The remote display subsystem 120 may be a display device, such as a portable display, a 4K display, or an 8K display.
本申请用以获取光谱数据的光谱成像装置和用以显示的显示设备实施为远程设置，通过本申请的技术方案可以以最大限度地为用户提供高色域和高保真的影像体验。例如，用以医疗治疗时，光谱成像装置可以设置于就诊室，而显示设备可以设置于就诊室之外的某诊室，该医生通过显示设备可以最大程度地获取真实医疗场景的空间颜色信息，有利于提高医生的诊断精度。The spectral imaging device used to obtain spectral data in this application and the display device used to display it are implemented as remote settings. Through the technical solution of this application, users can be provided with high color gamut and high fidelity image experience to the maximum extent. For example, when used for medical treatment, the spectral imaging device can be set up in the consultation room, and the display device can be set up in a certain consultation room outside the consultation room. The doctor can obtain the spatial color information of the real medical scene to the greatest extent through the display device, and there is It is helpful to improve the diagnostic accuracy of doctors.
本申请实施例提供的远程光谱成像系统还用于基于所述第一图像数据和所述第一图像数据的显示图像，获取所述第一图像数据的显示图像与第一图像数据的图像色差值；其中，第一图像数据的显示图像是远程显示子系统120基于第一图像数据进行显示的显示图像。示例性的，远程显示子系统120具有图像显示功能，可以将第一图像数据以图像的形式进行显示。The remote spectral imaging system provided by the embodiment of the present application is also used to obtain the image color difference between the display image of the first image data and the first image data based on the first image data and the display image of the first image data. value; wherein, the display image of the first image data is a display image displayed by the remote display subsystem 120 based on the first image data. For example, the remote display subsystem 120 has an image display function and can display the first image data in the form of an image.
所述第一图像数据的显示图像是指所述远程显示子系统在设备自身参数和外界环境因素影响(如外界光)等影响因素下，对第一图像数据进行实际显示的画面，所述第一图像数据的显示图像的显示内容为在设备自身参数和外界环境因素影响(如外界光)等影响因素影响下的第一图像数据。The display image of the first image data refers to the picture in which the first image data is actually displayed by the remote display subsystem under the influence of the device's own parameters and external environmental factors (such as external light). The display content of an image data is the first image data under the influence of the device's own parameters and external environmental factors (such as external light).
本申请实施例提供的远程光谱成像系统还用于基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，并由所 述远程显示子系统显示所述第二图像数据。The remote spectral imaging system provided by the embodiment of the present application is also used to perform color difference compensation on the first image data based on the image color difference value, obtain the second image data, and obtain the second image data based on the image color difference value. The remote display subsystem displays the second image data.
一个实施例中，使用色差检测装置(如本申请实施例提供的色差检测子系统)预先获取第一图像数据的显示图像与第一图像数据的图像色差值，第一图像数据中可以包括多种颜色，如第一图像数据可以为包括16777216种颜色的检测图片，通过对第一图像数据的显示图像与第一图像数据进行对比，可以获得16777216种颜色每种颜色对应的图像色差值。所述远程显示子系统通过预先获得的16777216种颜色中每种颜色对应的图像色差值，对第一图像数据的显示图像进行色差补偿。以上是为便于说明本申请进行的举例，不应对本申请构成任何限定。In one embodiment, a color difference detection device (such as the color difference detection subsystem provided in the embodiment of the present application) is used to pre-obtain the image color difference value of the display image of the first image data and the first image data. The first image data may include multiple For example, the first image data can be a detection picture including 16777216 colors. By comparing the display image of the first image data with the first image data, the image color difference value corresponding to each color of the 16777216 colors can be obtained. The remote display subsystem performs color difference compensation on the display image of the first image data by using the image color difference values corresponding to each of the 16777216 colors obtained in advance. The above examples are provided to facilitate the explanation of the present application and shall not constitute any limitation to the present application.
一个实施例中，使用色差检测装置(如本申请实施例提供的色差检测子系统)实时获取第一图像数据的显示图像与第一图像数据的图像色差值，通过对第一图像数据的显示图像与第一图像数据进行对比，可以获得第一图像数据的图像色差值，基于图像色差值对第一图像数据进行实时色差补偿。以上是为便于说明本申请进行的举例，不应对本申请构成任何限定。In one embodiment, a color difference detection device (such as the color difference detection subsystem provided in the embodiment of the present application) is used to obtain the display image of the first image data and the image color difference value of the first image data in real time. By displaying the first image data The image is compared with the first image data to obtain the image color difference value of the first image data, and real-time color difference compensation is performed on the first image data based on the image color difference value. The above examples are provided to facilitate the explanation of the present application and shall not constitute any limitation to the present application.
在一个实施例中，所述远程光谱成像系统还包括色差检测子系统，色差检测子系统用于基于第一图像数据和所述第一图像数据的显示图像，获取所述第一图像数据的显示图像与第一图像数据的图像色差值；In one embodiment, the remote spectral imaging system further includes a color difference detection subsystem, and the color difference detection subsystem is configured to obtain a display of the first image data based on the first image data and a display image of the first image data. The image color difference value between the image and the first image data;
在一个实施例中，所述远程光谱成像系统中的远程显示子系统可以用于基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，并由所述远程显示子系统显示所述第二图像数据。In one embodiment, the remote display subsystem in the remote spectral imaging system may be used to perform color difference compensation on the first image data based on the image color difference value, obtain second image data, and use the remote display subsystem to perform color difference compensation on the first image data based on the image color difference value. The display subsystem displays the second image data.
进一步，为了确保显示的精度，本申请进一步提供色差检测子系统，所述色差检测子系统130用于对所述第一图像数据的显示图像进行拍摄，获得对比图像，基于所述对比图像和所述第一图像数据获得图像色差值，向所述远程显示子系统120发送所述图像色差值，即可 通过色差检测子系统130，可以获取第一图像数据在所述远程显示子系统120上显示的图像(对比图像)，再与实际的第一图像数据进行对比，就可以获取两者的色差(色差值)；Furthermore, in order to ensure the accuracy of display, the present application further provides a color difference detection subsystem. The color difference detection subsystem 130 is used to shoot the display image of the first image data and obtain a comparison image. Based on the comparison image and the Obtain the image color difference value from the first image data, and send the image color difference value to the remote display subsystem 120. Through the color difference detection subsystem 130, the image (comparison image) of the first image data displayed on the remote display subsystem 120 can be obtained, and then compared with the actual first image data, the color difference (color difference) between the two can be obtained. difference);
在一个实施例中，第一图像数据的显示图像是指所述远程显示子系统120基于第一图像数据进行显示的显示内容，色差检测子系统130位于远程图像显示地，图像色差值可以包括RGB值或LAB色差值(L表示颜色的明亮度、A表示红绿值、B表示黄蓝值)等表示物体颜色差异的值色空间。In one embodiment, the displayed image of the first image data refers to the display content displayed by the remote display subsystem 120 based on the first image data. The color difference detection subsystem 130 is located at the remote image display location. The image color difference value may include RGB value or LAB color difference value (L represents the brightness of the color, A represents the red-green value, B represents the yellow-blue value) and other value color spaces that represent the color difference of objects.
所述远程显示子系统120还用于接收所述色差检测子系统130发送的所述图像色差值，基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，并显示所述第二图像数据。The remote display subsystem 120 is also configured to receive the image color difference value sent by the color difference detection subsystem 130, perform color difference compensation on the first image data based on the image color difference value, and obtain the second image data. , and display the second image data.
需要注意的是，本申请所述色差检测子系统可以集成于所述远程光谱成像系统；亦可以独立的通信连接于所述远程光谱成像系统，即仅用以拍摄第一图像数据获取对比图像，再通信地传输给远程光谱成像系统去获取图像色差值。It should be noted that the color difference detection subsystem described in this application can be integrated into the remote spectral imaging system; it can also be independently communicated and connected to the remote spectral imaging system, that is, it is only used to capture the first image data to obtain the comparison image. Then it is transmitted to the remote spectral imaging system through communication to obtain the image color difference value.
一个实施例中，第一图像数据颜色值为100，由于远程图像显示地环境光充足，提高了实际显示图像的颜色值，对比图像颜色值为105，图像色差值为第一图像数据与对比图像的差值-5，基于差值-5对第一图像数据进行色差补偿，远程显示子系统120获得并显示颜色值为95的第二图像数据，第二图像数据结合环境光后，实际显示更接近第一图像数据。In one embodiment, the color value of the first image data is 100. Since the ambient light of the remote image display place is sufficient, the color value of the actual displayed image is improved. The color value of the comparison image is 105. The color difference value of the image is the difference between the first image data and the contrast value. The difference value of the image is -5. Based on the difference value -5, the first image data is compensated for the color difference. The remote display subsystem 120 obtains and displays the second image data with a color value of 95. After the second image data is combined with the ambient light, the actual display closer to the first image data.
本申请实施例提供的远程光谱成像系统，一方面通过图像生成子系统110通过光谱成像进行画面采集，能够获得精确的光谱信息和色彩信息(即，获取精确的光谱数据)；还可以实现高保真的传输，并编码。另一方面通过色差检测子系统130将第一图像数据与对比图像进行匹配与校准，还原待成像场景的真实空间颜色信息，提高了远程图像显示的色彩还原度。如在医疗场景中，由于远程图像(第二图像 数据)的颜色真实性的提高，有利于提高用户对远程图像判断的准确性和有效性，提升远程医疗的效率、精准度和可信度。The remote spectral imaging system provided by the embodiment of the present application, on the one hand, can obtain accurate spectral information and color information (ie, obtain accurate spectral data) through image generation subsystem 110 through spectral imaging; it can also achieve high fidelity transmission and encoding. On the other hand, the first image data and the comparison image are matched and calibrated through the color difference detection subsystem 130 to restore the real spatial color information of the scene to be imaged, thereby improving the color restoration degree of remote image display. For example, in medical scenarios, due to the remote image (second image The improvement of the color authenticity of the data will help improve the accuracy and effectiveness of users' judgment of remote images, and improve the efficiency, accuracy and credibility of telemedicine.
下面，对上述子系统的可能的实现方式做进一步说明。Below, possible implementations of the above subsystems are further described.
在一个实施例中，所述图像生成子系统110包括In one embodiment, the image generation subsystem 110 includes
包括光谱成像装置、颜色处理模块和编码模块；Includes spectral imaging device, color processing module and encoding module;
所述光谱成像装置用于获取待成像场景的光谱数据，并向所述颜色处理模块发送所述光谱数据；The spectral imaging device is used to obtain spectral data of the scene to be imaged and send the spectral data to the color processing module;
所述颜色处理模块用于接收所述光谱成像装置发送的所述光谱数据，并对所述光谱数据进行颜色处理，获得颜色数据；所述颜色处理模块还用于向所述编码模块发送所述颜色数据；The color processing module is used to receive the spectral data sent by the spectral imaging device and perform color processing on the spectral data to obtain color data; the color processing module is also used to send the spectral data to the encoding module. color data;
所述编码模块用于接收所述颜色处理模块发送的所述颜色数据，并对所述颜色数据进行编码获得第一图像数据。The encoding module is configured to receive the color data sent by the color processing module, and encode the color data to obtain first image data.
一个实施例中，光谱成像装置能够获得待成像场景入射光的光强数据，光强数据与光谱成像装置中光谱像素位置一一对应，光谱成像装置的数据处理模块能够根据每个光谱像素对应的光谱透过率信息、光谱强度数据和光谱成像装置参数信息反演出待成像场景入射光的光谱数据。颜色处理模块通过颜色矩阵变换或颜色空间映射等方法对光谱数据进行颜色处理，并使得编码模块可编码获得第一图像数据。In one embodiment, the spectral imaging device can obtain the light intensity data of the incident light of the scene to be imaged. The light intensity data corresponds to the position of the spectral pixels in the spectral imaging device. The data processing module of the spectral imaging device can obtain the light intensity data corresponding to each spectral pixel. The spectral transmittance information, spectral intensity data and spectral imaging device parameter information invert the spectral data of the incident light of the scene to be imaged. The color processing module performs color processing on the spectral data through methods such as color matrix transformation or color space mapping, and enables the encoding module to encode and obtain the first image data.
图2是本申请实施例提供的光谱成像装置结构示意图，如图2所示，所述光谱成像装置包括滤光结构210、图像传感器220和数据处理单元230，所述滤光结构210设置于所述图像传感器220的感光路径上；所述滤光结构210的透射率曲线可以基于待成像场景的光谱特征确定；Figure 2 is a schematic structural diagram of a spectral imaging device provided by an embodiment of the present application. As shown in Figure 2, the spectral imaging device includes a filter structure 210, an image sensor 220 and a data processing unit 230. The filter structure 210 is disposed on the On the photosensitive path of the image sensor 220; the transmittance curve of the filter structure 210 can be determined based on the spectral characteristics of the scene to be imaged;
所述滤光结构210用于对待成像场景的入射光进行调制，获得调制光信号；The filter structure 210 is used to modulate the incident light of the scene to be imaged to obtain a modulated light signal;
所述图像传感器220用于接收所述调制光信号，并获得光强数据；The image sensor 220 is used to receive the modulated light signal and obtain light intensity data;
所述数据处理单元230用于接收所述图像传感器发送的光强数 据，并基于所述光强数据进行光谱恢复，获得所述待成像场景对应的光谱数据。The data processing unit 230 is configured to receive the light intensity data sent by the image sensor. and perform spectral recovery based on the light intensity data to obtain spectral data corresponding to the scene to be imaged.
在一个实施例中，对于滤光结构210，所述滤光结构210包括至少一个结构单元，所述结构单元包括至少二微纳结构。滤光结构210中的结构单元可以根据对应的透射率曲线出射目标波长范围的光，即入射光进入结构单元被调制后获取调制后的光信号。滤光结构210可以为频域或者波长域上的宽带滤光结构210。滤光结构210可以是超表面、光子晶体、纳米柱、多层膜、染料、量子点、微机电系统(MEMS)、FP标准具(FP etalon)、谐振腔层(cavity layer)、波导层(waveguide layer)或衍射元件等具有滤光特性的结构或材料。每个不同的结构单元的透射率曲线互不相同。结构单元的透射率曲线表示结构单元对不同波长入射光的光谱透过率。基于待成像场景的光谱特征确定所述结构单元的透射率曲线，结构单元的透射率在待成像场景吸收峰对应的波段较高，能够使滤光结构210对待成像场景不同状态光谱特征具有区分效果。In one embodiment, for the filter structure 210, the filter structure 210 includes at least one structural unit, and the structural unit includes at least two micro-nano structures. The structural units in the filter structure 210 can emit light in the target wavelength range according to the corresponding transmittance curve, that is, the incident light enters the structural unit and is modulated to obtain a modulated optical signal. The filter structure 210 may be a broadband filter structure 210 in the frequency domain or wavelength domain. The filter structure 210 can be a metasurface, a photonic crystal, a nanopillar, a multilayer film, a dye, a quantum dot, a microelectromechanical system (MEMS), an FP etalon, a cavity layer, a waveguide layer ( Structures or materials with light filtering properties such as waveguide layer) or diffractive elements. The transmittance curves of each different structural unit are different from each other. The transmittance curve of a structural unit represents the spectral transmittance of the structural unit to incident light of different wavelengths. The transmittance curve of the structural unit is determined based on the spectral characteristics of the scene to be imaged. The transmittance of the structural unit is higher in the band corresponding to the absorption peak of the scene to be imaged, which enables the filter structure 210 to have a distinguishing effect on the spectral characteristics of different states of the scene to be imaged. .
对于图像传感器220，图像传感器220可以是CMOS图像传感器(CMOS image sensor，CIS)、电荷耦合元件(Charge coupled Device，CCD)或阵列光探测器等。图像传感器220可以将光信号转换为电信号，获得检测图像和与结构单元一一对应的光强数据。检测图像包括多个检测位置，每个检测位置与一个或多个结构单元对应，因此每个检测位置与其对应的结构单元的光强数据相对应。For the image sensor 220, the image sensor 220 may be a CMOS image sensor (CMOS image sensor, CIS), a charge coupled device (Charge coupled device, CCD), or an array light detector, etc. The image sensor 220 can convert the optical signal into an electrical signal to obtain the detection image and light intensity data corresponding to the structural unit one-to-one. The detection image includes multiple detection positions, and each detection position corresponds to one or more structural units, so each detection position corresponds to the light intensity data of its corresponding structural unit.
在一个实施例中，所述光谱成像装置还包括光学系统，所述光学系统设置于所述图像传感器220的感光路径上；In one embodiment, the spectral imaging device further includes an optical system, which is disposed on the photosensitive path of the image sensor 220;
所述光学系统用于对入射光进行光路调整。The optical system is used to adjust the optical path of incident light.
在一个实施例中，光信号通过光学系统调整再经由滤光结构210进行调制后，被图像传感器220接收，获取光强数据；其中所述光学系统可以是透镜组件、匀光组件或准直组件等光学系统，能够对入射 光进行聚焦、匀光或准直。In one embodiment, the optical signal is adjusted by the optical system and then modulated by the filter structure 210, and then is received by the image sensor 220 to obtain the light intensity data; wherein the optical system can be a lens component, a uniform light component or a collimation component. and other optical systems, capable of incident Light is focused, homogenized or collimated.
对于所述光谱成像装置的数据处理单元，所述数据处理单元可以是微控制单元(Micro ControllerUnit，MCU)、中央处理器(Central Processing Unit，CPU)、图形处理器(Graphics Processing Unit，GPU)、现场可编程门阵列(FieldProgram mable GateArray，FPGA)、网络处理器(Neural network Processing Unit，NPU)或专用集成电路(Application Specific Integrated Circuit，ASIC)等处理单元。本申请中，所述数据处理单元对光谱数据进行预处理等。亦可以直接实现光谱数据的颜色处理，即实现了颜色处理模块的功能。For the data processing unit of the spectrum imaging device, the data processing unit may be a micro control unit (Micro Controller Unit, MCU), a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU), Processing units such as Field Programmable Gate Array (FPGA), Neural Network Processing Unit (NPU) or Application Specific Integrated Circuit (ASIC). In this application, the data processing unit performs preprocessing on spectral data, etc. Color processing of spectral data can also be directly implemented, that is, the function of the color processing module is realized.
上述光谱成像装置为基于计算光谱的光谱成像装置，尤其是，本申请基于计算光谱成像技术，可以利用透射谱曲线实现对入射光的宽谱调制，从而使得获取的空间信息更加丰富、更加精确。The above-mentioned spectral imaging device is a spectral imaging device based on computational spectrum. In particular, this application is based on computational spectral imaging technology, which can use the transmission spectrum curve to achieve broad-spectrum modulation of incident light, thereby making the acquired spatial information richer and more accurate.
一个实施例中，光谱成像装置测得光谱响应后，传入数据处理单元230进行光谱恢复计算。在一个实施例中，该过程描述如下：In one embodiment, after the spectral response is measured by the spectral imaging device, the spectral response is transmitted to the data processing unit 230 for spectral recovery calculation. In one embodiment, the process is described as follows:
将入射光在不同波长λ下的强度信号记为x(λ)，滤光结构210的透射谱曲线记为T(λ)，滤光结构210(也可以称为滤光片)上具有m个的结构单元，入射光经过一个结构组中每个结构单元得到的透射谱互不相同，整体来讲，滤光结构210可记为Ti(λ)(i＝1,2,3,…,m)。每一个结构单元下方都有相应的物理像素，物理像素用于探测经过滤光结构210调制的光信号的光强bi。一个物理像素可以对应一个结构单元，也可以是多个物理像素为一组对应于一个结构单元。The intensity signals of the incident light at different wavelengths λ are denoted as x(λ), and the transmission spectrum curve of the filter structure 210 is denoted as T(λ). Structural units, the transmission spectra obtained by incident light passing through each structural unit in a structural group are different from each other. Overall, the filter structure 210 can be recorded as Ti (λ) (i=1,2,3,…, m). There are corresponding physical pixels below each structural unit, and the physical pixels are used to detect the light intensity bi of the optical signal modulated by the filtered light structure 210. One physical pixel can correspond to a structural unit, or a group of multiple physical pixels can correspond to a structural unit.
本实施例中，至少二个不同的结构单元构成一个结构组，一个结构组和结构组对应的物理像素构成一个光谱像素。光谱成像装置可以用于进行光谱恢复的透射谱称为有效透射谱，有效透射谱是指不同结构单元组成的透射谱，在结构单元相同的情况下，会在系统响应系数A矩阵中出现相同的向量，造成无法恢复光谱的情况。In this embodiment, at least two different structural units constitute a structural group, and a structural group and the physical pixel corresponding to the structural group constitute a spectral pixel. The transmission spectrum that the spectral imaging device can use for spectral recovery is called the effective transmission spectrum. The effective transmission spectrum refers to the transmission spectrum composed of different structural units. When the structural units are the same, the same will appear in the system response coefficient A matrix. vector, resulting in a situation where the spectrum cannot be recovered.
所述滤光结构210的有效的透射谱Ti(λ)数量与结构单元数量可 以不一致，所述滤光结构210的透射谱根据待成像场景识别或光谱恢复的精度需求和速度需求人为的按照一定规则去设置、测试或计算获得，因此所述滤光结构210的有效透射谱的数量可以比结构单元数量少(如结构单元中具有重复情况，重复的结构单元的透射谱是无效透射谱)，甚至也可能比结构单元数量多(如，可以通过对结构单元的组合获得新的透射谱)。The number of effective transmission spectra Ti (λ) of the filter structure 210 and the number of structural units can be To be consistent, the transmission spectrum of the filter structure 210 is artificially set, tested or calculated according to certain rules according to the accuracy requirements and speed requirements of scene recognition or spectral recovery to be imaged. Therefore, the effective transmission spectrum of the filter structure 210 is The number can be less than the number of structural units (for example, if there are repetitions in the structural units, the transmission spectrum of the repeated structural units is an invalid transmission spectrum), or even more than the number of structural units (for example, new ones can be obtained by combining the structural units). transmission spectrum).
在一个实施例中，本申请可以用至少一个光谱像素去还原图像。即本申请中光谱装置可以根据光谱响应，去恢复光谱曲线并进行光谱成像。In one embodiment, this application can use at least one spectral pixel to restore the image. That is to say, the spectroscopic device in this application can restore the spectral curve and perform spectral imaging based on the spectral response.
入射光的频谱分布和图像传感器220的测量值之间的关系可以由下式表示：The relationship between the spectral distribution of the incident light and the measurement value of the image sensor 220 can be expressed by the following formula:
图像传感器220获得的光强数据为：
Bi＝∫x(λ)Ti(λ)R(λ)dλ；The light intensity data obtained by the image sensor 220 is:
B i =∫x(λ)T i (λ)R(λ)dλ;
对Bi进行离散化，得到离散后的光强数据为：
bi＝∑x(λ)Ti(λ)R(λ)dλ；Discretize B i , and obtain the discretized light intensity data as:
b i =∑x(λ)T i (λ)R(λ)dλ;
系统响应系数为：
Ai＝Ti(λ)R(λ)；The system response coefficient is:
A i =T i (λ)R(λ);
将上式拓展为矩阵形式：
Expand the above formula into matrix form:
其中，R(λ)为图像传感器220的量子效率，bi(i＝1，2，3……m)是待测光(即入射光)透过滤光结构210后图像传感器220的获得的光强数据，分别对应m个结构单元对应的图像传感器220的光强测量值，当一个物理像素对应一个结构单元时，可以理解为m个物理像素对应的光强测量值，其是一个长度为m的向量。系统响应系数A是系统对于不同波长的光响应，由滤光结构210透射率和图像传感器 220的量子效率两个因素决定。A是矩阵，每一个行向量对应一个结构单元对不同波长入射光的响应。对入射光进行离散、均匀的采样，共有n个采样点。A的列数与入射光的采样点数相同。这里，x(λ)即是入射光在不同波长λ的光强，也就是待测量的入射光光谱。Wherein, R(λ) is the quantum efficiency of the image sensor 220, b i (i=1, 2, 3...m) is the light obtained by the image sensor 220 after the light to be measured (ie, incident light) passes through the filter structure 210. Strong data respectively corresponds to the light intensity measurement values of the image sensor 220 corresponding to m structural units. When a physical pixel corresponds to a structural unit, it can be understood as the light intensity measurement values corresponding to m physical pixels, which is a length of m vector. The system response coefficient A is the light response of the system to different wavelengths, which is determined by the transmittance of the filter structure 210 and the image sensor The quantum efficiency of 220 is determined by two factors. A is a matrix, and each row vector corresponds to the response of a structural unit to incident light of different wavelengths. The incident light is sampled discretely and uniformly, with a total of n sampling points. The number of columns of A is the same as the number of sampling points of the incident light. Here, x(λ) is the intensity of the incident light at different wavelengths λ, which is the spectrum of the incident light to be measured.
在一个实施例中，所述颜色处理模块用于对所述光谱数据进行颜色处理，获得颜色数据包括：In one embodiment, the color processing module is used to perform color processing on the spectral data. Obtaining the color data includes:
所述颜色处理模块用于基于广色域标准对所述入射光光谱进行颜色处理，获得颜色数据。The color processing module is used to perform color processing on the incident light spectrum based on a wide color gamut standard to obtain color data.
对传输的入射光光谱数据进行颜色处理。在传统的数字图像处理过程中，往往要将图像信息转换到一定的颜色空间中进行处理，处理过程中会造成图像的颜色信息损失。在本申请一个实施例中，颜色处理选择色域较大的、设备无关的颜色空间，例如，将大于sRGB的颜色空间作为颜色处理模块进行颜色处理的颜色空间，例如在颜色处理过程中，可以采用BT2020色域空间。该颜色空间所能覆盖的颜色数量远大于其它的颜色空间，可以大大降低数据在传输和处理过程中的损失，最大限度的完成高保真颜色还原。Color processing is performed on the transmitted incident light spectral data. In the traditional digital image processing process, it is often necessary to convert the image information into a certain color space for processing, which will cause the loss of the color information of the image during the processing. In one embodiment of the present application, color processing selects a device-independent color space with a larger color gamut. For example, a color space larger than sRGB is used as a color space for color processing by the color processing module. For example, during the color processing process, you can Using BT2020 color gamut space. The number of colors that this color space can cover is much larger than other color spaces, which can greatly reduce the loss of data during transmission and processing and maximize high-fidelity color restoration.
本申请实施例提供的远程光谱成像系统，采用广色域标准颜色空间，广色域标准颜色空间所能覆盖的颜色数量远大于传统颜色空间，经过广色域标准颜色空间处理后，可以降低第一图像数据在传输和处理过程中的损失，最大限度的完成高保真颜色还原。The remote spectral imaging system provided by the embodiment of the present application adopts a wide color gamut standard color space. The number of colors that the wide color gamut standard color space can cover is much larger than that of the traditional color space. After processing in the wide color gamut standard color space, the second color space can be reduced. The loss of image data during transmission and processing is achieved to achieve high-fidelity color restoration to the maximum extent.
编码模块对颜色处理后的光谱数据(即颜色数据)进行编码获取第一图像数据。The encoding module encodes the color-processed spectral data (ie, color data) to obtain the first image data.
在一个实施例中，所述系统还包括数据传输模块，用以高比特地传输光谱数据；In one embodiment, the system further includes a data transmission module to transmit spectral data in a high-bit manner;
所述数据传输模块包括以下任一：第一传输模块、第二传输模块和第三传输模块；The data transmission module includes any of the following: a first transmission module, a second transmission module and a third transmission module;
所述第一传输模块与所述光谱成像装置和所述颜色处理模块相 连接，用于向所述颜色处理模块发送所述光谱成像装置获得的所述光谱数据；The first transmission module is connected to the spectral imaging device and the color processing module. Connection, used to send the spectral data obtained by the spectral imaging device to the color processing module;
所述第二传输模块与所述颜色处理模块和所述编码模块相连接，用于向所述编码模块发送所述颜色处理模块获得的所述颜色数据；The second transmission module is connected to the color processing module and the encoding module, and is used to send the color data obtained by the color processing module to the encoding module;
所述第三传输模块与所述编码模块和所述远程显示子系统相连接，用于向所述远程显示子系统发送所述编码模块获得的第一图像数据。一个实施例中，图像生成子系统110中的光谱成像装置设置于图像采集地，图像生成子系统110中的颜色处理模块设置于远程图像显示地，光谱成像装置与颜色处理模块之间通过第一传输模块相连接。The third transmission module is connected to the encoding module and the remote display subsystem, and is used to send the first image data obtained by the encoding module to the remote display subsystem. In one embodiment, the spectral imaging device in the image generation subsystem 110 is located at the image collection location, and the color processing module in the image generation subsystem 110 is located at the remote image display location. The spectrum imaging device and the color processing module are connected through a first transmission module is connected.
一个实施例中，图像生成子系统110中的颜色处理模块设置于图像采集地，图像生成子系统110中的编码模块设置于远程图像显示地，颜色处理模块与编码模块之间通过第二传输模块相连接。In one embodiment, the color processing module in the image generation subsystem 110 is located at the image collection location, and the encoding module in the image generation subsystem 110 is located at the remote image display location. The color processing module and encoding module are connected through a second transmission module. connected.
一个实施例中，编码模块设置于图像采集地，编码模块与远程显示子系统120之间通过第三传输模块相连接。In one embodiment, the encoding module is installed at the image collection location, and the encoding module and the remote display subsystem 120 are connected through a third transmission module.
本申请实施例对图像生成子系统中的数据处理模块和颜色处理模块的空间位置不作限定，用户在实际应用中可以根据图像采集地和远程图像显示地的硬件数据处理能力或网络传输速度等影响因素进行灵活设置。The embodiments of this application do not limit the spatial location of the data processing module and color processing module in the image generation subsystem. In actual applications, users can consider the hardware data processing capabilities or network transmission speed of the image collection place and the remote image display place. Factors can be set flexibly.
在一个实施例中，所述数据传输模块用于以不低于8bit/s的数据率进行数据传输。In one embodiment, the data transmission module is used to transmit data at a data rate of no less than 8 bit/s.
在一个实施例中，对图像生成子系统110中生成的光谱数据或第一图像数据进行高比特数据传输，即，在获取光谱数据或第一图像数据后，在最大限度地减少图像中颜色信息的丢失的前提下，进行高比特数据传输，在本领域内，通常不低于8比特每秒的数据传输速率都可以被称为高比特数据传输。示例性地，高比特数据传输可以是10比特、12比特或者14比特的数据传输。In one embodiment, high-bit data transmission is performed on the spectral data or first image data generated in the image generation subsystem 110, that is, after obtaining the spectral data or first image data, the color information in the image is minimized. Under the premise of loss, high-bit data transmission is performed. In this field, a data transmission rate of not less than 8 bits per second can usually be called high-bit data transmission. Exemplarily, the high-bit data transmission may be 10-bit, 12-bit, or 14-bit data transmission.
本申请实施例提供的远程光谱成像系统在信号传输过程中，可以 利用5G等远程通信技术实现高通量高时空分辨率的超光谱图像传输，实现高通量信息的实时传输，以确保远程图像传输的实时性。During the signal transmission process, the remote spectral imaging system provided by the embodiment of the present application can Use remote communication technologies such as 5G to achieve high-throughput and high-spatial-temporal resolution hyperspectral image transmission, and realize real-time transmission of high-throughput information to ensure the real-time nature of remote image transmission.
在一个实施例中，所述色差检测子系统130包括光谱成像模块、图像匹配模块和色差计算模块；In one embodiment, the color difference detection subsystem 130 includes a spectrum imaging module, an image matching module and a color difference calculation module;
所述光谱成像模块用于基于光谱成像对所述第一图像数据的显示图像进行拍摄，获得所述对比图像；The spectral imaging module is configured to capture the display image of the first image data based on spectral imaging to obtain the comparison image;
在一个实施例中，光谱成像模块可以为光谱成像装置或光谱仪等光谱设备，对第一图像数据的显示图像进行拍摄，获得第一图像数据的显示图像的光谱信息，并基于光谱信息获得对比图像。光谱成像模块和光谱成像装置采用的光谱设备和光谱成像方法可以不同。In one embodiment, the spectral imaging module can be a spectral imaging device or a spectrometer or other spectral equipment. It captures the display image of the first image data, obtains the spectral information of the display image of the first image data, and obtains the comparison image based on the spectral information. . The spectral equipment and spectral imaging methods adopted by the spectral imaging module and the spectral imaging device may be different.
所述图像匹配模块用于对所述对比图像和所述第一图像数据进行图像匹配，获得匹配像素点集；所述匹配像素点集中包括多个像素点对，每个所述像素点对包括所述第一图像数据中的第一像素点和所述对比图像中的第二像素点；The image matching module is used to perform image matching on the comparison image and the first image data to obtain a matching pixel point set; the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes The first pixel point in the first image data and the second pixel point in the comparison image;
在一个实施例中，图像匹配模块可以基于图像处理算法对对比图像和第一图像数据进行图像匹配，本申请实施例对匹配方法不做限定。匹配后的对比图像和第一图像数据的像素点一一对应。In one embodiment, the image matching module can perform image matching on the comparison image and the first image data based on an image processing algorithm. The embodiment of the present application does not limit the matching method. The matched comparison image has a one-to-one correspondence with the pixels of the first image data.
所述色差计算单元用于获取每个所述像素点对中的第一像素点与第二像素点的色差值，获得图像色差值。The color difference calculation unit is used to obtain the color difference value of the first pixel point and the second pixel point in each of the pixel point pairs, and obtain the image color difference value.
在一个实施例中，色差计算单元计算对比图像和第一图像数据一一对应的每个像素点的色差值，获得图像色差值。In one embodiment, the color difference calculation unit calculates the color difference value of each pixel corresponding to the comparison image and the first image data to obtain the image color difference value.
在一个实施例中，所述远程显示子系统120包括显示模块和色差补偿模块；In one embodiment, the remote display subsystem 120 includes a display module and a color difference compensation module;
所述显示模块用于显示所述第一图像数据和所述第二图像数据；The display module is used to display the first image data and the second image data;
所述色差补偿模块用于基于所述图像色差值对每个所述像素点对中的第一像素点进行色差补偿，获得第二图像数据。The color difference compensation module is configured to perform color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
在一个实施例中，所述图像色差值为RGB值。 In one embodiment, the image color difference values are RGB values.
一个实施例中，第一图像数据包括4个像素点：A、B、C和D。各像素点信息如下：In one embodiment, the first image data includes 4 pixel points: A, B, C and D. The information of each pixel is as follows:
A像素点坐标(1，1)，RGB值(255，182，193)；A pixel coordinate (1, 1), RGB value (255, 182, 193);
B像素点坐标(1，2)，RGB值(255，192，203)；B pixel point coordinates (1, 2), RGB value (255, 192, 203);
C像素点坐标(2，1)，RGB值(220，20，60)；C pixel coordinates (2, 1), RGB value (220, 20, 60);
D像素点坐标(2，2)，RGB值(255，140，245)；D pixel coordinates (2, 2), RGB value (255, 140, 245);
对比图像经过图像匹配处理后包括与4个与第一图像数据一一对应的像素点：A’、B’、C’和D’。各像素点信息如下：After image matching processing, the comparison image includes four pixels that correspond one-to-one with the first image data: A’, B’, C’ and D’. The information of each pixel is as follows:
A’像素点坐标(1，1)，RGB值(250，180，190)；A’ pixel point coordinates (1, 1), RGB value (250, 180, 190);
B’像素点坐标(1，2)，RGB值(265，197，203)；B’ pixel point coordinates (1, 2), RGB value (265, 197, 203);
C’像素点坐标(2，1)，RGB值(223，18，57)；C’ pixel point coordinates (2, 1), RGB value (223, 18, 57);
D’像素点坐标(2，2)，RGB值(245，130，248)；D’ pixel point coordinates (2, 2), RGB value (245, 130, 248);
图像色差值为：The image color difference value is:
A像素点RGB色差值(5，2，3)；A pixel RGB color difference value (5, 2, 3);
B像素点RGB色差值(-10，-5，0)；B pixel point RGB color difference value (-10, -5, 0);
C像素点RGB色差值(-3，2，3)；C pixel RGB color difference value (-3, 2, 3);
D像素点RGB色差值(10，10，-3)；D pixel point RGB color difference value (10, 10, -3);
色差补偿模块基于所述图像色差值对每个所述像素点对中的第一像素点进行色差补偿，即将图像色差值与第一图像数据相加，获得第二图像数据，第二图像数据包括4个像素点A”、B”、C”和D”。各像素点信息如下：The color difference compensation module performs color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value, that is, adding the image color difference value to the first image data to obtain the second image data. The second image The data includes 4 pixels A”, B”, C” and D”. The information of each pixel is as follows:
A”像素点坐标(1，1)，RGB值(260，184，196)；A" pixel coordinates (1, 1), RGB value (260, 184, 196);
B”像素点坐标(1，2)，RGB值(245，187，203)；B" pixel coordinates (1, 2), RGB value (245, 187, 203);
C”像素点坐标(2，1)，RGB值(217，22，57)；C” pixel coordinates (2, 1), RGB value (217, 22, 57);
D”像素点坐标(2，2)，RGB值(245，130，248)。D" pixel coordinates (2, 2), RGB value (245, 130, 248).
本申请实施例中显示模块显示的第二图像数据，在环境光或设备显示参数等环境因素的影响下，画面颜色显示效果跟接近待成像场景 的真实颜色。In the embodiment of the present application, the second image data displayed by the display module, under the influence of environmental factors such as ambient light or device display parameters, has a color display effect close to that of the scene to be imaged. true color.
在一个实施例中，在经过前端信号采集和数字图像处理之后，为了最大限度地减少图像色彩精度的损失，作为完整的全链路的高色域和高保真颜色管理方案，已处理的图像数据的编码和显示作为方案的最后一环，对于图像显示的效果也非常重要。In one embodiment, after front-end signal acquisition and digital image processing, in order to minimize the loss of image color accuracy, as a complete full-link high color gamut and high fidelity color management solution, the processed image data As the last link of the solution, the encoding and display are also very important for the effect of image display.
在本申请实施例中，需要针对整套颜色管理方案进行图像数据的压缩编码存储和显示屏幕的特性化，以最大限度地为用户提供高色域和高保真的影像体验。在本申请实施例中，图像数据在压缩编码时需要按照存储要求选择颜色空间，为实现全链路的高色域和高保真颜色管理，所以在图像压缩和编码过程中，仍然按照如上所述的广色域颜色空间进行颜色处理，例如BT.2020色域空间进行图片压缩和编码，且存储位深设置为高比特数，例如10比特，从而最大程度地保证图片颜色的层次过渡，然后传输给后端的显示系统进行显示。In this embodiment of the present application, it is necessary to compress, encode and store image data and characterize the display screen for a complete set of color management solutions, so as to provide users with a high-color gamut and high-fidelity imaging experience to the greatest extent. In the embodiment of this application, the color space needs to be selected according to the storage requirements when compressing and encoding image data. In order to achieve full-link high color gamut and high-fidelity color management, during the image compression and encoding process, the image data is still compressed and encoded as described above. Use a wide color gamut color space for color processing, such as the BT.2020 color gamut space for image compression and encoding, and the storage bit depth is set to a high number of bits, such as 10 bits, to ensure the level transition of the image color to the greatest extent, and then transmit Display to the back-end display system.
本申请实施例对光谱色域建模(即通过基于待成像场景的光谱信息获得第一图像数据)和显示端色域建模(即在远程图像显示端，通过将远程显示子系统显示的第一图像数据的实际画面与图像采集端的图像生成子系统110生成的第一图像数据进行校准)，在数据传输过程中采用实时高通量光谱图像数据传输，计算获得待成像场景的空间色彩信息在远程显示子系统中的色域表征(第二图像数据)，控制远程显示子系统显示第二图像数据，使显示后的画面尽可能逼近待成像场景的真实空间色彩。本申请实施例还可以将光谱成像装置采集获取的待成像场景空间不同位置的入射光光谱信息直接传输给远程显示子系统端，提供人眼视觉不易察觉的精细光谱特征，辅助用户进行远程图片判断。在医疗场景中，本申请是合理能够提升远程医疗的效率、精准度和可信度，促进医疗资源的平均分配。The embodiments of the present application model the spectral color gamut (that is, obtain the first image data based on the spectral information of the scene to be imaged) and the display end color gamut modeling (that is, at the remote image display end, by converting the first image data displayed by the remote display subsystem to The actual picture of the image data is calibrated with the first image data generated by the image generation subsystem 110 of the image acquisition end). During the data transmission process, real-time high-throughput spectral image data transmission is used to calculate and obtain the spatial color information of the scene to be imaged. The color gamut representation (second image data) in the remote display subsystem controls the remote display subsystem to display the second image data so that the displayed picture is as close as possible to the real space color of the scene to be imaged. Embodiments of the present application can also directly transmit incident light spectrum information at different locations in the scene space to be imaged collected by the spectrum imaging device to the remote display subsystem, providing fine spectral features that are not easily detectable by human vision, and assisting users in remote image judgment. . In medical scenarios, this application can reasonably improve the efficiency, accuracy and credibility of telemedicine and promote the equal distribution of medical resources.
在另一种应用场景中，可以通过快照式光谱芯片对实际场景进行光谱及图像信息的采集，然后通过高速数据传输，根据采集获取的光 谱信息，在显示终端进行图像的色彩显示。本申请实施例在实际场景画面采集端采用光谱芯片，图3是本申请实施例提供的光谱芯片的原理框图，图4是本申请实施例提供的光谱芯片的结构示意图，如图2、图3和图4所示，所述光谱芯片包括滤光结构和图像传感器，所述滤光结构形成于所述图像传感器的光学路径上，利用光谱成像芯片获取实地场景的空间各点光谱信息，即该步骤获取通过光谱成像芯片得到的空间光谱数据，一般而言，入射光经过滤光结构进行调制，再被图像传感器所接收获取到对应的光谱信息，再通过光谱恢复算法进行处理得到对应的光谱数据。如图4所示，滤光结构由多个结构单元构成，每个结构单元包括至少一微纳结构，微纳结构可以实施为如图4所示的调制孔。In another application scenario, the snapshot spectrum chip can be used to collect spectral and image information of the actual scene, and then through high-speed data transmission, based on the collected light Spectral information is used to display the color of the image on the display terminal. The embodiment of the present application uses a spectrum chip at the actual scene picture collection end. Figure 3 is a functional block diagram of the spectrum chip provided by the embodiment of the present application. Figure 4 is a schematic structural diagram of the spectrum chip provided by the embodiment of the present application, as shown in Figures 2 and 3 As shown in Figure 4, the spectrum chip includes a filter structure and an image sensor. The filter structure is formed on the optical path of the image sensor. The spectrum imaging chip is used to obtain spectral information of each point in space of the field scene, that is, the The step is to obtain the spatial spectral data obtained through the spectral imaging chip. Generally speaking, the incident light is modulated by the filtering light structure, and then received by the image sensor to obtain the corresponding spectral information, and then processed through the spectral recovery algorithm to obtain the corresponding spectral data. . As shown in Figure 4, the filter structure is composed of multiple structural units, each structural unit includes at least one micro-nano structure, and the micro-nano structure can be implemented as a modulation hole as shown in Figure 4.
在数据传输过程中，对所述光谱数据进行高比特数据传输，即在获取图像的每个像素的光谱数据以后，为了最大限度地减少图像中颜色信息的丢失。需要注意的是，也可以直接传输图像传感器获得的光谱信息。在信号传输过程中，利用5G等远程通信技术实现高通量高时空分辨率的超光谱图像传输，实现高通量信息的实时传输，以确保系统的实时性。During the data transmission process, the spectral data is subjected to high-bit data transmission, that is, after the spectral data of each pixel of the image is obtained, in order to minimize the loss of color information in the image. It should be noted that the spectral information obtained by the image sensor can also be transmitted directly. During the signal transmission process, remote communication technologies such as 5G are used to realize high-throughput and high-spatial-temporal resolution hyperspectral image transmission, and realize real-time transmission of high-throughput information to ensure the real-time nature of the system.
在本申请实施例中，在颜色处理过程中，全程可以采用BT.2020色域空间。In the embodiment of this application, during the color processing process, the BT.2020 color gamut space can be used throughout the process.
在信号画面重建端，首先建模显示终端的色域和光谱的色域，然后利用获取的实时高通量光谱数据，计算获取实地场景的空间色彩信息在显示系统色域的表征，控制显示系统的显示，使之尽可能逼近实地场景的空间色彩。On the signal screen reconstruction side, the color gamut and spectral color gamut of the display terminal are first modeled, and then the obtained real-time high-throughput spectral data is used to calculate and obtain the representation of the spatial color information of the field scene in the display system color gamut, and control the display system display to make it as close as possible to the spatial color of the actual scene.
需要注意的是，虽然所述光谱芯片和光谱数据传输技术可以获取高保真颜色还原，但是如果建模显示系统存在偏差，也会导致存在成像误差。It should be noted that although the spectral chip and spectral data transmission technology can obtain high-fidelity color reproduction, if there are deviations in the modeling display system, imaging errors will also occur.
进一步地，对上述成像系统进行分析，其误差来源主要有两个方 面：一方面来源于终端显示系统的成像误差，一方面来源于显示终端环境光对成像的影响。Further, analyzing the above imaging system, there are two main sources of error: Surface: On the one hand, it comes from the imaging error of the terminal display system, and on the other hand, it comes from the impact of the ambient light of the display terminal on the imaging.
示例性的，本申请可以应用于远程医疗画面。现有远程医疗画面无法做到色彩上的精准复现，从而会影响用户体验，如在医疗场景中画面颜色失真可能会导致用户误判。要确保远程画面的精度，需要从多个角度共同去实现，首先要在采集端确保获取的信息的精度，其次在传输过程中避免信息的丢失，最后在显示时也要确保精度，三者任一不足都有一定概率影响最后显示的精度。By way of example, this application can be applied to telemedicine scenarios. Existing telemedicine images cannot achieve accurate color reproduction, which will affect the user experience. For example, in medical scenarios, image color distortion may lead to misjudgment by users. To ensure the accuracy of remote images, it needs to be achieved from multiple angles. First, the accuracy of the acquired information must be ensured at the collection end. Secondly, the loss of information must be avoided during the transmission process. Finally, the accuracy must be ensured during display. All three must Any deficiencies have a certain probability of affecting the accuracy of the final display.
为解决上述问题，本申请实施例提供一种用于上述任一远程光谱成像系统的终端检测系统，用于检测远程光谱成像系统中的颜色显示误差，并进行校准和误差消除，进一步提升成像方案的精准度。示例性的，该终端检测系统，还可以应用于其他成像系统，如4K高清图像传输成像系统等。In order to solve the above problems, embodiments of the present application provide a terminal detection system for any of the above remote spectral imaging systems, which is used to detect the color display error in the remote spectral imaging system, and perform calibration and error elimination to further improve the imaging solution. accuracy. For example, the terminal detection system can also be applied to other imaging systems, such as 4K high-definition image transmission imaging systems.
图5是本申请实施例提供的用于远程光谱成像系统的终端检测系统的结构示意图，如图5所示，终端检测系统，包括：显示终端510、测色仪520和校准模块530；Figure 5 is a schematic structural diagram of a terminal detection system for a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 5, the terminal detection system includes: a display terminal 510, a colorimeter 520 and a calibration module 530;
在一个实施例中，显示终端510和测色仪520可以分立设置，校准模块530可以独立设置，也可以设置于显示终端510内部。In one embodiment, the display terminal 510 and the colorimeter 520 can be set up separately, and the calibration module 530 can be set up independently, or can be set up inside the display terminal 510 .
所述显示终端510用于显示第一图像数据；The display terminal 510 is used to display the first image data;
所述测色仪520用于获取第一图像数据的显示图像的颜色数据；The colorimeter 520 is used to obtain the color data of the display image of the first image data;
示例性的，显示终端510具有显示功能，可以将第一图像数据以图像的形式显示。在一个实施例中，第一图像数据的显示图像是显示终端510基于第一图像数据进行显示的显示图像。For example, the display terminal 510 has a display function and can display the first image data in the form of an image. In one embodiment, the display image of the first image data is a display image displayed by the display terminal 510 based on the first image data.
示例性的，测色仪520可以为色度检测仪或光谱仪等，测色仪用于测量第一图像数据的显示图像的RGB数值或色度值等测量参数数值。第一图像数据的显示图像是指显示终端510在设备自身和/或外界环境(如外界光)等影响因素下，实际显示第一图像数据所得的画 面，是用户可见的或其他设备可采集到的显示终端510在实际情况下显示的画面。第一图像数据的显示图像的颜色数据可以包括如色相数据、饱和度数据和亮度数据等用于表示图像颜色的量化数据。For example, the colorimeter 520 may be a colorimeter detector or a spectrometer, etc. The colorimeter is used to measure measurement parameter values such as RGB values or chromaticity values of the display image of the first image data. The display image of the first image data refers to the image obtained by the display terminal 510 actually displaying the first image data under the influence of factors such as the device itself and/or the external environment (such as external light). The screen is the screen displayed by the display terminal 510 under actual circumstances that is visible to the user or can be collected by other devices. The color data of the display image of the first image data may include quantified data representing the color of the image, such as hue data, saturation data, and brightness data.
所述校准模块530用于基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系；The calibration module 530 is configured to obtain a difference between the first image data and the display image of the first image data based on the color data of the first image data and the color data of the display image of the first image data. mapping relationship;
示例性地，第一图像数据的颜色数据是获取第一图像数据时预先获得的，由于第一图像数据与第一图像数据的显示图像是相对应的，因此第一图像数据中的每个像素的位置信息和颜色数据信息都与第一图像数据的显示图像一一对应，基于对应关系可以获得所述第一图像数据和所述第一图像数据的显示图像之间的映射关系，映射关系是指所述第一图像数据和所述第一图像数据的显示图像同一位置对应的颜色数据偏差关系。For example, the color data of the first image data is obtained in advance when acquiring the first image data. Since the first image data corresponds to the display image of the first image data, each pixel in the first image data The position information and color data information are in one-to-one correspondence with the display image of the first image data. Based on the corresponding relationship, the mapping relationship between the first image data and the display image of the first image data can be obtained. The mapping relationship is Refers to the deviation relationship of color data corresponding to the same position of the first image data and the display image of the first image data.
一个实施例中，第一图像数据分辨率信息为n*m像素，以第一图像数据的中心为原点建立坐标系，以1个像素为单位间距，可以获得n*m个位置，每个位置对应一个颜色数据；同理，对第一图像数据的显示图像也可以对应获得n*m个位置，每个位置对应一个颜色数据，第一图像数据的像素位置和第一图像数据的显示图像中的位置可以一一对应，因此第一图像数据的颜色数据和第一图像数据的显示图像中的颜色数据可以一一对应，从而构建映射关系，即根据第一图像数据中预先获得的参考参数数值与通过测量第一图像数据的显示图像获得的测量参数数值，建立映射关系。第一图像数据和第一图像数据的显示图像的对应方法可以包括但不限于AI图像识别、图像等距离划分等。In one embodiment, the resolution information of the first image data is n*m pixels. A coordinate system is established with the center of the first image data as the origin. With 1 pixel as the unit spacing, n*m positions can be obtained. Each position Corresponding to one color data; similarly, n*m positions can also be obtained correspondingly to the display image of the first image data, each position corresponds to one color data, the pixel position of the first image data and the display image of the first image data The positions of can correspond one to one, so the color data of the first image data and the color data in the display image of the first image data can correspond one to one, thereby constructing a mapping relationship, that is, according to the reference parameter values obtained in advance in the first image data A mapping relationship is established with the measurement parameter value obtained by measuring the display image of the first image data. The corresponding method of the first image data and the display image of the first image data may include but is not limited to AI image recognition, image equal distance division, etc.
示例性地，第一图像数据中坐标(5,36)的像素颜色数据为饱和度30，亮度10，色值36；第一图像数据的显示图像中坐标(5,36)的像素颜色数据为饱和度35，亮度15，色值56；可以获知像素(5,36) 实际显示时与第一图像数据偏离为饱和度+5，亮度+5，色值+20，即(5,36)处映射关系为：饱和度+5，亮度+5，色值+20。以上是为便于说明本申请实施例而进行的举例，不应对本申请实施例构成任何限定。For example, the pixel color data at coordinates (5,36) in the first image data is saturation 30, brightness 10, and color value 36; the pixel color data at coordinates (5,36) in the display image of the first image data is Saturation 35, brightness 15, color value 56; pixels (5,36) can be obtained During actual display, the deviation from the first image data is saturation +5, brightness +5, and color value +20. That is, the mapping relationship at (5,36) is: saturation +5, brightness +5, and color value +20. The above are examples for convenience of describing the embodiments of the present application, and should not constitute any limitation to the embodiments of the present application.
所述校准模块530还用于基于所述映射关系对所述显示终端进行颜色校准。The calibration module 530 is also used to perform color calibration on the display terminal based on the mapping relationship.
示例性地，颜色校准是指将实际显示的颜色数据修正为真实的颜色，真实的颜色是指数据生成时的颜色，即未受到设备自身或外界环境影响的颜色。如可以对显示终端进行光度补偿、白平衡补偿和色度补偿等。示例性地，对于像素(5,36)，可以基于映射关系，对显示终端中的像素(5,36)相同的颜色进行颜色调整：饱和度-5，亮度-5，色值-20，使得其实际显示颜色与真实颜色相同。示例性地，以光谱数据的形式传输图像的情况下，校准模块530可以基于映射关系对光谱数据进行调整，如对与像素(5,36)对应的颜色的光谱数据进行调整，使显示终端显示的显示图像更接近真实颜色。For example, color calibration refers to correcting the actual displayed color data to a real color. The real color refers to the color when the data is generated, that is, the color that is not affected by the device itself or the external environment. For example, photometric compensation, white balance compensation, and chromaticity compensation can be performed on the display terminal. For example, for pixel (5,36), the same color as pixel (5,36) in the display terminal can be adjusted based on the mapping relationship: saturation -5, brightness -5, color value -20, such that Its actual display color is the same as the real color. For example, when the image is transmitted in the form of spectral data, the calibration module 530 can adjust the spectral data based on the mapping relationship, such as adjusting the spectral data of the color corresponding to the pixel (5, 36), so that the display terminal displays The display image is closer to the real color.
本申请实施例提供的用于远程光谱成像系统的终端检测系统，以第一图像数据作为颜色参考，获取显示终端呈现的第一图像数据的显示图像，将两者进行对比，建立映射关系，基于映射关系对显示终端进行校准和误差消除，避免了显示终端的颜色显示误差，进一步提升成像系统的精准度。The terminal detection system for a remote spectral imaging system provided by embodiments of the present application uses the first image data as a color reference to obtain the display image of the first image data presented by the display terminal, compares the two, and establishes a mapping relationship based on The mapping relationship calibrates and eliminates errors on the display terminal, avoiding color display errors on the display terminal and further improving the accuracy of the imaging system.
在一个实施例中，以上所述颜色数据为色度值、RGB值和光谱数据中的任意一种。In one embodiment, the above-mentioned color data is any one of chromaticity values, RGB values, and spectral data.
一个实施例中，测色仪520测量第一图像数据的显示图像的色度值；测色仪520也可以是光谱仪，测色仪520直接测量第一图像数据的显示图像对应的图像光谱曲线(光谱数据)，再根据所述测量参数数值(第一图像数据的显示图像的颜色数据)与所述参考参数数值(第一图像数据的颜色数据)建立映射关系，再根据所述映射关系对远程 光谱成像系统的光谱数据进行校准，远程光谱成像系统的光谱数据用于生成目标图像，再通过所述显示终端显示所述目标图像，从而消除显示终端带来的成像误差。In one embodiment, the colorimeter 520 measures the chromaticity value of the displayed image of the first image data; the colorimeter 520 may also be a spectrometer, and the colorimeter 520 directly measures the image spectrum curve corresponding to the displayed image of the first image data ( Spectral data), and then establish a mapping relationship based on the measurement parameter value (color data of the display image of the first image data) and the reference parameter value (color data of the first image data), and then perform remote mapping based on the mapping relationship. The spectral data of the spectral imaging system is calibrated, the spectral data of the remote spectral imaging system is used to generate a target image, and then the target image is displayed through the display terminal, thereby eliminating imaging errors caused by the display terminal.
需要说明的是，第一图像数据的显示图像的颜色数据和第一图像数据的颜色数据不一定是同一种类型参数，例如，所述测量仪为屏幕测色仪的情况下，测量第一图像数据的显示图像的色度值，而第一图像数据提供的参考参数数值为RGB数值，则可以建立所述已知的RGB数值与所测量的色度值之间的映射关系，再通过所述映射关系去调整所述光谱数据，使显示终端显示的显示图像更接近真实颜色。It should be noted that the color data of the display image of the first image data and the color data of the first image data are not necessarily the same type of parameters. For example, when the measuring instrument is a screen colorimeter, the first image is measured. The data displays the chromaticity value of the image, and the reference parameter value provided by the first image data is the RGB value, then the mapping relationship between the known RGB value and the measured chromaticity value can be established, and then through the The mapping relationship is used to adjust the spectral data so that the display image displayed on the display terminal is closer to the real color.
在一个实施例中，所述显示终端还用于接收并显示目标图像。In one embodiment, the display terminal is also used to receive and display the target image.
一个实施例中，远程光谱成像系统中经过画面采集、传输和颜色处理后可以获得目标图像，显示终端可以接收经过处理后重建得到的目标图像；一个实施例中，远程光谱成像系统中经过画面采集和传输，将采集到的光谱数据传输给显示终端，校准模块530可以基于映射关系对光谱数据进行调整后，显示终端510基于调整后的光谱数据重建目标图像，并显示目标图像，使得显示的目标图像接近真实颜色。In one embodiment, the target image can be obtained after screen collection, transmission and color processing in the remote spectral imaging system, and the display terminal can receive the target image reconstructed after processing; in one embodiment, the target image can be obtained through screen collection in the remote spectral imaging system. and transmit the collected spectral data to the display terminal. After the calibration module 530 can adjust the spectral data based on the mapping relationship, the display terminal 510 reconstructs the target image based on the adjusted spectral data and displays the target image, so that the displayed target The image is close to the real color.
在一个实施例中，所述显示终端包括第一显示单元和第二显示单元：In one embodiment, the display terminal includes a first display unit and a second display unit:
所述第一显示单元用于显示所述目标图像；The first display unit is used to display the target image;
所述第二显示单元用于显示所述第一图像数据。The second display unit is used to display the first image data.
示例性地，第一显示单元和第二显示单元可以为两个独立的子显示终端，第一显示单元和第二显示单元的显示屏幕特性应为一致(或满足预设一致性阈值)，即需要所述第一显示单元和第二显示单元对应的显示屏幕的性能较为接近。在一个实施例中，可预先对第一显示单元和第二显示单元进行测试和显示参数调整。For example, the first display unit and the second display unit may be two independent sub-display terminals, and the display screen characteristics of the first display unit and the second display unit should be consistent (or meet a preset consistency threshold), that is, The performance of the display screens corresponding to the first display unit and the second display unit needs to be relatively close. In one embodiment, the first display unit and the second display unit may be tested and display parameters adjusted in advance.
一个实施例中，所述测色仪通过拍摄所述第二显示单元显示的第一图像数据，获取对应的测量参数数值，再基于已知的参考参数数值 获取实时映射关系，通过实时映射关系对光谱数据进行实时调整，再用第一显示单元进行显示，从而降低显示系统和环境光对光谱成像的影响。所述显示终端可以实施为显示屏幕。In one embodiment, the colorimeter obtains the corresponding measurement parameter value by photographing the first image data displayed by the second display unit, and then based on the known reference parameter value The real-time mapping relationship is obtained, the spectral data is adjusted in real time through the real-time mapping relationship, and then displayed on the first display unit, thereby reducing the impact of the display system and ambient light on spectral imaging. The display terminal may be implemented as a display screen.
本申请实施例利用第一图像数据作为对成像系统的检测标准，实现对显示系统的成像误差及环境光偏差的校准。利用第一图像数据建立显示终端实际成像色度值和参考标准的映射关系，并根据该映射关系调整显示；进一步地，在成像显示过程中设置显示单元用于第一图像数据的实时检测调整。Embodiments of the present application use the first image data as a detection standard for the imaging system to achieve calibration of the imaging error and ambient light deviation of the display system. The first image data is used to establish a mapping relationship between the actual imaging chromaticity value of the display terminal and the reference standard, and the display is adjusted according to the mapping relationship; further, during the imaging display process, a display unit is set for real-time detection and adjustment of the first image data.
在一个实施例中，所述第一显示单元为显示终端中的第一分屏区域，所述第二显示单元为显示终端中的第二分屏区域。In one embodiment, the first display unit is a first split-screen area in the display terminal, and the second display unit is a second split-screen area in the display terminal.
示例性地，第一显示单元和第二显示单元优选地实施为所述显示终端的两个区域，即对同一显示终端进行分屏或设置一区域用以显示第一图像数据。本申请实施例对分屏区域大小不作限定。Illustratively, the first display unit and the second display unit are preferably implemented as two areas of the display terminal, that is, the same display terminal is divided into screens or an area is set to display the first image data. The embodiment of the present application does not limit the size of the split-screen area.
本申请实施例提供的用于远程挂光谱成像系统的终端检测系统，通过对同一显示终端进行分屏，避免了第一显示单元和第二显示单元分立设置导致两个显示单元参数不同，而引入新的显示误差，保持了显示第一图像数据和显示第一图像数据的显示图像的设备一致性，进一步提升成像系统的精准度。The terminal detection system for remotely mounted spectral imaging system provided by the embodiment of the present application avoids the introduction of different parameters of the two display units due to the separate setting of the first display unit and the second display unit by splitting the screen of the same display terminal. The new display error maintains the consistency of the device displaying the first image data and the display image displaying the first image data, further improving the accuracy of the imaging system.
下面对本申请提供的用于远程光谱成像系统的终端检测方法进行描述，下文描述的用于远程光谱成像系统的终端检测方法与上文描述的用于远程光谱成像系统的终端检测系统可相互对应参照。The terminal detection method for a remote spectrum imaging system provided by this application is described below. The terminal detection method for a remote spectrum imaging system described below and the terminal detection system for a remote spectrum imaging system described above can correspond to each other. .
下面结合图6－图8描述本申请实施例提供的用于远程光谱成像系统的终端检测方法。The terminal detection method for the remote spectral imaging system provided by the embodiment of the present application is described below with reference to FIGS. 6-8 .
图6是本申请实施例提供的用于远程光谱成像系统的终端检测方法的流程示意图之一，如图6所示，本申请实施例提供一种用于远程光谱成像系统的终端检测方法，包括：Figure 6 is one of the flow diagrams of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 6, an embodiment of the present application provides a terminal detection method for a remote spectral imaging system, including :
步骤610，获取第一图像数据的显示图像的颜色数据； Step 610: Obtain the color data of the display image of the first image data;
步骤620，基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系；Step 620: Based on the color data of the first image data and the color data of the display image of the first image data, obtain the mapping relationship between the first image data and the display image of the first image data;
步骤630，基于所述映射关系对显示终端进行颜色校准。Step 630: Perform color calibration on the display terminal based on the mapping relationship.
对于第一图像数据、第一图像数据的显示图像和映射关系的介绍参考上文所述，此处不再赘述。For the introduction of the first image data, the display image of the first image data and the mapping relationship, refer to the above description and will not be described again here.
本申请实施例提供的用于远程光谱成像系统的终端检测方法，以第一图像数据作为颜色参考，获取显示终端实际呈现的第一图像数据的显示图像，将两者进行对比，建立映射关系，基于映射关系对显示终端进行校准和误差消除，避免了显示终端的颜色显示误差，进一步提升成像系统的精准度。The terminal detection method for a remote spectral imaging system provided by embodiments of the present application uses the first image data as a color reference to obtain a display image of the first image data actually presented by the display terminal, compare the two, and establish a mapping relationship. The display terminal is calibrated and errors are eliminated based on the mapping relationship, which avoids color display errors on the display terminal and further improves the accuracy of the imaging system.
在此需要说明的是，本申请实施例提供的上述方法，与上述系统实施例所实现的所有功能相对应，且能够达到相同的技术效果，在此不再对本实施例中与系统实施例相同的部分及有益效果进行具体赘述。It should be noted here that the above methods provided by the embodiments of the present application correspond to all the functions implemented by the above system embodiments and can achieve the same technical effects. The same as those in the system embodiments in this embodiment will no longer be used. The parts and beneficial effects will be described in detail.
下面，对上述步骤在具体实施例中的可能的实现方式做进一步说明。Possible implementations of the above steps in specific embodiments are further described below.
在一个实施例中，所述颜色数据为色度值、RGB值和光谱数据中的任意一种。In one embodiment, the color data is any one of chromaticity values, RGB values, and spectral data.
关于第一图像数据的显示图像的颜色数据和第一图像数据的颜色数据的介绍，参考上文所述，此处不再赘述。Regarding the introduction of the color data of the display image of the first image data and the color data of the first image data, refer to the above description and will not be described again here.
在一个实施例中，所述目标图像是基于目标成像对象的光谱数据获得的。In one embodiment, the target image is obtained based on spectral data of the target imaging object.
示例性地，目标成像对象为待传输画面的实际场景，即待传输画面(在成像前可以称为待成像画面，两者是同一的)未受影响的真实画面。目标成像对象的光谱数据可以通过本申请实施例介绍的快照式光谱芯片获取，参照上文以及对图3和图4的介绍，此处不再赘述。 For example, the target imaging object is the actual scene of the picture to be transmitted, that is, the real picture that is not affected by the picture to be transmitted (before imaging, it can be called the picture to be imaged, the two are the same). The spectral data of the target imaging object can be obtained through the snapshot spectrum chip introduced in the embodiment of the present application. Refer to the above and the introduction of Figures 3 and 4, which will not be described again here.
在一个实施例中，所述目标图像是通过对所述光谱数据进行颜色处理获得的；和/或In one embodiment, the target image is obtained by color processing the spectral data; and/or
所述目标图像是通过不低于8bit/s的数据率进行数据传输获得的。The target image is obtained through data transmission at a data rate of no less than 8 bit/s.
示例性地，对所述光谱数据进行颜色处理参照上文对“对传输的光谱数据进行颜色处理”的介绍，目标图像是通过不低于8bit/s的数据率进行数据传输获得的参照上文对“对所述光谱数据进行高比特数据传输”的介绍，此处不再赘述。Illustratively, for performing color processing on the spectral data, refer to the above introduction to "Color processing of transmitted spectral data". The target image is obtained through data transmission at a data rate of no less than 8 bit/s. Refer to the above description. The introduction of "high-bit data transmission of the spectral data" will not be described again here.
本申请实施例对颜色处理和数据传输的先后顺序不做限定。The embodiment of the present application does not limit the order of color processing and data transmission.
在一个实施例中，图7是本申请实施例提供的用于远程光谱成像系统的终端检测方法的流程示意图之二，如图7所示，所述获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系之后，还包括：In one embodiment, FIG. 7 is a schematic flow chart of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in FIG. 7 , the first image data and the third image data are obtained. After displaying the mapping relationship between images, the image data also includes:
显示终端接收目标图像；The display terminal receives the target image;
所述基于所述映射关系对显示终端进行颜色校准，之后还包括：The color calibration of the display terminal based on the mapping relationship also includes:
所述显示终端显示所述目标图像。The display terminal displays the target image.
一个实施例中，本申请实施例中可以预先设置并由显示终端显示第一图像数据，并通过图6所示的用于远程光谱成像系统的终端检测方法，预先获得映射关系。在画面采集端，利用光谱芯片对待传输画面的实际场景进行光谱图像数据(即光谱数据)获取，对所述光谱数据进行高比特数据传输，对传输的光谱数据进行颜色处理，获得已进行颜色处理的图像数据(即目标图像)，对已进行颜色处理的图像数据根据映射关系进行调整，再编码和显示目标图像。In one embodiment, the first image data can be preset and displayed on the display terminal in the embodiment of the present application, and the mapping relationship can be obtained in advance through the terminal detection method for the remote spectral imaging system shown in FIG. 6 . At the picture acquisition end, a spectrum chip is used to obtain spectral image data (i.e., spectral data) of the actual scene of the picture to be transmitted, high-bit data transmission is performed on the spectral data, color processing is performed on the transmitted spectral data, and the color processed data is obtained The image data (that is, the target image) is adjusted according to the mapping relationship according to the color-processed image data, and then the target image is encoded and displayed.
需要说明的是，本申请实施例中的映射关系是预先获得的，预先获得映射关系，主要是基于显示终端本身显示会存在偏差、以及获取映射关系时的环境光对成像的影响构建的，在显示过程中显示终端本身的性能变动不大的情况下，显示终端本身带来的偏差较为稳定，若环境光也比较稳定的情况下，所述映射关系只需要在成像显示前获得， 即可应用于后续显示终端的颜色校正，如显示终端显示远程光谱成像或光谱视频的场景中的颜色校正。It should be noted that the mapping relationship in the embodiment of the present application is obtained in advance. The pre-obtained mapping relationship is mainly based on the display deviation of the display terminal itself and the impact of ambient light on imaging when obtaining the mapping relationship. When the performance of the display terminal itself does not change much during the display process, the deviation caused by the display terminal itself is relatively stable. If the ambient light is also relatively stable, the mapping relationship only needs to be obtained before imaging and display. It can be applied to color correction of subsequent display terminals, such as color correction in scenarios where the display terminal displays remote spectral imaging or spectral video.
不同型号的显示终端根据相同图像数据(包括空间信息和光谱信息)显示出的图像颜色可能会有偏差，与实际场景的人眼感知颜色也存在一定的偏差；同一显示终端在不同环境光下，最终的显示颜色效果也会有所不同。本申请实施例提供的用于远程光谱成像系统的终端检测方法，对第一图像数据的显示图像进行测量，根据第一图像数据获取映射关系，能够排除显示终端本身显示性能和环境光对显示终端成像效果的影响。The image color displayed by different models of display terminals based on the same image data (including spatial information and spectral information) may deviate, and there is also a certain deviation from the color perceived by the human eye in the actual scene; the same display terminal under different ambient lights, Final display color effects will also vary. The terminal detection method for the remote spectral imaging system provided by the embodiment of the present application measures the display image of the first image data and obtains the mapping relationship according to the first image data, which can eliminate the effects of the display terminal itself and the ambient light on the display terminal. Imaging effect.
但是实际应用中，某些应用场景中环境光保持不变难度比较大，因此，映射关系需要根据环境光等变动进行实时调整，才不会影响光谱成像，尤其是光谱视频输出。即不考虑显示系统和环境光的变动，在显示过程中，所述映射关系可以保持不变；若需要考虑显示系统和环境光的变动则需要时刻保持对映射关系的调整。在一个实施例中，图8是本申请实施例提供的用于远程光谱成像系统的终端检测方法的流程示意图之三，如图8所示，所述第一图像数据由第二显示单元显示；However, in actual applications, it is difficult to keep the ambient light constant in some application scenarios. Therefore, the mapping relationship needs to be adjusted in real time according to changes in ambient light, etc., so as not to affect spectral imaging, especially spectral video output. That is, regardless of changes in the display system and ambient light, the mapping relationship can remain unchanged during the display process; if changes in the display system and ambient light need to be considered, the mapping relationship needs to be adjusted at all times. In one embodiment, Figure 8 is a third schematic flowchart of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 8, the first image data is displayed by the second display unit;
所述方法还包括：The method also includes:
显示终端接收目标图像，所述目标图像由第一显示单元显示；The display terminal receives a target image, and the target image is displayed by the first display unit;
所述获取第一图像数据的显示图像的颜色数据包括：The obtaining the color data of the display image of the first image data includes:
获取所述第一图像数据的显示图像的当前颜色数据；Obtain the current color data of the display image of the first image data;
所述基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系，包括：Obtaining a mapping relationship between the first image data and the display image of the first image data based on the color data of the first image data and the color data of the display image of the first image data includes: :
基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的当前颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的实时映射关系； Based on the color data of the first image data and the current color data of the display image of the first image data, obtain a real-time mapping relationship between the first image data and the display image of the first image data;
所述基于所述映射关系对显示终端进行颜色校准包括：The color calibration of the display terminal based on the mapping relationship includes:
基于所述实时映射关系对所述显示终端进行实时颜色校准。Perform real-time color calibration on the display terminal based on the real-time mapping relationship.
示例性地，对于第一显示单元和第二显示单元的介绍，参考上文所述，此处不再赘述。For example, for the introduction of the first display unit and the second display unit, refer to the above description and will not be described again here.
本申请实施例中由第二显示单元显示第一图像数据，在显示终端参数设置和环境光影响下，第一图像数据的显示图像的颜色数据可能存在变化，获取所述第一图像数据的显示图像的当前颜色数据是指实时获取第一图像数据的显示图像的颜色数据；示例性地，需要对显示终端进行颜色校正时，获取第一图像数据的显示图像的颜色数据。实时映射关系是基于实时获得的第一图像数据的显示图像的当前颜色数据和第一图像数据的颜色数据获得的，能够反应当前时刻第一图像数据的显示图像和第一图像数据的显示偏差。In the embodiment of the present application, the first image data is displayed by the second display unit. Under the influence of display terminal parameter settings and ambient light, the color data of the displayed image of the first image data may change. Obtaining the display of the first image data The current color data of the image refers to the color data of the display image of the first image data obtained in real time; for example, when color correction needs to be performed on the display terminal, the color data of the display image of the first image data is obtained. The real-time mapping relationship is obtained based on the current color data of the display image of the first image data and the color data of the first image data obtained in real time, and can reflect the display deviation of the display image of the first image data and the first image data at the current moment.
一个实施例中，在画面采集端，利用光谱芯片对待传输画面的实际场景进行光谱图像数据(即光谱数据)获取，对所述光谱数据进行高比特数据传输，对传输的光谱数据进行颜色孤立，获得已进行颜色处理的图像数据(即目标图像)。另一方面，在第一显示单元显示高保真传输图像，当前显示的高保真传输图像可以为上一时刻的目标图像；在第二显示单元显示第一图像数据，通过图6所示的用于远程光谱成像系统的终端检测方法，实时获得第一图像数据与第一图像数据的显示图像之间的实时映射关系。通过本申请实施例，可以同时获得已进行颜色处理的图像数据和实时映射关系，对已进行颜色处理的图像数据根据实时映射关系进行调整，再编码和显示目标图像。In one embodiment, at the picture acquisition end, a spectrum chip is used to obtain spectral image data (i.e., spectral data) of the actual scene of the picture to be transmitted, high-bit data transmission is performed on the spectral data, and color isolation is performed on the transmitted spectral data. Obtain the color-processed image data (i.e., the target image). On the other hand, the high-fidelity transmission image is displayed on the first display unit, and the currently displayed high-fidelity transmission image may be the target image at the previous moment; the first image data is displayed on the second display unit, and the high-fidelity transmission image shown in Figure 6 is used to display the first image data. The terminal detection method of the remote spectral imaging system obtains the real-time mapping relationship between the first image data and the display image of the first image data in real time. Through the embodiments of the present application, the color-processed image data and the real-time mapping relationship can be obtained at the same time, the color-processed image data can be adjusted according to the real-time mapping relationship, and then the target image can be encoded and displayed.
不同型号显示终端根据相同图像数据(包括空间信息和光谱信息)显示出的图像颜色可能会有偏差，与实际场景的人眼感知颜色也存在一定的偏差；同一显示终端在不同环境光下，最终的显示颜色效果也会有所不同。本申请实施例提供的用于远程光谱成像系统的终端检测方法，由第二显示单元显示第一图像数据，根据第一图像数据和第一 图像数据的显示图像，获取实时映射关系，能够实时排除显示终端本身显示性能和环境光对成像效果的影响。The image colors displayed by different models of display terminals based on the same image data (including spatial information and spectral information) may deviate, and there is also a certain deviation from the color perceived by the human eye in the actual scene; the same display terminal under different ambient lights will ultimately The display color effects will also be different. In the terminal detection method for a remote spectral imaging system provided by embodiments of the present application, the first image data is displayed by the second display unit. According to the first image data and the first The display image of the image data can obtain the real-time mapping relationship, which can eliminate the influence of the display performance of the display terminal itself and the ambient light on the imaging effect in real time.
下面对本申请提供的远程光谱成像方法进行描述，下文描述的远程光谱成像方法与上文描述的远程光谱成像系统可相互对应参照。The long-range spectral imaging method provided by the present application is described below. The long-range spectral imaging method described below and the long-range spectral imaging system described above can be mutually referenced.
图9是本申请实施例提供的远程光谱成像方法的流程示意图，如图9所示，本申请实施例提供一种基于上述远程光谱成像系统进行的远程光谱成像方法，包括：Figure 9 is a schematic flowchart of a remote spectral imaging method provided by an embodiment of the present application. As shown in Figure 9, an embodiment of the present application provides a remote spectral imaging method based on the above-mentioned remote spectral imaging system, including:
步骤910，基于光谱成像(尤其是计算光谱成像技术)对待成像场景进行拍摄，获得第一图像数据；Step 910: Shoot the scene to be imaged based on spectral imaging (especially computational spectral imaging technology) to obtain first image data;
步骤920，显示所述第一图像数据，基于所述第一图像数据的显示图像和所述第一图像数据，获取图像色差值；Step 920: Display the first image data, and obtain an image color difference value based on the display image of the first image data and the first image data;
步骤930，基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据。Step 930: Perform color difference compensation on the first image data based on the image color difference value to obtain second image data.
在一个实施例中，所述基于所述第一图像数据的显示图像和所述第一图像数据，获取图像色差值包括：In one embodiment, obtaining the image color difference value based on the display image of the first image data and the first image data includes:
对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值。The display image of the first image data is photographed to obtain a comparison image, and the image color difference value is obtained based on the comparison image and the first image data.
在一个实施例中，所述对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值包括：In one embodiment, photographing the display image of the first image data, obtaining a comparison image, and obtaining the image color difference value based on the comparison image and the first image data includes:
基于光谱成像对所述第一图像数据的显示图像进行拍摄，获得所述对比图像；Capture the display image of the first image data based on spectral imaging to obtain the comparison image;
对所述对比图像和所述第一图像数据进行图像匹配，获得匹配像素点集；所述匹配像素点集中包括多个像素点对，每个所述像素点对包括所述第一图像数据中的第一像素点和所述对比图像中的第二像素点；Perform image matching on the comparison image and the first image data to obtain a matching pixel point set; the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes a pair of pixel points in the first image data. The first pixel point and the second pixel point in the comparison image;
获取每个所述像素点对中的第一像素点与第二像素点的色差值， 获得图像色差值。Obtain the color difference value of the first pixel point and the second pixel point in each pair of pixel points, Get the image color difference value.
在一个实施例中，所述基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，包括：In one embodiment, performing color difference compensation on the first image data based on the image color difference value to obtain the second image data includes:
基于所述图像色差值对每个所述像素点对中的第一像素点进行色差补偿，获得第二图像数据。Color difference compensation is performed on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
在一个实施例中，所述图像色差值为RGB值。In one embodiment, the image color difference values are RGB values.
在一个实施例中，所述基于光谱成像对待成像场景进行拍摄，获得第一图像数据，包括：In one embodiment, the scene to be imaged is photographed based on spectral imaging to obtain first image data, including:
获取待成像场景的光谱数据；Obtain spectral data of the scene to be imaged;
对所述光谱数据进行颜色处理，获得颜色数据；Perform color processing on the spectral data to obtain color data;
对所述颜色数据进行编码获得第一图像数据。The color data is encoded to obtain first image data.
在一个实施例中，所述对所述光谱数据进行颜色处理，获得颜色数据，包括：In one embodiment, performing color processing on the spectral data to obtain color data includes:
基于广色域标准对所述入射光光谱进行颜色处理，获得所述颜色数据。Color processing is performed on the incident light spectrum based on a wide color gamut standard to obtain the color data.
在一个实施例中，所述方法包括以下任一数据传输步骤：In one embodiment, the method includes any of the following data transmission steps:
第一传输模块向颜色处理模块发送所述光谱成像装置获得的所述光谱数据；The first transmission module sends the spectral data obtained by the spectral imaging device to the color processing module;
第二传输模块向编码模块发送所述颜色处理模块获得的所述颜色数据；The second transmission module sends the color data obtained by the color processing module to the encoding module;
第三传输模块向远程显示子系统发送所述编码模块获得的第一图像数据。The third transmission module sends the first image data obtained by the encoding module to the remote display subsystem.
在一个实施例中，所述数据传输步骤中以不低于8bit/s的数据率进行数据传输。In one embodiment, in the data transmission step, data transmission is performed at a data rate of no less than 8 bit/s.
需要说明的是，本申请所述图像色差值既可以是根据任一所述第一图像数据和对应的所述显示图像进行实时的调整，从而获得所述第二图像数据。亦可以，通过一次或多次测量获得所述图像色差值，再 根据所述图像色差值对任一所述第一图像数据进行补偿，从而获得对应的第二图像数据。即所述图像色差值可以是定值亦可以是动态变动。It should be noted that the image color difference value described in this application can be adjusted in real time according to any of the first image data and the corresponding display image, thereby obtaining the second image data. Alternatively, the image color difference value can be obtained through one or more measurements, and then Any first image data is compensated according to the image color difference value to obtain corresponding second image data. That is, the image color difference value can be a fixed value or a dynamic change.
需要说明的是，本申请中所述第一图像数据和所述第二图像数据可以在不同的远程显示子系统上得以显示，即远程显示子系统可以实施为两个屏幕去实现显示，第一屏幕用以显示第一图像数据，第二屏幕用以显示最终所需要的高保真的第二图像数据，优选地第一屏幕和第二屏幕的性能接近，即对相同图像在相同环境下显示效果接近。第一屏幕和第二屏幕也可以为同一屏幕的不同区域。It should be noted that the first image data and the second image data in this application can be displayed on different remote display subsystems, that is, the remote display subsystem can be implemented as two screens to achieve display. The first The screen is used to display the first image data, and the second screen is used to display the final required high-fidelity second image data. Preferably, the performance of the first screen and the second screen is close, that is, the same image can be displayed in the same environment. near. The first screen and the second screen may also be different areas of the same screen.
本申请实施例提供的远程光谱成像方法利用光谱成像直接获取待成像场景的空间光谱信息，避免了RGB彩色相机等传统相机对光谱维度信息获取时降维所导致的信息丢失，以及不同光照和环境色温下图像白平衡的偏差，同时在画面重建端(远程显示子系统)进行实际显示效果的校准匹配，进一步提升远程图像的画面色彩还原精度，最大程度地还原待成像场景的空间颜色信息。The remote spectral imaging method provided by the embodiment of the present application uses spectral imaging to directly obtain the spatial spectral information of the scene to be imaged, avoiding the information loss caused by dimensionality reduction when obtaining spectral dimension information by traditional cameras such as RGB color cameras, as well as different lighting and environments. Deviations in image white balance under color temperature, and at the same time, the actual display effect is calibrated and matched at the image reconstruction end (remote display subsystem) to further improve the accuracy of image color restoration of remote images and restore the spatial color information of the scene to be imaged to the greatest extent.
在此需要说明的是，本申请实施例提供的上述方法，能够实现上述系统实施例所实现的所有方法步骤，且能够达到相同的技术效果，在此不再对本实施例中与系统实施例相同的部分及有益效果进行赘述。It should be noted here that the above method provided by the embodiment of the present application can implement all the method steps implemented by the above system embodiment, and can achieve the same technical effect. The same as the system embodiment in this embodiment will no longer be used. The parts and beneficial effects will be described in detail.
图10示例了一种电子设备的实体结构示意图，如图10所示，该电子设备可以包括：处理器(processor)1010、通信接口(Communications Interface)1020、存储器(memory)1030和通信总线1040，其中，处理器1010，通信接口1020，存储器1030通过通信总线1040完成相互间的通信。处理器1010可以调用存储器1030中的逻辑指令，以执行上述任一种方法。Figure 10 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 10, the electronic device may include: a processor (processor) 1010, a communications interface (Communications Interface) 1020, a memory (memory) 1030 and a communication bus 1040. Among them, the processor 1010, the communication interface 1020, and the memory 1030 complete communication with each other through the communication bus 1040. The processor 1010 can call logic instructions in the memory 1030 to perform any of the above methods.
此外，上述的存储器1030中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者 说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。In addition, the above-mentioned logical instructions in the memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of this application is essentially or The part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions to enable a computer device (which can be a personal computer). Computer, server, or network device, etc.) executes all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .
另一方面，本申请还提供一种计算机程序产品，所述计算机程序产品包括计算机程序，计算机程序可存储在非暂态计算机可读存储介质上，所述计算机程序被处理器执行时，计算机能够执行上述任一方法。On the other hand, the present application also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can Perform any of the above methods.
又一方面，本申请还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现以执行上述各方法提供的以执行上述任一方法。In another aspect, the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. The computer program is implemented when executed by a processor to perform any of the above methods.
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所 述的方法。Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the part of the above technical solution that essentially contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute various embodiments or certain parts of the embodiments. method described.
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。 Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the scope of the technical solutions of the embodiments of the present application.

Claims (28)

  1. 一种远程光谱成像系统，包括：图像生成子系统和远程显示子系统；A remote spectral imaging system, including: an image generation subsystem and a remote display subsystem;
    所述图像生成子系统用于基于光谱成像获取待成像场景的第一图像数据，并向所述远程显示子系统发送所述第一图像数据，由所述远程显示子系统显示所述第一图像数据；The image generation subsystem is configured to obtain first image data of the scene to be imaged based on spectral imaging, and send the first image data to the remote display subsystem, and the remote display subsystem displays the first image. data;
    基于所述第一图像数据和所述第一图像数据的显示图像获取所述第一图像数据的显示图像与所述第一图像数据的图像色差值，所述第一图像数据的显示图像是所述第一图像数据在所述远程显示子系统上的显示图像；The image color difference value of the display image of the first image data and the first image data is obtained based on the first image data and the display image of the first image data, and the display image of the first image data is The display image of the first image data on the remote display subsystem;
    基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，并由所述远程显示子系统显示所述第二图像数据。Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data, and the second image data is displayed by the remote display subsystem.
  2. 根据权利要求1所述的远程光谱成像系统，还包括色差检测子系统；The remote spectral imaging system according to claim 1, further comprising a color difference detection subsystem;
    所述色差检测子系统用于对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值。The color difference detection subsystem is used to shoot the display image of the first image data, obtain a comparison image, and obtain the image color difference value based on the comparison image and the first image data.
  3. 根据权利要求2所述的远程光谱成像系统，其中，所述色差检测子系统包括光谱成像模块、图像匹配模块和色差计算模块；The remote spectral imaging system according to claim 2, wherein the color difference detection subsystem includes a spectrum imaging module, an image matching module and a color difference calculation module;
    所述光谱成像模块用于基于光谱成像对所述第一图像数据的显示图像进行拍摄，获得所述对比图像；The spectral imaging module is configured to capture the display image of the first image data based on spectral imaging to obtain the comparison image;
    所述图像匹配模块用于对所述对比图像和所述第一图像数据进行图像匹配，获得匹配像素点集；所述匹配像素点集中包括多个像素点对，每个所述像素点对包括所述第一图像数据中的第一像素点和所述对比图像中的第二像素点；The image matching module is used to perform image matching on the comparison image and the first image data to obtain a matching pixel point set; the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes The first pixel point in the first image data and the second pixel point in the comparison image;
    所述色差计算单元用于获取每个所述像素点对中的第一像素点 与第二像素点的色差值，获得所述图像色差值。The color difference calculation unit is used to obtain the first pixel point in each pair of pixel points. and the color difference value of the second pixel point to obtain the image color difference value.
  4. 根据权利要求3所述的远程光谱成像系统，其中，所述远程显示子系统包括显示模块和色差补偿模块；The remote spectral imaging system according to claim 3, wherein the remote display subsystem includes a display module and a color difference compensation module;
    所述显示模块用于显示所述第一图像数据和所述第二图像数据；The display module is used to display the first image data and the second image data;
    所述色差补偿模块用于基于所述图像色差值对每个所述像素点对中的所述第一像素点进行色差补偿，获得所述第二图像数据。The color difference compensation module is configured to perform color difference compensation on the first pixel point in each of the pixel point pairs based on the image color difference value to obtain the second image data.
  5. 根据权利要求1-4任一项所述的远程光谱成像系统，其中，所述图像色差值为RGB值。The remote spectral imaging system according to any one of claims 1 to 4, wherein the image color difference value is an RGB value.
  6. 根据权利要求1所述的远程光谱成像系统，其中，所述图像生成子系统包括光谱成像装置、颜色处理模块和编码模块；The remote spectral imaging system of claim 1, wherein the image generation subsystem includes a spectral imaging device, a color processing module and an encoding module;
    所述光谱成像装置用于获取待成像场景的光谱数据，并向所述颜色处理模块发送所述光谱数据；The spectral imaging device is used to obtain spectral data of the scene to be imaged and send the spectral data to the color processing module;
    所述颜色处理模块用于接收所述光谱数据，并对所述光谱数据进行颜色处理，获得颜色数据；所述颜色处理模块还用于向所述编码模块发送所述颜色数据；The color processing module is used to receive the spectrum data and perform color processing on the spectrum data to obtain color data; the color processing module is also used to send the color data to the encoding module;
    所述编码模块用于接收所述颜色数据，并对所述颜色数据进行编码获得所述第一图像数据。The encoding module is configured to receive the color data and encode the color data to obtain the first image data.
  7. 根据权利要求6所述的远程光谱成像系统，其中，所述光谱成像装置包括滤光结构、图像传感器和数据处理单元，所述滤光结构设置于所述图像传感器的感光路径上；所述滤光结构的透射率曲线是基于待成像场景的光谱特征确定的；The remote spectral imaging system according to claim 6, wherein the spectral imaging device includes a filter structure, an image sensor and a data processing unit, the filter structure is disposed on the photosensitive path of the image sensor; the filter The transmittance curve of the light structure is determined based on the spectral characteristics of the scene to be imaged;
    所述滤光结构用于对待成像场景的入射光进行调制，获得调制光信号；The filter structure is used to modulate the incident light of the scene to be imaged to obtain a modulated light signal;
    所述图像传感器用于接收所述调制光信号，并获得光强数据；The image sensor is used to receive the modulated light signal and obtain light intensity data;
    所述数据处理单元用于接收所述图像传感器发送的光强数据，并基于所述光强数据进行光谱恢复，获得所述待成像场景对应的所述光谱数据。 The data processing unit is configured to receive the light intensity data sent by the image sensor, and perform spectral recovery based on the light intensity data to obtain the spectral data corresponding to the scene to be imaged.
  8. 根据权利要求6所述的远程光谱成像系统，其中，所述颜色处理模块用于对所述入射光光谱进行颜色处理，获得颜色数据，包括：The remote spectral imaging system according to claim 6, wherein the color processing module is used to perform color processing on the incident light spectrum to obtain color data, including:
    所述颜色处理模块用于基于广色域标准对所述入射光光谱进行颜色处理，获得所述颜色数据。The color processing module is used to perform color processing on the incident light spectrum based on a wide color gamut standard to obtain the color data.
  9. 根据权利要求6-8任一项所述的远程光谱成像系统，还包括：数据传输模块；The remote spectral imaging system according to any one of claims 6-8, further comprising: a data transmission module;
    所述数据传输模块包括以下任一：第一传输模块、第二传输模块和第三传输模块；The data transmission module includes any of the following: a first transmission module, a second transmission module and a third transmission module;
    所述第一传输模块与所述光谱成像装置和所述颜色处理模块相连接，用于向所述颜色处理模块发送所述光谱成像装置获得的所述光谱数据；The first transmission module is connected to the spectral imaging device and the color processing module, and is used to send the spectral data obtained by the spectral imaging device to the color processing module;
    所述第二传输模块与所述颜色处理模块和所述编码模块相连接，用于向所述编码模块发送所述颜色处理模块获得的所述颜色数据；The second transmission module is connected to the color processing module and the encoding module, and is used to send the color data obtained by the color processing module to the encoding module;
    所述第三传输模块与所述编码模块和所述远程显示子系统相连接，用于向所述远程显示子系统发送所述编码模块获得的所述第一图像数据。The third transmission module is connected to the encoding module and the remote display subsystem, and is used to send the first image data obtained by the encoding module to the remote display subsystem.
  10. 根据权利要求9所述的远程光谱成像系统，其中，所述数据传输模块用于以不低于8bit/s的数据率进行数据传输。The remote spectral imaging system according to claim 9, wherein the data transmission module is used for data transmission at a data rate of no less than 8 bit/s.
  11. 根据权利要求1所述的远程光谱成像系统，其中，所述远程显示子系统包括终端检测系统，所述终端检测系统包括：显示终端、测色仪和校准模块；The remote spectral imaging system according to claim 1, wherein the remote display subsystem includes a terminal detection system, and the terminal detection system includes: a display terminal, a colorimeter and a calibration module;
    所述显示终端用于显示所述第一图像数据；The display terminal is used to display the first image data;
    所述测色仪用于获取所述第一图像数据的显示图像的颜色数据；The colorimeter is used to obtain the color data of the display image of the first image data;
    所述校准模块用于基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系；The calibration module is configured to obtain, based on the color data of the first image data and the color data of the display image of the first image data, the difference between the first image data and the display image of the first image data. Mapping relations;
    所述校准模块还用于基于所述映射关系对所述显示终端进行颜 色校准。The calibration module is also used to color the display terminal based on the mapping relationship. Color calibration.
  12. 根据权利要求11所述的远程光谱成像系统，其中，所述颜色数据为色度值、RGB值和光谱数据中的任意一种。The remote spectral imaging system according to claim 11, wherein the color data is any one of chromaticity values, RGB values and spectral data.
  13. 根据权利要求11或12所述的远程光谱成像系统，其中，所述显示终端还用于接收并显示目标图像。The remote spectral imaging system according to claim 11 or 12, wherein the display terminal is also used to receive and display the target image.
  14. 根据权利要求13所述的远程光谱成像系统，其中，所述显示终端包括第一显示单元和第二显示单元：The remote spectral imaging system according to claim 13, wherein the display terminal includes a first display unit and a second display unit:
    所述第一显示单元用于显示所述目标图像；The first display unit is used to display the target image;
    所述第二显示单元用于显示所述第一图像数据。The second display unit is used to display the first image data.
  15. 根据权利要求14所述的远程光谱成像系统，其中，所述第一显示单元为所述显示终端中的第一分屏区域，所述第二显示单元为所述显示终端中的第二分屏区域。The remote spectrum imaging system according to claim 14, wherein the first display unit is a first split-screen area in the display terminal, and the second display unit is a second split-screen area in the display terminal. area.
  16. 一种基于权利要求11-15任一项所述的远程光谱成像系统的终端检测方法，包括：A terminal detection method based on the remote spectral imaging system according to any one of claims 11-15, including:
    获取第一图像数据的显示图像的颜色数据；Obtain the color data of the display image of the first image data;
    基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系；Based on the color data of the first image data and the color data of the display image of the first image data, obtain a mapping relationship between the first image data and the display image of the first image data;
    基于所述映射关系对显示终端进行颜色校准。Color calibration is performed on the display terminal based on the mapping relationship.
  17. 根据权利要求16所述的终端检测方法，其中，所述颜色数据为色度值、RGB值和光谱数据中的任意一种。The terminal detection method according to claim 16, wherein the color data is any one of chromaticity value, RGB value and spectral data.
  18. 根据权利要求16所述的终端检测方法，其中，所述获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系之后，还包括：The terminal detection method according to claim 16, wherein after obtaining the mapping relationship between the first image data and the display image of the first image data, it further includes:
    显示终端接收目标图像；The display terminal receives the target image;
    所述基于所述映射关系对显示终端进行颜色校准，之后还包括：The color calibration of the display terminal based on the mapping relationship also includes:
    所述显示终端显示所述目标图像。 The display terminal displays the target image.
  19. 根据权利要求16所述的终端检测方法，其中，所述第一图像数据由第二显示单元显示；The terminal detection method according to claim 16, wherein the first image data is displayed by a second display unit;
    所述方法还包括：The method also includes:
    显示终端接收目标图像，所述目标图像由第一显示单元显示；The display terminal receives a target image, and the target image is displayed by the first display unit;
    所述获取第一图像数据的显示图像的颜色数据包括：The obtaining the color data of the display image of the first image data includes:
    获取所述第一图像数据的显示图像的当前颜色数据；Obtain the current color data of the display image of the first image data;
    所述基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的映射关系，包括：Obtaining a mapping relationship between the first image data and the display image of the first image data based on the color data of the first image data and the color data of the display image of the first image data includes: :
    基于所述第一图像数据的颜色数据与所述第一图像数据的显示图像的当前颜色数据，获取所述第一图像数据和所述第一图像数据的显示图像之间的实时映射关系；Based on the color data of the first image data and the current color data of the display image of the first image data, obtain a real-time mapping relationship between the first image data and the display image of the first image data;
    所述基于所述映射关系对显示终端进行颜色校准包括：The color calibration of the display terminal based on the mapping relationship includes:
    基于所述实时映射关系对所述显示终端进行实时颜色校准。Perform real-time color calibration on the display terminal based on the real-time mapping relationship.
  20. 根据权利要求18或19所述的终端检测方法，其中，所述目标图像是基于目标成像对象的光谱数据获得的。The terminal detection method according to claim 18 or 19, wherein the target image is obtained based on spectral data of the target imaging object.
  21. 根据权利要求20所述的终端检测方法，其中，所述目标图像是通过对所述光谱数据进行颜色处理获得的；和/或The terminal detection method according to claim 20, wherein the target image is obtained by color processing the spectral data; and/or
    所述目标图像是通过不低于8bit/s的数据率进行数据传输获得的。The target image is obtained through data transmission at a data rate of no less than 8 bit/s.
  22. 一种基于权利要求1-10任一项所述的远程光谱成像系统的远程光谱成像方法，包括：A remote spectral imaging method based on the remote spectral imaging system according to any one of claims 1 to 10, including:
    基于光谱成像对待成像场景进行拍摄，获得第一图像数据；Shoot the scene to be imaged based on spectral imaging to obtain the first image data;
    显示所述第一图像数据，基于所述第一图像数据的显示图像和所述第一图像数据，获取图像色差值；Display the first image data, and obtain an image color difference value based on the display image of the first image data and the first image data;
    基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据。Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data.
  23. 根据权利要求22所述的远程光谱成像方法，其中，所述基 于所述第一图像数据的显示图像和所述第一图像数据，获取图像色差值包括：The remote spectral imaging method according to claim 22, wherein the base From the display image of the first image data and the first image data, obtaining the image color difference value includes:
    对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值。The display image of the first image data is photographed to obtain a comparison image, and the image color difference value is obtained based on the comparison image and the first image data.
  24. 根据权利要求23所述的远程光谱成像方法，其中，所述对所述第一图像数据的显示图像进行拍摄，获取对比图像，基于所述对比图像和所述第一图像数据获取所述图像色差值包括：The remote spectral imaging method according to claim 23, wherein the display image of the first image data is photographed to obtain a comparison image, and the image color is obtained based on the comparison image and the first image data. Differences include:
    基于光谱成像对所述第一图像数据的显示图像进行拍摄，获得所述对比图像；Capture the display image of the first image data based on spectral imaging to obtain the comparison image;
    对所述对比图像和所述第一图像数据进行图像匹配，获得匹配像素点集；所述匹配像素点集中包括多个像素点对，每个所述像素点对包括所述第一图像数据中的第一像素点和所述对比图像中的第二像素点；Perform image matching on the comparison image and the first image data to obtain a matching pixel point set; the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes a pair of pixel points in the first image data. The first pixel point and the second pixel point in the comparison image;
    获取每个所述像素点对中的第一像素点与第二像素点的色差值，获得图像色差值。The color difference value of the first pixel point and the second pixel point in each pair of pixel points is obtained to obtain the image color difference value.
  25. 根据权利要求24所述的远程光谱成像方法，其中，所述基于所述图像色差值对所述第一图像数据进行色差补偿，获得第二图像数据，包括：The remote spectral imaging method according to claim 24, wherein the performing color difference compensation on the first image data based on the image color difference value to obtain the second image data includes:
    基于所述图像色差值对每个所述像素点对中的第一像素点进行色差补偿，获得所述第二图像数据。Color difference compensation is performed on the first pixel point in each pair of pixel points based on the image color difference value to obtain the second image data.
  26. 根据权利要求22所述的远程光谱成像方法，其中，所述基于光谱成像对待成像场景进行拍摄，获得第一图像数据，包括：The remote spectral imaging method according to claim 22, wherein the step of photographing the scene to be imaged based on spectral imaging and obtaining the first image data includes:
    获取待成像场景的光谱数据；Obtain spectral data of the scene to be imaged;
    对所述光谱数据进行颜色处理，获得颜色数据；Perform color processing on the spectral data to obtain color data;
    对所述颜色数据进行编码获得所述第一图像数据。The color data is encoded to obtain the first image data.
  27. 根据权利要求26所述的远程光谱成像方法，其中，所述对所述光谱数据进行颜色处理，获得颜色数据，包括： The remote spectral imaging method according to claim 26, wherein performing color processing on the spectral data to obtain color data includes:
    基于广色域标准对所述入射光光谱进行颜色处理，获得所述颜色数据。Color processing is performed on the incident light spectrum based on a wide color gamut standard to obtain the color data.
  28. 根据权利要求26或27所述的远程光谱成像方法，包括以下任一数据传输步骤：The remote spectral imaging method according to claim 26 or 27, including any of the following data transmission steps:
    第一传输模块向颜色处理模块发送光谱成像装置获得的所述光谱数据；The first transmission module sends the spectral data obtained by the spectral imaging device to the color processing module;
    第二传输模块向编码模块发送所述颜色处理模块获得的所述颜色数据；The second transmission module sends the color data obtained by the color processing module to the encoding module;
    第三传输模块向远程显示子系统发送所述编码模块获得的所述第一图像数据。 The third transmission module sends the first image data obtained by the encoding module to the remote display subsystem.
PCT/CN2023/083095 2022-04-29 2023-03-22 Remote spectral imaging system and method WO2023207443A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202210476278.0A CN117014586A (en) 2022-04-29 2022-04-29 Remote spectral imaging system and method
CN202210476278.0 2022-04-29
CN202210869460.2 2022-07-22
CN202210869460.2A CN117478867A (en) 2022-07-22 2022-07-22 Terminal detection system and method for imaging system and remote spectrum imaging system

Publications (1)

Publication Number Publication Date
WO2023207443A1 true WO2023207443A1 (en) 2023-11-02

Family

ID=88517281

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/083095 WO2023207443A1 (en) 2022-04-29 2023-03-22 Remote spectral imaging system and method

Country Status (1)

Country Link
WO (1) WO2023207443A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090268953A1 (en) * 2008-04-24 2009-10-29 Apteryx, Inc. Method for the automatic adjustment of image parameter settings in an imaging system
CN103765503A (en) * 2011-08-29 2014-04-30 高通股份有限公司 Fast calibration of displays using spectral-based colorimetrically calibrated multicolor camera
CN105791783A (en) * 2016-03-10 2016-07-20 广州视睿电子科技有限公司 Adjusting method and system for imaging color of camera
CN111866318A (en) * 2019-04-29 2020-10-30 北京小米移动软件有限公司 Multispectral imaging method, multispectral imaging device, mobile terminal and storage medium
CN112309317A (en) * 2020-10-22 2021-02-02 北京虚拟动点科技有限公司 Screen color difference compensation method, device and system, electronic equipment and storage medium
CN112712475A (en) * 2020-12-17 2021-04-27 珠海格力电器股份有限公司 Image color difference compensation method and device, electronic equipment and storage medium
CN112967193A (en) * 2021-03-02 2021-06-15 Oppo广东移动通信有限公司 Image calibration method and device, computer readable medium and electronic equipment

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090268953A1 (en) * 2008-04-24 2009-10-29 Apteryx, Inc. Method for the automatic adjustment of image parameter settings in an imaging system
CN103765503A (en) * 2011-08-29 2014-04-30 高通股份有限公司 Fast calibration of displays using spectral-based colorimetrically calibrated multicolor camera
CN105791783A (en) * 2016-03-10 2016-07-20 广州视睿电子科技有限公司 Adjusting method and system for imaging color of camera
CN111866318A (en) * 2019-04-29 2020-10-30 北京小米移动软件有限公司 Multispectral imaging method, multispectral imaging device, mobile terminal and storage medium
CN112309317A (en) * 2020-10-22 2021-02-02 北京虚拟动点科技有限公司 Screen color difference compensation method, device and system, electronic equipment and storage medium
CN112712475A (en) * 2020-12-17 2021-04-27 珠海格力电器股份有限公司 Image color difference compensation method and device, electronic equipment and storage medium
CN112967193A (en) * 2021-03-02 2021-06-15 Oppo广东移动通信有限公司 Image calibration method and device, computer readable medium and electronic equipment

Similar Documents

Publication Publication Date Title
US11750785B2 (en) Video signal processing method and apparatus
CN105046646B (en) A kind of color visualization method of high spectrum image
US11615734B2 (en) Method and apparatus for colour imaging
JP2000152269A (en) Color reproduction system
CN113132696B (en) Image tone mapping method, image tone mapping device, electronic equipment and storage medium
WO2023231583A1 (en) Image processing method and related device thereof
CN106683082B (en) It is a kind of complete with reference to color image quality evaluation method based on quaternary number
CN107465903B (en) Image white balance method, device and computer readable storage medium
JP4677699B2 (en) Image processing method, image processing device, photographing device evaluation method, image information storage method, and image processing system
CN115665565A (en) Online tobacco leaf image color correction method, system and device
CN114584752B (en) Image color restoration method and related equipment
US8164650B2 (en) Image processing apparatus and method thereof
WO2023207443A1 (en) Remote spectral imaging system and method
Jiayun et al. Tongue image color correction method based on root polynomial regression
EP3182691B1 (en) Method of encoding raw color coordinates provided by a camera representing colors of a scene having two different illuminations
CN108174183A (en) A kind of VR image data control systems based on Internet of Things
CN115550575B (en) Image processing method and related device
WO2023088186A1 (en) Image processing method and apparatus based on spectral imaging, and electronic device
CN117014586A (en) Remote spectral imaging system and method
CN117478867A (en) Terminal detection system and method for imaging system and remote spectrum imaging system
CN116152359A (en) Image processing method and device based on spectral imaging and electronic equipment
TWI747039B (en) Multi-angle mobile electronic device spectrum shooting system and image display method
WO2023088188A1 (en) Image white balance optimization method and apparatus, and electronic device
US20240037795A1 (en) Color calibration method and apparatus, computer device, and computer-readable storage medium
WO2022198436A1 (en) Image sensor, image data acquisition method and imaging device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23794878

Country of ref document: EP

Kind code of ref document: A1