WO2023207443A1 - Système et procédé d'imagerie spectrale à distance - Google Patents

Système et procédé d'imagerie spectrale à distance Download PDF

Info

Publication number
WO2023207443A1
WO2023207443A1 PCT/CN2023/083095 CN2023083095W WO2023207443A1 WO 2023207443 A1 WO2023207443 A1 WO 2023207443A1 CN 2023083095 W CN2023083095 W CN 2023083095W WO 2023207443 A1 WO2023207443 A1 WO 2023207443A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
data
color
display
image data
Prior art date
Application number
PCT/CN2023/083095
Other languages
English (en)
Chinese (zh)
Inventor
崔开宇
徐晟�
黄翊东
张巍
冯雪
刘仿
Original Assignee
清华大学
北京与光科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202210476278.0A external-priority patent/CN117014586A/zh
Priority claimed from CN202210869460.2A external-priority patent/CN117478867A/zh
Application filed by 清华大学, 北京与光科技有限公司 filed Critical 清华大学
Publication of WO2023207443A1 publication Critical patent/WO2023207443A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present application relates to the field of computer technology, and in particular to a remote spectral imaging system and method.
  • the transmission process of remote images can be divided into three steps: image acquisition, signal transmission, and image reconstruction.
  • the image reconstruction step can use display technologies such as laser liquid crystal display to achieve high-definition and high-fidelity color reproduction based on the different color channel intensity values obtained by the color camera.
  • RGB and other multi-channel cameras are used to collect color image information of actual scenes, and then the terminal screen is displayed based on the transmitted image information.
  • current imaging technology focuses on how to obtain more color image information and high-performance transmission, but does not consider terminal display, so the accuracy of image display in actual situations cannot be guaranteed.
  • This application provides a remote spectral imaging system and method to solve the defect of color distortion of remote images in the existing technology and improve the color restoration of remote images.
  • This application provides a terminal detection system and method for a remote spectral imaging system to solve the defect of color distortion in actual image display in the prior art and improve the color accuracy of image display.
  • This application provides a remote spectral imaging system, including: an image generation subsystem and a remote display subsystem;
  • the image generation subsystem is configured to obtain first image data of the scene to be imaged based on spectral imaging, and send the first image data to the remote display subsystem, and the remote display subsystem displays the first image data;
  • Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data, and the second image data is displayed by the remote display subsystem.
  • the remote spectral imaging system further includes a color difference detection subsystem
  • the color difference detection subsystem is used to shoot the display image of the first image data, obtain a comparison image, and obtain the image color difference value based on the comparison image and the first image data.
  • the color difference detection subsystem includes a spectrum imaging module, an image matching module and a color difference calculation module;
  • the spectral imaging module is configured to capture the display image of the first image data based on spectral imaging to obtain the comparison image;
  • the image matching module is used to perform image matching on the comparison image and the first image data to obtain a matching pixel point set;
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes The first pixel point in the first image data and the second pixel point in the comparison image;
  • the color difference calculation unit is used to obtain the color difference value of the first pixel point and the second pixel point in each of the pixel point pairs, and obtain the image color difference value.
  • the remote display subsystem includes a display module and a color difference compensation module
  • the display module is used to display the first image data and the second image data
  • the color difference compensation module is configured to perform color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the image generation subsystem includes a spectral imaging device, a color processing module and an encoding module;
  • the spectral imaging device is used to obtain spectral data of the scene to be imaged and send the spectral data to the color processing module;
  • the color processing module is used to receive the spectrum data and perform color processing on the spectrum data to obtain color data; the color processing module is also used to send the color data to the encoding module;
  • the encoding module is used to receive the color data and encode the color data to obtain first image data.
  • the spectral imaging device includes a filter structure, an image sensor and a data processing unit, the filter structure is disposed on the photosensitive path of the image sensor; the transmittance curve of the filter structure is based on The spectral characteristics of the scene to be imaged are determined;
  • the filter structure is used to modulate the incident light of the scene to be imaged to obtain a modulated light signal
  • the image sensor is used to receive the modulated light signal and obtain light intensity data
  • the data processing unit is configured to receive light intensity data sent by the image sensor, and perform spectral recovery based on the light intensity data to obtain spectral data corresponding to the scene to be imaged.
  • the color processing module is used to perform processing on the incident light spectrum Color processing, obtaining color data, including:
  • the color processing module is used to perform color processing on the incident light spectrum based on a wide color gamut standard to obtain color data.
  • system further includes: a data transmission module
  • the data transmission module includes any of the following: a first transmission module, a second transmission module and a third transmission module;
  • the first transmission module is connected to the spectral imaging device and the color processing module, and is used to send the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module is connected to the color processing module and the encoding module, and is used to send the color data obtained by the color processing module to the encoding module;
  • the third transmission module is connected to the encoding module and the remote display subsystem, and is used to send the first image data obtained by the encoding module to the remote display subsystem.
  • the data transmission module is used to transmit data at a data rate of no less than 8 bit/s.
  • the remote display subsystem includes a terminal detection system, which includes: a display terminal, a colorimeter and a calibration module;
  • the display terminal is used to display the first image data
  • the colorimeter is used to obtain the color data of the display image of the first image data
  • the calibration module is configured to obtain, based on the color data of the first image data and the color data of the display image of the first image data, the difference between the first image data and the display image of the first image data. Mapping relations;
  • the calibration module is also used to perform color calibration on the display terminal based on the mapping relationship.
  • the color data is any one of chromaticity values, RGB values, and spectral data.
  • the display terminal is also used to receive and display the target image.
  • the display terminal includes a first display unit and a second display unit:
  • the first display unit is used to display the target image
  • the second display unit is used to display the first image data.
  • the first display unit is a first split-screen area in the display terminal
  • the second display unit is a second split-screen area in the display terminal.
  • This application also provides a terminal detection method based on the above-mentioned remote spectral imaging system, including:
  • Color calibration is performed on the display terminal based on the mapping relationship.
  • the color data is any one of chromaticity values, RGB values, and spectral data.
  • the method further includes:
  • the display terminal receives the target image
  • the color calibration of the display terminal based on the mapping relationship also includes:
  • the display terminal displays the target image.
  • the first image data is displayed by a second display unit
  • the method also includes:
  • the display terminal receives a target image, and the target image is displayed by the first display unit;
  • the obtaining the color data of the display image of the first image data includes:
  • the color data based on the first image data and the first image data Display the color data of the image, and obtain the mapping relationship between the first image data and the display image of the first image data, including:
  • the color calibration of the display terminal based on the mapping relationship includes:
  • the target image is obtained based on spectral data of the target imaging object.
  • the target image is obtained by color processing the spectral data; and/or
  • the target image is obtained through data transmission at a data rate of no less than 8 bit/s.
  • This application also provides a remote spectral imaging method based on the above-mentioned remote spectral imaging system, including:
  • Color difference compensation is performed on the first image data based on the image color difference value to obtain second image data.
  • obtaining the image color difference value based on the display image of the first image data and the first image data includes:
  • the display image of the first image data is photographed to obtain a comparison image, and the image color difference value is obtained based on the comparison image and the first image data.
  • photographing the display image of the first image data, obtaining a comparison image, and obtaining the image color difference value based on the comparison image and the first image data includes:
  • the display image of the first image data is photographed to obtain the Described comparison image
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes a pair of pixel points in the first image data.
  • the color difference value of the first pixel point and the second pixel point in each pair of pixel points is obtained to obtain the image color difference value.
  • performing color difference compensation on the first image data based on the image color difference value to obtain the second image data includes:
  • Color difference compensation is performed on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the scene to be imaged is photographed based on spectral imaging to obtain first image data, including:
  • the color data is encoded to obtain first image data.
  • encoding the color data to obtain color data includes:
  • Color processing is performed on the incident light spectrum based on a wide color gamut standard to obtain color data.
  • the method includes any of the following data transmission steps:
  • the first transmission module sends the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module sends the color data obtained by the color processing module to the encoding module
  • the third transmission module sends the first image data obtained by the encoding module to the remote display subsystem.
  • data transmission is performed at a data rate of no less than 8 bit/s.
  • the remote spectral imaging system and method provided by this application can, on the one hand, collect pictures through spectral imaging through the image generation subsystem, and obtain accurate spectral information and color information; on the other hand, the first image data can be compared with the color difference detection subsystem through the color difference detection subsystem. The images are matched and calibrated to restore the real spatial color information of the scene to be imaged, improving the color restoration of remote image display.
  • the terminal detection system and method for a remote spectral imaging system uses the first image data as a color reference to obtain a display image of the first image data actually presented by the display terminal, compare the two, and establish a mapping Relationship, the display terminal is calibrated and errors are eliminated based on the mapping relationship, which avoids the color display error of the display terminal and further improves the accuracy of the imaging system.
  • Figure 1 is a schematic structural diagram of a remote spectral imaging system provided by an embodiment of the present application
  • Figure 2 is a schematic structural diagram of a spectral imaging device provided by an embodiment of the present application.
  • Figure 3 is a functional block diagram of a spectrum chip provided by an embodiment of the present application.
  • Figure 4 is a schematic structural diagram of a spectrum chip provided by an embodiment of the present application.
  • Figure 5 is a schematic structural diagram of a terminal detection system for a remote spectral imaging system provided by an embodiment of the present application
  • Figure 6 is one of the flow diagrams of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application
  • Figure 7 is a terminal detection for a remote spectral imaging system provided by an embodiment of the present application. Flowchart 2 of the method;
  • Figure 8 is the third schematic flowchart of the terminal detection method for the remote spectral imaging system provided by the embodiment of the present application.
  • Figure 9 is a schematic flowchart of a remote spectral imaging method provided by an embodiment of the present application.
  • Figure 10 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the accuracy of the acquired information must be ensured at the collection end.
  • the loss of information must be avoided during the transmission process.
  • the accuracy must be ensured during the display process. Any deficiency has a certain probability of affecting the accuracy of the final display.
  • Figure 1 is a schematic structural diagram of a remote spectral imaging system provided by an embodiment of the present application.
  • the remote spectral imaging system provided by an embodiment of the present application includes: an image generation subsystem 110 and a remote display subsystem 120;
  • the image generation subsystem 110 is configured to obtain first image data of the scene to be imaged based on spectral imaging, and send the first image data to the remote display subsystem 120, and the remote display subsystem 120 displays the first image data;
  • the scene to be imaged is located at the image collection site, and the image generation subsystem 110 captures the scene to be imaged through a spectral imaging device to obtain spectral data; further, the spectral data is transmitted, preferably during the transmission process.
  • the spectral data is subjected to color processing, converted into a color space, and then compressed and encoded to obtain a display image.
  • the image generation subsystem described in this application may include a collection end for collecting data and a transmission end for transmitting to the display system, that is, the collection end may be implemented as the spectrum imaging device to Collect spectral data; and then further use the transmission end to transmit the data with high fidelity, for example, perform high-bit data transmission on the spectral data. That is, after acquiring the spectral data of each pixel of the image, in order to minimize the loss of color information in the image, high-bit data transmission is performed in the embodiment of the present application. In this field, usually no less than 8 Bit data transmission can be called high-bit data transmission.
  • the high-bit data transmission may be 10-bit, 12-bit, or 16-bit data transmission.
  • color processing of the transmitted spectral data may include: the image processing process is performed in a gamut-independent color space, and there is no compression or loss of color gamut during the image processing process.
  • Spectral data needs to be converted into a color space, such as the BT.2020 color gamut space, through colorimetric calculation and color characterization.
  • color characterization can use polynomial transformation, lookup table or neural network methods.
  • color characterization refers to establishing a correspondence between chromaticity values and camera output values.
  • Encode color-processed image data (spectral data). That is to say, after front-end signal collection and digital image processing, the loss of image color accuracy can be minimized.
  • the processed image data Encoding results in high-fidelity display images.
  • the remote display subsystem 120 is configured to receive the image sent by the image generation subsystem 110
  • the first image data and display the first image data that is, the remote display subsystem 120 displays based on the first image data; generally speaking, encoding can be performed on the remote display subsystem, also It can be performed in the image generation subsystem; further explanation is needed.
  • the first image data generated by the image generation subsystem 110 of this application can be conventional RGB, black and white pictures, or spectral data or spectral imaging.
  • the remote display subsystem 120 is provided at a remote image display location.
  • the remote display subsystem 120 may be a display device, such as a portable display, a 4K display, or an 8K display.
  • the spectral imaging device used to obtain spectral data in this application and the display device used to display it are implemented as remote settings.
  • users can be provided with high color gamut and high fidelity image experience to the maximum extent.
  • the spectral imaging device can be set up in the consultation room, and the display device can be set up in a certain consultation room outside the consultation room.
  • the doctor can obtain the spatial color information of the real medical scene to the greatest extent through the display device, and there is It is helpful to improve the diagnostic accuracy of doctors.
  • the remote spectral imaging system provided by the embodiment of the present application is also used to obtain the image color difference between the display image of the first image data and the first image data based on the first image data and the display image of the first image data. value; wherein, the display image of the first image data is a display image displayed by the remote display subsystem 120 based on the first image data.
  • the remote display subsystem 120 has an image display function and can display the first image data in the form of an image.
  • the display image of the first image data refers to the picture in which the first image data is actually displayed by the remote display subsystem under the influence of the device's own parameters and external environmental factors (such as external light).
  • the display content of an image data is the first image data under the influence of the device's own parameters and external environmental factors (such as external light).
  • the remote spectral imaging system provided by the embodiment of the present application is also used to perform color difference compensation on the first image data based on the image color difference value, obtain the second image data, and obtain the second image data based on the image color difference value.
  • the remote display subsystem displays the second image data.
  • a color difference detection device (such as the color difference detection subsystem provided in the embodiment of the present application) is used to pre-obtain the image color difference value of the display image of the first image data and the first image data.
  • the first image data may include multiple
  • the first image data can be a detection picture including 16777216 colors.
  • the image color difference value corresponding to each color of the 16777216 colors can be obtained.
  • the remote display subsystem performs color difference compensation on the display image of the first image data by using the image color difference values corresponding to each of the 16777216 colors obtained in advance.
  • a color difference detection device (such as the color difference detection subsystem provided in the embodiment of the present application) is used to obtain the display image of the first image data and the image color difference value of the first image data in real time. By displaying the first image data The image is compared with the first image data to obtain the image color difference value of the first image data, and real-time color difference compensation is performed on the first image data based on the image color difference value.
  • the remote spectral imaging system further includes a color difference detection subsystem, and the color difference detection subsystem is configured to obtain a display of the first image data based on the first image data and a display image of the first image data. The image color difference value between the image and the first image data;
  • the remote display subsystem in the remote spectral imaging system may be used to perform color difference compensation on the first image data based on the image color difference value, obtain second image data, and use the remote display subsystem to perform color difference compensation on the first image data based on the image color difference value.
  • the display subsystem displays the second image data.
  • the present application further provides a color difference detection subsystem.
  • the color difference detection subsystem 130 is used to shoot the display image of the first image data and obtain a comparison image. Based on the comparison image and the Obtain the image color difference value from the first image data, and send the image color difference value to the remote display subsystem 120.
  • the image (comparison image) of the first image data displayed on the remote display subsystem 120 can be obtained, and then compared with the actual first image data, the color difference (color difference) between the two can be obtained. difference);
  • the displayed image of the first image data refers to the display content displayed by the remote display subsystem 120 based on the first image data.
  • the color difference detection subsystem 130 is located at the remote image display location.
  • the image color difference value may include RGB value or LAB color difference value (L represents the brightness of the color, A represents the red-green value, B represents the yellow-blue value) and other value color spaces that represent the color difference of objects.
  • the remote display subsystem 120 is also configured to receive the image color difference value sent by the color difference detection subsystem 130, perform color difference compensation on the first image data based on the image color difference value, and obtain the second image data. , and display the second image data.
  • the color difference detection subsystem described in this application can be integrated into the remote spectral imaging system; it can also be independently communicated and connected to the remote spectral imaging system, that is, it is only used to capture the first image data to obtain the comparison image. Then it is transmitted to the remote spectral imaging system through communication to obtain the image color difference value.
  • the color value of the first image data is 100. Since the ambient light of the remote image display place is sufficient, the color value of the actual displayed image is improved.
  • the color value of the comparison image is 105.
  • the color difference value of the image is the difference between the first image data and the contrast value.
  • the difference value of the image is -5. Based on the difference value -5, the first image data is compensated for the color difference.
  • the remote display subsystem 120 obtains and displays the second image data with a color value of 95. After the second image data is combined with the ambient light, the actual display closer to the first image data.
  • the remote spectral imaging system can obtain accurate spectral information and color information (ie, obtain accurate spectral data) through image generation subsystem 110 through spectral imaging; it can also achieve high fidelity transmission and encoding.
  • the first image data and the comparison image are matched and calibrated through the color difference detection subsystem 130 to restore the real spatial color information of the scene to be imaged, thereby improving the color restoration degree of remote image display.
  • the remote image second image
  • the improvement of the color authenticity of the data will help improve the accuracy and effectiveness of users' judgment of remote images, and improve the efficiency, accuracy and credibility of telemedicine.
  • the image generation subsystem 110 includes
  • the spectral imaging device is used to obtain spectral data of the scene to be imaged and send the spectral data to the color processing module;
  • the color processing module is used to receive the spectral data sent by the spectral imaging device and perform color processing on the spectral data to obtain color data; the color processing module is also used to send the spectral data to the encoding module. color data;
  • the encoding module is configured to receive the color data sent by the color processing module, and encode the color data to obtain first image data.
  • the spectral imaging device can obtain the light intensity data of the incident light of the scene to be imaged.
  • the light intensity data corresponds to the position of the spectral pixels in the spectral imaging device.
  • the data processing module of the spectral imaging device can obtain the light intensity data corresponding to each spectral pixel.
  • the spectral transmittance information, spectral intensity data and spectral imaging device parameter information invert the spectral data of the incident light of the scene to be imaged.
  • the color processing module performs color processing on the spectral data through methods such as color matrix transformation or color space mapping, and enables the encoding module to encode and obtain the first image data.
  • FIG. 2 is a schematic structural diagram of a spectral imaging device provided by an embodiment of the present application.
  • the spectral imaging device includes a filter structure 210, an image sensor 220 and a data processing unit 230.
  • the filter structure 210 is disposed on the On the photosensitive path of the image sensor 220; the transmittance curve of the filter structure 210 can be determined based on the spectral characteristics of the scene to be imaged;
  • the filter structure 210 is used to modulate the incident light of the scene to be imaged to obtain a modulated light signal
  • the image sensor 220 is used to receive the modulated light signal and obtain light intensity data
  • the data processing unit 230 is configured to receive the light intensity data sent by the image sensor. and perform spectral recovery based on the light intensity data to obtain spectral data corresponding to the scene to be imaged.
  • the filter structure 210 includes at least one structural unit, and the structural unit includes at least two micro-nano structures.
  • the structural units in the filter structure 210 can emit light in the target wavelength range according to the corresponding transmittance curve, that is, the incident light enters the structural unit and is modulated to obtain a modulated optical signal.
  • the filter structure 210 may be a broadband filter structure 210 in the frequency domain or wavelength domain.
  • the filter structure 210 can be a metasurface, a photonic crystal, a nanopillar, a multilayer film, a dye, a quantum dot, a microelectromechanical system (MEMS), an FP etalon, a cavity layer, a waveguide layer ( Structures or materials with light filtering properties such as waveguide layer) or diffractive elements.
  • the transmittance curves of each different structural unit are different from each other.
  • the transmittance curve of a structural unit represents the spectral transmittance of the structural unit to incident light of different wavelengths.
  • the transmittance curve of the structural unit is determined based on the spectral characteristics of the scene to be imaged.
  • the transmittance of the structural unit is higher in the band corresponding to the absorption peak of the scene to be imaged, which enables the filter structure 210 to have a distinguishing effect on the spectral characteristics of different states of the scene to be imaged. .
  • the image sensor 220 may be a CMOS image sensor (CMOS image sensor, CIS), a charge coupled device (Charge coupled device, CCD), or an array light detector, etc.
  • the image sensor 220 can convert the optical signal into an electrical signal to obtain the detection image and light intensity data corresponding to the structural unit one-to-one.
  • the detection image includes multiple detection positions, and each detection position corresponds to one or more structural units, so each detection position corresponds to the light intensity data of its corresponding structural unit.
  • the spectral imaging device further includes an optical system, which is disposed on the photosensitive path of the image sensor 220;
  • the optical system is used to adjust the optical path of incident light.
  • the optical signal is adjusted by the optical system and then modulated by the filter structure 210, and then is received by the image sensor 220 to obtain the light intensity data; wherein the optical system can be a lens component, a uniform light component or a collimation component. and other optical systems, capable of incident Light is focused, homogenized or collimated.
  • the data processing unit may be a micro control unit (Micro Controller Unit, MCU), a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU), Processing units such as Field Programmable Gate Array (FPGA), Neural Network Processing Unit (NPU) or Application Specific Integrated Circuit (ASIC).
  • MCU Micro Controller Unit
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • FPGA Field Programmable Gate Array
  • NPU Neural Network Processing Unit
  • ASIC Application Specific Integrated Circuit
  • the above-mentioned spectral imaging device is a spectral imaging device based on computational spectrum.
  • this application is based on computational spectral imaging technology, which can use the transmission spectrum curve to achieve broad-spectrum modulation of incident light, thereby making the acquired spatial information richer and more accurate.
  • the spectral response is transmitted to the data processing unit 230 for spectral recovery calculation.
  • the process is described as follows:
  • the intensity signals of the incident light at different wavelengths ⁇ are denoted as x( ⁇ ), and the transmission spectrum curve of the filter structure 210 is denoted as T( ⁇ ).
  • Structural units the transmission spectra obtained by incident light passing through each structural unit in a structural group are different from each other.
  • One physical pixel can correspond to a structural unit, or a group of multiple physical pixels can correspond to a structural unit.
  • At least two different structural units constitute a structural group, and a structural group and the physical pixel corresponding to the structural group constitute a spectral pixel.
  • the transmission spectrum that the spectral imaging device can use for spectral recovery is called the effective transmission spectrum.
  • the effective transmission spectrum refers to the transmission spectrum composed of different structural units. When the structural units are the same, the same will appear in the system response coefficient A matrix. vector, resulting in a situation where the spectrum cannot be recovered.
  • the number of effective transmission spectra Ti ( ⁇ ) of the filter structure 210 and the number of structural units can be to be consistent, the transmission spectrum of the filter structure 210 is artificially set, tested or calculated according to certain rules according to the accuracy requirements and speed requirements of scene recognition or spectral recovery to be imaged. Therefore, the effective transmission spectrum of the filter structure 210 is The number can be less than the number of structural units (for example, if there are repetitions in the structural units, the transmission spectrum of the repeated structural units is an invalid transmission spectrum), or even more than the number of structural units (for example, new ones can be obtained by combining the structural units). transmission spectrum).
  • this application can use at least one spectral pixel to restore the image. That is to say, the spectroscopic device in this application can restore the spectral curve and perform spectral imaging based on the spectral response.
  • R( ⁇ ) is the quantum efficiency of the image sensor 220
  • Strong data respectively corresponds to the light intensity measurement values of the image sensor 220 corresponding to m structural units.
  • a physical pixel corresponds to a structural unit, it can be understood as the light intensity measurement values corresponding to m physical pixels, which is a length of m vector.
  • the system response coefficient A is the light response of the system to different wavelengths, which is determined by the transmittance of the filter structure 210 and the image sensor
  • the quantum efficiency of 220 is determined by two factors.
  • A is a matrix, and each row vector corresponds to the response of a structural unit to incident light of different wavelengths.
  • the incident light is sampled discretely and uniformly, with a total of n sampling points.
  • the number of columns of A is the same as the number of sampling points of the incident light.
  • x( ⁇ ) is the intensity of the incident light at different wavelengths ⁇ , which is the spectrum of the incident light to be measured.
  • the color processing module is used to perform color processing on the spectral data.
  • Obtaining the color data includes:
  • the color processing module is used to perform color processing on the incident light spectrum based on a wide color gamut standard to obtain color data.
  • Color processing is performed on the transmitted incident light spectral data.
  • it is often necessary to convert the image information into a certain color space for processing, which will cause the loss of the color information of the image during the processing.
  • color processing selects a device-independent color space with a larger color gamut.
  • a color space larger than sRGB is used as a color space for color processing by the color processing module.
  • the number of colors that this color space can cover is much larger than other color spaces, which can greatly reduce the loss of data during transmission and processing and maximize high-fidelity color restoration.
  • the remote spectral imaging system provided by the embodiment of the present application adopts a wide color gamut standard color space.
  • the number of colors that the wide color gamut standard color space can cover is much larger than that of the traditional color space.
  • the second color space can be reduced. The loss of image data during transmission and processing is achieved to achieve high-fidelity color restoration to the maximum extent.
  • the encoding module encodes the color-processed spectral data (ie, color data) to obtain the first image data.
  • system further includes a data transmission module to transmit spectral data in a high-bit manner;
  • the data transmission module includes any of the following: a first transmission module, a second transmission module and a third transmission module;
  • the first transmission module is connected to the spectral imaging device and the color processing module. Connection, used to send the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module is connected to the color processing module and the encoding module, and is used to send the color data obtained by the color processing module to the encoding module;
  • the third transmission module is connected to the encoding module and the remote display subsystem, and is used to send the first image data obtained by the encoding module to the remote display subsystem.
  • the spectral imaging device in the image generation subsystem 110 is located at the image collection location, and the color processing module in the image generation subsystem 110 is located at the remote image display location.
  • the spectrum imaging device and the color processing module are connected through a first transmission module is connected.
  • the color processing module in the image generation subsystem 110 is located at the image collection location, and the encoding module in the image generation subsystem 110 is located at the remote image display location.
  • the color processing module and encoding module are connected through a second transmission module. connected.
  • the encoding module is installed at the image collection location, and the encoding module and the remote display subsystem 120 are connected through a third transmission module.
  • the embodiments of this application do not limit the spatial location of the data processing module and color processing module in the image generation subsystem. In actual applications, users can consider the hardware data processing capabilities or network transmission speed of the image collection place and the remote image display place. Factors can be set flexibly.
  • the data transmission module is used to transmit data at a data rate of no less than 8 bit/s.
  • high-bit data transmission is performed on the spectral data or first image data generated in the image generation subsystem 110, that is, after obtaining the spectral data or first image data, the color information in the image is minimized.
  • high-bit data transmission is performed.
  • a data transmission rate of not less than 8 bits per second can usually be called high-bit data transmission.
  • the high-bit data transmission may be 10-bit, 12-bit, or 14-bit data transmission.
  • the remote spectral imaging system provided by the embodiment of the present application can Use remote communication technologies such as 5G to achieve high-throughput and high-spatial-temporal resolution hyperspectral image transmission, and realize real-time transmission of high-throughput information to ensure the real-time nature of remote image transmission.
  • the color difference detection subsystem 130 includes a spectrum imaging module, an image matching module and a color difference calculation module;
  • the spectral imaging module is configured to capture the display image of the first image data based on spectral imaging to obtain the comparison image;
  • the spectral imaging module can be a spectral imaging device or a spectrometer or other spectral equipment. It captures the display image of the first image data, obtains the spectral information of the display image of the first image data, and obtains the comparison image based on the spectral information. .
  • the spectral equipment and spectral imaging methods adopted by the spectral imaging module and the spectral imaging device may be different.
  • the image matching module is used to perform image matching on the comparison image and the first image data to obtain a matching pixel point set;
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes The first pixel point in the first image data and the second pixel point in the comparison image;
  • the image matching module can perform image matching on the comparison image and the first image data based on an image processing algorithm.
  • the embodiment of the present application does not limit the matching method.
  • the matched comparison image has a one-to-one correspondence with the pixels of the first image data.
  • the color difference calculation unit is used to obtain the color difference value of the first pixel point and the second pixel point in each of the pixel point pairs, and obtain the image color difference value.
  • the color difference calculation unit calculates the color difference value of each pixel corresponding to the comparison image and the first image data to obtain the image color difference value.
  • the remote display subsystem 120 includes a display module and a color difference compensation module
  • the display module is used to display the first image data and the second image data
  • the color difference compensation module is configured to perform color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the first image data includes 4 pixel points: A, B, C and D.
  • the information of each pixel is as follows:
  • the comparison image includes four pixels that correspond one-to-one with the first image data: A’, B’, C’ and D’.
  • the information of each pixel is as follows:
  • the image color difference value is:
  • the color difference compensation module performs color difference compensation on the first pixel point in each pair of pixel points based on the image color difference value, that is, adding the image color difference value to the first image data to obtain the second image data.
  • the second image The data includes 4 pixels A”, B”, C” and D”.
  • the information of each pixel is as follows:
  • the second image data displayed by the display module under the influence of environmental factors such as ambient light or device display parameters, has a color display effect close to that of the scene to be imaged. true color.
  • the processed image data As the last link of the solution, the encoding and display are also very important for the effect of image display.
  • the color space needs to be selected according to the storage requirements when compressing and encoding image data.
  • the image data is still compressed and encoded as described above.
  • the storage bit depth is set to a high number of bits, such as 10 bits, to ensure the level transition of the image color to the greatest extent, and then transmit Display to the back-end display system.
  • the embodiments of the present application model the spectral color gamut (that is, obtain the first image data based on the spectral information of the scene to be imaged) and the display end color gamut modeling (that is, at the remote image display end, by converting the first image data displayed by the remote display subsystem to The actual picture of the image data is calibrated with the first image data generated by the image generation subsystem 110 of the image acquisition end).
  • the color gamut representation (second image data) in the remote display subsystem controls the remote display subsystem to display the second image data so that the displayed picture is as close as possible to the real space color of the scene to be imaged.
  • Embodiments of the present application can also directly transmit incident light spectrum information at different locations in the scene space to be imaged collected by the spectrum imaging device to the remote display subsystem, providing fine spectral features that are not easily detectable by human vision, and assisting users in remote image judgment. .
  • this application can reasonably improve the efficiency, accuracy and credibility of telemedicine and promote the equal distribution of medical resources.
  • the snapshot spectrum chip can be used to collect spectral and image information of the actual scene, and then through high-speed data transmission, based on the collected light Spectral information is used to display the color of the image on the display terminal.
  • the embodiment of the present application uses a spectrum chip at the actual scene picture collection end.
  • Figure 3 is a functional block diagram of the spectrum chip provided by the embodiment of the present application.
  • Figure 4 is a schematic structural diagram of the spectrum chip provided by the embodiment of the present application, as shown in Figures 2 and 3
  • the spectrum chip includes a filter structure and an image sensor. The filter structure is formed on the optical path of the image sensor.
  • the spectrum imaging chip is used to obtain spectral information of each point in space of the field scene, that is, the The step is to obtain the spatial spectral data obtained through the spectral imaging chip.
  • the incident light is modulated by the filtering light structure, and then received by the image sensor to obtain the corresponding spectral information, and then processed through the spectral recovery algorithm to obtain the corresponding spectral data.
  • the filter structure is composed of multiple structural units, each structural unit includes at least one micro-nano structure, and the micro-nano structure can be implemented as a modulation hole as shown in Figure 4.
  • the spectral data is subjected to high-bit data transmission, that is, after the spectral data of each pixel of the image is obtained, in order to minimize the loss of color information in the image. It should be noted that the spectral information obtained by the image sensor can also be transmitted directly.
  • remote communication technologies such as 5G are used to realize high-throughput and high-spatial-temporal resolution hyperspectral image transmission, and realize real-time transmission of high-throughput information to ensure the real-time nature of the system.
  • the BT.2020 color gamut space can be used throughout the process.
  • the color gamut and spectral color gamut of the display terminal are first modeled, and then the obtained real-time high-throughput spectral data is used to calculate and obtain the representation of the spatial color information of the field scene in the display system color gamut, and control the display system display to make it as close as possible to the spatial color of the actual scene.
  • this application can be applied to telemedicine scenarios.
  • Existing telemedicine images cannot achieve accurate color reproduction, which will affect the user experience.
  • image color distortion may lead to misjudgment by users.
  • To ensure the accuracy of remote images it needs to be achieved from multiple angles. First, the accuracy of the acquired information must be ensured at the collection end. Secondly, the loss of information must be avoided during the transmission process. Finally, the accuracy must be ensured during display. All three must Any deficiencies have a certain probability of affecting the accuracy of the final display.
  • embodiments of the present application provide a terminal detection system for any of the above remote spectral imaging systems, which is used to detect the color display error in the remote spectral imaging system, and perform calibration and error elimination to further improve the imaging solution. accuracy.
  • the terminal detection system can also be applied to other imaging systems, such as 4K high-definition image transmission imaging systems.
  • FIG. 5 is a schematic structural diagram of a terminal detection system for a remote spectral imaging system provided by an embodiment of the present application.
  • the terminal detection system includes: a display terminal 510, a colorimeter 520 and a calibration module 530;
  • the display terminal 510 and the colorimeter 520 can be set up separately, and the calibration module 530 can be set up independently, or can be set up inside the display terminal 510 .
  • the display terminal 510 is used to display the first image data
  • the colorimeter 520 is used to obtain the color data of the display image of the first image data
  • the display terminal 510 has a display function and can display the first image data in the form of an image.
  • the display image of the first image data is a display image displayed by the display terminal 510 based on the first image data.
  • the colorimeter 520 may be a colorimeter detector or a spectrometer, etc.
  • the colorimeter is used to measure measurement parameter values such as RGB values or chromaticity values of the display image of the first image data.
  • the display image of the first image data refers to the image obtained by the display terminal 510 actually displaying the first image data under the influence of factors such as the device itself and/or the external environment (such as external light).
  • the screen is the screen displayed by the display terminal 510 under actual circumstances that is visible to the user or can be collected by other devices.
  • the color data of the display image of the first image data may include quantified data representing the color of the image, such as hue data, saturation data, and brightness data.
  • the calibration module 530 is configured to obtain a difference between the first image data and the display image of the first image data based on the color data of the first image data and the color data of the display image of the first image data. mapping relationship;
  • the color data of the first image data is obtained in advance when acquiring the first image data. Since the first image data corresponds to the display image of the first image data, each pixel in the first image data The position information and color data information are in one-to-one correspondence with the display image of the first image data. Based on the corresponding relationship, the mapping relationship between the first image data and the display image of the first image data can be obtained.
  • the mapping relationship is Refers to the deviation relationship of color data corresponding to the same position of the first image data and the display image of the first image data.
  • the resolution information of the first image data is n*m pixels.
  • a coordinate system is established with the center of the first image data as the origin. With 1 pixel as the unit spacing, n*m positions can be obtained.
  • Each position Corresponding to one color data; similarly, n*m positions can also be obtained correspondingly to the display image of the first image data, each position corresponds to one color data, the pixel position of the first image data and the display image of the first image data.
  • the positions of can correspond one to one, so the color data of the first image data and the color data in the display image of the first image data can correspond one to one, thereby constructing a mapping relationship, that is, according to the reference parameter values obtained in advance in the first image data
  • a mapping relationship is established with the measurement parameter value obtained by measuring the display image of the first image data.
  • the corresponding method of the first image data and the display image of the first image data may include but is not limited to AI image recognition, image equal distance division, etc.
  • the pixel color data at coordinates (5,36) in the first image data is saturation 30, brightness 10, and color value 36;
  • the pixel color data at coordinates (5,36) in the display image of the first image data is Saturation 35, brightness 15, color value 56;
  • pixels (5,36) can be obtained During actual display, the deviation from the first image data is saturation +5, brightness +5, and color value +20. That is, the mapping relationship at (5,36) is: saturation +5, brightness +5, and color value +20.
  • the calibration module 530 is also used to perform color calibration on the display terminal based on the mapping relationship.
  • color calibration refers to correcting the actual displayed color data to a real color.
  • the real color refers to the color when the data is generated, that is, the color that is not affected by the device itself or the external environment.
  • photometric compensation, white balance compensation, and chromaticity compensation can be performed on the display terminal.
  • the same color as pixel (5,36) in the display terminal can be adjusted based on the mapping relationship: saturation -5, brightness -5, color value -20, such that Its actual display color is the same as the real color.
  • the calibration module 530 can adjust the spectral data based on the mapping relationship, such as adjusting the spectral data of the color corresponding to the pixel (5, 36), so that the display terminal displays The display image is closer to the real color.
  • the terminal detection system for a remote spectral imaging system uses the first image data as a color reference to obtain the display image of the first image data presented by the display terminal, compares the two, and establishes a mapping relationship based on The mapping relationship calibrates and eliminates errors on the display terminal, avoiding color display errors on the display terminal and further improving the accuracy of the imaging system.
  • the above-mentioned color data is any one of chromaticity values, RGB values, and spectral data.
  • the colorimeter 520 measures the chromaticity value of the displayed image of the first image data; the colorimeter 520 may also be a spectrometer, and the colorimeter 520 directly measures the image spectrum curve corresponding to the displayed image of the first image data ( Spectral data), and then establish a mapping relationship based on the measurement parameter value (color data of the display image of the first image data) and the reference parameter value (color data of the first image data), and then perform remote mapping based on the mapping relationship.
  • the spectral data of the spectral imaging system is calibrated, the spectral data of the remote spectral imaging system is used to generate a target image, and then the target image is displayed through the display terminal, thereby eliminating imaging errors caused by the display terminal.
  • the color data of the display image of the first image data and the color data of the first image data are not necessarily the same type of parameters.
  • the measuring instrument is a screen colorimeter
  • the first image is measured.
  • the data displays the chromaticity value of the image
  • the reference parameter value provided by the first image data is the RGB value
  • the mapping relationship between the known RGB value and the measured chromaticity value can be established, and then through the The mapping relationship is used to adjust the spectral data so that the display image displayed on the display terminal is closer to the real color.
  • the display terminal is also used to receive and display the target image.
  • the target image can be obtained after screen collection, transmission and color processing in the remote spectral imaging system, and the display terminal can receive the target image reconstructed after processing; in one embodiment, the target image can be obtained through screen collection in the remote spectral imaging system. and transmit the collected spectral data to the display terminal.
  • the calibration module 530 can adjust the spectral data based on the mapping relationship
  • the display terminal 510 reconstructs the target image based on the adjusted spectral data and displays the target image, so that the displayed target The image is close to the real color.
  • the display terminal includes a first display unit and a second display unit:
  • the first display unit is used to display the target image
  • the second display unit is used to display the first image data.
  • the first display unit and the second display unit may be two independent sub-display terminals, and the display screen characteristics of the first display unit and the second display unit should be consistent (or meet a preset consistency threshold), that is, The performance of the display screens corresponding to the first display unit and the second display unit needs to be relatively close.
  • the first display unit and the second display unit may be tested and display parameters adjusted in advance.
  • the colorimeter obtains the corresponding measurement parameter value by photographing the first image data displayed by the second display unit, and then based on the known reference parameter value
  • the real-time mapping relationship is obtained, the spectral data is adjusted in real time through the real-time mapping relationship, and then displayed on the first display unit, thereby reducing the impact of the display system and ambient light on spectral imaging.
  • the display terminal may be implemented as a display screen.
  • Embodiments of the present application use the first image data as a detection standard for the imaging system to achieve calibration of the imaging error and ambient light deviation of the display system.
  • the first image data is used to establish a mapping relationship between the actual imaging chromaticity value of the display terminal and the reference standard, and the display is adjusted according to the mapping relationship; further, during the imaging display process, a display unit is set for real-time detection and adjustment of the first image data.
  • the first display unit is a first split-screen area in the display terminal
  • the second display unit is a second split-screen area in the display terminal.
  • the first display unit and the second display unit are preferably implemented as two areas of the display terminal, that is, the same display terminal is divided into screens or an area is set to display the first image data.
  • the embodiment of the present application does not limit the size of the split-screen area.
  • the terminal detection system for remotely mounted spectral imaging system avoids the introduction of different parameters of the two display units due to the separate setting of the first display unit and the second display unit by splitting the screen of the same display terminal.
  • the new display error maintains the consistency of the device displaying the first image data and the display image displaying the first image data, further improving the accuracy of the imaging system.
  • the terminal detection method for a remote spectrum imaging system provided by this application is described below.
  • the terminal detection method for a remote spectrum imaging system described below and the terminal detection system for a remote spectrum imaging system described above can correspond to each other. .
  • the terminal detection method for the remote spectral imaging system provided by the embodiment of the present application is described below with reference to FIGS. 6-8 .
  • Figure 6 is one of the flow diagrams of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 6, an embodiment of the present application provides a terminal detection method for a remote spectral imaging system, including :
  • Step 610 Obtain the color data of the display image of the first image data
  • Step 620 Based on the color data of the first image data and the color data of the display image of the first image data, obtain the mapping relationship between the first image data and the display image of the first image data;
  • Step 630 Perform color calibration on the display terminal based on the mapping relationship.
  • the terminal detection method for a remote spectral imaging system uses the first image data as a color reference to obtain a display image of the first image data actually presented by the display terminal, compare the two, and establish a mapping relationship.
  • the display terminal is calibrated and errors are eliminated based on the mapping relationship, which avoids color display errors on the display terminal and further improves the accuracy of the imaging system.
  • the color data is any one of chromaticity values, RGB values, and spectral data.
  • the target image is obtained based on spectral data of the target imaging object.
  • the target imaging object is the actual scene of the picture to be transmitted, that is, the real picture that is not affected by the picture to be transmitted (before imaging, it can be called the picture to be imaged, the two are the same).
  • the spectral data of the target imaging object can be obtained through the snapshot spectrum chip introduced in the embodiment of the present application. Refer to the above and the introduction of Figures 3 and 4, which will not be described again here.
  • the target image is obtained by color processing the spectral data; and/or
  • the target image is obtained through data transmission at a data rate of no less than 8 bit/s.
  • the embodiment of the present application does not limit the order of color processing and data transmission.
  • FIG. 7 is a schematic flow chart of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in FIG. 7 , the first image data and the third image data are obtained. After displaying the mapping relationship between images, the image data also includes:
  • the display terminal receives the target image
  • the color calibration of the display terminal based on the mapping relationship also includes:
  • the display terminal displays the target image.
  • the first image data can be preset and displayed on the display terminal in the embodiment of the present application, and the mapping relationship can be obtained in advance through the terminal detection method for the remote spectral imaging system shown in FIG. 6 .
  • a spectrum chip is used to obtain spectral image data (i.e., spectral data) of the actual scene of the picture to be transmitted, high-bit data transmission is performed on the spectral data, color processing is performed on the transmitted spectral data, and the color processed data is obtained
  • the image data (that is, the target image) is adjusted according to the mapping relationship according to the color-processed image data, and then the target image is encoded and displayed.
  • mapping relationship in the embodiment of the present application is obtained in advance.
  • the pre-obtained mapping relationship is mainly based on the display deviation of the display terminal itself and the impact of ambient light on imaging when obtaining the mapping relationship.
  • the deviation caused by the display terminal itself is relatively stable. If the ambient light is also relatively stable, the mapping relationship only needs to be obtained before imaging and display. It can be applied to color correction of subsequent display terminals, such as color correction in scenarios where the display terminal displays remote spectral imaging or spectral video.
  • the image color displayed by different models of display terminals based on the same image data may deviate, and there is also a certain deviation from the color perceived by the human eye in the actual scene; the same display terminal under different ambient lights, Final display color effects will also vary.
  • the terminal detection method for the remote spectral imaging system provided by the embodiment of the present application measures the display image of the first image data and obtains the mapping relationship according to the first image data, which can eliminate the effects of the display terminal itself and the ambient light on the display terminal. Imaging effect.
  • Figure 8 is a third schematic flowchart of a terminal detection method for a remote spectral imaging system provided by an embodiment of the present application. As shown in Figure 8, the first image data is displayed by the second display unit;
  • the method also includes:
  • the display terminal receives a target image, and the target image is displayed by the first display unit;
  • the obtaining the color data of the display image of the first image data includes:
  • Obtaining a mapping relationship between the first image data and the display image of the first image data based on the color data of the first image data and the color data of the display image of the first image data includes: :
  • the color calibration of the display terminal based on the mapping relationship includes:
  • the first image data is displayed by the second display unit.
  • the color data of the displayed image of the first image data may change.
  • the current color data of the image refers to the color data of the display image of the first image data obtained in real time; for example, when color correction needs to be performed on the display terminal, the color data of the display image of the first image data is obtained.
  • the real-time mapping relationship is obtained based on the current color data of the display image of the first image data and the color data of the first image data obtained in real time, and can reflect the display deviation of the display image of the first image data and the first image data at the current moment.
  • a spectrum chip is used to obtain spectral image data (i.e., spectral data) of the actual scene of the picture to be transmitted, high-bit data transmission is performed on the spectral data, and color isolation is performed on the transmitted spectral data.
  • spectral image data i.e., spectral data
  • color isolation is performed on the transmitted spectral data.
  • the color-processed image data i.e., the target image.
  • the high-fidelity transmission image is displayed on the first display unit, and the currently displayed high-fidelity transmission image may be the target image at the previous moment; the first image data is displayed on the second display unit, and the high-fidelity transmission image shown in Figure 6 is used to display the first image data.
  • the terminal detection method of the remote spectral imaging system obtains the real-time mapping relationship between the first image data and the display image of the first image data in real time.
  • the color-processed image data and the real-time mapping relationship can be obtained at the same time, the color-processed image data can be adjusted according to the real-time mapping relationship, and then the target image can be encoded and displayed.
  • the image colors displayed by different models of display terminals based on the same image data may deviate, and there is also a certain deviation from the color perceived by the human eye in the actual scene; the same display terminal under different ambient lights will ultimately The display color effects will also be different.
  • the first image data is displayed by the second display unit. According to the first image data and the first The display image of the image data can obtain the real-time mapping relationship, which can eliminate the influence of the display performance of the display terminal itself and the ambient light on the imaging effect in real time.
  • the long-range spectral imaging method provided by the present application is described below.
  • the long-range spectral imaging method described below and the long-range spectral imaging system described above can be mutually referenced.
  • FIG. 9 is a schematic flowchart of a remote spectral imaging method provided by an embodiment of the present application. As shown in Figure 9, an embodiment of the present application provides a remote spectral imaging method based on the above-mentioned remote spectral imaging system, including:
  • Step 910 Shoot the scene to be imaged based on spectral imaging (especially computational spectral imaging technology) to obtain first image data;
  • Step 920 Display the first image data, and obtain an image color difference value based on the display image of the first image data and the first image data;
  • Step 930 Perform color difference compensation on the first image data based on the image color difference value to obtain second image data.
  • obtaining the image color difference value based on the display image of the first image data and the first image data includes:
  • the display image of the first image data is photographed to obtain a comparison image, and the image color difference value is obtained based on the comparison image and the first image data.
  • photographing the display image of the first image data, obtaining a comparison image, and obtaining the image color difference value based on the comparison image and the first image data includes:
  • the matching pixel point set includes a plurality of pixel point pairs, and each of the pixel point pairs includes a pair of pixel points in the first image data.
  • performing color difference compensation on the first image data based on the image color difference value to obtain the second image data includes:
  • Color difference compensation is performed on the first pixel point in each pair of pixel points based on the image color difference value to obtain second image data.
  • the image color difference values are RGB values.
  • the scene to be imaged is photographed based on spectral imaging to obtain first image data, including:
  • the color data is encoded to obtain first image data.
  • performing color processing on the spectral data to obtain color data includes:
  • Color processing is performed on the incident light spectrum based on a wide color gamut standard to obtain the color data.
  • the method includes any of the following data transmission steps:
  • the first transmission module sends the spectral data obtained by the spectral imaging device to the color processing module;
  • the second transmission module sends the color data obtained by the color processing module to the encoding module
  • the third transmission module sends the first image data obtained by the encoding module to the remote display subsystem.
  • data transmission is performed at a data rate of no less than 8 bit/s.
  • the image color difference value described in this application can be adjusted in real time according to any of the first image data and the corresponding display image, thereby obtaining the second image data.
  • the image color difference value can be obtained through one or more measurements, and then Any first image data is compensated according to the image color difference value to obtain corresponding second image data. That is, the image color difference value can be a fixed value or a dynamic change.
  • the first image data and the second image data in this application can be displayed on different remote display subsystems, that is, the remote display subsystem can be implemented as two screens to achieve display.
  • the first The screen is used to display the first image data
  • the second screen is used to display the final required high-fidelity second image data.
  • the performance of the first screen and the second screen is close, that is, the same image can be displayed in the same environment. near.
  • the first screen and the second screen may also be different areas of the same screen.
  • the remote spectral imaging method uses spectral imaging to directly obtain the spatial spectral information of the scene to be imaged, avoiding the information loss caused by dimensionality reduction when obtaining spectral dimension information by traditional cameras such as RGB color cameras, as well as different lighting and environments. Deviations in image white balance under color temperature, and at the same time, the actual display effect is calibrated and matched at the image reconstruction end (remote display subsystem) to further improve the accuracy of image color restoration of remote images and restore the spatial color information of the scene to be imaged to the greatest extent.
  • Figure 10 illustrates a schematic diagram of the physical structure of an electronic device.
  • the electronic device may include: a processor (processor) 1010, a communications interface (Communications Interface) 1020, a memory (memory) 1030 and a communication bus 1040.
  • the processor 1010, the communication interface 1020, and the memory 1030 complete communication with each other through the communication bus 1040.
  • the processor 1010 can call logic instructions in the memory 1030 to perform any of the above methods.
  • the above-mentioned logical instructions in the memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
  • the technical solution of this application is essentially or The part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes a number of instructions to enable a computer device (which can be a personal computer). Computer, server, or network device, etc.) executes all or part of the steps of the methods described in various embodiments of this application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .
  • the present application also provides a computer program product.
  • the computer program product includes a computer program.
  • the computer program can be stored on a non-transitory computer-readable storage medium.
  • the computer program can Perform any of the above methods.
  • the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is implemented when executed by a processor to perform any of the above methods.
  • the device embodiments described above are only illustrative.
  • the units described as separate components may or may not be physically separated.
  • the components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.
  • each embodiment can be implemented by software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute various embodiments or certain parts of the embodiments. method described.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

La présente demande concerne un système et un procédé d'imagerie spectrale à distance. Le système comprend : un sous-système de génération d'image et un sous-système d'affichage à distance. Le sous-système de génération d'image est utilisé pour obtenir, sur la base d'une imagerie spectrale, des premières données d'image d'une scène à représenter et le sous-système d'affichage à distance affiche les premières données d'image ; sur la base des premières données d'image et d'une image d'affichage des premières données d'image, une valeur d'aberration chromatique d'image entre l'image d'affichage des premières données d'image et les premières données d'image est obtenue, une compensation d'aberration chromatique est effectuée sur les premières données d'image en fonction de la valeur d'aberration chromatique d'image pour obtenir des secondes données d'image et le sous-système d'affichage à distance affiche les secondes données d'image ; et la valeur d'aberration chromatique peut être obtenue par un sous-système de détection d'aberration chromatique effectuant un calcul à partir des premières données d'image et de l'image d'affichage dans le sous-système d'affichage à distance. Le système et le procédé d'imagerie spectrale à distance décrits dans la présente demande permettent au degré de rendu de couleur d'un affichage d'image à distance d'être amélioré.
PCT/CN2023/083095 2022-04-29 2023-03-22 Système et procédé d'imagerie spectrale à distance WO2023207443A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202210476278.0A CN117014586A (zh) 2022-04-29 2022-04-29 远程光谱成像系统及方法
CN202210476278.0 2022-04-29
CN202210869460.2A CN117478867A (zh) 2022-07-22 2022-07-22 用于成像系统的终端检测系统及方法和远程光谱成像系统
CN202210869460.2 2022-07-22

Publications (1)

Publication Number Publication Date
WO2023207443A1 true WO2023207443A1 (fr) 2023-11-02

Family

ID=88517281

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/083095 WO2023207443A1 (fr) 2022-04-29 2023-03-22 Système et procédé d'imagerie spectrale à distance

Country Status (1)

Country Link
WO (1) WO2023207443A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090268953A1 (en) * 2008-04-24 2009-10-29 Apteryx, Inc. Method for the automatic adjustment of image parameter settings in an imaging system
CN103765503A (zh) * 2011-08-29 2014-04-30 高通股份有限公司 使用经基于光谱的色度校准的彩色相机的显示器的快速校准
CN105791783A (zh) * 2016-03-10 2016-07-20 广州视睿电子科技有限公司 摄像头成像色彩调整方法和系统
CN111866318A (zh) * 2019-04-29 2020-10-30 北京小米移动软件有限公司 多光谱成像方法、多光谱成像装置、移动终端和存储介质
CN112309317A (zh) * 2020-10-22 2021-02-02 北京虚拟动点科技有限公司 屏幕色差补偿方法、装置、系统、电子设备及存储介质
CN112712475A (zh) * 2020-12-17 2021-04-27 珠海格力电器股份有限公司 图像色差补偿方法、装置、电子设备和存储介质
CN112967193A (zh) * 2021-03-02 2021-06-15 Oppo广东移动通信有限公司 图像校准方法及装置、计算机可读介质和电子设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090268953A1 (en) * 2008-04-24 2009-10-29 Apteryx, Inc. Method for the automatic adjustment of image parameter settings in an imaging system
CN103765503A (zh) * 2011-08-29 2014-04-30 高通股份有限公司 使用经基于光谱的色度校准的彩色相机的显示器的快速校准
CN105791783A (zh) * 2016-03-10 2016-07-20 广州视睿电子科技有限公司 摄像头成像色彩调整方法和系统
CN111866318A (zh) * 2019-04-29 2020-10-30 北京小米移动软件有限公司 多光谱成像方法、多光谱成像装置、移动终端和存储介质
CN112309317A (zh) * 2020-10-22 2021-02-02 北京虚拟动点科技有限公司 屏幕色差补偿方法、装置、系统、电子设备及存储介质
CN112712475A (zh) * 2020-12-17 2021-04-27 珠海格力电器股份有限公司 图像色差补偿方法、装置、电子设备和存储介质
CN112967193A (zh) * 2021-03-02 2021-06-15 Oppo广东移动通信有限公司 图像校准方法及装置、计算机可读介质和电子设备

Similar Documents

Publication Publication Date Title
US11750785B2 (en) Video signal processing method and apparatus
JP4212165B2 (ja) 色再現システム
US11615734B2 (en) Method and apparatus for colour imaging
WO2023088186A1 (fr) Procédé et appareil de traitement d'image basés sur l'imagerie spectrale et dispositif électronique
CN113132696B (zh) 图像色调映射方法、装置、电子设备和存储介质
CN107465903B (zh) 图像白平衡方法、装置和计算机可读存储介质
WO2023231583A1 (fr) Procédé de traitement d'image et son dispositif associé
CN106683082A (zh) 一种基于四元数的全参考彩色图像质量评价方法
JP4677699B2 (ja) 画像処理方法、画像処理装置、撮影装置評価方法、画像情報保存方法および画像処理システム
CN115665565A (zh) 在线烟叶图像颜色校正方法、系统及装置
CN114584752B (zh) 图像颜色还原方法及相关设备
US8164650B2 (en) Image processing apparatus and method thereof
WO2023207443A1 (fr) Système et procédé d'imagerie spectrale à distance
EP3182691B1 (fr) Procédé de codage de coordonnées de couleurs brutes fournies par une caméra représentant les couleurs d'une scène comportant deux éclairages différents
CN108174183A (zh) 一种基于物联网的vr影像数据控制系统
CN115550575B (zh) 图像处理方法及其相关设备
CN116350177A (zh) 基于光谱的医疗图像处理方法
CN117014586A (zh) 远程光谱成像系统及方法
CN117478867A (zh) 用于成像系统的终端检测系统及方法和远程光谱成像系统
CN116152359A (zh) 基于光谱成像的图像处理方法、装置和电子设备
TWI747039B (zh) 多角度行動電子裝置頻譜拍攝系統及影像顯示方法
WO2023088188A1 (fr) Procédé et appareil d'optimisation d'équilibrage des blancs d'une image, et dispositif électronique
US20240037795A1 (en) Color calibration method and apparatus, computer device, and computer-readable storage medium
WO2022198436A1 (fr) Capteur d'image, procédé d'acquisition de données d'image et dispositif d'imagerie
WO2023124165A1 (fr) Procédé de traitement d'image et dispositif électronique associé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23794878

Country of ref document: EP

Kind code of ref document: A1