WO2023040721A1

WO2023040721A1 - Method, apparatus and system for calibrating two spatial light modulation devices, and electronic device

Info

Publication number: WO2023040721A1
Application number: PCT/CN2022/117542
Authority: WO
Inventors: 弓殷强; 陈彦哲; 赵鹏; 李屹
Original assignee: 深圳光峰科技股份有限公司
Priority date: 2021-09-14
Filing date: 2022-09-07
Publication date: 2023-03-23
Also published as: CN115808291A

Abstract

A method, apparatus and system for calibrating two spatial light modulation devices, and an electronic device. The method comprises: acquiring a pixel distribution of a second spatial light modulator (S210); turning on pixels in a first spatial light modulator according to a predetermined rule, such that the second spatial light modulator outputs first image light according to output light of the turned-on pixels in the first spatial light modulator (S220); acquiring a first image to be processed that includes a first imaged image, wherein the first imaged image is obtained by means of imaging by the first image light (S230); according to a pre-processed first image to be processed, determining position information of the first imaged image (S240); and according to the pixel distribution of the second spatial light modulator and the position information of the first imaged image, determining and storing a correspondence between each pixel in the first spatial light modulator and each pixel in the second spatial light modulator (S250). Therefore, the calibration efficiency of two spatial light modulation devices can be improved, thereby ensuring the accuracy of a calibration result.

Description

Calibration method, device, system and electronic equipment for dual spatial light modulation equipment

technical field

The present application relates to the field of optical technology, in particular, to a calibration method, device, system and electronic equipment for a dual spatial light modulation device.

Background technique

With the development of projection technology, it has been widely used in people's daily life, for example, it can be seen everywhere in conferences, teaching or entertainment places. In the current technical solution, in order to improve the dynamic range (HDR) of the projection system, two spatial light modulators are generally used for mutual cooperation. However, before using the dual spatial light modulator, the pixel relationship between the two spatial light modulators in the dual spatial light modulator needs to be calibrated. Therefore, how to improve the pixel calibration efficiency between the dual spatial light modulators and ensure the accuracy of the calibration results has become an urgent technical problem to be solved.

Contents of the invention

Embodiments of the present application provide a calibration method, device, system and electronic equipment for dual spatial light modulators, which can improve the calibration efficiency of dual spatial light modulators at least to a certain extent and ensure the accuracy of calibration results.

Other features and advantages of the present application will become apparent from the following detailed description, or in part, be learned by practice of the present application.

According to an aspect of an embodiment of the present application, a calibration method for a dual spatial light modulation device is provided, the dual spatial light modulation device includes a first spatial light modulator and a second spatial light modulator connected in series, including:

acquiring pixel distribution of the second spatial light modulator;

Turn on the pixels in the first spatial light modulator according to a predetermined rule, so that the second spatial light modulator outputs the first image light according to the output light of the turned-on pixels in the first spatial light modulator;

Acquiring a first image to be processed including a first imaged image obtained by imaging the first imaged image;

determining position information of the first imaged image according to the preprocessed first image to be processed;

According to the pixel distribution of the second spatial light modulator and the position information of the first imaged image, determine and store each pixel in the first spatial light modulator and each pixel in the second spatial light modulator Correspondence between.

According to an aspect of an embodiment of the present application, a calibration device for a dual spatial light modulation device is provided, the dual spatial light modulation device includes a first spatial light modulator and a second spatial light modulator connected in series, and the device includes:

a first acquiring module, configured to acquire the pixel distribution of the second spatial light modulator;

A pixel turning-on module, configured to turn on pixels in the first spatial light modulator according to a predetermined rule, so that the second spatial light modulator outputs light according to the output light of the turned-on pixels in the first spatial light modulator first image light;

A second acquisition module, configured to acquire a first image to be processed including a first imaged image, the first imaged image is obtained by imaging the first image light;

The first determination module determines the position information of the first imaged image according to the preprocessed first image to be processed;

The second determining module is configured to determine and store the relationship between each pixel in the first spatial light modulator and the second image based on the pixel distribution of the second spatial light modulator and the position information of the first imaged image. Correspondence between pixels in the spatial light modulator.

According to an aspect of an embodiment of the present application, a calibration system for a dual spatial light modulation device is provided, and the system includes:

a first spatial light modulator and a second spatial light modulator connected in series;

a light source for providing source light to the first spatial light modulator;

an image processing device, configured to acquire and process the projection image output by the second spatial light modulator;

Wherein, the image processing apparatus can execute the calibration method of the dual spatial light modulation device as described in the above embodiments.

According to an aspect of the embodiments of the present application, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the calibration of the dual spatial light modulation device as described in the above-mentioned embodiments is realized method.

According to an aspect of the embodiments of the present application, an electronic device is provided, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more When multiple processors execute, the one or more processors implement the calibration method for the dual spatial light modulation device as described in the foregoing embodiments.

In the technical solution provided by some embodiments of the present application, by obtaining the pixel distribution of the second spatial light modulator, the pixels in the first spatial light modulator are turned on according to predetermined rules, so that the second spatial light modulator The output light of the turned-on pixel in a spatial light modulator outputs the first image light, acquires the first image to be processed including the first imaged image, the first imaged image is obtained by imaging the first image light, and Determine the position information of the first imaged image according to the preprocessed first image to be processed, and then determine and store the first spatial light modulation according to the pixel distribution of the second spatial light modulator and the position information of the first imaged image The corresponding relationship between each pixel in the second spatial light modulator and each pixel in the second spatial light modulator. In this way, the corresponding relationship between each pixel in the first spatial light modulator and each pixel in the second spatial light modulator can be quickly determined to complete the calibration, and at the same time, the accuracy of the calibration result can be guaranteed.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application. Apparently, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative efforts. In the attached picture:

Fig. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.

Fig. 2 shows a schematic flowchart of a calibration method for a dual spatial light modulation device according to an embodiment of the present application.

Fig. 3 shows a schematic flowchart of determining a center position of a first imaged image in a calibration method for a dual spatial light modulation device according to an embodiment of the present application.

FIG. 4 shows a schematic flowchart of step S250 in the calibration method of the dual spatial light modulation device in FIG. 2 according to an embodiment of the present application.

Fig. 5 shows a schematic flowchart of step S410 in the calibration method of the dual spatial light modulation device in Fig. 4 according to an embodiment of the present application.

Fig. 6 shows a schematic flowchart of determining the topography of the first imaged image further included in the calibration method of the dual spatial light modulation device according to an embodiment of the present application.

Fig. 7 shows a schematic flowchart of determining shape information of an unimaged image further included in the calibration method for a dual spatial light modulation device according to an embodiment of the present application.

FIG. 8 shows a schematic flowchart of step S210 in the calibration method for the dual spatial light modulation device in FIG. 2 according to an embodiment of the present application.

Fig. 9a to Fig. 9e show a schematic processing flow diagram of a calibration method for a dual spatial light modulation device according to an embodiment of the present application.

Fig. 10 shows a block diagram of a calibration device of a dual spatial light modulation device according to an embodiment of the present application.

FIG. 11 shows a schematic structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concepts of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of the embodiments of the application. However, those skilled in the art will appreciate that the technical solutions of the present application may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other instances, well-known methods, apparatus, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.

The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices entity.

The flow charts shown in the drawings are only exemplary illustrations, and do not necessarily include all contents and operations/steps, nor must they be performed in the order described. For example, some operations/steps can be decomposed, and some operations/steps can be combined or partly combined, so the actual order of execution may be changed according to the actual situation.

As shown in FIG. 1 , the system architecture may include an imaging carrier 110 , a dual spatial light modulation device 120 , an image acquisition device 130 and a processor 140 . Wherein, the dual spatial light modulation device 120 and the processor 140, and the image acquisition device 130 and the processor 140 can be connected through a network, and the network can include various connection types, such as wired communication links, wireless communication links etc. It should be noted that the image acquisition device 130 may be a camera, or an imaging luminance meter disposed on the outgoing light path of the second spatial light modulator, which is not specifically limited in this application.

It should be noted that the processor 140 may be a processor independent of the dual spatial light modulation device 120, such as a processing server, a processing terminal (such as a smart phone, a tablet computer, a portable computer or a desktop computer, etc.), a processor 140 may also be a processor provided in the dual spatial light modulation device 120, which is not specifically limited in this application.

It should be understood that the numbers of imaging carriers, dual spatial light modulation devices, image acquisition devices and processors in Fig. 1 are only illustrative. According to the implementation requirements, there may be any number of imaging carriers, dual spatial light modulation devices, image acquisition devices, and processors.

In a specific application scenario of the present application, the processor 140 obtains the pixel distribution of the second spatial light modulator, and turns on the pixels in the first spatial light modulator according to predetermined rules, so that the second spatial light modulator The output light of the turned-on pixel in the light modulator outputs the first image light, and the first image to be processed including the first imaged image is acquired by the image acquisition device 130, and the first imaged image is imaged by the first image light obtained, and according to the preprocessed first image to be processed, determine the position information of the first imaged image, and then determine and store the first image according to the pixel distribution of the second spatial light modulator and the position information of the first imaged image The corresponding relationship between each pixel in the spatial light modulator and each pixel in the second spatial light modulator is used to complete the calibration of the dual spatial light modulation device.

The implementation details of the technical solutions of the embodiments of the present application are described in detail below:

Fig. 2 shows a schematic flowchart of a calibration method for a dual spatial light modulation device according to an embodiment of the present application. Referring to Figure 2, the calibration method of the dual spatial light modulation device includes at least step S210 to step S250, which are described in detail as follows:

In step S210, pixel distribution of the second spatial light modulator is acquired.

Wherein, the pixel distribution may be the distribution of the projection positions of the pixels in the second spatial light modulator.

In an exemplary embodiment of the present application, a projection image for determining the pixel distribution of the second spatial light modulator may be pre-stored in the processor, and the processor may control the pixels in the first spatial light modulator to perform turn on, so that the second spatial light modulator outputs the projected image according to the output light of each pixel in the first spatial light modulator. The processor may determine the pixel distribution of the second spatial light modulator based on the projection image output by the second spatial light modulator, that is, the distribution positions of the projection images output by each pixel in the second spatial light modulator. It should be noted that, in the actual calibration process, the processor controls each pixel of the first spatial light modulation to be fully open, collects the projection image output by the second spatial light modulator, and processes the projection image The pixel distribution of the second spatial light modulator is obtained.

In step S220, turn on the pixels in the first spatial light modulator according to a predetermined rule, so that the second spatial light modulator outputs the first An image light.

Wherein, the predetermined rule may be preset by those skilled in the art, and is used to control the turn-on of each pixel in the first spatial light modulator so as to realize the pixel turn-on rule of pixel calibration of the dual spatial light modulation device. For example, the predetermined rule may be to turn on the pixels in the first spatial light modulator one by one in order, or the predetermined rule may also be to turn on the pixels in the first spatial light modulator at predetermined intervals, etc. Those skilled in the art may The corresponding predetermined rule is determined according to actual implementation requirements, which is not specifically limited in the present application.

In an exemplary embodiment of the present application, the first spatial light modulator and the second spatial light modulator connected in series each include a plurality of pixels, and the plurality of pixels may be arranged in a certain order. The pixels in the first spatial light modulator can be used as the illumination pixels of the pixels in the second spatial light modulator, that is, the source light emitted by the light source is transmitted from the pixels in the first spatial light modulator to the The outgoing light is formed outside, and the pixels in the second spatial light modulator can receive the outgoing light, and output image light outside. Thus, a pixel in the first spatial light modulator can serve as an illuminated pixel for a pixel in the second spatial light modulator, and a pixel in the second spatial light modulator can serve as a display pixel for a pixel in the first spatial light modulator.

The processor can turn on the pixels in the first spatial light modulator according to a predetermined rule, so that the first spatial light modulator can turn on the light-passing ability, and at the same time, turn on all the pixels in the second spatial light modulator, so that the second spatial light modulator can The pixels in the modulator can receive the outgoing light of the pixels in the first spatial light modulator, so as to output the first image light, and the first image light can be imaged on the imaging carrier to form an image spot.

In step S230, a first image to be processed including a first imaged image obtained by imaging with the first image light is acquired.

In an exemplary embodiment of the present application, those skilled in the art can set a receiving screen (ie, an imaging carrier), and the first image light output by the second spatial light modulator is projected on the screen to form an image spot (ie, first imaged image). The processor may use the image acquisition device to capture the screen on which the image spot is formed, so as to acquire the first image to be processed including the first imaged image.

It should be noted that the image acquisition device may be a camera, or an electronic device with an image acquisition function such as an imaging luminance meter, which is not specifically limited in this application.

In step S240, the location information of the first imaged image is determined according to the preprocessed first image to be processed.

Wherein, the preprocessing may be a processing process of straightening the image. It should be understood that, due to the influence of the shooting angle of the image capturing device, the image captured by the image capturing device may be tilted and not directly facing the imaging carrier. Therefore, by preprocessing the first image to be processed, the first image to be processed taken from an oblique angle of view can be adjusted, for example, if the figure formed by the sides of the first image to be processed is a trapezoid, then the first image to be processed can be adjusted The sides of the image to be processed are adjusted to obtain a rectangular first image to be processed.

The location information may be location information of the first imaged image on the screen. In an example, a corresponding screen coordinate system may be established based on the screen, so as to determine position information of the first imaged image on the screen, that is, coordinate information of the first imaged image in the screen coordinate system.

In an exemplary embodiment of the present application, the processor performs preprocessing on the first image to be processed according to the acquired first image to be processed, so as to obtain a converted first image to be processed. The processor may perform image processing on the converted first image to be processed, so as to determine the position information of the first imaged image. In an example, the processor may determine the center position of the first imaged image according to brightness information of the first imaged image included in the image to be processed. Specifically, the processor can identify the range of the first imaged image based on the brightness information of each first imaged image in the first image to be processed. For example, if the difference between the brightness information of two adjacent points reaches a certain threshold, it means that the two points are on the boundary of the first imaged image, thereby identifying the range of the first imaged image. The processor then uses the position information of the brightest point in the range as the center position of the first imaged image, and uses the center position as the position information of the first imaged image.

In another example, the processor may also perform edge recognition according to the first image to be processed, so as to determine the edge information of the first imaged image included in the first image to be processed, and then determine the first image based on the edge information. The geometric center of the imaged image is used to determine the position information of the first imaged image.

In an example, the preprocessing may further include threshold denoising, Gaussian filtering, etc., so as to improve the accuracy of the determined position information of the first imaged image.

Please continue to refer to FIG. 2, in step S250, according to the pixel distribution of the second spatial light modulator and the position information of the first imaged image, determine and store the relationship between each pixel in the first spatial light modulator and Correspondence between pixels in the second spatial light modulator.

In an exemplary embodiment of the present application, the pixel distribution of the second spatial light modulator may include the projection position of each pixel in the second spatial light modulator, and the processor may combine the position information of the first imaged image with the second spatial light modulator The projection position of each pixel in the light modulator is matched, so as to determine that the position of the first imaged image corresponds to the projection position corresponding to which pixel in the second spatial light modulator, thus, when the processor knows the current first The pixel turned on by the first optical spatial light modulator and the pixel whose projection position matches the position information of the first imaged image in the second spatial light modulator can determine that the two pixels are in a corresponding relationship, and then can determine the first Correspondence between pixels in the spatial light modulator and pixels in the second spatial light modulator. After the corresponding relationship is determined, the processor may store the corresponding relationship, so as to complete the calibration.

It should be noted that the pixels in the first spatial light modulator and the pixels in the second spatial light modulator may have a one-to-one correspondence or a one-to-many correspondence, which is not specifically limited in this application.

Therefore, in the embodiment shown in FIG. 2 , the pixel distribution of the second spatial light modulator is obtained in advance, and then the pixels in the first spatial light modulator are turned on according to predetermined rules, based on the pixel distribution of the second spatial light modulator and the position information of the first imaged image are compared to determine the corresponding relationship between the pixels of the first spatial light modulator and the second spatial light modulator, which improves the calibration efficiency and ensures the accuracy of the calibration results sex.

Based on the embodiment shown in FIG. 2, in an exemplary embodiment of the present application, turning on the pixels in the first spatial light modulator according to a predetermined rule includes:

Pixels in the first spatial light modulator are turned on at predetermined pixel intervals.

In this embodiment, the predetermined pixel interval may be pre-configured, and is used to determine the quantity information of pixels separated between turned-on pixels in the first spatial light modulator. For example, the predetermined pixel interval can be 1, which means that there is one pixel between adjacent turned-on pixels, and if the predetermined pixel interval is 2, it means that there are two pixels between adjacent turned-on pixels, and so on. The above numbers are only illustrative examples, and the present application does not make any special limitation thereto.

It should be noted that the setting of the predetermined pixel interval should be based on the fact that the initial patterns formed by the second spatial light modulator do not interfere with each other, that is, there will be no boundary between the images formed by the second spatial light modulator , so that the position information of each image spot can be accurately identified in the follow-up, thereby ensuring the accuracy of subsequent calibration.

It should be understood that the predetermined pixel interval may be the pixel interval between horizontal pixels, or the interval between vertical pixels. In another example, the pixel interval corresponding to each dimension may also be set according to multiple dimensions of pixel arrangement, which is not specifically limited in the present application.

Based on the foregoing embodiments, in an exemplary embodiment of the present application, the determining the position information of the first imaged image according to the preprocessed first image to be processed includes:

The center position of each of the first imaged images is determined according to the brightness information of each pixel in the preprocessed first image to be processed.

In this embodiment, the processor may perform image recognition on the preprocessed first image to be processed, and acquire brightness information of each pixel in the first image to be processed. It should be understood that the luminance information of the pixels in the area where the first imaged image is located should be significantly greater than that in the surrounding areas. Thus, based on the brightness information of each pixel in the first image to be processed, the region where the first imaged image is located can be identified, and then the position information of the pixel with the largest brightness information in the region where the first imaged image is located as the center position of the first imaged image.

It should be understood that when the pixels of the first spatial light modulator are turned on according to a predetermined pixel interval, a plurality of first imaged images may be formed, so that the processor may respectively determine the regions where the plurality of first imaged images are located, and The center positions of the multiple first imaged images are determined, so that the subsequent calibration efficiency can be improved.

Based on the foregoing embodiments, FIG. 3 shows a schematic flowchart of determining a center position of a first imaged image in a calibration method for a dual spatial light modulation device according to an embodiment of the present application. As shown in FIG. 3, the first imaged image includes three sub-spots of red, green and blue, then determining the center position of the first imaged image includes at least steps S310 to S320, which are described in detail as follows:

In step S310, the center position of each sub-spot in each of the first imaged images is determined according to the brightness information of each pixel in the first image to be processed after preprocessing.

In an exemplary embodiment of the present application, the first imaged image may be an image spot in RGB format, which may include three sub-spots of red, green and blue. The processor respectively obtains the pixel points with the highest brightness of the three colors in the determined area where the first imaged image is located, as the center positions of the three sub-spots of the first imaged image.

In step S320, the center position of each of the first imaged images is determined by taking an average value according to the center positions of each of the sub-spots in each of the first imaged images.

In an exemplary embodiment of the present application, the processor may average the center positions of the three sub-spots included in each imaged image, so as to determine the center position of the first imaged image. For example, the center positions of the three sub-spots contained in the first imaged image A are respectively (x ₁ , y ₁ ), (x ₂ , y ₂ ) and (x ₃ , y ₃ ), then the center of the first imaged image A The position is ((x ₁ +x ₂ +x ₃ )/3, (y ₁ +y ₂ +y ₃ )/3).

Thus, the processor can determine the center position of the first imaged image according to the center positions of the three sub-spots included in the first imaged image, which ensures the accuracy of the center position of the first imaged image.

Based on the foregoing embodiments, FIG. 4 shows a schematic flowchart of step S250 in the calibration method for the dual spatial light modulation device in FIG. 2 according to an embodiment of the present application. Referring to FIG. 4, step S250 includes at least step S410 to step S420, which are described in detail as follows:

In step S410, according to the center position of each of the first imaged images, determine the center position of the unimaged image, the unimaged image is related to other pixels except the turned-on pixels in the first spatial light modulator corresponding to pixels.

Wherein, the non-imaged image may be an image spot corresponding to a pixel of which the first spatial light modulator is not turned on, that is, an image spot corresponding to other pixels except the pixel for which the first spatial light modulator is turned on. Since the pixels of the first spatial light modulator are turned on according to a predetermined pixel interval, the unimaged images should be respectively located between two adjacent first imaged images. At the same time, it should be understood that the central position of the unimaged image should have a certain degree of correlation with the central positions of the two adjacent first imaged images, so the unimaged image adjacent to it can be determined according to the central position of the first imaged image. Like the center position of the image.

In an exemplary embodiment of the present application, the processor may determine a center position of an unimaged image located between the two adjacent first imaged images according to the center positions of two adjacent first imaged images. Specifically, based on the center positions of two adjacent first imaged images, the distance between two adjacent first imaged images can be determined, and according to the distance and the predetermined pixel interval, it can be determined that the distance between the two adjacent imaged images is The center position of the unimaged image between images.

For example, according to the center positions of two adjacent first imaged images, the distance between the center positions of two adjacent first imaged images can be determined. If the predetermined pixel interval is n, it means that between two adjacent There should be n unimaged images between the first imaged images, and the distance can be divided into (n+1) equal parts, and the nodes of each equal part are the central positions of the unimaged images.

In step S420, according to the pixel distribution of the second spatial light modulator, the center position of the first imaged image, and the center position of the unimaged image, determine and store the Correspondence between each pixel and each pixel in the second spatial light modulator.

In this embodiment, the processor may match the center position of the unimaged image, the center position of the first imaged image, and the pixel distribution of the second spatial light modulator. It should be understood that because the unimaged image corresponds to the unturned-on pixels in the first spatial light modulator, and the first imaged image corresponds to the turned-on pixels in the first spatial light modulator. Thus, through matching, the corresponding relationship between all the pixels in the first spatial light modulator and the pixels in the second spatial light modulator can be determined, so as to complete the calibration between the two pixels. Moreover, the calibration can be completed without turning on all the pixels in the first spatial light modulator, which improves the calibration efficiency and ensures the accuracy of the calibration result.

Based on the embodiment shown in FIG. 4 , FIG. 5 shows a schematic flowchart of step S410 in the calibration method for the dual spatial light modulation device in FIG. 4 according to an embodiment of the present application. Referring to FIG. 5, step S410 includes at least step S510 to step S520, which are described in detail as follows:

In step S510, the distance between adjacent first imaged images is determined according to the center position of each of the first imaged images.

In an exemplary embodiment of the present application, the processor may determine the distance between adjacent first imaged images according to the center position of each first imaged image. For example, the first imaged image A and the first imaged image B are two adjacent image spots, the center position of the first imaged image A is (x ₁ , y ₁ ), and the center position of the first imaged image B is is (x ₂ , y ₂ ), then the distance between the first imaged image A and the first imaged image B is

In step S520, according to the distance between the adjacent first imaged images and the predetermined pixel interval, the center position of the non-imaged image between the adjacent first imaged images is determined.

In an exemplary embodiment of the present application, the processor may determine the distance between two adjacent first imaged images based on the center positions of the two adjacent first imaged images, and according to the distance and the predetermined pixel interval, Then the position information of the unimaged image located between the two first imaged images can be determined.

For example, according to the center positions of two adjacent first imaged images, the distance between the center positions of two adjacent first imaged images can be determined. If the predetermined pixel interval is n, it means that between two adjacent There should be n unimaged spots (i.e., unimaged images) between the first imaged images, and the distance can be equally divided into (n+1), and each equally divided node is the central position of the unimaged image, by This determines the location information of the unimaged image.

In an exemplary embodiment of the present application, when the image spot is in RGB format, according to the distance between two adjacent first imaged images and the predetermined pixel interval, determine the non-imaged spot between adjacent first imaged images The center position of the image, including:

According to the distance between the center positions of the sub-spots of the same type in the adjacent first imaged images and the predetermined pixel interval, the center positions of the sub-spots of the same type in the non-imaged images between the adjacent first imaged images are determined.

In an exemplary embodiment of the present application, the processor may determine the distance between the center positions of the same type of sub-spots in the adjacent imaged images and the predetermined pixel interval to determine the unformed spot between the adjacent first imaged images. Center positions of sub-spots of the same category in the image. For example, the center position of the red sub-spots of the unimaged images between the adjacent first imaged images may be determined according to the distance between the central positions of the red sub-spots in the adjacent first imaged images and the predetermined pixel interval. For specific determination steps, reference may be made to the above description, and the present application will not repeat them here.

Based on the embodiment shown in FIG. 4 , FIG. 6 shows a schematic flow chart of determining the topography of the first imaged image further included in the calibration method for a dual spatial light modulation device according to an embodiment of the present application. Referring to FIG. 6, determining the topography of the first imaged image includes at least steps S610 to S630, which are described in detail as follows:

In step S610, the distance between adjacent first imaged images is determined according to the center position of each of the first imaged images.

In step S620, according to the distance between the adjacent first imaged images, with the center position of each of the first imaged images as the center, acquire sub-images respectively including each of the first imaged images.

In an exemplary embodiment of the present application, the processor may take the center position of each first imaged image as the center according to the distance between adjacent first imaged images, and use the distance as the width and height of the sub-image , perform segmentation from the first image to be processed to obtain sub-images respectively including each first imaged image. For example, the first imaged image A and the first imaged image B are two adjacent first imaged images, the center position of the first imaged image A is (x ₁ , y ₁ ), the first imaged image The center position of the image B is (x ₂ , y ₂ ), and the distance between the first imaged image A and the imaged image B is 10, then the acquired four corners of the sub-image containing the first imaged image A The positions of the points are (x ₁ -5, y ₁ -5), (x ₁ +5, y ₁ -5), (x ₁ +5, y ₁ +5) and (x ₁ -5, y ₁ +5 ).

In step S630, according to each of the sub-images, the topography of the first imaged image included in the sub-images is determined.

The topography may be information related to the display state of the image spot, for example, the topography of the image spot may include but not limited to information such as shape, brightness, and color of the image spot. According to the shape of each image spot, the display state of the image spot on the screen can be determined.

In an exemplary embodiment of the present application, the processor may perform image analysis on each sub-image, so as to determine the shape of the first imaged image included in the sub-image. Thus, by acquiring sub-images containing each first imaged image, and then performing image analysis on each sub-image, the accuracy of the acquired first imaged image can be improved, avoiding the problem from a single first imaged image. In the processed image, a large error occurs in the topography of the acquired multiple first imaged images. The processor can store the acquired topography of the first imaged image for subsequent recall.

In an exemplary embodiment of the present application, the shape includes brightness information, and the processor may also determine the difference between the adjacent first imaged images according to the brightness information of each first imaged image in the adjacent first imaged images. The brightness information of the unimaged image in between.

In an example, the processor may calculate an average value of brightness according to the brightness information of two adjacent first imaged images, and use it as the brightness information of the unimaged image. In another example, when the predetermined pixel interval is at least two, the processor may also calculate the difference according to the difference between the brightness information of two adjacent first imaged images according to the predetermined pixel interval Aliquoting is performed to determine brightness information of the unimaged image. For example, if the difference in brightness information between two adjacent first imaged images A and B is 9, and the predetermined pixel interval is 2, it means that there should be two unimaged images between the first imaged images A and B . Therefore, the difference can be equally divided, 9/(2+1)=3, so the brightness information of the unimaged image near A should be the brightness of the imaged image A plus 3, and the brightness of the unimaged image near B The information should be the luminance information of the first imaged image A plus 6, ie (3+3), and so on. Thus, it can be ensured that the determined shape of the unimaged image complies with the gradual change rule of the image spot, so as to ensure the accuracy of the determined shape of the unimaged image.

It should be noted that the luminance information of the first imaged image can be represented by the luminance information of its center position, for example, the luminance information of the center position of the first imaged image is a, then the luminance information of the first imaged image is a. If the first imaged image includes three sub-spots, the brightness information of the central positions of the three sub-spots may be used as the brightness information of the first imaged image.

Thus, one first imaged image can correspond to three luminance information, and when subsequently determining the luminance information of the unimaged image, the brightness information of the sub-spots of the same type in the unimaged image can be calculated according to the luminance information of the sub-spots of the same type. The brightness information, for example, according to the brightness information of the red sub-spots of two adjacent first imaged images, the brightness information of the red sub-spots of the unimaged image between the two adjacent first imaged images can be calculated, and the calculation For the method, reference may be made to the above description, and the present application will not repeat it here.

Based on the above embodiments, FIG. 7 shows a schematic flowchart of determining shape information of an unimaged image further included in the calibration method for a dual spatial light modulation device according to an embodiment of the present application. Referring to FIG. 7 , before determining the brightness information of the non-imaged images between the adjacent first imaged images according to the brightness information of each first imaged image in the adjacent first imaged images, determine The shape information of the unimaged image includes at least step S710 to step S720, which are described in detail as follows:

In step S710, according to the shape information of each of the first imaged images in the adjacent first imaged images, the size difference between the adjacent first imaged images is determined.

In an exemplary embodiment of the present application, the processor may compare the shape information of the adjacent first imaged images according to the shape information of the adjacent first imaged images, so as to determine the size difference between them. It should be noted that the shape information may be size information related to the shape of the image spot, for example, the shape information may include but not limited to information such as width, height, and diameter of the image spot. The processor may subtract shape information of the same category of adjacent first imaged images to determine a size difference between the adjacent first imaged images.

In step S720, the shape information of the non-imaged image between adjacent first imaged images is determined according to the size difference.

In an exemplary embodiment of the present application, the processor may also equally divide the size difference between two adjacent first imaged images according to a predetermined pixel interval, so as to determine the unimaged image shape information. Taking the width as an example, if the width difference between two adjacent first imaged images A and B is 9, and the predetermined pixel interval is 2, it means that there should be two unimaged images between the first imaged images A and B , therefore, the width difference can be equally divided, 9/(2+1)=3. Thus, the width of the unimaged image near A should be the width of the imaged image A plus 3, the width of the unimaged image near B should be the width of the imaged image A plus 6 (3+3), and so on. Thus, it can be ensured that the determined shape information of the unimaged image complies with the gradual change rule of the image spot, so as to ensure the accuracy of the determined shape information of the unimaged image.

Based on the embodiment shown in FIG. 2 , FIG. 8 shows a schematic flowchart of step S210 in the calibration method for the dual spatial light modulation device in FIG. 2 according to an embodiment of the present application. Referring to FIG. 8, step S210 includes at least step S810 to step S830, which are described in detail as follows:

In step S810, turn on a designated pixel in the first spatial light modulator for determining the projection position of the second spatial light modulator, so that the second spatial light modulator The emitted light outputs the second image light.

Wherein, the specified pixel may be a pixel used to determine the pixel distribution of the second spatial light modulator. In an example, the specified pixel may be a pixel at an edge position among the pixels of the first spatial light modulator. For example, if the first The pixels of the spatial light modulator are arranged in a square, and the designated pixels are the pixels on the four sides of the square, and so on. In another example, the designated pixel may also be a pixel located at a corner position among the pixels of the first spatial light modulator. For example, if the pixels of the first spatial light modulator are arranged in a square, the designated pixel is located in the square. Pixels at the four corner locations, etc. In yet another example, all pixels in the first spatial light modulator may also be turned on, that is, all pixels in the first spatial light modulator are specified pixels. Those skilled in the art can determine the corresponding designated pixel according to the implementation requirements, which is not specifically limited in the present application.

In an exemplary embodiment of the present application, the processor may turn on a designated pixel in the first spatial light modulator, so that the pixels in the second spatial light modulator may receive the outgoing light of the designated pixel, thereby outputting the second image light . In an example, the processor may project an easily identifiable figure (such as a straight line or a square image, etc.) through the first spatial light modulator and the second spatial light modulator, so that the second spatial light modulator outputs corresponding to the second image light.

In step S820, a second image to be processed including a second imaged image obtained by imaging with the second image light is acquired.

In an exemplary embodiment of the present application, those skilled in the art can set a receiving screen (ie, an imaging carrier), and the second image light output by the second spatial light modulator is projected on the screen to form an image spot (ie, second imaged image). The processor may use the image acquisition device to capture the screen on which the image spot is formed, so as to acquire the second image to be processed including the second imaged image.

In step S830, pixel distribution in the second spatial light modulator is determined according to the preprocessed second image to be processed.

In an exemplary embodiment of the present application, the preprocessing may be a process of performing conversion processing on an image. It should be understood that, due to the influence of the shooting angle of the image capturing device, the image captured by the image capturing device may be tilted and not directly facing the imaging carrier. Therefore, by preprocessing the second image to be processed, the second image to be processed taken from an oblique angle of view can be adjusted, for example, if the figure formed by the sides of the second image to be processed is a trapezoid, then the second image to be processed can be The image to be processed is adjusted to obtain a rectangular second image to be processed.

The processor may perform image recognition on the second image to be processed based on the preprocessed second image to be processed, so as to identify the position information of the second image to be imaged included in the preprocessed second image to be processed, and then A pixel distribution of the second spatial light modulation device is determined. For the determination method, reference may be made to the content described above, which will not be repeated here in this application.

Based on the technical solutions of the above embodiments, a specific application scenario of the embodiments of the present application is introduced as follows:

As shown in Fig. 9a, the processor may turn on the pixels of the first spatial light modulator at predetermined intervals, and turn on all the pixels in the second spatial light modulator. The second spatial light modulator may receive the light emitted by the turned-on pixels of the first spatial light modulator, and project the light on the screen to form an imaged image. The image acquiring device can acquire the image to be processed including the imaged image and transmit the image to be processed to the processor for processing.

As shown in FIG. 9b, the processor may perform image processing based on the image to be processed, and determine the center position of the sub-spot contained in each imaged image, thereby determining the position information of each imaged image. Thereby, the distance between adjacent imaged images can be determined based on the position information of the imaged images.

As shown in Fig. 9c, the processor can take the center position of each imaged image as the center, and according to the determined distance between adjacent imaged images, segment the image to be processed, so as to obtain subimage of the . The processor may perform image processing based on each sub-image to determine the topography of each imaged image.

As shown in Figure 9d, the processor can interpolate the center position of the unimaged image between adjacent imaged images according to the distance between adjacent imaged images and the predetermined pixel interval, so as to obtain the corresponding first spatial light The center position of the image spot for all pixels in the modulator. And based on the position information of the imaged image and the position information of the unimaged image, match the pixel distribution of the second spatial light modulator acquired in advance, so as to determine the pixel distribution of the first spatial light modulator and the second spatial light modulator The corresponding relationship between the pixels in and stored for subsequent calls to complete the calibration.

As shown in Figure 9e, the processor can also determine the topography between the unimaged images located between the adjacent imaged images according to the topography of the adjacent imaged images, so as to obtain the topography of the imaged images and the unimaged images. The shape of the image. The processing terminal can perform normalization processing on the topography of all image spots, and then store the normalized topography of the image spots for subsequent calling.

Therefore, the calibration can be completed without turning on all the pixels of the first spatial light modulator, which improves the calibration efficiency. At the same time, mutual interference between image spots can be avoided, ensuring the accuracy of calibration results.

The apparatus embodiments of the present application are introduced below, which can be used to implement the calibration method for the dual spatial light modulation device in the foregoing embodiments of the present application. For the details not disclosed in the device embodiment of the present application, please refer to the embodiment of the calibration method for the dual spatial light modulation device mentioned above in the present application.

Referring to FIG. 10 , according to an embodiment of the present application, a calibration device for a dual spatial light modulation device includes a first spatial light modulator and a second spatial light modulator connected in series, and the device includes:

A first acquiring module 1010, configured to acquire the pixel distribution of the second spatial light modulator;

A pixel turning-on module 1020, configured to turn on pixels in the first spatial light modulator according to a predetermined rule, so that the second spatial light modulator outputs light according to the turned-on pixels in the first spatial light modulator outputting the first image light;

The second acquisition module 1030 is configured to acquire a first image to be processed including a first imaged image, the first imaged image is obtained by imaging the first image light;

The first determination module 1040 is configured to determine the position information of the first imaged image according to the preprocessed first image to be processed;

The second determination module 1050 is configured to determine and store the relationship between each pixel in the first spatial light modulator and the first imaged image according to the pixel distribution of the second spatial light modulator and the position information of the first imaged image. Correspondence between pixels in two spatial light modulators.

The embodiment of the present application also provides a calibration system for a dual spatial light modulation device, the system comprising:

a light source for providing source light to the first spatial light modulator;

Wherein, the image processing apparatus can execute the calibration method of the dual spatial light modulation device as described in the above-mentioned embodiments.

It should be noted that the computer system of the electronic device shown in FIG. 11 is only an example, and should not limit the functions and scope of use of the embodiments of the present application.

As shown in Figure 11, the computer system includes a central processing unit (Central Processing Unit, CPU) 1101, which can be stored in a program in a read-only memory (Read-Only Memory, ROM) 1102 or loaded to random access from a storage part 1108 Various appropriate actions and processes are executed by programs in the RAM (Random Access Memory, RAM) 1103, for example, the methods described in the above-mentioned embodiments are executed. In RAM 1103, various programs and data necessary for system operation are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other via a bus 1104. An input/output (Input/Output, I/O) interface 1105 is also connected to the bus 1104 .

The following components are connected to the I/O interface 1105: an input part 1106 including a keyboard, a mouse, etc.; an output part 1107 including a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc., and a speaker ; the storage part 1108 including hard disk etc.; and the communication part 1109 including the network interface cards such as LAN (Local Area Network, local area network) card, modem etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I/O interface 1105 as needed. A removable medium 1111 such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc. is mounted on the drive 1110 as necessary so that a computer program read therefrom is installed into the storage section 1108 as necessary.

In particular, according to the embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 1109, and/or installed from removable media 1111. When this computer program is executed by a central processing unit (CPU) 1101, various functions defined in the system of the present application are performed.

It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable one of the above The combination. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which a computer-readable computer program is carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . A computer program embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Wherein, each block in the flowchart or block diagram may represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or part of the code includes one or more executable instruction. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a A combination of dedicated hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software or by hardware, and the described units may also be set in a processor. Wherein, the names of these units do not constitute a limitation of the unit itself under certain circumstances.

As another aspect, the present application also provides a computer-readable medium. The computer-readable medium may be included in the electronic device described in the above-mentioned embodiments; or it may exist independently without being assembled into the electronic device. middle. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by an electronic device, the electronic device is made to implement the methods described in the above-mentioned embodiments.

It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. Actually, according to the embodiment of the present application, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided to be embodied by a plurality of modules or units.

Through the description of the above implementations, those skilled in the art can easily understand that the example implementations described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, server, touch terminal, or network device, etc.) execute the method according to the embodiment of the present application.

Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any modification, use or adaptation of the application, these modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the technical field not disclosed in the application .

It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

A calibration method for a dual spatial light modulation device, the dual spatial light modulation device comprising a first spatial light modulator and a second spatial light modulator connected in series, characterized in that it includes:

acquiring pixel distribution of the second spatial light modulator;

Turn on the pixels in the first spatial light modulator according to a predetermined rule, so that the second spatial light modulator outputs the first image light according to the output light of the turned-on pixels in the first spatial light modulator;

Acquiring a first image to be processed including a first imaged image obtained by imaging the first imaged image;

determining position information of the first imaged image according to the preprocessed first image to be processed;

According to the pixel distribution of the second spatial light modulator and the position information of the first imaged image, determine and store each pixel in the first spatial light modulator and each pixel in the second spatial light modulator Correspondence between.
The method according to claim 1, wherein the turning on the pixels in the first spatial light modulator according to a predetermined rule comprises:

Pixels in the first spatial light modulator are turned on at predetermined pixel intervals.
The method according to claim 2, wherein the determining the position information of the first imaged image according to the preprocessed first image to be processed comprises:

The center position of each of the first imaged images is determined according to the brightness information of each pixel in the preprocessed first image to be processed.
The method according to claim 3, wherein the first imaged image comprises three sub-spots of red, green and blue;

The determining the center position of each of the first imaged images according to the brightness information of each pixel in the preprocessed first image to be processed includes:

determining the center position of each of the sub-spots in each of the first imaged images according to the brightness information of each pixel in the first image to be processed after preprocessing;

The center position of each of the first imaged images is determined by taking an average value according to the center positions of each of the sub-spots in each of the first imaged images.
The method according to claim 3, wherein the first spatial light modulator is determined and stored according to the pixel distribution of the second spatial light modulator and the position information of the first imaged image. The corresponding relationship between each pixel in the second spatial light modulator and each pixel in the second spatial light modulator includes:

determining the center position of an unimaged image according to the center position of each of the first imaged images, the unimaged image corresponding to other pixels in the first spatial light modulator except the turned-on pixel;

According to the pixel distribution of the second spatial light modulator, the central position of the first imaged image, and the central position of the unimaged image, determine and store the relationship between each pixel in the first spatial light modulator and the Correspondence between pixels in the second spatial light modulator.
The method according to claim 5, wherein said determining the center position of the non-imaged image according to the center position of each said first imaged image comprises:

determining the distance between adjacent first imaged images according to the center position of each of the first imaged images;

According to the distance between the adjacent first imaged images and the predetermined pixel interval, determine the center position of the non-imaged image between the adjacent first imaged images.
The method according to claim 5, wherein the method further comprises:

determining the distance between adjacent first imaged images according to the center position of each of the first imaged images;

Acquiring sub-images respectively including each of the first imaged images with the center position of each of the first imaged images as the center according to the distance between the adjacent first imaged images;

From each of the sub-images, the topography of the first imaged image contained in the sub-images is determined.
The method according to claim 7, wherein the profile includes brightness information, and the method further comprises:

According to the brightness information of each of the first imaged images in the adjacent first imaged images, the brightness information of the non-imaged images between the adjacent first imaged images is determined.
The method according to claim 8, wherein the profile further includes shape information;

Before determining the brightness information of the non-imaged images between the adjacent first imaged images according to the brightness information of each first imaged image in the adjacent first imaged images, it also includes:

According to the shape information of each of the first imaged images in the adjacent first imaged images, determine the size difference between the adjacent first imaged images;

Determine shape information of non-imaged images between adjacent first imaged images according to the size difference.
The method according to claim 1, wherein the acquiring the pixel distribution of the second spatial light modulator comprises:

Turn on a designated pixel in the first spatial light modulator for determining the projection position of the second spatial light modulator, so that the second spatial light modulator outputs a second image according to the emitted light of the designated pixel Light;

Acquiring a second image to be processed that includes a second imaged image, the second imaged image is obtained by imaging with the second image light;

Determine pixel distribution in the second spatial light modulator according to the preprocessed second image to be processed.
A calibration device for a dual spatial light modulation device, the dual spatial light modulation device includes a first spatial light modulator and a second spatial light modulator connected in series, characterized in that it includes:

a first acquiring module, configured to acquire the pixel distribution of the second spatial light modulator;

A pixel turning-on module, configured to turn on pixels in the first spatial light modulator according to a predetermined rule, so that the second spatial light modulator outputs light according to the output light of the turned-on pixels in the first spatial light modulator first image light;

A second acquisition module, configured to acquire a first image to be processed including a first imaged image, the first imaged image is obtained by imaging the first image light;

The first determination module determines the position information of the first imaged image according to the preprocessed first image to be processed;

The second determining module is configured to determine and store the relationship between each pixel in the first spatial light modulator and the second image based on the pixel distribution of the second spatial light modulator and the position information of the first imaged image. Correspondence between pixels in the spatial light modulator.
A calibration system for a dual spatial light modulation device, characterized in that it includes:

a first spatial light modulator and a second spatial light modulator connected in series;

a light source for providing source light to the first spatial light modulator;

an image processing device, configured to acquire and process the projection image output by the second spatial light modulator;

Wherein, the image processing device is capable of executing the calibration method for a dual spatial light modulation device according to any one of claims 1-10.
An electronic device, characterized in that it comprises:

one or more processors;

A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors are configured to implement any one of claims 1 to 10 A method for calibrating a dual spatial light modulation device described in one item.