WO2022179109A1

WO2022179109A1 - Projection correction method and apparatus, storage medium and electronic device

Info

Publication number: WO2022179109A1
Application number: PCT/CN2021/122447
Authority: WO
Inventors: 胡震宇; 付啸天; 张聪
Original assignee: 深圳市火乐科技发展有限公司
Priority date: 2021-02-24
Filing date: 2021-09-30
Publication date: 2022-09-01
Also published as: CN113365040B; CN113365041A; CN112584116A; CN112584116B; CN113365041B; CN113365040A; WO2022179108A1

Abstract

The present disclosure relates to a projection correction method and apparatus, a storage medium and an electronic device. The method comprises: projecting a correction image onto an imaging medium to obtain a projection correction image, and acquiring a collection image; determining first position information of each second mark pattern in the collection image; determining a first projection area corresponding to the projection correction image according to the first position information of each second mark pattern and second position information of each first mark pattern; determining a second projection area corresponding to the first projection area; determining third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern according to the second position information; determining a target correction matrix corresponding to each first mark pattern according to the first position information and the third position information under the condition that the offset parameter does not meet a preset condition; and correcting an image to be projected according to the target correction matrix, and projecting said corrected image onto the imaging medium.

Description

Projection correction method, device, storage medium and electronic device

CROSS-REFERENCE TO RELATED APPLICATIONS

This disclosure claims the priority of a Chinese patent application with application number 202110206396.5 and entitled "Projection Correction Method, Apparatus, Storage Medium and Electronic Device" filed with the China Patent Office on February 24, 2021, the entire contents of which are incorporated by reference in in this disclosure.

technical field

The present disclosure relates to the field of projection technology, and in particular, to a projection correction method, device, storage medium and electronic device.

Background technique

In order to meet people's viewing needs for large-size screens, projection equipment is widely used in homes, offices, schools and entertainment venues. The projection device may project onto an imaging medium to display a corresponding projected image on the imaging medium, where the imaging medium may be, for example, a projection screen or a wall. However, when the surface of the imaging medium is not flat (for example, the projection screen is curved or the wall has depressions and protrusions), the projected image projected by the projection device on the imaging medium may be distorted. This distortion is random and Unpredictable, this will affect the projection effect of the projection equipment.

SUMMARY OF THE INVENTION

In order to solve the problems existing in the related art, the present disclosure provides a projection correction method, apparatus, storage medium and electronic device.

According to a first aspect of the embodiments of the present disclosure, there is provided a projection correction method applied to a projection device, the method comprising:

Projecting a preset correction image onto an imaging medium to obtain a projected correction image displayed on the imaging medium, and acquiring a captured image; the corrected image includes a plurality of first mark patterns, and the captured image includes and each a second marking pattern corresponding to the first marking pattern, and the captured image is determined according to an image obtained by photographing the projection correction image by an image capturing device;

determining first position information of each of the second marking patterns in the captured image;

determining, according to the first position information of each of the second marking patterns and the second position information of each of the first marking patterns, a first projection area corresponding to the projection correction image in the captured image;

determining a second projection area corresponding to the first projection area on the imaging medium, the second projection area including a third mark pattern corresponding to each of the first mark patterns;

According to the second position information, determine the third position information of each of the third marking patterns and the offset parameter corresponding to the third marking pattern;

In the case that the offset parameter does not meet the preset condition, according to the first position information and the third position information, determine a target correction matrix corresponding to each of the first marking patterns;

The to-be-projected image is corrected according to the target correction matrix, and the corrected to-be-projected image is projected onto the imaging medium.

According to a second aspect of the embodiments of the present disclosure, there is provided a projection correction apparatus, which is applied to a projection device, the apparatus comprising:

an acquisition module, configured to project a preset corrected image onto an imaging medium to obtain a projected corrected image displayed on the imaging medium, and acquire a captured image; the corrected image includes a plurality of first mark patterns, and the captured image The image includes a second marking pattern corresponding to each of the first marking patterns, and the acquired image is determined according to an image obtained by photographing the projection correction image by an image acquisition device;

a processing module, configured to determine the first position information of each of the second marking patterns in the captured image;

The processing module is further configured to determine, according to the first position information of each of the second marking patterns and the second position information of each of the first marking patterns, whether the projection correction image corresponds to the captured image. The first projection area of ;

The processing module is further configured to determine a second projection area corresponding to the first projection area on the imaging medium, where the second projection area includes a third mark pattern corresponding to each of the first mark patterns ;

The processing module is further configured to determine, according to the second position information, third position information of each of the third marking patterns and an offset parameter corresponding to the third marking pattern;

The processing module is further configured to determine a target corresponding to each of the first marking patterns according to the first position information and the third position information when the offset parameter does not meet a preset condition correction matrix;

The correction module is used for correcting the to-be-projected image according to the target correction matrix, and projecting the corrected to-be-projected image onto the imaging medium.

According to a third aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the method in any one of the above first aspects.

According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic device, comprising:

a memory on which a computer program is stored;

A processor for executing the computer program in the memory to implement the steps of any one of the methods in the first aspect.

Through the above technical solutions, the present disclosure first obtains the projected corrected image displayed on the imaging medium by projecting the preset corrected image onto the imaging medium, acquires the captured image, and then determines the size of each second mark pattern in the captured image. the first position information, and according to the first position information of each second mark pattern and the second position information of each first mark pattern, determine the first projection area corresponding to the projection correction image in the captured image, and then determine the first The projection area corresponds to the second projection area on the imaging medium, and according to the second position information and the second projection area, the third position information of each third mark pattern and the offset parameter corresponding to the third mark pattern are determined, and then In the case that the offset parameter does not meet the preset condition, the target correction matrix corresponding to each first mark pattern is determined according to the first position information and the third position information, and finally the projected image is corrected according to the target correction matrix, and the The corrected image to be projected is projected onto the imaging medium. The present disclosure corrects the to-be-projected image through the determined target correction matrix corresponding to each first mark pattern, and projects the corrected to-be-projected image onto the imaging medium, so as to ensure that the to-be-projected image is not projected onto the imaging medium when projected onto the imaging medium. Distortion will occur, the efficiency of correcting the projection is high, and the projection effect of the projection device is improved.

Other features and advantages of the present disclosure will be described in detail in the detailed description that follows.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present disclosure, and constitute a part of the specification, and together with the following detailed description, are used to explain the present disclosure, but not to limit the present disclosure. In the attached image:

1 is a flowchart of a projection correction method according to an exemplary embodiment;

FIG. 2 is a schematic diagram of a corrected image according to an exemplary embodiment;

Fig. 3 is a flow chart of step 101 shown in the embodiment shown in Fig. 1;

Fig. 4 is a flow chart of step 103 shown in the embodiment shown in Fig. 1;

Fig. 5 is a flow chart of step 104 shown in the embodiment shown in Fig. 1;

Fig. 6 is a flow chart of step 105 shown in the embodiment shown in Fig. 1;

7 is a block diagram of a projection correction apparatus according to an exemplary embodiment;

Fig. 8 is a block diagram of an electronic device according to an exemplary embodiment.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

Before introducing the projection correction method, device, storage medium, and electronic device provided by the present disclosure, the application scenarios involved in various embodiments of the present disclosure are first introduced, and the application scenarios may include projection devices, image acquisition devices, and imaging media. The projection device can project the image or video played by itself onto the imaging medium to display the corresponding projected image on the imaging medium. The projection device may include, but is not limited to, a projector, a projection television, and the like. For example, the projection device may be a short-throw or ultra-short-throw projector, or a telephoto projector. The image acquisition device is used to capture the projection image displayed on the imaging medium. The image acquisition device can be a camera module (such as a camera, an image sensor, etc.) mounted on the projection device itself, or it can be connected to the projection device through a wired or wireless connection. The separated and movable camera module can also be a terminal with a shooting function (ie, third-party hardware), for example, the terminal can be a smart phone, a tablet computer, a smart watch, a smart bracelet, a PDA (English: Personal Digital Assistant, Chinese: personal digital assistant) and other terminals. The imaging medium may be a projection screen, a wall, a hard screen, or any other device capable of displaying the projection of the projection device, which is not specifically limited in the present disclosure.

In a scenario, when the interior of the projected image projected by the projection device on the imaging medium is distorted due to the uneven surface of the imaging medium, the projected image can be corrected in the manner shown in the following embodiment:

Fig. 1 is a flowchart of a projection correction method according to an exemplary embodiment. As shown in Figure 1, applied to a projection device, the method may include the following steps:

Step 101: Project a preset corrected image onto an imaging medium to obtain a projected corrected image displayed on the imaging medium, and acquire a captured image.

The corrected image includes a plurality of first marking patterns, the collected image includes a second marking pattern corresponding to each of the first marking patterns, and the collected image is determined according to an image obtained by photographing the projected corrected image by an image collecting device.

For example, when a projection device projects an image onto an imaging medium, if the user finds that the projection image corresponding to the image is distorted (for example, distortion, translation, and deformation appear in the projection image), the user can send the message to the projection device. Correction commands to correct the projected image. For example, a corresponding correction button may be provided on the projection device, and the user may send a correction instruction to the projection device by pressing the correction button. Afterwards, the projection device can send corresponding prompt information to the user to prompt the user to adjust the position of the image capture device to a suitable position, so that the image capture device can accurately capture the projected image of the projection device on the imaging medium, so as to more accurately capture the image. Correct the projected image.

Specifically, when the image acquisition device is a movable camera module separated from the projection device, after receiving the correction instruction, the projection device can first project a prompt image on the imaging medium to prompt the user to move to the designated location Mobile camera module. For example, the camera module can shoot the prompt image, and send the image obtained by shooting the prompt image to the projection device, and the projection device determines whether the image obtained by the camera module shooting the prompt image and the prompt image meet the preset adjustment conditions. If the preset adjustment conditions are met, the projection device may determine that the user has adjusted the camera module to an appropriate position. Otherwise, the projection device may determine that the user has not adjusted the camera module to a proper position, and the projection device needs to project a new prompt image again to prompt the user to continue adjusting the camera module. The preset adjustment condition may be, for example, that the image obtained by the camera module shooting the prompt image includes the prompt image, and the proportion of the prompt image to the image obtained by the camera module shooting the prompt image is greater than or equal to a threshold (eg 50%). After the camera module is adjusted to an appropriate position, the projection device can project the corrected image including a plurality of first marking patterns onto the imaging medium to obtain the projected corrected image displayed on the imaging medium, and then the projected corrected image is displayed by the camera module. Shooting is performed, and the image obtained by shooting the projection-corrected image is sent to the projection device. The image obtained by shooting the projection correction image is preprocessed by the projection device (for example, the preprocessing may be: image format conversion, segmentation processing, and image size conversion, etc.) to obtain the captured image. Wherein, the captured image includes a second marking pattern corresponding to each first marking pattern.

When the image acquisition device is a mobile phone with a shooting function, after receiving the correction instruction, the projection device can first project a prompt image on the imaging medium to prompt the user to use the mobile phone to pair the mobile phone with the projection device. After successful pairing, the projection device can project the corrected image onto the imaging medium to obtain the projected corrected image, and then prompts the user to align the camera interface of the mobile phone with the projected corrected image, and prompts the user to use the mobile phone to shoot the projected corrected image. Afterwards, the projection device may acquire the image obtained by the image acquisition device shooting the projection correction image, and preprocess the image obtained by shooting the projection correction image to obtain the acquisition image. Further, in order to reduce the amount of data that the projection device needs to process, the mobile phone can also preprocess the image obtained by shooting the projection correction image to obtain the captured image, and send the obtained captured image to the projection device.

The corrected image may be any image including a plurality of first marking patterns, and the position of each first marking pattern may be used as a reference for correcting the projected image. As shown in FIG. 2 , when the corrected image is a checkerboard image, the first marking pattern can be any square in the checkerboard, and the projection device projects the corrected image onto the imaging medium, actually projecting the checkerboard onto the imaging medium superior. At this time, the image obtained by the image acquisition device shooting the projection-corrected image also includes a checkerboard, then the captured image obtained by preprocessing the image obtained by shooting the projection-corrected image also includes a checkerboard (that is, the captured image is also a checkerboard image) , the second mark pattern can be any square in the checkerboard.

Step 102: Determine the first position information of each second mark pattern in the captured image.

In this step, the projection device may detect each second marker pattern in the captured image, and determine the first position information of each detected second marker pattern. Taking the corrected image as a checkerboard image and the first marking pattern as any square in the checkerboard as an example, the projection device can detect the checkerboard in the captured image, and determine each square included in the checkerboard in the captured image For example, the coordinates of the four corners of each square can be used as the location information of the square. Afterwards, the projection device may use the position information of each square included in the checkerboard in the captured image as the first position information.

Step 103 , according to the first position information of each second mark pattern and the second position information of each first mark pattern, determine a first projection area corresponding to the projection correction image in the captured image.

For example, prior information of the corrected image may be pre-stored in the projection device, and the prior information may include position information of the boundary of the corrected image and second position information of each first marker pattern in the corrected image. For example, when the corrected image is a checkerboard image, the coordinates of the four corners of each square (ie, the first marking pattern) included in the checkerboard in the corrected image may be used as the second position information of the square. Then, the projection device may determine the positional relationship between each first marking pattern and the boundary of the corrected image according to the second position information of each first marking pattern. Since the collected image is determined according to the image obtained by shooting the projected corrected image by the image collecting device, and the projected corrected image is obtained by projecting the corrected image. Therefore, the positional relationship between each second marker pattern and the boundary formed by the projected corrected image in the captured image is the same as the positional relationship between each first marker pattern and the boundary of the corrected image. Then, the projection device can determine the boundary formed by the projection correction image in the captured image according to the positional relationship between each first marking pattern and the boundary of the correction image, and use the area enclosed by the boundary as the first projection area.

Step 104: Determine a second projection area corresponding to the first projection area on the imaging medium, where the second projection area includes a third mark pattern corresponding to each first mark pattern.

For example, after determining the first projection area, the projection apparatus may first calculate a homography matrix between the imaging plane of the image capturing device and the projection plane where the imaging medium is located. For example, when the main optical axis of the projector of the projection device is parallel to the normal vector of the imaging medium, the projection plane and the projection plane of the projector can be regarded as the same plane. Therefore, the homography matrix between the imaging plane and the projection plane can be calculated using the first position information of each second marker pattern and the second position information of each first marker pattern. Wherein, for calculating the homography matrix between the imaging plane and the projection plane by using the first position information and the second position information, reference may be made to the method described in the related art, and details are not repeated here. Then, the projection device can map the first projection area on the projection plane through the homography matrix to obtain the second projection area. Wherein, the second projection area includes a third mark pattern corresponding to each of the first mark patterns. The second projection area can be understood as: the area corresponding to the projection correction image simulated by the projection device according to the acquired image on the projection plane, and the third mark pattern can be understood as: the first mark pattern simulated by the projection device according to the acquired image on the projection plane The corresponding pattern above, that is, the third marking pattern corresponding to each first marking pattern is: when the first marking pattern is projected onto the projection plane, the pattern formed by the first marking pattern on the projection plane.

Step 105: Determine the third position information of each third mark pattern and the offset parameter corresponding to the third mark pattern according to the second position information.

Further, when the projection plane and the projection plane of the light projector are the same plane, since the second projection area is obtained by mapping the first projection area on the imaging plane on the projection plane, the second projection area Has the same pixel size as the acquired image. For example, when the pixel size of the captured image is 2K, the pixel size of the second projection area is also 2K. However, in practical situations, the pixel size of the projected corrected image may be different from that of the second projection area (the projected corrected image has the same pixel size as the corrected image). The pixel size is converted into the same pixel size as the second projection area, and the converted corrected image is compared with the second projection area to determine the third position information of each third mark pattern in the second projection area, and each The offset parameter corresponding to the third mark pattern.

For example, the projection device may determine the proportional relationship between the corrected image and the captured image according to the pixel size of the corrected image and the pixel size of the captured image, and according to the proportional relationship, in combination with the second position information of each first marking pattern, determine The ideal position of the third mark pattern corresponding to the first mark pattern in the second projection area when the projected corrected image is not distorted. Then, according to the ideal position of the third marking pattern corresponding to the first marking pattern, the third position information of each third marking pattern and the offset parameter corresponding to each third marking pattern are determined. Wherein, the offset parameter corresponding to each third mark pattern is used to represent the deviation between the third position information of the third mark pattern and the ideal position.

Step 106 , in the case that the offset parameter does not meet the preset condition, according to the first position information and the third position information, determine a target correction matrix corresponding to each first mark pattern.

For example, after determining the offset parameter corresponding to each third marking pattern, the projection device may first determine whether the offset parameter satisfies the preset condition. For example, when the offset parameter is the pixel distance, the preset condition may be: the average value of the pixel distances (ie the offset parameter) corresponding to all the third marking patterns is less than the average threshold (the average threshold may be, for example, 3 pixels), and The largest pixel distance among the pixel distances corresponding to all the third marking patterns is smaller than the distance threshold (for example, the distance threshold may be 5 pixels).

In the case where the offset parameter does not meet the preset condition, the projection device may, for each first marking pattern, use the first position information of the first marking pattern and the third marking pattern of the third marking pattern corresponding to the first marking pattern. The position information is used to determine the initial correction matrix corresponding to the first marking pattern. The initial calibration matrix can be a homography matrix, for example, when the first marking pattern and the third marking pattern are any squares included in a checkerboard, the second position information of each first marking pattern is the first marking pattern ( In the case where the coordinates of the four corner points of the third marking pattern (that is, the square), and the third position information of each third marking pattern is the coordinates of the four corners of the third marking pattern (that is, the square), it can be determined according to each The coordinates of the four corners of the first marking pattern and the four corners of the third marking pattern corresponding to the first marking pattern are used to calculate the initial correction matrix of the first marking pattern. Afterwards, the projection device can correct the corrected image according to the initial correction matrix, that is, adjust the first marking pattern according to the initial correction matrix corresponding to each first marking pattern, so as to reduce the amount of noise when the first marking pattern is projected onto the imaging medium. Distortion of the first marking pattern. Then, the projection device may repeatedly perform the steps of projecting the preset corrected image onto the imaging medium according to the corrected corrected image, so as to obtain the projected corrected image displayed on the imaging medium, to the step of correcting the corrected image according to the initial correction matrix, until The offset parameter satisfies the preset condition. Finally, the initial correction matrix is used as the target correction matrix.

Step 107: Correct the to-be-projected image according to the target correction matrix, and project the corrected to-be-projected image onto the imaging medium.

In this step, the projection device may determine the first target area corresponding to each first marker pattern in the corrected image, and use the area in the image to be projected that is the same as the first target area corresponding to the first marker pattern as the The first marking pattern corresponds to the second target area in the to-be-projected image. Then, the projection device can correct the second target area corresponding to the first mark pattern in the to-be-projected image according to the target correction matrix corresponding to each first mark pattern, so as to offset the projection of the second target area corresponding to the first mark pattern to the target area. On the imaging medium, the second target area corresponding to the first marking pattern is distorted, so that when the projection device projects the to-be-projected image on the imaging medium, the projected image corresponding to the to-be-projected image is no longer distorted.

FIG. 3 is a flowchart of step 101 shown in the embodiment shown in FIG. 1 . As shown in FIG. 3, step 101 may include the following steps:

Step 1011: Convert the image obtained by the image acquisition device to shoot the projection corrected image into a specified format to obtain a first image.

Step 1012 , perform segmentation processing on the first image to remove the area of the first image except the projection corrected image to obtain a second image.

Step 1013: Convert the second image to a specified image size to obtain a captured image.

For example, after receiving the image obtained by shooting the projection correction image sent by the image acquisition device, the projection device needs to preprocess the image obtained by shooting the projection correction image, so as to obtain a standardized acquisition image, so that the projection image can be subsequently processed. Make corrections. Specifically, the image obtained by shooting the projection correction image may be converted into a specified format to obtain the first image, and the specified format may be, for example, a BMP (English: Bitmap, Chinese: Bitmap File) graphic file format. The image obtained by shooting the projection-corrected image may also include other regions than the projection-corrected image. If the first image is directly used to correct the projection image, noise and data volume in subsequent processing will be increased. In order to reduce the noise and data volume in the subsequent processing, the projection device may perform segmentation processing on the first image (for example, a coarse-grained segmentation method may be used) to remove the area of the first image except the projection corrected image, and obtain the first image. Second image. Then, the projection device may convert the second image into a specified image size (for example: 720P), and use the converted second image as a captured image. Further, the projection device can also perform histogram equalization processing on the acquired image, and adjust the white balance and exposure time of the acquired image, so as to improve the quality of the acquired image.

It should be noted that, the second image obtained after the first image is segmented may still contain other regions than the projection corrected image. That is to say, preprocessing the image obtained by shooting the projection correction image is actually a coarse-grained processing method, and the acquired captured image does not only contain the projection correction image.

FIG. 4 is a flowchart of step 103 shown in the embodiment shown in FIG. 1 . As shown in FIG. 4, step 103 may include the following steps:

Step 1031: Determine the positional relationship between each first marking pattern and the first boundary of the corrected image according to each second positional information, where the positional relationship is used to represent the distance between the first marking pattern and the first boundary.

Step 1032 , according to the positional relationship and each first positional information, determine the first region corresponding to the projection correction image in the captured image.

Illustratively, the projection device may calculate the distance between each first marking pattern and the first boundary according to the position information of the first boundary of the corrected image and the second position information of each first marking pattern in the prior information of the corrected image, to obtain the positional relationship between the first marking pattern and the first boundary. Then, according to the positional relationship between each first marking pattern and the first boundary, the projection device can use the proportional relationship between the pixel size of the corrected image and the pixel size of the captured image to determine the second marking pattern corresponding to the first marking pattern and the The positional relationship of the pending boundaries. Afterwards, the projection device can use the positional relationship between each second marking pattern and the pending boundary to determine the pending boundary corresponding to the projection correction image in the captured image. Then, the undetermined boundary is expanded outward by a specified pixel distance (for example: 20 pixels), and the area enclosed by the outwardly expanded undetermined boundary is used as the first area. Wherein, the undetermined boundary can be understood as the boundary formed by the pre-assumed projection correction image in the acquired image, and the first area can be understood as the approximate range of the projection correction image in the acquired image.

Step 1033: Determine the second area from the first area according to the color feature of the first area.

Step 1034: Perform edge recognition on the second area to obtain a second boundary, and use the area enclosed by the second boundary as the first projection area.

Further, the projection device can extract the color feature of the first area, and according to the color feature of the first area, in combination with the color feature difference between the first boundary and the surrounding area in the prior information of the corrected image, search for the sum from the first area. The area with the same color feature difference is used as the second area. Then, the projection device may perform edge recognition on the second area to identify the boundary corresponding to the second area, and then use the boundary as the second boundary, and use the area enclosed by the second boundary as the first projection area. Wherein, to obtain the second boundary by performing edge recognition on the second region, reference may be made to the manner described in the related art, which will not be repeated here. It should be noted that the process of obtaining the first projection area is actually a fine-grained processing manner to further remove other areas in the captured image except the projection corrected image.

FIG. 5 is a flowchart of step 104 shown in the embodiment shown in FIG. 1 . As shown in FIG. 5, step 104 may include the following steps:

Step 1041: Determine a transformation matrix according to each first position information and each second position information.

Step 1042: Determine the second projection area according to the transformation matrix.

For example, after determining the first projection area, the projection device may first calculate the transformation between the imaging plane and the projection plane according to the first position information of each second marker pattern and the second position information of each first marker pattern. matrix. The transformation matrix is used to convert the imaging plane into the projection plane, and the transformation matrix may be, for example, a homography matrix. Afterwards, the projection device may map the first projection area on the projection plane according to the transformation matrix to obtain the second projection area.

FIG. 6 is a flowchart of step 105 shown in the embodiment shown in FIG. 1 . As shown in FIG. 6 , the first mark pattern is any first square included in the first checkerboard, the third mark pattern is any second square included in the second checkerboard, and the second position information includes the first square The coordinates of each first corner of the grid. Step 105 may include the following steps:

Step 1051: Determine the proportional relationship between the corrected image and the collected image according to the pixel size of the corrected image and the pixel size of the collected image.

For example, when the corrected image is a checkerboard image, the captured image and the second projection area obtained by the projection device are actually checkerboard images, and the checkerboard in the corrected image can be set as the first checkerboard, Sets the checkerboard in the second projected area as the second checkerboard. In this case, the first mark pattern is any first square included in the first checkerboard, and the first square includes 4 first corner points, so the second position information of the first mark pattern may include every first square of the first square. The coordinates of the first corner points, the third mark pattern is any second square included in the second checkerboard, and the second square includes four second corners. After determining the second position information, the projection device may determine the proportional relationship between the corrected image and the captured image according to the pixel size of the corrected image and the pixel size of the captured image. For example, when the pixel size of the corrected image is 1080P, and the pixel size of the captured image is 2K, the scale relationship is 2.

Step 1052, for each first square, according to the coordinates of each first corner point of the first square and using the proportional relationship, determine that each first corner point of the first square is in the second projection area The coordinates of the ideal corner point of .

Step 1053 , according to the coordinates of the ideal corner point, determine whether the ideal corner point is the second corner point of the target second square, which is the second square corresponding to the first square in the second projection area.

Step 1054, if the ideal corner point is the second corner point of the target second square, use the coordinates of the ideal corner point as the third position information of the target second square.

Step 1055, if the ideal corner point is not the second corner point of the target second square, the coordinates of the adjacent corner point closest to the ideal corner point are taken as the third position information of the target second square, and the adjacent corner point is used as the third position information. The pixel distance from the ideal corner as the offset parameter for the second square of the target.

Specifically, for each first square, the projection device may determine, according to the coordinates of each first corner of the first square, and by using a proportional relationship, that each first corner of the first square is in the second square. Coordinates of ideal corner points in the projected area. The ideal corner point of each first corner point is the coordinate corresponding to the first corner point in the second projection area under ideal conditions (ie, when the projection correction image is not distorted). For example, when the coordinates of the first corner point are (1, 1) and the proportional relationship is 2, the coordinates of the ideal corner point may be (2, 2). Then, the projection device can determine whether the ideal corner point of the first corner point is the second corner point of the target second square according to the coordinates of the ideal corner point of each first corner point of the first square, wherein , the target second square is the second square corresponding to the first square in the second projection area.

If the ideal corner of the first corner is the second corner of the target second square, it means that the first corner is not distorted when projected onto the imaging medium, and the ideal corner of the first corner is the second corner point of the target second square, the projection device may use the coordinates of the ideal corner point of the first corner point as the third position information of the target second square. If the ideal corner point of the first corner point is not the second corner point of the target second square, it means that the first corner point is distorted when projected onto the imaging medium, and the projection device can project the closest corner point to the ideal corner point. The adjacent corner point of , is used as the second corner point of the second target square, and the coordinates of the adjacent corner point are used as the third position information of the target second square. At the same time, the projection device can use the pixel distance between the adjacent corner point and the ideal corner point as the offset parameter of the second square of the target. It should be noted that the third position information of each second square includes the coordinates of the four second corners, and the coordinates of the four second corners may only include the coordinates of the ideal corners (that is, the four second corners). There is no distortion at all points), or it can only include the coordinates of the adjacent corner points (that is, the four second corner points are all distorted), or it can include both the coordinates of the ideal corner point and the coordinates of the adjacent corner points (that is, 4 Some of the second corners may be distorted).

To sum up, the present disclosure first obtains the projected corrected image displayed on the imaging medium by projecting the preset corrected image onto the imaging medium, acquires the captured image, and then determines the size of each second mark pattern in the captured image. the first position information, and according to the first position information of each second mark pattern and the second position information of each first mark pattern, determine the first projection area corresponding to the projection correction image in the captured image, and then determine the first The projection area corresponds to the second projection area on the imaging medium, and according to the second position information and the second projection area, the third position information of each third mark pattern and the offset parameter corresponding to the third mark pattern are determined, and then In the case that the offset parameter does not meet the preset condition, the target correction matrix corresponding to each first mark pattern is determined according to the first position information and the third position information, and finally the projected image is corrected according to the target correction matrix, and the The corrected image to be projected is projected onto the imaging medium. The present disclosure corrects the to-be-projected image through the determined target correction matrix corresponding to each first mark pattern, and projects the corrected to-be-projected image onto the imaging medium, so as to ensure that the to-be-projected image is not projected onto the imaging medium when projected onto the imaging medium. Distortion will occur, the efficiency of correcting the projection is high, and the projection effect of the projection device is improved.

Fig. 7 is a block diagram of a projection correction apparatus according to an exemplary embodiment. As shown in FIG. 7, applied to projection equipment, the apparatus 200 includes:

The acquiring module 201 is configured to project a preset corrected image onto an imaging medium to obtain a projected corrected image displayed on the imaging medium, and acquire a captured image. The corrected image includes a plurality of first marking patterns, the collected image includes a second marking pattern corresponding to each of the first marking patterns, and the collected image is determined according to an image obtained by photographing the projected corrected image by an image collecting device.

The processing module 202 is configured to determine the first position information of each second mark pattern in the captured image.

The processing module 202 is further configured to determine, according to the first position information of each second mark pattern and the second position information of each first mark pattern, a first projection area corresponding to the projection correction image in the captured image.

The processing module 202 is further configured to determine a second projection area corresponding to the first projection area on the imaging medium, where the second projection area includes a third mark pattern corresponding to each first mark pattern.

The processing module 202 is further configured to determine, according to the second position information, third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern.

The processing module 202 is further configured to determine a target correction matrix corresponding to each first mark pattern according to the first position information and the third position information when the offset parameter does not meet the preset condition.

The correction module 203 is configured to correct the to-be-projected image according to the target correction matrix, and project the corrected to-be-projected image onto the imaging medium.

Optionally, the obtaining module 201 includes:

The first conversion sub-module is used to convert the image obtained by the image acquisition device shooting the projection corrected image into a specified format to obtain the first image.

The segmentation sub-module is used to perform segmentation processing on the first image, so as to remove the area in the first image except the projection corrected image to obtain the second image.

The second conversion submodule is used to convert the second image into a specified image size to obtain the captured image.

Optionally. The processing module 202 includes:

The detection submodule is used to detect the checkerboard in the collected image, and determine the position information of each square included in the checkerboard in the collected image.

The first determination submodule is used for taking the position information of each square included in the checkerboard in the collected image as the first position information.

Optionally, the processing module 202 includes:

The second determination sub-module is configured to determine the positional relationship between each first marking pattern and the first boundary of the corrected image according to each second positional information, where the positional relationship is used to characterize the distance between the first marking pattern and the first boundary .

The second determination submodule is configured to determine, according to the positional relationship and each first positional information, the first region corresponding to the projection correction image in the captured image.

The second determination submodule is configured to determine the second area from the first area according to the color feature of the first area.

The identification sub-module is used for performing edge identification on the second area to obtain the second boundary, and the area enclosed by the second boundary is used as the first projection area.

Optionally, the processing module 202 includes:

The calculation submodule is configured to determine the transformation matrix according to each of the first position information and each of the second position information.

The third determination sub-module is configured to determine the second projection area according to the transformation matrix.

Optionally, the first mark pattern is any first square included in the first checkerboard, the third mark pattern is any second square included in the second checkerboard, and the second position information includes the first square. For the coordinates of each first corner point, the processing module 202 is used to:

According to the pixel size of the corrected image and the pixel size of the acquired image, the proportional relationship between the corrected image and the acquired image is determined.

For each first square, according to the coordinates of each first corner point of the first square, and using proportional relationship, determine the ideal angle of each first corner point of the first square in the second projection area the coordinates of the point.

According to the coordinates of the ideal corner point, it is determined whether the ideal corner point is the second corner point of the second target square, which is the second square corresponding to the first square in the second projection area.

If the ideal corner point is the second corner point of the target second square, the coordinates of the ideal corner point are used as the third position information of the target second square.

If the ideal corner is not the second corner of the second grid of the target, the coordinates of the adjacent corner closest to the ideal corner are taken as the third position information of the second grid of the target, and the adjacent corner and the ideal corner are compared with each other. The pixel distance between the points, as the offset parameter of the second square of the target.

Optionally, the processing module 202 is used to:

For each first marking pattern, an initial correction matrix corresponding to the first marking pattern is determined according to the first position information of the first marking pattern and the third position information of the third marking pattern corresponding to the first marking pattern.

The corrected image is corrected according to the initial correction matrix, and the preset corrected image is projected onto the imaging medium repeatedly according to the corrected corrected image to obtain the projected corrected image displayed on the imaging medium, until the corrected image is corrected according to the initial correction matrix. The steps of correction are performed until the offset parameters meet the preset conditions.

Take the initial correction matrix as the target correction matrix.

Regarding the apparatus in the above-mentioned embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.

To sum up, the present disclosure first obtains the projected corrected image displayed on the imaging medium by projecting the preset corrected image onto the imaging medium, acquires the captured image, and then determines the size of each second mark pattern in the captured image. first position information, and according to the first position information of each second mark pattern and the second position information of each first mark pattern, determine the first projection area corresponding to the projection correction image in the captured image, and then determine the first The projection area corresponds to the second projection area on the imaging medium, and according to the second position information and the second projection area, the third position information of each third mark pattern and the offset parameter corresponding to the third mark pattern are determined, and then In the case that the offset parameter does not meet the preset condition, the target correction matrix corresponding to each first mark pattern is determined according to the first position information and the third position information, and finally the projected image is corrected according to the target correction matrix, and the The corrected image to be projected is projected onto the imaging medium. The present disclosure corrects the to-be-projected image through the determined target correction matrix corresponding to each first mark pattern, and projects the corrected to-be-projected image onto the imaging medium, so as to ensure that the to-be-projected image is not projected onto the imaging medium when projected onto the imaging medium. Distortion will occur, the efficiency of correcting the projection is high, and the projection effect of the projection device is improved.

FIG. 8 is a block diagram of an electronic device 300 according to an exemplary embodiment. As shown in FIG. 8 , the electronic device 300 may include: a processor 301 and a memory 302 . The electronic device 300 may also include one or more of a multimedia component 303 , an input/output (I/O) interface 304 , and a communication component 305 .

Wherein, the processor 301 is used to control the overall operation of the electronic device 300 to complete all or part of the steps in the above-mentioned projection correction method. The memory 302 is used to store various types of data to support operations on the electronic device 300, such data may include, for example, instructions for any application or method operating on the electronic device 300, and application-related data, Such as contact data, messages sent and received, pictures, audio, video, and so on. The memory 302 can be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM for short), electrically erasable programmable read-only memory ( Electrically Erasable Programmable Read-Only Memory (EEPROM for short), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read-Only Memory, ROM for short), magnetic memory, flash memory, magnetic disk or optical disk. Multimedia components 303 may include screen and audio components. Wherein the screen can be, for example, a touch screen, and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in memory 302 or transmitted through communication component 305 . The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 304 provides an interface between the processor 301 and other interface modules, and the above-mentioned other interface modules can be a keyboard, a mouse, a button, and the like. These buttons can be virtual buttons or physical buttons. The communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or one or more of them The combination is not limited here. Therefore, the corresponding communication component 305 may include: Wi-Fi module, Bluetooth module, NFC module and so on.

In an exemplary embodiment, the electronic device 300 may be implemented by one or more application-specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), digital signal processors (Digital Signal Processor, DSP for short), digital signal processing devices (Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic components Implementation is used to perform the above-mentioned projection correction method.

In another exemplary embodiment, there is also provided a computer-readable storage medium comprising program instructions, the program instructions implement the steps of the above-mentioned projection correction method when executed by a processor. For example, the computer-readable storage medium can be the above-mentioned memory 302 including program instructions, and the above-mentioned program instructions can be executed by the processor 301 of the electronic device 300 to complete the above-mentioned projection correction method.

In another exemplary embodiment, there is also provided a computer program product comprising a computer program executable by a programmable apparatus, the computer program having, when executed by the programmable apparatus, for performing the above The code section of the projection correction method.

The preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings. However, the present disclosure is not limited to the specific details of the above-mentioned embodiments. Various simple modifications can be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure. These simple modifications all fall within the protection scope of the present disclosure.

In addition, it should be noted that the various specific technical features described in the above-mentioned specific embodiments can be combined in any suitable manner unless they are inconsistent. In order to avoid unnecessary repetition, the present disclosure provides The combination method will not be specified otherwise.

In addition, the various embodiments of the present disclosure can also be arbitrarily combined, as long as they do not violate the spirit of the present disclosure, they should also be regarded as the contents disclosed in the present disclosure.

Claims

A projection correction method, characterized in that, applied to projection equipment, the method comprising:

Projecting a preset correction image onto an imaging medium to obtain a projected correction image displayed on the imaging medium, and acquiring a captured image; the corrected image includes a plurality of first mark patterns, and the captured image includes and each a second marking pattern corresponding to the first marking pattern, and the captured image is determined according to an image obtained by photographing the projection correction image by an image capturing device;

determining first position information of each of the second marking patterns in the captured image;

determining, according to the first position information of each of the second marking patterns and the second position information of each of the first marking patterns, a first projection area corresponding to the projection correction image in the captured image;

determining a second projection area corresponding to the first projection area on the imaging medium, the second projection area including a third mark pattern corresponding to each of the first mark patterns;

According to the second position information, determine the third position information of each of the third marking patterns and the offset parameter corresponding to the third marking pattern;

In the case that the offset parameter does not meet the preset condition, according to the first position information and the third position information, determine a target correction matrix corresponding to each of the first marking patterns;

The to-be-projected image is corrected according to the target correction matrix, and the corrected to-be-projected image is projected onto the imaging medium.
The method according to claim 1, wherein the acquiring the captured image comprises:

Converting the image obtained by the image acquisition device shooting the projection correction image into a specified format to obtain a first image;

Performing segmentation processing on the first image to remove the area of the first image except the projection correction image to obtain a second image;

Converting the second image to a specified image size to obtain the acquired image.
The method according to claim 1, wherein the corrected image is a checkerboard image, and the first mark pattern is any square in the checkerboard; the determining of each of the collected images The first position information of the second marking pattern includes:

Detecting the checkerboard in the collected image, and determining the position information of each square included in the checkerboard in the collected image;

The position information of each square included in the checkerboard in the acquired image is used as the first position information.
The method according to claim 1, wherein the projection correction image is determined according to the first position information of each of the second marking patterns and the second position information of each of the first marking patterns The corresponding first projection area in the captured image includes:

According to each of the second position information, the positional relationship between each of the first marking patterns and the first boundary of the corrected image is determined, and the positional relationship is used to characterize the first marking pattern and the first boundary the distance;

determining, according to the positional relationship and each of the first positional information, a first region corresponding to the projection correction image in the captured image;

determining a second area from the first area according to the color feature of the first area;

Perform edge recognition on the second area to obtain a second boundary, and use the area enclosed by the second boundary as the first projection area.
The method according to claim 1, wherein the determining the second projection area corresponding to the first projection area on the imaging medium comprises:

determining a transformation matrix according to each of the first position information and each of the second position information;

According to the transformation matrix, the second projection area is determined.
The method according to claim 1, wherein the first mark pattern is any first square included in the first checkerboard, and the third mark pattern is any second checkerboard included in the second checkerboard. a square, the second position information includes the coordinates of each first corner of the first square; the third position information of each of the third mark patterns is determined according to the second position information The offset parameters corresponding to the third marking pattern include:

determining a proportional relationship between the corrected image and the acquired image according to the pixel size of the corrected image and the pixel size of the acquired image;

For each of the first squares, according to the coordinates of each of the first corners of the first square, and using the proportional relationship, it is determined that each of the first corners of the first square is in the second the coordinates of the ideal corners in the projected area;

According to the coordinates of the ideal corner point, it is determined whether the ideal corner point is the second corner point of the target second square, and the target second square is the corresponding first square in the second projection area. second square;

If the ideal corner point is the second corner point of the target second square, use the coordinates of the ideal corner point as the third position information of the target second square;

If the ideal corner point is not the second corner point of the target second square, the coordinates of the adjacent corner point closest to the ideal corner point are used as the third position information of the target second square, And the pixel distance between the adjacent corner point and the ideal corner point is used as the offset parameter of the second target square.
The method according to claim 1, wherein the determining a target correction matrix corresponding to each of the first marking patterns according to the first position information and the third position information comprises:

For each of the first marking patterns, an initial calibration matrix corresponding to the first marking pattern is determined according to the first position information of the first marking pattern and the third position information of the third marking pattern corresponding to the first marking pattern ;

Correct the correction image according to the initial correction matrix, and repeat the projecting the preset correction image onto the imaging medium according to the corrected correction image, so as to obtain the projection correction displayed on the imaging medium image, to the step of correcting the corrected image according to the initial correction matrix, until the offset parameter satisfies the preset condition;

The initial correction matrix is used as the target correction matrix.
A projection correction device, characterized in that, when applied to projection equipment, the device comprises:

an acquisition module, configured to project a preset corrected image onto an imaging medium to obtain a projected corrected image displayed on the imaging medium, and acquire a captured image; the corrected image includes a plurality of first mark patterns, and the captured image The image includes a second marking pattern corresponding to each of the first marking patterns, and the acquired image is determined according to an image obtained by photographing the projection correction image by an image acquisition device;

a processing module, configured to determine the first position information of each of the second marking patterns in the captured image;

The processing module is further configured to determine, according to the first position information of each of the second marking patterns and the second position information of each of the first marking patterns, whether the projection correction image corresponds to the captured image. The first projection area of ;

The processing module is further configured to determine a second projection area corresponding to the first projection area on the imaging medium, where the second projection area includes a third mark pattern corresponding to each of the first mark patterns ;

The processing module is further configured to determine, according to the second position information, third position information of each of the third marking patterns and an offset parameter corresponding to the third marking pattern;

The processing module is further configured to determine a target corresponding to each of the first marking patterns according to the first position information and the third position information when the offset parameter does not meet a preset condition correction matrix;

The correction module is used for correcting the to-be-projected image according to the target correction matrix, and projecting the corrected to-be-projected image onto the imaging medium.
A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the steps of the method according to any one of claims 1-7 are implemented.
An electronic device, comprising:

a memory on which a computer program is stored;

A processor for executing the computer program in the memory to implement the steps of the method of any one of claims 1-7.